APPLICATION OF ESCO TO DOCUMENT LIFELONG LEARNING - CASE STUDIES IN GERMAN CONTINUING EDUCATION INSTITUTIONS

The rapidly changing labor market and the promotion of labor mobility lead to a demand for lifelong learning as well as the need for transparency and comparability of competencies. The application of skill classifications to describe competencies on educational certificates provides a way to document lifelong learning in a standardized way. In this paper, case studies regarding the application of the ESCO classification in three continuing education institutions are described. The results show that ESCO is a useful tool to document learning outcomes of continuing education courses in a comparable and transparent way but the mapping of competencies requires a lot of effort and differs in quality. The presented analysis identifies steps for further development of ESCO and for the change process when it comes to the standardized documentation of skills and competencies


Documentation of lifelong learning pathways via digital certificates
The global need for qualified workers in a rapidly changing labor market that includes new job profiles as well as new kinds of key skills and competencies regarding digitalization or sustainability creates a high demand for lifelong learning.At the same time, there is a variety of education and training opportunities that is not easy to evaluate or compare while assessing the suitability of employees for specific professional tasks or job profiles.
There are political initiatives to address this problem using standardized descriptions of occupations, skills and competencies or qualifications.Currently, these are mainly used to establish skills monitoring systems that provide information on labor market needs (Siekmann & Fowler, 2017).The classifications are intended to ensure statistical and research-based processes for decision-making on labor market strategies, but are not yet widespread in practice of the continuing education system.
In the project "MyEduLife" we assume that using standardized skills descriptions on educational certificates helps to enable the documentation of lifelong learning and to ensure a comparability and permeability in the continuing education and training system.For that purpose, we decided to use the European classification for Skills, Competences, Qualifications and Occupations (ESCO) that has been introduced by the European Commission to strengthen European labor market mobility and transparency (Europäische Kommission, 2019).In this paper, we describe the application of the ESCO classification in three German continuing education institutions.Case studies were conducted and accompanied the process of identifying relevant ESCO concepts to be used for standardized representation of learning outcomes on educational certificates.

Regulations and standards in the field of continuing education and training in Germany
The German system of continuing education and training is regulated at the national level with different types of regulations for the various educational offers.The field of vocational continuing education and training is separated into further professional education and further vocational training, both of which have different regulatory conditions.On the one hand, further education courses aim to provide higher professional qualifications and are regulated by the state through so-called framework plans, which specify learning objectives, content and scope.There are around 400 state-regulated further education courses (Bundesinstitut für Berufsbildung, 2013).On the other hand, further training courses aim at maintaining and expanding existing skills to meet current occupational requirements and are developed based on market conditions without specific state regulations.In some cases, there are regulations by ordinance or professional associations that affect those further training courses instead.
Especially in the field of further education, there are changes in the respective regulations on learning objectives and content that are negotiated by the relevant federal ministries, the social partners, and the trade associations responsible for the respective occupation under the direction of the Federal Institute for Vocational Education and Training (BIBB) (Bundesinstitut für Berufsbildung, 2013).
So far, the German Federal Employment Agency (BA) provides a standard for the description of competencies using a specific classification called Dokumentationskennzahlen (DKZ, documentation indicators).This classification is specific for the German national system using own labels and description for competencies that may be acquired during continuing education programs.The DKZ is linked to ESCO through a mapping conducted by experts of the corresponding competencies to the ESCO-pillar of skills and competencies.
For the issuance of certificates, there are no regulations regarding the information contained.Therefore, it is not clear to what extend the more than 60,000 existing educational institutions (Schrader & Martin, 2021) provide information on learning outcomes and whether or not standards are applied.

Application of ESCO in German continuing education and training
The ESCO classification provides a structured vocabulary to describe occupations as well as knowledge and skills in a standardized way and is available in 27 different languages (including German).Since the beginning of this year it is available in version 1.1.1.The ESCO is organized along three pillars: occupations (based on ISCO-08), skills and competencies and qualifications, with the latter not being completed yet, due to conflicting national interests.There are connections between the pillar for occupations and the pillar for skills and competencies linking typical skills and competencies for every occupation.
The skills and competencies pillar is further structured by the categories knowledge, skills, languages and transversal skills.More than 13,000 entries, organized by hierarchical levels, are located in this pillar.The number of levels differs for each subcategory.It is possible to connect to the classification via API.The database can be searched via web interface as well.
The introduction of ESCO into the German system is viewed skeptically regarding the governance structures especially in terms of the development of curricula for continuing education and training (Annen et al., 2020).National systems cannot be affected by EU regulations due to the principle of subsidiarity and the prohibition of harmonization (Annen et al., 2020).
There have been project-based initiatives to apply ESCO for the documentation of learning outcomes (Rentzsch et al., 2020;TALQ, 2018) that already identified difficulties in practical applicability.This involves the level of detail of knowledge and skills, which is not of the same quality for all professions.The TALQ project used the occupation of stage technicians to show that sector-specific additions are necessary in order to make a realistic assessment of the knowledge and skills possessed by employees (TALQ, 2018).In the context of continuing academic education, existing learning objectives were successfully linked to ESCO entries using the example of continuing education courses in museum education, digital forensics, and nursing (Rentzsch et al., 2020).Nevertheless, more research is needed to assess the applicability of ESCO for a widespread practical use and acceptance to document learning outcomes.

Research Design
To answer our research questions "To what extent can ESCO be used to document learning outcomes accurately and meaningful?" (RQ1) and "How do education and training institutions assess the applicability and usefulness of the ESCO classification?"(RQ2) we conducted case studies in three continuing education and training institutions as testing partners.Relevant ESCO concepts for their specific courses in the fields of trading, welding and voltage switching were identified and the whole process of applying the standard was systematically documented and evaluated based on a document analysis of the results and a group discussion.Figure 1 illustrates the procedure of the scientific monitoring of the application process.

Intervention
The application of ESCO for the documentation of learning outcomes on educational certificates was performed by the testing partners themselves.They are familiar with the respective educational content and are able to describe the contents, results and learning objectives of their continuing education programs in detail.Moreover, they are able to map ESCO concepts based on their adequate professional expertise.A joint preparatory workshop of 3.5h was held to introduce the procedure and ensure uniform documentation of the results.In this workshop, the courses suitable for the testing were determined and the testing partners were instructed on how to use the ESCO database.
Research and mapping of ESCO concepts was done manually by searching the public ESCO database for keywords or links between concepts of skills and/or occupations.For documentation purposes of the change process in the institutions, the testing partners were asked to fill in a template and provide information about (1) what information about the course content is currently shown on the certificate, (2) how this can be defined as a learning objective or learning outcome and (3) which equivalent can be found for this in the pillar of skills and competences of the ESCO database.Further, the testing partners were asked to document (4) any difficulties or specifics in applying the ESCO standard.It was possible to assign several ESCO concepts to one learning objective or one ESCO concept to several learning objectives.
The testing partners had approximately eleven weeks to complete the task (in addition to their regular work) with the possibility to receive help in case questions or uncertainties occurred.The scope of the task exceeded the time resources of the responsible persons, so that some of the completed templates could not be made available until 7 weeks later.

Document Analysis
The analysis of the documents created in the work and transfer phase was carried out by applying further criteria to qualitatively assess the respective mapping: − 5 -the learning objective is too specific to identify a match for it in ESCO.Accordingly, no ESCO concept was assigned at this point, so this category refers to the learning objective instead.

• Classification of the ESCO concept within the ESCO hierarchy
The aim of a more detailed description of the identified ESCO concepts based on their classification in the ESCO hierarchy was to identify overlaps or focal points within the hierarchy in order to find out whether individual concepts can be grouped together for a clearer presentation on the educational certificates.

Group discussion
To answer RQ2, a group discussion was conducted with four leading questions that addressed the initial situation in the educational institutions and the application of the ESCO standard.The group discussion was analyzed according to the rules of structuring content analysis (Mayring, 1994) using MAXQDA software.The main categories were derived from the interview guideline and revised by inductively developed categories and subcategories.The following category system emerged, as shown in Table 1.

Mapping ESCO concepts to learning outcomes
The testing institutions chose different courses and used learning objectives on different levels (coarse vs. fine learning objectives) and from different sources (regulations vs. own definition).This approach led to different numbers and quality of learning objectives as well as mapped ESCO concepts.The detailed results are shown in Table 2. None of the courses has a concordance concerning the number of learning objectives and the number of ESCO concepts.This means that there are no direct expressions of learning objectives in ESCO concepts, but there is the possibility to describe one learning objective using several ESCO concepts, at least for the two cases where coarse objectives were used for the mapping process.That is why, in these cases category 1 and category 3 allocations can be labeled good matchings.
For the 'trading' course, this means that 80 % of the identified ESCO concepts have a good quality, while there are 9 % that are even coarser and another 11 % that do not fit in terms of the definition of the learning objective.
Here, the mapping is likely due to the detailed knowledge of the subject matter experts, who know that further knowledge and skills are to be expected behind the learning objective in the context of the specific training.In this case, using ESCO may allow for greater transparency about the actual contents of the educational courses.
Regarding the course 'voltage switching', learning outcomes were used for the mapping that were defined by the training institution itself.The quality of the allocations is rated less specific.Only 48 % of the identified concepts can be labeled good quality.In contrast, 20 % do not match the descriptions of the learning outcomes and 32 % are coarser than the learning outcome.In this case, the validity on an educational credential is weakened in that the ESCO concept promises a greater skill set than was taught in the continuing education course.
Differences regarding the quality of the allocations might have various reasons, such as the quality of the learning objectives or the type and subject of the education or training course, which have to be analyzed in further research.
The example of the 'welding' course shows that for learning objectives defined at a very detailed level hardly suitable ESCO concepts are mapped.Not only does the huge number of learning objectives create a high workload in the mapping process, but the descriptions also lead to a reduced number of relevant ESCO concepts and a large number of duplicates.Overall, 35 learning objectives could not be allocated at all and 52 % were brought to a coarser level.Nevertheless, the number of allocations rated good quality is similar to that of the course 'voltage switching' (43 %).
These findings confirm the findings from other projects (TALQ, 2018) that more sector-specific concepts may be needed to create good quality mapping.With this in mind, it can be assumed that the results rated 'good quality' regarding the course 'trading' are based on a better representation of this sector in the database.However, other explanations might apply to this fact as well, such as the source of the learning objectives, which were derived from a well-defined state regulation.Overall, there is still a need to improve the quality of allocations and to enable sector-specific extensions.Thus, the use of ESCO can be viewed as "different" and it depends on the acceptance of the users (training institutions, participants, employers) to what extent it is established in practical use to describe skills acquired in training.

Application and acceptance of the classification
The acceptance and willingness to adopt ESCO in the certification processes of continuing education and training has been analyzed by a group discussion about the application of ESCO.None of the three institutions had used descriptions of learning objectives in general or in a standardized way using ESCO before.Only in some cases they record some areas of action or course content on the corresponding certificates.
The presentation of the learning outcomes of the exemplarily selected further education offers with the competence descriptions of the ESCO classification turned out to be more difficult than expected, as the testing partners had to face several challenges in the process.Among other things, the transformation process was difficult because certain prerequisites for the description of competencies according to ESCO were not given on the part of the educational institutions.Moreover, the classification and competence descriptions of ESCO did not always match the actual circumstances in the educational institutions.Therefore, the allocations often are vague and inconclusive.The ESCO descriptions are only partially fitting and can be assigned to several given competencies or they do not reflect actual requirements of the professions and the competencies taught in the training and continuing education courses.In addition to these content-related challenges, it was also difficult to deal with the ESCO classification as a whole, which was expressed as a feeling of disorientation and a lack of acceptance.Due to the enormous amount of time and effort involved, ESCO is not perceived as practical and is rejected as a standard for competency documentation beyond the project context by the testing partners.
In addition to these challenges, potentials for the testing partners as well as for training participants and employers were identified as well.A detailed presentation of competencies for continuing education offerings increases the significance of certificates and in turn helps to place and advertise the offering to the target group.
Based on the data, suggestions for further training or professional activities can be generated and comparisons with job vacancies are possible, which is also useful for participants and employers.
The general idea to use standardized descriptions for the documentation of learning outcomes is accepted by the testing partners.However, improvements regarding ESCO, such as supporting structures or technical and content improvements to the database, are needed to successfully apply this process to other courses as well.In this regard, the need for an AI-based system to suggest links to ESCO concepts was mentioned.The EU Commission is already developing such an AI-Tool with the expected release date in 2023.

Discussion: Next steps for change
The paper addresses the application of the ESCO classification to describe learning outcomes of continuing education programs in a standardized way, based on three case studies.It shows that the use of the ESCO may not only lead to a more transparent and comparable documentation of learning objectives, but can also improve the level of detail of the representation.While this approach is generally requested by the testing institutions, the use of the ESCO as a standardized way for describing learning outcomes is not accepted yet.The mapping effort seems to be too high and the mapping quality too low.Therefore, the continuing education institutions wish to have this process simplified and supported.
Furthermore, the testing shows that the use of broad objectives in defining learning outcomes and the use of predefined learning objectives can lead to a higher mapping quality.At the same time, sector-specific differences cannot be ruled out, so that demands for further development of the content of the ESCO classification remain.In this context, it should be examined to what extent a direct participation of sector experts can be facilitated.Further development of the ESCO -both in terms of content and in terms of possible application -seems desirable, since the potential of a standardized documentation of learning outcomes are manifold and have been confirmed by the testing institutions.

Figure
Figure 1.Research Design

•
Number of defined learning objectives • Number of assigned ESCO concepts • Number of ESCO concepts identified more than once (duplications) • Assessment of mapping quality based on five evaluation levels: − 1 -exact match, − 2 -ESCO concept covers larger area than the originally defined learning objective − 3 -ESCO concept covers only part of the originally defined learning objective, − 4 -no apparent match,

Table 1 :
Category system

Table 2 :
Number and quality of learning outcomes and ESCO concepts in the piloting courses (for categories of quality see chapter 2: Document analysis) category 5 is related to the learning objective, not to the ESCO concept.Therefore, it is not included in the calculation of percentage. *