Assessment of Attitudes, ethics and communication skills in a competency-based undergraduate medical curriculum

Affective learning outcomes are difficult to objectively analyze and assess as they refer to feelings and internal processes of the mind and heart. These cannot be quantified by traditional testing methods. The available methods rely on qualitative self-reflection and the observation of simple to complex internally consistent qualities of character. Just as in the other domains of learning, the validity and reliability of these assessments need to be ensured. The assessment of the affective domain requires careful observation of specified behaviours by multiple observers. Miller’s pyramid, Krathwohl’s levels and Epstein’s conceptual frameworks can be used to devise suitable assessment plans for the complex and varied competencies in this domain. Many techniques including paper-based tests, self-administered rating scales, faculty and peer assessment, simulations, reflections, portfolios, observed clinical encounters, collated views of co-workers, records of incidents of unprofessionalism, critical incident reports, patient surveys, and global views of supervisor have been used for assessment of affective domain. The effective utilization of these assessment tools needs to factor in the ground realities that exist in different medical colleges. The acceptability and feasibility of using resource-intensive assessment methods need to be carefully balanced against their contributions toward increasing validity and reliability.


Introduction
The affective domain of learning encompasses aspects such as attitudes, communication skills, ethics, professionalism, empathy and compassion (Singh et al., 2013).This domain is as important as the cognitive and psychomotor domains in contributing to the quality of patient care (Rogers et al., 2017).Previous studies have shown that students who exhibit unprofessional behaviour during their training are more likely to do so in their subsequent clinical practice (Modi et al., 2014).It is therefore critical that an Indian Medical Graduate (IMG) attains competence in the affective domain and that the associated value systems become internalized.
The AETCOM (Attitudes, EThics, and COMmunication) modules introduced by the National Medical Council in India as a part of the new Competency Based Curriculum (CBC) is a step in this direction (Medical Council of India, 2018).
Teaching faculty across medical colleges in India face challenges in the teaching and assessment of the affective domain.Most faculty have limited training in the application of teaching-learning (TL) and assessment methods used for the affective domain.A toolkit containing different TL methods that can be used for the affective domain has been published earlier (Fathima et al., 2022).The aim of this article is to introduce some theoretical constructs and methods that can be applied for the assessment of the affective domain so that these can be effectively utilized by faculty members to implement the AETCOM modules. https://doi.org/10.4038/seajme.v17i1.544

South-East Asian Journal of Medical Education
Vol.17, no.1, 2023

Need and challenges in assessment of the affective domain
There are compelling reasons for assessing the affective domain with as much rigor as the cognitive and psychomotor domains (Modi et al., 2014).When learning objectives in the affective domain are assessed, they are taken more seriously by both teachers and students.
If the affective domain is not assessed, there is a likelihood that ill-prepared students are allowed to graduate, thereby contributing to suboptimal patient care and its attendant consequences.The assessment of the affective domain requires careful observation of specified behaviours by multiple observers.This contributes to a greater emphasis being placed on these behaviours and consequently a positive change in the learning culture (Ten Cate and & De Haes, 2000).
Affective learning outcomes are difficult to objectively analyze and assess as they refer to feelings and internal processes of the mind and heart.These cannot be quantified by traditional testing methods (Wu et al., 2019).The available methods rely on qualitative self-reflection and the observation of simple to complex internally consistent qualities of character.Just as in the other domains of learning, the validity and reliability of these assessments need to be ensured.The threats to validity in the affective domain include fake responses in self-reported assessments which may cushion failures by maximizing virtues and minimizing faults.Additionally, students with superior language skills may have an undue advantage over others (Ngozi, 2018).

Theoretical frameworks for assessing the affective domain
Miller's pyramid is a widely used four-level framework for assessment (Miller, 1990).These levels include 'knows', 'knows how', 'shows how', and 'does'.The level of competence being assessed increases as one moves from the 'knows' to the 'does'.
Competencies that contain learning objectives related to the affective domain can be assessed at these four levels.An example of taking informed consent can be used to demonstrate how these four levels can be used.The basic theoretical components of taking informed consent ('knows') can be assessed using any of the numerous written assessment tools available such as multiple choice, short answer, and essay questions.A contextual application of this knowledge ('knows how') can also be assessed using the methods mentioned for the 'knows' component.The questions could be modified to include realistic case scenarios for taking informed consent.An objective structured clinical examination (OSCE) using a simulated patient from whom informed consent for a medical or surgical procedure must be taken could be utilized to assess the 'shows how' component.When students reach their internship, they could be observed taking informed consent from an actual patient ('does') using a workplace-based assessment method such as the mini-clinical evaluation exercise (mini-CEX).
Krathwohl described five levels of affective learning, namely receiving, responding, valuing, organization, and characterization (Rogers et al., 2017).At the receiving level, students become aware of certain aspects of an experience that may eventually result in effective learning.When students respond, some amount of reflection about an experience occurs and they exhibit some intellectual and emotional reactions to it.This may then lead to insight about oneself, termed valuing.The organization occurs when this insight can be related to future professional practice.Finally, characterization refers to a translation of a value system into professional behaviour.(Rogers et al., 2017) These levels of learning have been used to develop a scale for assessing student reflections (Rogers et al., 2018).
Epstein's conceptual framework identifies three stages in affective domain development (Stephens & Ormandy, 2019).Stage 1 is compliance.In this stage, students exhibit certain attitudes or behaviours to earn praise or avoid punishment.The next stage (stage 2) is identification, where attitudes and behaviours are assumed to maintain rewarding relationships with individuals or a group.In stage 3 (internalization), new attitudes and

South-East Asian Journal of Medical Education
Vol.17, no.1, 2023 behaviours are imbibed as they are intrinsically fulfilling (Stephens & Ormandy, 2019).This framework has been used to identify progress made by learners in affective domain development.(Stephens & Ormandy, 2019).

Toolkit for assessment of the affective domain
Wilkinson classified assessment tools for professionalism into nine broad areas (Wilkinson et al., 2009).This classification has since been widely adopted by many authors (Goldie, 2013;Guraya et al., 2016;Li et al., 2017;Smith et al., 2021).A similar classification could be used for assessment tools in the affective domain (Table 1).It was noted that many of these tools were used to assess residents and doctors rather than medical students.Miller's pyramid provides a convenient framework for selecting appropriate assessment tools based on the stage in the course where the student is placed (Modi et al., 2014).It must be emphasized that specific learning objectives for each of the competencies mentioned in the AETCOM module need to be formulated for the selection of the most appropriate assessment tools (Andrusyszyn, 1989).The conceptual frameworks of Krathwohl and Epstein are useful to define and assess the depth of learning in the affective domain (Rogers et al., 2017;Stephens & Ormandy, 2019;Yanofsky et al., 2010).The effective utilization of these assessment tools needs to factor in the ground realities that exist in different medical colleges.The acceptability and feasibility of using resource intensive assessment methods needs to be carefully balanced against their contributions towards increasing validity and reliability.An example of an assessment plan using some of the previously mentioned tools for an AETCOM module is shown in Table 2. 2. Students could be asked to complete a self-administered questionnaire on empathy such as the Interpersonal reactivity index during the SDL phase.
3. The anchoring lecture could have a quiz with MCQs' incorporated into it.
4. After the discussion and closure of the case, students could be asked to write their reflections.This could form part of an AETCOM portfolio.
5. An end-of-clinical posting OSCE could be conducted to assess the relevant communication skills.
6.During the internship, a workplacebased assessment tool such as a mini-CEX could be used.
Summative assessment As mentioned in the previous column.Some components of the formative assessment could contribute to the summative assessment scores.
While formative assessment forms the mainstay in the affective domain, summative assessment too has a role to play (Andrusyszyn, 1989;Haes et al., 2005;Miller, 2014;Ten Cate & De Haes, 2000).Domains of assessment that can be assigned grades or scores are more amenable to summative assessments than qualitative methods (Andrusyszyn, 1989).Previous studies have shown that it is possible to calculate a composite score to make pass or fail decisions regarding communication skills and attitudes (Haes et al., 2005;Ten Cate & De Haes, 2000).It is important to note that five or six assessments with multiple assessors are required to arrive at a precise composite score.A global rating scale with nine categories was used for the observations (Haes et al., 2005;Ten Cate & De Haes, 2000).Portfolios have been shown to be a useful tool to document the learning progress of medical students (Joshi et al., 2015).It has been suggested that aspects of the affective domain be included in the portfolio and used for decision-making in summative assessments (Mueller, 2009).
Checklists, rating scales, rubrics and questionnaires are often used in the assessment of the affective domain (Table 1).(Haes et al., 2005;Li et al., 2017;Nittur & Kibble, 2017;Smith et al., 2021;Wilkinson et al., 2009).Checklists state the behaviours or steps that need to be performed and assessors indicate on the checklist whether these are

South-East Asian Journal of Medical Education
Vol.17, no.1, 2023 performed.Rating scales indicate not only the behaviours but also the extent to which they are exhibited.The most frequently used scale is the 5-point Likert scale.An example of a rating scale that is has been suggested for assessing the professionalism of first-year medical students in India is shown in Table 3 (RGUHS, 2019).A rubric is a scoring guide used to assess constructed responses like reflections.
It contains specific criteria and performance levels for every criterion.Each performance level has a detailed description which assists assessors in accurately assigning a grade.Questionnaires are a convenient and flexible tool to gather information.The responses could be dichotomous, in the form of a rating, qualitative in nature, or a combination of one or more of these (Violato, 2019).

 Student awareness
Students need to be informed in advance that the affective domain is going to be periodically assessed along with the other domains.

 Clear objectives
Specific objectives need to be framed that are in alignment with the competencies stated in the AETCOM modules.The competency framework of the CBC provides ample opportunities to frame specific learning objectives in the affective domain outside the ambit of the AETCOM modules as well.

 Formative and summative assessment
A categorization of the specific learning objectives that are suitable for formative and summative assessment needs to be made.The assessment tools used for formative and summative assessments are likely to be different.

 Remediation
Professional lapses that have been noted in formative assessment require early remediation.It has been observed that unprofessional behaviour among clinicians is often preceded by such behaviour during their training period.

 Tool revalidation
Cultural contexts are especially important in the affective domain.It is therefore important that tools that have been validated in one setting be revalidated prior to their use in different cultural contexts.

Conclusion
The affective domain is as important as the cognitive and psychomotor domains in patient care.This hitherto neglected domain has now been formally introduced into the undergraduate medical curriculum in India in the form of the AETCOM modules.Although there is a consensus that assessment of the affective domain is important, there are many challenges that need to be surmounted for its effective implementation.Miller's pyramid, Krathwohl's levels, and Epstein's conceptual frameworks can be used to devise suitable assessment plans for the complex and varied competencies in this domain.Assessment tools for every level in Miller's pyramid are available ranging from written tests to workplace-based assessments.The numerous opportunities that the CBC provides for formative assessment of the affective domain need to be utilized by students and faculty members.The quality of these assessments can be improved if certain evidence-based guidelines are followed.It is hoped that this article provides some pointers to healthcare educationists working in contexts like those in India to improve the quality of assessments of the affective domain.