TRANSFORMING THE CULTURE AND INNOVATING THE PRACTICES OF DIGITAL ASSESSMENT: HOW MOOCS CAN SCALE UP THE IMPACT

The emerging new learning scenarios made possible by the rapid dissemination of adaptive digital technologies are deeply changing higher education processes. They provide educators and learners with new opportunities for transforming their educational practices. encouraging learning innovation and challenging traditional teaching methods. This transformation is particularly significant in how learning achievement is measured, assessed, and certified. The use of learning analytics and adaptive digital technologies not only contribute to consolidate an assessment for learning approach as call for it. In the face of this challenge, educators must speedily adapt their practices to ensure that they are effectively using digital tools to support this new assessment culture. In the framework of the Erasmus+ project PROF-XXI, a Massive Open Online Course (MOOC) entitled "The 21st Century Teacher: Learning to Teach with Digital Technologies" was designed and delivered to facilitate teachers’ access to good practices in this field. The MOOC included four modules covering various aspects of digital education and was designed taking as a reference the competences defined in the DigCompEdu framework. It was delivered on an Open edX instance. A total of 5,660 participants, mainly from Latin America, took this course in two cohorts, with 681 participants completing it. In this paper we describe the experience of designing a learning module specifically dedicated to digital assessment innovation for this MOOC, The aim was to inspire educational change and promote a more sustainable approach to assessment for learning based on the PrACT framework. The main results of this experience are presented and discussed, as well as lessons learnt.


Introduction
Traditional classroom teaching falls short of providing an immediate learning environment, engaging and personalised, flexible enough to accommodate the increasing diversity of learners, their needs, and expectations (Haleem et al., 2022).As digital technologies rapidly disseminate across educational institutions, its impact is immediately felt, leading to the transformation of teaching and learning.This creates a need for teachers and educators in general, as well as students and non-formal learners, to develop digital competencies.The Erasmus+ PROF-XXI Project (http://www.profxxi.org/)focuses on enhancing the capacity of Teaching and Learning Centers (TLCs) in Latin American Higher Education Institutions (HEIs) to fulfil the requirements of the twenty-first century (Delgado Kloos et al., 2021;Pérez-Sanagustín et al, 2022).Digital competence involves the confident, critical and responsible use of, and engagement with, digital technologies for learning, at work, and for participation in society.It includes information and data literacy, communication and collaboration, media literacy, digital content creation (including programming), safety (including digital well-being and competences related to cybersecurity), intellectual property related questions, problem solving and critical thinking (Vuorikari et al., 2022).The European Framework for the Digital Competence of Educators (DigCompEdu) characterizes it more specifically: "As professionals dedicated to teaching, they need, in addition to the general digital competences for life and work, educator-specific digital competences to be able to effectively use digital technologies […] to foster efficient, inclusive and innovative teaching and learning strategies" (Redecker, 2017, 15-16).Developing these competences is therefore critical to ensure that educators can use digital tools effectively and provide innovative teaching methods.This transformation is particularly significant in how learning achievement is measured, assessed, and certified.The implementation of learning analytics and adaptive digital technologies have been contributing to consolidate a new approach to assessment.However, teachers must have the necessary competences to integrate digital tools into their teaching practices.As demonstrated in the context of the higher education system to the Covid-19 pandemic, this difficulty is aggravated by the fact that digital transformation in education refers to the incorporation of digital technologies into teaching and learning practices to improve educational outcomes, which not only involves the adoption of new technologies, but also the development of new teaching methodologies, strategies, and pedagogies (Abad-Segura et al., 2020).To address this massive challenge, the PROF-XXI partnership developed a Massive Open Online Course (MOOC) entitled "The 21st Century Teacher: Learning to Teach with Digital Technologies."One of the modules of this MOOC was specifically dedicated to digital assessment.The option for an open education format was based on its suitability to deliver a highly scalable, flexible, inclusive and speedy teacher training experience.In fact, MOOCs provide a space for problematizing and analyzing practices in an open and collaborative working environment (Teixeira et al., 2018).The outcomes and insights learned from designing and delivering this course and specifically the course unit on digital assessment are detailed in the following sections.

A MOOC on the 21st century teacher: learning to teach with digital technologies
The MOOC developed by the PROF-XXI partnership was aligned with the DigCompEdu framework and it targeted higher education teachers and other educators from Latin American countries.It provided participants with a significant learning experience in which they had access to research-validated concepts, methodologies, strategies, and good practices on technology-enhanced teaching and learning.The purpose of the MOOC was to contribute to the development of teachers' and other educators' digital competencies.The open course was deployed on the Campus MOOC-Maker platform (http://campus.moocmaker.org)which resulted from the Erasmus+ project MOOC-Maker (Alario-Hoyos, 2018).This decision reflected the fact that most PROF-XXI partners had been also part of the MOOC-Maker consortium.Therefore, it added to the sustainability of the initiative taken.MOOC participants learned on how effectively leveraging digital technologies to enhance their teaching practices.It encouraged them to promote active learning and to create more engaging and effective learning environments for their students.The MOOC was implemented in two cohorts, the first one from 8 th November to 13 th December 2022 and the second cohort took place from 26 th February to 10 th April 2023.In total, 5,660 participants registered in this MOOC, with 681 of them obtaining a certificate of completion (see table 1).It should be noted that 55.44% of the participants were female, while 43.99% were male, which represents a proportionate gender representation.Of the other participants, 10 of them declared to be non-binary and 22 did not provide information.The MOOC was developed collaboratively by an international multidisciplinary team of experts from Spain, Guatemala, Portugal and France.To ensure a coherent learning design, a common strategy adjusted to the features of the Open edX platform was devised and shared among the partners.However, each one acted autonomous in the way it implemented those general principles to the production of the different modules.Each module covered a distinct digital competence, as follows: active teaching methodologies; instructional design for active and hybrid learning; digital assessment of learning, and evidence-based continuous improvement.They included multiple educational resources, as videos, exercises, formative activities, forums, a podcast, and a final exam.
Each module started with an introduction video lecture which presented the key concepts, explained them and gave concrete examples to facilitate comprehension.Videos had closed captions for easy understanding and accessibility.Participants were given access to a list of complementary references on each module.They were also required to answer a series of questions at the end of each module to receive feedback on what they had learned.The non-scored formative activities were a useful tool for students to apply the module's concepts and deepen their understanding of the topics.Forums were interactive spaces where students could share their experiences and perspectives on the module's topics.
The MOOC also included a course presentation module with a welcome video, a course syllabus, a learning path, faculty information, a section on frequently asked questions, the initial survey, and the welcome forum.The participants were required to spend a minimum of 4 hours and a maximum of 16 hours per week in this course.

Promoting a new digital assessment culture
The traditional perspective of "measuring" learning at given points in the learning process, with a special focus on the use of written tests, has long been inadequate and insufficient.Especially so when dealing with the assessment of complex and diversified knowledge resulting from the new work dynamics demanded by the challenges of the contemporary world (Pereira et al.2021).Assessment is still generally linked to tasks of little relevance, disconnected from reality, driven by the purpose of obtaining a qualification and not as meaningful evidence of how competences acquired may be applied to respond to real problems and challenges.As a result of very transmissive teaching methods, traditionally assessment was centred on content.It appeared as something almost exclusively linked with the role of classification and certification, highlighting therefore the concern with "measuring" learning.The need to transpose curricula based on mere content to competence-based curricula making it possible to face current challenges associated with pedagogical proposals that go beyond the transmission of knowledge, determined the evolution of conceptions of assessment and its various roles.As a result, the final assessment role associated with giving a mark (summative role) has been giving way to a greater importance of the formative function of assessment, i.e. to the processes of regulating learning.Considering this perspective, a new understanding of assessment has been emerging, giving rise to the so-called 'Culture of Assessment' (Pereira et al.,2015;Birenbaum, 1996).From this perspective, it is worth highlighting some key ideas or concepts, such as the concept of competence.This concept refers to the ability to respond successfully to a personal/social demand or to perform a task or activity that requires the convergence of knowledge (implicit and/or explicit), skills, abilities, capacities, skills, attitudes, emotions and values (Pereira et al.2021).It is developed through action and interaction both in educational contexts (formal, non-formal or informal) and in professional contexts, and requires the mobilisation of knowledge rather than just its reproduction.The assessment of competences therefore requires a new approach, integrating knowledge, skills and attitudes.As such, it implies the use of a variety of assessment strategies and modes of assessment that go beyond the traditional written test.This new approach requires assessing performance in authentic tasks, close to real contexts, which provide the application of the competences to be developed.Another key concept refers to the 'digital alternative assessment' strategy.The expansion of the use of electronic tools in assessment, from multiple-choice tests to questionnaires, blogs and e-portfolios, among others, calls for the need to clarify the understanding of e-assessment.In this regard, a concept is proposed that incorporates the technological dimension, but is specifically aimed at competence assessment.The concept of digital alternative assessment strategy (Amante et al., 2017) refers to a set of alternative assessment proposals, where design, implementation and feedback are mediated by technologies.The design involves the definition of the competences to be assessed, the indications about the activity to be carried out by the student and the use of electronic devices for its implementation.In this perspective, the traditional idea of assessment of learning gives way to the concept of assessment for learning.In assessment of learning, the data collected are used to make judgments about the level of performance of learners.The data collected are used to make judgements about the level of achievement of learners.The information collected by the assessment process is basically summative and, when translated into a numerical or qualitative rating, certifies the learning achieved.This type of assessment may also indicate learning that has not been achieved, but it is not intended to be remedial and occurs on an ad hoc basis, usually at the end of the learning process.Error is understood as failure.Assessment for learning focuses on certification.By contrast, in assessment for learning, the formative and empowering role of assessment is emphasised in contrast with the measurement role.Teachers use the information from the assessment process, including errors, to change their teaching practice, with the aim of helping the learner to improve his or her performance.Rather than being seen as something that happens at the end of the learning process, assessment is now considered as an integral part of the learning process.Consequently, similarly, to learning it is a process of social interaction in which all stakeholders should be involved (Pinto, 2016).In the networked society a new culture of learning is emerging and, with it, necessarily also a new culture of evaluation (Birenbaum, 1996).One that is characterised by: -Emphasis on the interconnection of assessment with teaching and learning.
-Student participation in the elaboration of their own assessment in dialogue with the teacher.
-Evaluation of process and product.
-Assumption of a diversity of forms of assessment, associated and interconnected with teaching practices.
-Use of assessment tasks close to real-life situations.
-Continuous and ongoing process.
-Use of feedback that reinforces students' reflection on their learning.
-Valuing qualitative assessment over simple grading.

The MOOC module on "Digital Assessment for Learning"
The course module on "Digital Assessment for Learning" was developed by a multidisciplinary team from the Laboratory of Distance and eLearning of Universidade Aberta, Portugal.It involved 6 researchers and 4 technicians.The team had a large experience in designing MOOCs for teacher training.As such, it applied some of the general principles of its own pedagogical model for massive open online education, the iMOOC (Teixeira & Mota, 2015), to an Open edX environment.The MOOC module had the following learning objectives: -Characterise the new assessment culture, differentiating it from the traditional one.
-Identify the main dimensions and parameters of the PrACT framework for digital assessment.
-To reflect on the use of self and peer assessment in Higher Education -Distinguish digital assessment tools, instruments and media.
-To design an assessment plan The course module included five (5) topics, as follows: -A new culture of assessment.
-The PrACT framework for digital assessment.
-Self-assessment and peer assessment in higher education.
-Digital tools, instruments and means of assessment.
-Planning digital assessment and feedback.The first topic of the course module focused on the new culture of assessment, as analysed in the previous section of the paper.The second topic related to the PrACT framework for digital assessment.The PrACT model corresponds to a conceptual matrix designed to support an alternative model of digital assessment, based on four main dimensions: authenticity, consistency, transparency and practicality.The idea behind it is to tackle the emerging challenges of the digital society in what refers to competence development, particularly in higher education (Pereira et al., 2015).These dimensions make it possible to characterise the determining features of the digital assessment strategies to be developed and are also essential for the quality of assessment in the educational process, each of them being characterised by a set of parameters.It is important to mention that these dimensions are articulated, representing different degrees of reciprocal influence, and the alteration of a parameter in one of them can affect another or others: practicability, for example, often neglected, can decisively influence the level of implementation of the remaining dimensions.The third topic related to self and peer Assessment in Higher Education.The aim was to build teachers' awareness of the importance of giving learners co-responsibility for their own learning pathways, which requires engaging their participation in the assessment process.Beyond the concept of hetero assessment, i.e. the evaluation carried out by the teacher, teachers are trained and stimulated to introduce practices of self-assessment and peer assessment, in order to apply a co-assessment approach (Souza & Amante, 2021;Relvas et al., 2020;Amante & Oliveira, 2019).As for the fourth topic, teachers were familiarised with a wide range of technological resources and tools which characterise the new learning contexts, online, hybrid and even face-to-face.These resources favour the rethinking of assessment practices, making them closer to authentic situations in which learners will have to know how to act and intervene, thus replacing the traditional idea of test/examination.Finally, the fifth topic was an activity dedicated to allowing participants to train assessment and feedback planning.The new assessment strategies aim to focus on the competences required in real-life/professional practice, ensuring that the ways of assessing them enable the knowledge, skills and attitudes that need to be mobilised in these contexts.They also aim to consider clear and transparent assessment criteria, as well as to promote a communication of results that favours the regulation of learning (Panadero et al., 2019;Panadero, Jonsson & Strijbos, 2016).This perspective raises the need to design an assessment plan that responds to these new concerns, as follows: -what do we want to assess?-how to evaluate?-who assesses?
-by what standards is it evaluated?-how are the results communicated?

Impact of the massive open learning experience
At the end of each cohort, all participants who completed the MOOC were invited to fill in an online satisfaction survey conformed of Likert Scale and open-ended questions.The survey intended to collect relevant information on the demographic characteristics, levels of satisfaction, perceived utility of the course content, and overall learning experience of the participants.The survey was electronically disseminated via the MOOC platform.
The data collected and participant feedback (Alario Hoyos et al. 2022, 2023) allow us to conclude that most of the participants (70,25%) considered the course to be moderately difficult to follow and complete.They have found a good balance in the course design between providing meaningful challenges and facilitating comprehension.There was also a minor percentage of participants (12,33%) who have found the course to be extremely simple.This may result from prior knowledge or their familiarity with the topic.A similarly small number of participants (13,11%) identified difficulties in understanding some concepts and mastering the course materials.
In total, 97,3 % of participants strongly agree (73,97%) or agree (24,46%) with the statement that the course helped them to develop their competences to understand how to teach with digital technologies.Almost all the participants strongly agree (80%) or agree (19,96%) with the statement that the course helped them to develop their competencies to recognize the importance of active learning in the teaching-learning process.
In what regards the learning materials and learning activities, most participants found them very useful and aligned with their perception of the learning goals (78%).These results demonstrate the effectiveness of the course in fostering substantial student learning and the success in providing valuable educational content and engaging teaching strategies.The positive perception of learning outcomes indicates that the course met its educational objectives and provided participants with meaningful knowledge and skills.

Conclusions
Teaching and learning methodologies are rapidly changing as a result of the dissemination of new digital technologies.Accelerated by the dramatic context of the Covid-19 pandemic and also by new educational policy, technology-enhanced learning has become mainstream in Higher Education.Assessment of learning outcomes is one of the areas in which the impact of new digital technologies, such as learning analytics and artificial intelligence, is leading to a major reengineering of educational processes and a deep transformation of concepts, methods and tools.A new culture based on a assessment for learning is emerging However, as educators worldwide embrace digital education, they face a major challenge: how to speedily develop new competences, update teaching methodologies and innovate educational practices.The speed and scope required by such massive effort of training teaching staff can only be meet in a sustainable and successful way by fully exploring the affordances of open education.
In this paper we describe an experience of designing a course module on digital assessment in the framework of a MOOC produced and delivered to support the development of digital education competences of Latin American educators.The aim was to inspire educational change and the innovation of teaching practices, promoting a new more sustainable approach to assessment for learning based on the PrACT framework.The evidence collected from the two iterations of the MOOC seem to demonstrate a significant impact of this open education format for enabling innovation transfer in teaching training contexts.The high number and diversity of participants show a wider outreach than could be expected from using traditional in presence formats.Additionally, the learner's achievement measured by the completion rates is in line with standards for this course format.Naturally, this will need to be corroborated by the actual transformation that may occur in the field of educational practice as a result of this training experience.Nevertheless, the very positive feedback received from participants evidence the perceived high quality of the topics, notably the ones related to the innovation of learning assessment.This is in line with the results of previous research conducted in Europe (Teixeira et al, 2018) and it seems to suggest that the MOOC module may have succeeded in inspiring Latin American teachers to transform their learning assessment culture and practices.The success of this initiative has led the PROF-XXI partners to plan the relaunch of the MOOC in the edX platform which should increase even further the outreach of the course.