Assessment in Open and Distance Learning System (ODL): A Challenge

Assessment is an integral part of the learning process. The traditional practice of assessment has changed to meet the need of the contemporary society. In this paper assessment strategies used in Open and Distance Education are discussed and constructive suggestions are given to meet the challenges of assessment. Recently we experience a paradigm shift in assessment both in face-to-face and ODL system. Content-based testing has shifted to performance-based assessment. Assessment is no longer used for grading and certification, rather it has linked with learning and skill development of the students. Instead of a single paper pencil test, a variety of techniques and methods are being increasingly conducted. In this context assessment in the ODL system has adopted a new shape to provide better assessment judgments to its students and at the same time helping teachers and administrators. Coping with the changing scenario in ODL we face challenges addressed extensively in this article.


Introduction
Let us start the discussion of the issues raised in this paper by analysing three cases related to assessment of students' performance in the ODL system.

Case I: Sudhir -A student of a Bachelor Degree Programme in Arts
Sudhir is a student of a Bachelor Degree Programme in an Open University.Recently he completed his three years Bachelor Degree Programme in Arts.He was dissatisfied with the grades awarded to him by the University.He completed the programme with a lower grade in few courses.He was dissatisfied with the assessment system in the Open University, in the sense that he experienced confusing results in some courses.Interestingly, he managed to get a higher grade in the course in which he had not done well, and on the contrary he was awarded a lower grade in the course in which he really did well.
The experience of Sudhir left many questions in the mind of teachers, academics, and administrators.
-Why did this happen with Sudhir?-Can we say it is an unexpected event, which occurred only with Sudhir, or has it been happening with many students like him?-Is the system of assessment in ODL capable enough to do justice to the student like Sudhir? -What measures do we need to take to overcome such practices?
In the next part of this article we will come forward with some remedies to arrest such difficulties of the students.

Case II: Gurpreet never gets timely information about course transaction
Gurpreet is a student of a Master Degree Programme in Management in an Open University.She is motivated to participate in all the academic activities and hopeful to do well in her programme although practicing a part time job in a corporate organisation.She felt it difficult to understand all the courses (self learning material) supplied to her during her studentship in the programme.She was expecting certain types of academic deliberations relating to her courses either face-to-face or by using other modes of course transaction, such as teleconferencing, interactive radio counselling, etc.After completion of first year of her programme, when she talked with her friends and the academic counsellors, she came to know that this type of course transaction is often practiced by the University.But she was not informed about the schedule of the academic activities.In the second year, she became more conscious and managed to get all the information about the academic activities, participated and tried to clear all her learning difficulties.Ultimately she did well in second term-end examination in comparison to first term-end examination.
The above experience of Gurpreet again left many questions about the management of course delivery in the ODL system.
-Did we inform the students about the nature of course transaction well in advance?-Whose responsibility is it?-Do you think the experience of Gurpreet is well connected with assessment and her performance in the programme?
What measures can we take to avoid such difficulties of the students, like Gurpreet?

Case III: Rehman-A student of a Bachelor Degree Programme in Education
Rehman has been teaching in a secondary school for the last five years and has just completed his bachelor's degree in Education from an Open University.He was a regular and committed student during his studentship period and actively participated in all the school-based activities, workshop-based practice, teaching practice, assignments, and preparation for term-end examinations.Though he completed the programme with a high grade, still he was dissatisfied for some reasons.He observed that students who were not regular in carrying out course activities managed to earn a higher grade than him.
The experience of Rehman also makes us think about assessment strategies and puts forth various questions before us like: -How can one irregular student manage to get a similar or higher grade in practical courses in comparison to a regular student?-Whose fault is it?-Do you justify a student gets higher grade without actively participating in practical courses?
We have to reflect on how assessment measures can liberate students from de-motivation to inspiration.
The above three cases can force us to discuss the assessment practices conducted in the ODL system.Sometime we fail to achieve our assessment goal.As we know, the most important function of any university, open or conventional, is to work for the students and to satisfy their expectations.Students expect better teaching-learning environments, pedagogic dialogue, practice research and extension activities, and a suitable system of assessment.The latter is linked with the quality of education imparted in an Open University.In other words, a sound assessment system can motivate students to achieve their learning objectives effectively.Assessment in Open and Distance Learning System (ODL): A Challenge Open Praxis, vol. 5 issue 3, July-September 2013, pp. 207-216 Considering the above three cases related to teaching-learning and assessment in ODL, the present paper aims to achieve the following objectives: • To discuss the importance of assessment in the ODL system; • To elaborate present assessment practices conducted in the ODL institutes; • To discuss challenges relating to students' assessment in the ODL system; • To deliberate an innovative practice of students' assessment; • To examine the use of ICT tools in assessment practices of the ODL system; and • To suggest good assessment practices in the ODL system.

Importance of Assessment in Open and Distance Learning
The three cases analysed in the introduction section of this article give us enough input to think and renovate our assessment practices in the ODL system.As discussed earlier, assessment occupies a vital position in the process of teaching-learning, certification and acquiring knowledge and skills, whether in a conventional system, or an ODL system.Assessment is a sub-system of the total ODL system.A vast mechanism works for managing students' assessment.
As teachers, we hope that our students understand the concepts of deliberation and use knowledge gained in their life.Teaching cannot be effective without stating its purpose and framing specific objectives.A conscious teacher always tries to form the specific objectives before starting teaching and achieve that objectives at the end of his/her discussion in the classroom.So we can say that continuous assessment practices in the teaching-learning process in terms of formative and summative evaluation should be practiced.If we closely analyse the main purpose of assessment, we find that its role is not only to simply rank or grade students, which is secondary in the process of assessment, but to increase student's learning and development.Effective assessment strategies help the students to understand their difficulties and to improve further accordingly.It helps the learners to gain mastery over the task and widen one's horizon of knowledge.Assessment is key in helping students to achieve the real purpose of learning.
In India, reform of the examination system was connected to the National Curriculum Framework (NCERT, 2005) which suggests a paradigm shift of content-based testing to problem-solving and competency-based assessment, examination of shorter duration, flexible time limit, open book examination; self-assessment, peer assessment and feedback, maintaining a daily diary, emphasis on continuous evaluation and use of ICT.NCERT (2005) also suggests various assessment tools such as observation, assignments, projects, portfolio, e-portfolio, checklist, rating scale, anecdotal records, etc.The suggestions of NCERT (2005) and the tools described above are being used for assessing students' performance.
Answers to the following four questions will throw light on assessment in the ODL system (The Commonwealth of Learning, 1999, pp.78-79): 1. Why assessment?
• We conduct assessment in the ODL system mainly: to provide feedback to learners starting from assignments to the term-end examination; to get an idea about what they are really doing and what they are supposed to do to complete the programme; and to generate a spirit of consciousness to connect the concept of written materials with the varieties of practices carried out by the learner.• To reach at summative evaluation based on formative assessment, which helps learners to examine their performance at every stage of their study and progress successfully to attain the course objectives.
Open Praxis, vol. 5 issue 3, July-September 2013, pp.207-216 • Assessment in ODL is not only meant for students to earn a grade, it is equally helpful for monitoring the effectiveness of academic programmes and adopting appropriate strategies to accomplish institutional objectives.• The practice of assessment in ODL is not only for certifying students, it also impacts their learning improvement and helps learners to develop a positive attitude towards the institutional system.2. Assessment is for whom?
Assessment in ODL is NOT to satisfy a single stakeholder; rather it is for the multiple users working in the system such as: students, teachers, institutions, and community.Let us discuss how assessment benefits the stakeholders: • Assessment helps students to be aware of progress in their studies, to achieve mastery of the concept, to find out the causes of the difficulties and to get the remediation to overcome the learning difficulties.It also helps learners to improve their learning in the positive direction if they fail to achieve the required standard.Further assessment satisfies the purpose of certification and award of degree.• It helps teachers to assess the effectiveness of the instructional strategies, communication, and involvement of the students.In case teachers fail to achieve specific instructional objectives, they can modify their teaching-learning strategies.It also helps teachers to judge the suitability and effectiveness of self-learning material supplied to the students.It helps teachers to modify learning behaviour of the students and make teachinglearning process effective and outcome oriented.• As in the case of learners and teachers, assessment satisfies the institution.Assessment of the students in terms of level of attainment helps the academic administrators and programme/course coordinators to make certain decisions in revising the programme.
The institution can decide whether the very objectives of the programme have fulfilled their vision.• Assessment also satisfies the need of society at large -satisfying the broad and long term need of the society, providing appropriate job opportunities for the students, maintaining social cohesiveness and brotherhood, and satisfying the need of economic development of society.

Who assesses?
• Assessment in the ODL system is a multitask activity unlike in the face-to-face system.
• Unique practices are managed to assess students' performance such as: selfassessment (students assess themselves by evaluating their performance when they read SLM), peer assessment (mutual assessment can be practiced), academic tutors to assess assignment (formative evaluation), external examiner for project and practical assessment (part of summative evaluation) in term-end examination.4. How to assess?
• Several assessment techniques are used in the ODL system, such as essay type examination, short answer type and objective tests, oral assessment in tutorials, practical examination, field-based practice and assessment in some specific programmes, project and dissertation, observation of performance in extended contact programme and personal contact programme, etc.

Assessment Practices in some existing ODL System
To have an idea of assessment practices being conducted in Open Universities worldwide, the main features of assessment in five universities are presented in Analysing the assessment practices of the above Open Universities, we find common components of students' assessment.All universities have a two-tier system of assessment: formative and summative.More weighing has been given to summative assessment to certify the students' performance.In formative assessment, most universities use assignment and in-text questions and activities for self-assessment.In summative assessment, most universities prefer term-end There are two types of evaluation at BOU such as continuous evaluation through tutor marked/practical assignments and the projects, and semester-end evaluation through semester-end examination.
Allama Iqbal Open University (AIOU), Islamabad, Pakisthan www.aiou.edu.pkAssessment of students in distance education system in AIOU is done in two ways: continuous assessment and term-final examination.In continuous assessment students are required to do two assignments for each half-credit course and four assignments for each full credit course.The marks obtained in the assignments contribute to the final course result.Term-final examination is another component of overall assessment system of a course.Term-final examination helps the students to review their studies and see the course as a whole.At the end of each semester the University arranges a final three-hour written examination in each course.
The Open University of China (OUC), Beijing, China http://en.crtvu.edu.cnEvery learner has his/her own individual learning space and enjoys personalised support services.Learning progress can be evaluated constantly, and formative and summative evaluation are used together to ensure the learning process and quality of learning.The OUC operates a credit bank with the functions of credit accreditation, transfer, deposit and withdrawal.The bank helps learners establish lifelong learning portfolios and accredit or receive certificates for various learning achievements.The bank carries out accreditation and transfer of credits between degree and non-degree continuing education, and bridges and connects different types of learning achievements.
Source: University websites of the above five Open Universities retrieved on 13.04.2013.
Open Praxis, vol. 5 issue 3, July-September 2013, pp.207-216 examinations at the end of a session.It has been observed that the existing system of assessment in ODL fails to satisfy students' expectations and their development in learning.In the following section we discuss specific issues and challenges needed to carry out effective students' assessment practices.

Students' Assessment in ODL: Issues and Challenges
Credibility and reliability of assessment procedures is a question of serious debate in the ODL system.There are three types of assessment qualities, which are essential for effective assessment.They are validity, reliability and fairness (Makamane, 2011).For an assessment system to be valid, care should be taken to verify whether the purpose of assessment has been achieved or not.Reliability entails the extent to which assessment is free from errors of measurement.An error free assessment system is treated as a reliable system.Fairness in assessment speaks about the objectivity of assessment and making assessment free of subjective judgment (UNESCO, 2006).
Let us discuss some of the major challenges faced by the ODL system in making assessment valid and reliable.

Observation of students' activities in personal contact programmes and practice of performance based assessment
Careful observation of all aspects of students' assessment can help to organise it constructively.If there is the provision of compulsory attendance in certain assessment related activities, it should be done meticulously and with full of gravity.Provision of certain grades/credits should be there for attendance in the programme.There is also the need to compare students in a particular component of assessment criteria.The academic counsellors and the tutors need to be associated wholeheartedly with the assessment system and should fulfil their assessment responsibilities without any bias and pre-mind set up.
Performance based assessment in the personal contact programme needs to be strengthened.Performance assessments are viewed as providing more valid inferences concerning learning than the traditional pencil and paper test, because they focus on the types of performance tasks that are being taught (Gronlund, 2003).This concept needs to be practiced in students' performance assessment in the personal contact programme in an ODL system.On the contrary, Nitko and Brookhart (2007), propose that there be a balance between the traditional and new system of assessment.

Open-book examination system
Realising the changing scenario of the ODL system, open book examination may be seriously considered.The philosophy of ODL encourages students to develop their comprehension and analytical skills.For example, the Central Board of Secondary Education (CBSE, 2013) in India is going to add open book examination on experimental basis in annual examination of secondary school students (Thakkar, 2013).In the ODL system students may also be provided with study materials and other reference books within a specified period.Fully open book examination may also be executed in a distance education system in which students may be allowed to enter the examination hall with their notes and books.They may be allowed to answer the questions by referring to the material they have brought with them.If the institutes adopt an open book examination system, they have to be more careful in constructing questions for term-end examination, mostly analytical and synthesis type.But it remains a question whether the ODL system is able to manage it properly or not (NCERT, 2012).Assessment in Open and Distance Learning System (ODL): A Challenge Open Praxis, vol. 5 issue 3, July-September 2013, pp.207-216

Conducting formative evaluation in ODL
All the distance education institutes practice Unit or Course wise assignments.It has often been observed that the tutors assess the assignments and grade them without putting any remarks.In this case, the purpose of formative evaluation remains unfulfilled, if we fail to link assessment with students learning development.It has also been observed that without going thorough the details of the assignment, tutors award a grade.Quite frequently, tutors assign the same grade to all/most students just to satisfy them.It has also been observed that the copies of assignments are evaluated not by the real appointed tutors but by a person who is not specialised in that field or is unauthorised to do that.All the above cases left room for the system to be blamed.A careful monitoring is needed to check such type of evaluation lacunas in ODL systems (Shidong, 2011).

Term-end examination (summative evaluation)
Almost all the ODL institutes give a higher weighing to the final or term-end examination, which is conducted annually.Many a time, students respond to the subject matter with a very low standard but manage to get a higher grade, and sometimes a good writing gets a lower grade.It is difficult to find out the real cause of this as it is a matter of subjective evaluation.Expert to expert scoring may vary, but we can try to attain objectivity by adopting certain important steps such as carefully phrasing the essay items, make the response restricted in words/time/pages, by preparing standard answers of the questions and comparing them with students' writing, by asking various types of questions in the form of essay, short answer, and objective questions, by moderating the checked answers (randomly/fully) by the experts, by supplying reasonable copies to the examiners, etc.

Integrating formative and summative evaluation
Integration of formative and summative evaluation components is also another important issue in ODL systems.Most ODL institutes follow the integration approach of formative and summative evaluation by providing definite weighing for the final certification of the students.But this is not the sense of true integration.Integration in true sense is to give equal weighing to both formative and summative evaluation i.e. 50% to each type.Formative evaluation can also bifurcate to assignments and open book examination and the like.Students may be asked to read related information and materials collected from various sources through outside research and carry out a project of substantial size and quality in a given particular time period for obtaining a score/grade for the course.This may soften the learning and examination load of the students and involve students in learning (Renkun, Tingting & Lina, 2011).

Linking assessment with learning
Assessment of learning plays an important part in the measuring of the quality in education (Makamane, 2011).Students' performance assessment cannot be separated from learning.As we know, ODL institutions follow diversified approaches to assessing performance of the students.The components of assessment in ODL systems should never be separated from students' learning experiences.For example, strategies of student's self-evaluation, peer evaluation, instructors/tutors evaluation, and external evaluation may well be linked with the learning strategies of the students.The detailed assessment process in the ODL system should be informed to the students well in advance.As we know, most distance students initially come from the conventional system, having experienced certain types of face-to-face activities.They are used to the practice of learning styles in the face-to-face system.But in the ODL system, assessment is conducted differently according to the learning strategies of ODL systems.Keeping these things in mind, a careful detailed Open Praxis, vol. 5 issue 3, July-September 2013, pp. 207-216 orientation is needed for the distance learners-on how to develop their study habits, read the self learning materials, and do the self assessment, peer assessment and preparation for term-end examination.Open learning models, while providing a variety of content, routes for accessibility, interaction and instruction, have not yet reckoned with the question of assessment and its corollary challenges of probability and recognition (Conrad, 2013).

Using ICT as an assessment tool
It is the age of Information and Communication Technology.ICT plays a very important role in transacting curriculum and students' assessment in ODL systems.Many open universities, including IGNOU, have started online programmes and practicing online assessment by using ICT.But it has been realised that the use of ICT in students' assessment is very limited.It is confined to few academic programmes.Nowadays, time demands to use more and more ICT tools in students' assessment.Students need a very innovative and friendly assessment system in ODL, which can be fulfilled by using ICT assessment tools in a large scale.Students' portfolio and e-portfolio can be better used as an assessment tool in various programmes of ODL system.Few Open Universities have started to motivate the learners by maintaining an e-portfolio and using it as an evaluation tool, especially in the online academic programmes.Today there is the need to use it on a wide scale in all Open Universities.Though ICT has been used as an assessment tool for programmes with small groups of students, its use can be broadened by carefully grouping students and assigning them an academic counsellor/tutor/manager.Open universities should think it over and implement it on a large scale.

Renovation of Conventional Practices of Students' Assessment
The above points might have given enough input to consider renovating conventional practices of students' assessment in ODL.The existing system in ODL cannot be called a failure, rather there is the need for renovation in the existing practices.Assessment mechanisms in ODL require a more cautious and vigilant approach to discharge their duties.We sometime fail to bring quality in assessment given a large number of students.In ODL systems, there is no bar of limitations; it is widely spread in terms of geographical region, entry qualification of the students, experiences, age, sex, and recruitment of students.It is not as if 20-30 students are coming to class every day like in a face-to-face system and accordingly we prepare a suitable assessment strategy for that group.Things are purely different in ODL systems.As an example, in IGNOU more than 0.6 millions students enrol in a year, i.e. in both the January and July session.It is simply difficult to cater to them in terms of developing learning attitudes, curriculum practices and developing comprehension in the ODL system, and assess their learning constructively.To ensure quality in a said quantity, we need to renovate the practices.Let us discuss some of the important points to renovate assessment in ODL.
• Justified bifurcation in assessment is needed in ODL programmes; that is to say, equal contribution of formative and summative evaluation to the final certification of the students.• Integration of formative and summative evaluation components needs to be done carefully.
The above two evaluation strategies cannot be separated fully, rather each type of evaluation, in-house or external, needs to be depended upon each other.Students should feel that every action and activity which has been practiced in the system of assessment has a definite contribution to the final certification, whether formative or summative, or both.• ODL Institutes should consider including partially or fully open book examination systems in their assessment components.A careful system needs to be developed in this regard.
• A team of supervisors needs to be appointed to observe, supervise, vet, and compare the answer prepared by the students and evaluated by the tutors or instructors.• Questions in term-end examination need to be more analytical-type, so that the real objective of assessment may be fulfilled in ODL system.• No distance education institution should neglect regularity of the students in various assessment activities.There should be the provision of a grade and credit to students' attendance rate in the personal contact programme and practical sessions contributing to final certification.• Assignment questions should be carefully prepared and timely distributed to the students.
Again, the authorised tutors specialised in the field should assess them.Suggestions for improvement need to be mentioned in each question of the assignment.More over one moderator should supervise and moderate the checked answers of the students' assignments.
The process of peer assessment of assignments could also be introduced.• Accordingly, an external and an internal expert needs to be appointed to moderate and supervise the assessed copies of term-end examination either full or randomly.In case any irregularities are found, it should be communicated to the concerned evaluator and necessary action should be taken.• For students from overseas countries, online assessment facilities should be practiced.There should be a special assessment cell in each Open University for this purpose.• Suitable question banks of each course and programme for the term-end examination may also be developed to reduce examination tension of the students.A limited percentage of questions for term-end examination could be drawn randomly from question banks.• Academic counsellors and tutors should be trained not only to tackle the academic matters but also to link in-house and external examination with the learning experiences of the students.• ICT may be effectively used as an assessment tool in various academic programmes in the ODL system in the form of portfolio, e-portfolio, student's journal and online examination.• Students' development and involvement in studies and other activities throughout the programme could be recorded by the students themselves through an e-portfolio, which should be evaluated on a terminal basis and certain credit should be assigned to it.5-10 percent of total credits of the entire programme could be kept for this purpose.Open Universities need to develop a definitive format of e-portfolio and ICT proficient teachers should be engaged in evaluating it.• Open-book examination (partial/fully) could be introduced in ODL systems after carefully observing and monitoring the system.Before implementing it, there is the need to explore whether open book examination is needed for that programme or not.Such facilities could be provided to that programme where the nature of examination is purely subjective and analytical but not objective.

Suggestions and Implications
Assessment is an integral part of any system of education.It is closely linked with learning experience of students (Black et al. 2003).Assessment in the ODL system is a complex process.It is a challenge to assess and manage quality in a widely spread system of education.To sustain quality in ever expanding quantity can be possible if the system functions in order.ODL systems should ensure that students earn marks/grades according to their competency.More so, it should not make them feel neglected and de-motivated by earning a lower grade, in spite of performing better.

Table 1 : Assessment practices in Open Universities Open Universities Assessment Practices
It conducts both formative and summative evaluation with a greater weighing to summative assessment and lesser weighing to formative assessment.Assignment in each course, term-end examination, projects, dissertation, seminar, group discussion, observation records, practical assignments; field work, contact sessions, hands on activities are the common components of students assessment.