Development of an Online Teaching Competency Scale for University Instructors

In providing effective online education, it is crucial that the instructors have the competence to teach online. The aim of this study is to develop a valid and reliable online teaching competency scale for online instructors. The data were collected from 392 instructors working in a big state university in Turkey (Istanbul University-Cerrahpasa). The instructors have conducted online courses through synchronous and asynchronous methods during the pandemic process. The development and evaluation process of the scale included exploratory factor analysis and convergent validity. The scale consists of 15 items and represents four factors of online instructors’ competencies: pedagogy, facilitation, technology, and course administration. The total variance of the scale was found 64%. Internal consistency coefficient of the whole scale was found to be .83 according to reliability analysis. The results of the study revealed that the scale is valid and reliable for measuring instructors’ online teaching competency. The implications of the study were discussed in detail.


INTRODUCTION
Being a symbol of modern culture today, technology has become the most effective and indispensable element of daily life (Menz, 2009;Ozturk & Can, 2013;Zhang, 2017).As in many other fields, new technological concepts are added to the field of education every day.Due to developments in the field of information and communication, the changes in the needs of people, the increasing number of students and the amount of information, the low-cost Internet access and the widespread use of mobile devices that allow access from anywhere can be considered among the main reasons for the change and transformation in education.The reflection of the changes in the social order onto the learning environments due to these factors has been particularly effective in the differentiation of the teaching methods used (Gurley, 2018;Schmid, & Petko, 2019).Concepts related to new teaching methods and techniques such as web/computer-supported/based teaching, e-learning, virtual/cyber classroom, distance teaching have gathered prominence in the education world.
The recent technological developments and changes in social life have caused online teaching to become more widespread all over the world.Especially after the Corona Virus (COVID19), the disease that emerged in the last quarter of 2019 and spread worldwide in a very short time, was classified by The World Health Organization's (WHO) as a global epidemic, the popularity of distance education went off charts.In order to reduce the rate of spread of this epidemic affecting the whole world, the majority of countries have adopted many strict rules to prevent social interaction.Restrictions such as completely banning closed places like cinemas, theaters, and shopping malls where people are gathering in large numbers, working from home, and organizing flexible working hours are some of the measures taken.At this point, it is inevitable that the sudden changes in the social and cultural fields affect the education systems of the countries.According to Telli Yamamoto and Altun (2020), education is the most affected field by COVID19 after health.According to UNESCO (2020) data, while the education life of approximately 300 million students (17.1% of students receiving education) was restricted in March due to the epidemic, this number reached approximately 1.5 billion (84.3%) within a month.As a result, educational institutions have had to urgently stop face-to-face education at all levels from kindergartens to higher education in this process and switch to distance education practices.Machynska and Dzikovska (2020) stated that educational institutions that try to carry out their activities by taking urgent measures during the pandemic process face various difficulties.One of them is to decide on the learning platforms to be used in distance education and to ensure that teachers reach the competence to teach in these environments.In providing effective online teaching, it is very important that the instructors have the competence to teach in online environments.In this direction, studies have been carried out in the literature to reveal the competencies of online tutors.

ONLINE TEACHING COMPETENCIES AND SELF-EFFICACY
Online teaching competencies consist of many categories.The categories that are found in the literature include technology/technical skills, online communication skills, pedagogical knowledge, teaching methods and strategies, online education and content; field expertise, personal characteristics, process management and facilitation, planning and preparation, course management, and evaluation (Aydin, 2005;Denis et al., 2004;Klein & Fox, 2004;Reid, 2002;Richey et al., 2001;Salmon, 2012;Shank, 2004).For effective online teaching, instructors must be proficient in these dimensions.At this point, the self-efficacy of the tutors is very important.In the educational process, it is important to understand self-efficacy related to various academic practices because self-efficacy has a significant impact on participants' goals, efforts, and achievements (Kundu, 2020).Studies that examined the importance of teachers' self-efficacy in the online teaching process revealed a strong link between self-efficacy and technology use potential (Corry & Stella, 2018;Sun & Chen, 2016).Tutors tend to feel less selfefficient about online teaching compared to physical and online classroom settings (Johnson et al., 2020).However, instructive self-efficacy is malleable (Bandura, 1997), and the research has shown its relationship to student outcomes (Goddard et al., 2000;Tschannen-Moran & Hoy, 2001).In addition to the importance of quality in both technology and curriculum of online education, more research is needed in defining and determining the instructional self-efficacy structure in online education (Corry & Stella, 2018;Ma et al., 2021).

LITERATURE REVIEW
A Great amount of valuable information could be extracted from evaluations of distance and online education and instructors, which then could be of benefit to the learners and could be used to enhance better learning.With these evaluations, course designing and administration could be optimized for the best learning experience as well.However, according to Thomas and Graham (2018) given the immense growth of online education, the standardized evaluation of online courses and instructors is extremely impoverished.Berk (2013) posits that the available measures and the quality of measures are lagging far behind course production in the context of the assessment of online courses and the instructors that are conducting them.
Later, broadly speaking, checklists and rubrics were either developed in-house or acquired from other institutions to evaluate the online courses, the instructors, and especially the course design (Piña & Bohn, 2014).In evaluating the online courses and instructor's student evaluations have always been among the most common forms of evaluations in online higher education courses (Thomas, 2018).For instance, to evaluate the online instructors, Loveland (2007) adjusted the Student Evaluation of Teaching (SET) used widely and accepted as a valid and reliable instrument to evaluate instructors in a face-to-face classroom by altering "oral" communication skills with "written" communication skills.Another measure that is adapted from traditional evaluations to evaluate distance education is Electronic Student Instructional Report II (e-SIR II) that is administered by Educational Testing Services (Klieger et al., 2014).
The scale features planning and course organization, interaction, specific course activities like grading, exams, and assignments, teacher's instruction and course material, course outcomes, student effort and engagement, and the amount of work, pace, and difficulty of the course (Liu, 2012).
In the context of evaluating online teaching, Northcote et al. (2011) tried to define the scale of knowledge and skills required for an effective online educator.Bigatel et al. (2012) also searched to find out the competencies for online teaching success.They identified attitude/philosophy, building a learning community, class administration, faculty workload management, teaching and learning, and technology-use abilities as effective online teaching competencies.Kavrat & Turel (2013) developed a scale to determine teacher competencies for online teaching.They identified the teacher's roles as communicative, technical, social, and pedagogical.Furthermore, in their study, Gosselin et al. (2016) examined the threshold concepts, threshold attitudes, and threshold skills of the instructors.These included how instructors perceived the course design, facilitated interaction, engaging meaningfully in online learning contexts, selfefficacy, and confidence in online teaching, management of assessment process, setting up and modifying online learning, tracking student attendance and progress.However, they still believe that there is a need for more research to clarify the threshold concepts and self-efficacy levels in academic staff within the context of online teaching and learning.Reyes-Fournier et al. ( 2020) as well concluded that the available measures and scales that evaluate online and distance teaching efficiency have significant limitations.Thus, online teaching competencies scales cannot shed light comprehensively on instructors' online teaching competencies from the online teaching process (Wang et al., 2019).Reyes-Fournier et al. (2020) add to that by stressing the lack of research and appropriate tools for evaluating online teaching and asserting that reliability and validity data are insufficient.

RATIONALE AND IMPORTANCE OF THE STUDY
Online teaching activities can generally be carried out synchronously or asynchronously.In asynchronous learning, students and instructors participate in teaching activities at different times/places, while in synchronous distance learning, all participants perform learning activities at the same or different locations (Allen & Seaman, 2008).However, in the online teaching process, most of the instructors stated that online education was considered as transferring the existing teaching materials to the online environment (Wang et al., 2019).Today, all educational institutions have urgently switched to distance education, and in this process, teachers have generally focused on performing face-to-face (F2F) educational activities through live sessions.At this point, video conferencing applications such as Google Meet, Microsoft Teams, Zoom, storage areas such as Google Drive, Dropbox, Yandex Drive, learning management systems such as Moodle, Google Classroom, Canvas or various Web 2.0 applications are used to increase interaction in the course.It is important that the instructors have the technical competencies to use such tools in the online education process.Gang & Shanxi (2015) also stated that technology competencies are generally taken into consideration in determining the competencies of teachers in online teaching.However, the online instructor should have the competence to guide students by organizing their learning activities in the online learning process, as well as bringing educational content to the online environment (Allen & Seaman, 2008;Wang et al., 2019).In this respect, it is clear that it is necessary to focus on the problem of "reaching the proficiency of teachers in teaching platforms to be used in distance education" stated by Machynska & Dzikovska (2020).While determining these competencies, the pedagogical competencies of online instructors (Machynska & Dzikovska, 2020), their ability to prepare themselves and students for online education, to choose the right tools with appropriate teaching methods and techniques, to facilitate learning, and to manage online courses should also be taken into consideration (Wang et al., 2019).In today's world, where all educators from preschool to higher education assume the role of online teacher, determining the online teaching competencies of teachers is important in improving the online education process.This study was focused on determining the online teaching proficiency levels of higher education instructors based on their self-efficacy and confidence in online teaching.The aim of this study is to develop a valid and reliable online teaching competency scale for online instructors.

METHOD
The correlational research design employing factor analysis techniques was used in the study.More specifically exploratory factor analysis (EFA) technique was used.EFA technique is used to reduce the number of variables by grouping according to moderate or high correlation with each other (Fraenkel et al., 2012).

PARTICIPANTS
The data were collected from the instructors working in a big state university in Turkey: Istanbul University-Cerrahpasa.These instructors have experienced online teaching along with the pandemic process.The face-to-face instructors that move to remote teaching due to the emergency situation have conducted online courses through synchronous and asynchronous methods during the Covid-19 pandemic process.The participants have been teaching their face-to-face courses with online methods since March 2019 due to the pandemic.The data were collected from the instructors in the 2020-2021 fall semester mid-term.For EFA, the data were collected from 392 online instructors (209 females, 183 males; aged between 25 and 64 years) who have been working in various departments of health, engineering, and social sciences faculties.The online survey was e-mailed to all instructors in the university and the volunteers filled the scale.

DATA COLLECTION AND ANALYSIS
Previous studies in the literature were examined to determine the online instructor competencies in the study.Based on the previous studies (Bangert, 2006;Kavrat & Turel, 2013;Reyes-Fournier et al., 2020;Wang et al., 2019) and theoretical frameworks (Bawane & Spector, 2009;Klein & Fox, 2004;Richey et al., 2001;Shank, 2004) in the literature, an item pool was created including 32 items.The scale items are designed in 5-Likert type questions (from 1: strongly disagree to 5: strongly agree).For content and face validity, three field experts and one language expert checked the created items, and the necessary revisions were made.The scale was applied to 10 instructors as a pilot.A total of 32 items were sent to 1197 instructors working at the university via email.A total of 392 instructors voluntarily participated in the research.The scale consisting of 32 items was applied to 392 instructors for EFA.In addition, convergent validity was checked for the construct validity of the scale in order to verify the factor structure that emerged in the EFA.Cronbach's alpha reliability was applied for the reliability of the scale.For construct validity, EFA analysis was conducted.The composite reliability (CR) and the average variance extracted (AVE) values were calculated for convergent validity.IBM SPSS AMOS 24.0 software was employed to scrutinize the data.First, exploratory factor analysis assumptions were checked (Field, 2013;Pallant, 2020;Tabachnick et al., 2007).Outliers and missing data, normality of data set, sample size, and sampling adequacy, inter-correlations between variables and linearity were examined.Overall, the data met all the assumptions.

FINDINGS THE RESULTS OF EXPLORATORY FACTOR ANALYSIS AND RELIABILITY
The EFA was employed using principal components approach.The adequacy of sample has been decided as statistically significant since the KMO coefficient was found to be .846and the Bartlett's Sphericity test's χ 2 indicated a value of 2417,037 (p < .05).In the context of the study, the correlation matrix was examined to determine the relationships between the items and it was determined that there were relationships between the items.For this reason, the promax  rotation technique, which is used when there is a relationship between factors, was preferred (Brown, 2009).In the literature, it is expressed that the cut-off value can be taken as .512for a sample of 100 and .364for a sample of 200 (Field, 2013).In this study, the cut-off value was determined to be .40.It is not appropriate to handle items with a value less than .30 in the communalities table under any factor (Pallant, 2016).In addition, the value of the anti-image correlation should be greater than .50(Field, 2013).The anti-image matrix of correlation and communalities values were checked, and it was observed that the assumptions were satisfied.
In the first factor analysis, 34 items were collected under 7 factors.The analysis was repeated by checking the items and removing the items that went under two or more factors.There should be a difference greater than .10 between the factor loads of the items that appear in two factors (Field, 2013).By controlling the communalities and factor load values each time, the items that need to be removed were removed sequentially, considering the significance value in the scale.As a result, a structure consisting of 15 items under 4 factors emerged.The scree plot graph of the breakpoint was presented in Figure 2.
The total explained variance of the scale gathered under 4 factors is 64.244%.According to items' meanings, the first factor has been named "pedagogy", the second "facilitation", the third "technology", and the fourth "course administration".The Cronbach's alpha coefficient of the scale was calculated for reliability (Cronbach's α = .834).The reliability of the factors of the scale took values between .83 and .68.The reliability coefficients of the factors that are around .70 is sufficient for internally consistent scores (Nunnally, 1994).Therefore, the reliability of the scale is good.The communalities of the scale items, standardized factor loads, eigenvalues, explained total variance of the factors and their reliability values were presented in

ITEM-FACTOR CORRELATION ANALYSIS
The means, standard deviations and inter-correlations between the factors are presented in Table 3.While the pedagogy factor has the highest mean score (M = 4.68; SD = .40),the course administration factor has the lowest mean score (M = 3.36; SD = .89).According to Pearson Correlation analysis, there were significant relationships among all factors (p < .01).The results show that pedagogy, facilitation, technology, and course administration were positively correlated.The scale has internal consistency, and each factor measures a discrete property serving the aim of the scale as a whole.

CONVERGENT VALIDITY
A convergent validity study was conducted for the construct validity of the scale.AVE values were explained by the dimensions in the scale, CR coefficients and finally, whether the composite reliability coefficients for convergent validity are greater than the average variance values explained were examined.Bernstein , 1994), and CR coefficients should be greater than AVE values (Byrne, 2016).In

DISCUSSION
As a result of this scale development study conducted to determine online teaching competencies, a 4-factor structure consisting of 15 items emerged.These factors are "Pedagogy", "Facilitation", "Technology", and "Course Administration".The variance total that the scale explains has been determined to be 64.244% and the reliability coefficient of the scale was found to be Cronbach's α = .834.In addition, convergent validity, which was carried out to check the structural validity of the scale, provided appropriate values.The scale developed according to the obtained results has turned out to be a valid and reliable scale.
When the factor structure of the scale is examined, the explained variance of the "pedagogy" dimension, which expresses the pedagogical knowledge level of the teachers, is 35.5% and it has the most important place in the structure of the scale.Instructors with a high level of pedagogical competence should clearly state the important goals and subjects of the course in the online course process.On the other hand, it is very important for the instructors to make the necessary preparations before the courses, conduct their courses willingly and use the teaching methods and techniques that will ensure the participation of the students in the course process (Bawane & Spector, 2009).As a matter of fact, the importance of the pedagogical competence of instructors in the online education process is mentioned in the literature (González-Sanmamed et al., 2014;Gosselin et al., 2016;McAllister & Graham, 2016;Murphy et al., 2011;Wang et al., 2019).For this reason, the pedagogical competence factor of the scale is paramount.
In online teaching, it is very important for instructors to facilitate the process.The facilitation aspect of instructors' support for students to form and maintain a learning community, contributing to their development of a positive perspective towards online teaching, and enabling students to establish online learning relationships were discussed in this study.Reid (2002), Shank (2004), Bawane and Spector (2009) highlighted the importance of facilitation in their study in which they determined the Online Instructors Competencies categories.Facilitation includes "regular, active, and thoughtful classroom interactions executing planned activities, managing communications, and supervising learning processes" (Blackman et al., 2020).The facilitation dimension of the scale has an important place in revealing how the instructors perceive themselves in this aspect.This dimension, which has the same items in the study of Kavrat and Turel (2013), was named as social role.However, in this study, based on the literature, it was concluded that the factor explains the facilitation better (Al-Salman, 2011; Bawane & Spactor, 2009;Blackman et al., 2020;Reyes-Fournier et al., 2020).
In the teaching from distance process, instructors have to use software and hardware tools.
The technology dimension of the scale includes the ability of instructors to effectively use the hardware and software tools required by distance education and to solve the technical problems they encounter on their own.As a matter of fact, in the online teaching process, video conferencing applications such as Google Meet, Microsoft Teams, Zoom, online storage spaces such as Google Drive, Dropbox, Yandex Drive, learning management systems such as Moodle, Google Classroom, Canvas, and various Web 2.0 applications are frequently used to increase interaction in the course.Therefore, it is very important for the instructors to have the technical competencies to use such tools in the online education process.Gang & Shanxi (2015) draws attention to the fact that technology competencies should be taken into consideration in determining the competencies of instructors in online teaching.Similarly, Roberts (2018) suggested that technology competence is very important in his study on the competencies of online tutors and that trainers should be given training to improve their skills in this field.When the studies on e-Teacher Competencies in the literature are examined, concepts such as technical knowledge (Reid, 2002), technical skills (Salmon, 2012), technology knowledge (Denis et al., 2004), Skills with Internet tools Bailie (2011) are especially emphasized in technology dimension of instructors' competencies.Similarly, in the literature, it is seen that the technology dimension is also included in the studies of instructors' roles (Aydin, 2005;Egan & Akdere, 2005;Lee Dong Yub, 2011;Thach & Murphy, 1995;Whitehead, 2018;Wiesenberg & Hutton, 1995;Williams, 2003).
In the online teaching process, courses are conducted over various learning management systems.The Course Administration dimension of the scale includes the applications performed by online tutors on LMS.Learning management systems have features such as module, calendar, homework, online exam, discussion, integration with live course systems.Effective use of such features of the learning management system in structuring the online teaching process will facilitate the Course Administration process.On the other hand, activities carried out over LMS are very important for effective communication and interaction processes.In online teaching, administration is to manage the time and course, demonstrate leadership qualities, establish rules and regulations (Bawane & Spector, 2009).Nowadays, course administration can be carried out easily via LMS.
When the correlation between the factors of the scale was examined, moderate positive relationships were found between factors.As a matter of fact, it can be said that competencies in all these factors interact with each other in the effective online teaching process.When the online proficiency levels of the instructors were examined, it was found that they see themselves the most competent in pedagogy dimension.This situation may be related to the fact that the instructors in the sample of the study have been teaching face to face at the university for years.The pedagogical principles in the face-to-face teaching process are naturally similar to those in the online teaching process.However, it has emerged that they consider themselves less adequate in Course Administration over LMS.It can be said that this is due to the fact that the online teaching experiences of the instructors started with the pandemic process.As the training started to be given completely online, the instructors started to manage their courses on LMSs such as Canvas and Google Classroom.

CONCLUSION
As a result, in this study, a valid and reliable scale was developed to determine the online teaching competencies of instructors in higher education.The goal of effective survey design is to measure constructs with short, concise, user-friendly questions that produce high response rates (Saleh & Bista, 2017).The developed scale offers an advantageous structure in terms of usability because it contains few items.The factor structure of the scale (pedagogy, facilitation, technology, and course administration) overlaps with the theoretical structures that reveal online teaching competencies.In the online teaching process, instructors need to make use of many technological tools.However, the high level of technological knowledge of the instructors alone does not guarantee an effective online teaching process.In this context, it is a very important result that the pedagogy dimension of the scale came to the fore.Thus, the pedagogical competence level of the instructors is one of the most important requirements of qualified and effective teaching.
The strength of the study is that a scale was developed with the data obtained from a large number of participants working in different disciplines at higher education level.However, although the sample was sufficient in the study, it has limitations in terms of collecting data from only one university, not having CFA, and gathering data from convenient sample.In the light of the data obtained from the research, various suggestions for practitioners and researchers are presented.By using this scale, online teaching proficiency levels of instructors who teach online at university level can be revealed.Faculties can carry out studies to improve the competencies of instructors according to the results obtained from the scale.In future studies, it may be recommended to verify the factor structure of the scale with CFA.
The developed scale can also be used to determine the online teaching competencies of teachers at the K-12 level.Based on the items in this scale, a student evaluation form can be created, and a student version of the scale can be developed.In this way, both the instructors' self-assessment and the students' evaluation of the teachers can be provided in parallel.
Figure 1.summarizes the operations performed during the scale development process.

Figure 1
Figure 1 The development and validation process of the scale.
Simsek et al.
Simsek et al.

Table 1
presents the demographic characteristics of the participants.
According to the criteria, Items should have standardized factor loads and AVE values greater than .50,CR coefficients should be greater than .70(Nunnally &

Table 4 ,
the AVE and CR values of the factors are given.The scale developed in the study meets the AVE and CR criteria.