Open and Distance Education/QA in Open and Distance Learning

Introduction

"The concept of quality in online and distance education can be elusive and complex" (ICDE 2015). Uvalic-Trumbic & Daniel (2013, 2014) argued that quality and reality in online and distance education have an overbearing effect towards the implementation of e-learning. However, most researchers emphasized that quality is not anything which is, but something which can be created nor caused. (ICDE 2015).

However, according to Pirsing (1994, p 241, In: Uvalic-Trumbic & Daniel 2014) stated that quality is not a thing but it can be formed in the due process. As a famous saying states that "quality is in the eye of the beholder." (ICDE 2015)

According to Nordkvele, Fossland & Netteland (2013) quality can be categorized into three main levels such as:

1. Macro-level: national or global dimension
2. Meso-level: institutional dimension
3. Micro-level: individual practice

Discussion of quality assurance in open and distance education are fairly recent.This comparative study in Quality Assurance in Open and Distance Learning between Europe and Asia will highlight the differences between those two regions in the way that how they maintain the quality assurance. Also, we will discuss Quality Assurance in Mooc.

What is Quality Assurance?

Quality Assurance of online or e-learning has been a topic of educational debate for recent years which have culminated in the broad literature review. Frydenberg (2002) critically analyzed quality dimensions in a range of e-learning quality models and highlighted nine domains. These quality domains include executive commitment, instruction and instructor services, financial health, program delivery, legal and regulatory requirements and program evaluation (Ossiannilsson 2012 p65). However, Shelton (2011) stated that quality involves support, motivation and overall policies which is more important than focussing on technology, cost-efficiency, and management (in Ossiannilsson 2012 p68). Ossiannilsson (2012) reviewed e-learning benchmarking in higher education and also international Quality Standard models. Also, she highlighted the importance of the conceptual and holistic approach to quality and quality must be considered as a dynamic process since it is an educational area which revolutionized rapidly. Additionally, she stated the importance of creating a culture of quality within all educational levels such as institutional, staff and students rather than just satisfying certain standards as stipulated in quality models. Furthermore, she highlighted the importance to include learners in the quality assurance work and mission. Jung and Latchem (2012) focus on quality assurance processes within institutions. They highlighted the importance of engaging in a systematic approach to quality assurance and to consider quality assurance as a gradual process of continuous improvement, which will allow the institution to be transformed from external controls to an internal culture of quality. Also, they stated that outcomes are the leading measures of quality, thus, poor quality can be very costly. Therefore an investment to improve quality in e-learning is beneficial.

ICDE's 2011 study of quality assurance proves that majority of the countries in the Asia Pacific region had accreditation and quality assurance on higher education which specifically addresses e-learning. Two comparative studies conducted by ICDE which documents QA systems while IDRC study focuses on the institutional level. IDRC comprehensively focuses on countries such as India, China, Thailand, Indonesia, Sri Lanka,  the Philippines, Malaysia, Japan, Singapore, Hong Kong, Pakistan, and Mongolia. This study highlights varieties of approaches towards developing and improving their quality assurance system in their own context (Jung, Wong & Belawati 2013).Specifically, this particular study highlights the importance of creating a culture of quality assurance since it will be very difficult to create in such culture in a top-down approach. Such culture of quality can only be implemented and become sustainable if staff take personal ownership of such processes. However, it has been proven that countries and institutions with national quality assurance framework have the tendency to improve and maintain such culture of quality in the future. Also, in the survey stated that strong institutional leadership is an important component for provision of infrastructure, resources, training, and support, includes incentives for staff and learners.

In the Asian survey (Jung, Wong & Belawati 2013) stated that many institutions use the ISO 9000 system which demonstrates the applicability of approaches to quality used in the commercial world to higher education. In such cases, the Open University of China (OUC) quality assurance framework consists of 35 quality indicators within five quality areas: teaching resources development and management; teaching process management; learning support services; teaching management, and teaching and learning environment (Du, Yang, Yin, and Zhang 2009, in Jung, Wong & Belawati 2013).

The Open University of Sri Lanka highlights the challenges for quality development and quality assurance in which they argued that such framework must be flexible, dynamic and have the capacity to adapt to different e-learning contexts. Also, they stated three main areas in which e-learning can be successful which includes, human resource development for knowledge, training, and research in online learning. Jung, Wong & Belawati (2013) stated:

• Adopt a balanced, systemic approach
• Focus on pedagogy, learner support, and management
• Move towards a performance and outcomes-based approach
• Promote a culture of quality and continuous improvement

Quality Standard Models

Ossiannilsson (2012) argued that majority quality standard models focus on three to six main dimensions as in the E-xcellence by EADTU (William, Kear & Rosewell 2012) and was supported by Bacsich (2009-2011).

The three main domains are.

1. Services: includes staff and student  support
2. Products: curriculum and course design & course delivery
3. Management: strategic planning and development

Nature of Quality Interventions

Many researchers argued that quality assurance system varies and also they application depends on the institution maturity with e-learning processes. Major differences include the type of quality criteria, methods employed, the role of quality managers,  reviewers and external assessors.

Previous Studies of Quality Assurance in Open and Distance Education

Recent research and reports have proven the existence of numerous schemes and quality assurance model of online and distance education with common features which allow flexibility for institutions to adopt in at national and institutional context. This commonality includes criteria such as performance, curriculum design, learners support and other important educational elements. These criteria can be categorized as management, products, and services. Also, the concepts of quality assurance can be considered at Macro (national/international) Meso (institutional) and Micro (individual practice) levels. (ICDE 2015)

Previous studies highlighted the characteristics for quality assurance and quality enhancement as listed below:

1. Multifaceted
2. Dynamic
3. Mainstreamed
4. Representative
5. Multifunctional

ICDE Quality Standard Study 2014

The objective of ICDE Quality Standard Study 2014

• Highlight existing global QA standards and guidelines
• Benchmarks and quality assurance models for open distance, flexible, online education, and e-learning, MOOCs and OER

International Research Centre Openness and Quality in Asian Distance Education

Findings highlight the various quality assurance approach approaches in higher education within the region in the respective countries which have been studied. (Jung, Wong & Belawati 2013). Creation of a culture of maintaining quality has been emphasized in the survey. Such practices may encounter early barriers and obstacles if "top-down" approach is adopted in an institution. Collective and collaborative efforts of those involved in online and distance education play an important role in the scalability and sustainability of such initiative.

Countries with grounded quality assurance framework and policies have the tendency to effectively implement it at an institutional level. However, it was stated in the survey that institutional leadership must ensure accessibility to infrastructure, resources, training, and incentives for learners and facilitators. Many institutions in the Asian regions uses ISO 9000 system which highlights the applicability of commercial concepts to higher education (Jung, Wong & Belawati 2013)

The Open University of China (OUC) quality assurance framework resembles such commercial concepts since it has 35 quality indicators such as teaching resources development & management, teaching and learning environment. (Du, Yang, Yin, and Zhang 2009, in Jung, Wong & Belawati 2013). However, the Open University of Sri Lanka stated three important components of quality assurance framework are flexibility, dynamic, and adaptability to changing learning environment. The importance of investment in human resources for knowledge, training, and research was emphasized which guaranteed the successful e-learning implementation.

CALED 2003 (Latin America)

The review of quality assurance for distance and online education has proven that numerous countries in the  Latin American region with existing national quality assurance agencies. The establishment of a consortium of countries and institutions results in the development and adoption of quality assurance system and criteria. However, the criteria as stipulated in CALED 2003 favors distance education in bimodal institutions rather than that of unimodal institutions. Collaboration with other regional systems is highly prioritized in the near future.

African Council for Distance Education

COL/DEMP framework was developed by the African Council for Distance Education has been adopted by numerous African Open University. However, the AVU Quality Assurance Framework has seven performance criteria for ODL and community capacity building, development and engagement. The capacity building factors focus on the accessibility of higher education in the African region. The commonality features between ACDE and AVU frameworks will positively impact developments in the Francophone and Lusophone countries.

European University Association (EAU 2014)

EAU 2014 stated that the rationale for online and distance education is due to pedagogy and monetary value, flexibility in time and space for learners, residential students and lifelong learners. However, participants in the survey demand curriculum and assessment methods to be amended accordingly. Most institutions review their online educational strategy in order to allow mainstreaming and effective implementation of e-learning.

The European Commission report (EC 2014) recommendations on quality in higher education:

1. Integration of digital technologies and pedagogies in teaching and learning
2. Continous PD and training on relevant digital technologies and pedagogies for teaching staff
3. Collaboration between the central government and academic institutions in order to increase accessibility to open educational resources

Quality Assurance in Mooc

Quality Frameworks and Quality Processes

The quality of MOOCs can be considered from the following perspectives.

1. Quality from the learner's point of view.

Considering quality from the perspective of learners requires engaging with diverse goals, expectations, learning behaviors, and abilities of learners to facilitate their own learning.

2. Quality connected to the pedagogical framework of the MOOC.

Downes (2013) has formulated four key success factors in this area: autonomy, diversity, openness, and interactivity.

3. Quality related to the input elements.

These may include aspects such as instructional design, the content and resources, multiple choice questions and assessment, the technology employed, and the quality of the teacher (Margaryan et al. 2015; Lowenthal & Hodges 2015)

4. Quality based on outcome measures.

Taking completion rate as a measure of the quality of a MOOC has therefore been criticised (Weller 2013; Clark 2016). It is argued that low values of conventional measures, such as retention and completion, may not signal poor quality.

The conventional measures may not be appropriate if the intentions of MOOC learners differ from those of a conventional university student (Ehlers et al. 2013)

Butcher and Hoosen (2014) also question whether tightly structured frameworks for quality assurance can be applicable to MOOCs, because openness and flexibility are primary characteristics of these new approaches.

However, Jansen, D, Rosewell J & Kear K (2016) suggest that since both conventional HEIs and MOOCs offer higher education, quality principles developed for HE could be used to improve the quality of MOOCs and OER.

Quality can also be viewed at three levels: macro (national), meso (institutional) and micro (course) level (Nordkvelle et al. 2013)

Future Research of Quality Assurance in Open and Distance Education

Recommendation 1   Mainstream e-learning quality into traditional institutional quality assurance

Previous studies with stakeholders emphasized two related problems. The first one is exhaustion with existing quality assurance processes, indicating that they are already too resource intensive when evaluated on a cost-benefit basis. The second is the concern of overlap, or in some cases even contradictions, between different quality labels and systems.

The mainstreaming of ‘e-learning’ into quality assurance requires disaggregation of the standard components of e-learning quality systems and integrating each of these into the appropriate part of the quality assurance process. They are shown below.

• E-learning as technology-enhanced learning: TEL here mainly refers to technology-supported optimization of learning and teaching processes, but also of governance/administration processes. This could include the addition of criteria on providing access to high-quality learning resources electronically, having the appropriate technological infrastructure to support classroom-based teaching and/or optimizing administrative processes through a paperless workflow to existing quality assurance criteria.

• E-learning as a mode of provision: distance learning, blended learning, problem-based learning, lectures, work-based learning, and simulation are all distinct forms of provision, which require distinct measures of quality. Rather than present an entire quality label for each, a mainstreamed quality system would set out a common core of criteria applicable to all teaching and learning, and augment these with modules for each mode of learning – with each module containing only those criteria which are specific to the mode in question.

• E-learning as a driver for innovation: most of the e-learning quality assurance systems reviewed include a focus on innovation. This is displayed through requirements for innovation strategies, rapid iterative review, the connection between research and pedagogy and/or an emphasis on learning design (which requires knowledge of latest innovations to select the most appropriate means to reach learning objectives). Mainstream quality assurance systems tend to focus on guaranteeing a set level of quality rather than on acting as a tool for pushing innovation and could thus benefit from the inclusion of such criteria.

Recommendation 2   Support the contextualisation of quality systems

According to the ICDE reports, quality systems make socio-economic-cultural assumptions that are not equally true in all contexts. Examples of such assumptions include some points are shown below.

・Access to high-bandwidth Internet: a stress is often put on interactivity without considering the access implications of such interactivity in all places.

・Existence of a multi-stakeholder, participatory governance environment: a strong feature of western democracies, this does not necessarily apply in the case of top-down, less participative governance.

・Personal Computers (PC) are the main way to interact with e-learning, when in fact mobile devices are the connection method of choice in many countries.

・Academia universally speaks English or a national language: most quality assurance models are available only in English or in the language of the country from where they originate.

It would be mistaken that high-quality e-learning can only be delivered over the high-speedInternet to a Personal Computer, in an English speaking western democracy. This would suggest a need for stakeholders in below.

・Translate existing quality systems to allow for wider access (along with providing for multi-lingual reviewers with knowledge of the cultures, etc.).

・Work with quality assurance model providers to widen the interpretation of specific criteria within existing systems, to allow for more international applicability, and regional sensitivity.

・Support the development of regional quality labels, derived from existing tools and adapted for context.

・Explore possibilities of ICDE and COL working together with the mobile telecom sector and educational experts to create a set of quality criteria specifically related to mobile learning systems delivered by feature-phone/low-end smartphones. This is an area in which there is little current experience and could yield major benefits.

Recommendation 3   Support professional development, in particular through documentation of best practice and exchange of information

All of the systems reviewed took an approach to quality, which required significant knowledge on the part of the reviewers, as in many cases, the criteria on their own did not specify the best practice being required. Criteria were littered with terms such as ‘appropriate, suitable, relevant, standard-practice’ without giving examples of what would be considered to match any of these terms. We see this as a substantial risk inherent to many of the systems, as the quality of the reviews conducted is directly correlated to and heavily dependent on the knowledge and experience of the reviewers. A secondary risk is that where the standards leave scope for value and/or experience-based judgments, there can be a significant difference between the opinions of different reviewers, depending on their background. Practically all review schemes offer some training for reviewers, but follow-up on this is usually limited. For this reason, it is critical to support the professional development of reviewers by building up databases of reference materials, which they can use to understand their role better and improve their skills.

From an institutional perspective, Tony Bates points out15 that “it is one thing to have a set of standards

for e-learning; it´s quite another to implement them. Even rarer are studies that attempt to measure the

impact of a quality assurance process on the actual quality of teaching and learning”.

In particular, it is believed that the need for better documentation and reference materials could be achieved by stakeholders through two initiatives.

・Creation of an e-learning quality resource hub – an online collection of research papers, quality tools, training materials, etc., which could be useful to both institutions seeking to improve their quality systems, and to quality assurance reviewers. We would recommend that such a hub would be carefully curated to ensure that it focuses exclusively on the best research and tools, without creating unnecessary ‘noise.’ Examples of more general hubs in related areas include the OER Research Portal,16 and the Open Education Europa Portal.17 Also, the EADTU’s EMPOWERING Universities initiative guides universities in considering all aspects related to online and open education.18

・Create a best practice database, containing curated examples of best practices in institutions around the world, contributed by quality reviewers from examples in real reviews. The database would not necessarily be mapped to any particular quality label/scheme, but rather contain best-in-class examples under a number of categories, such as “institutional policies, Human Resource development, media design, learning design, learning environment,” etc. The database would be of particular use to reviewers in writing recommendations for improvement (since it would provide examples of what those recommendations could achieve), and for institutions trying to benchmark themselves against others, in preparation for a review, or in the process of implementing recommendations. It could be complemented by social functionality in the form of forums, wiki or other similar tools.

・Compile and maintain a register of professional development programmes and training materials appropriate for use by institutions and Quality Assurance Agencies.

Recommendation 4    Communicate and promote general principles

It is believed that a holistic and conceptual approach to quality management is essential to the implementation of well-functioning quality infrastructure. To this end, in section three, this study set forth a set of universally applicable characteristics for e-learning quality systems, derived from a survey of the principles as included within existing quality systems, from stakeholder opinions/positions, and policy documents. We find that each of the characteristics is essential and intrinsic to a well-operating quality system, whether at programme, institutional or regional/national/international levels.

The stakeholders should take a primary role in communicating and promoting these general quality principles, through a combination of actions:

・Train reviewers on the quality principles, and providing information about the principles

・Promote the quality principles to distance education institutions, as a foundation on which to build their quality policies and write their quality manuals.

・Stakeholders should work towards international adoption and the mainstream of the principles. In particular, we recommend that ICDE engage with ISO, and propose these principles for incorporation in the upcoming ISO 21001 standard on Quality Management Systems for Educational Organisations.

Recommendation 5   Assist institutions in designing a personalized quality management system

The study aims that a wide variety of quality tools are available for e-learning. It is including tools for evaluation, quality assurance, quality enhancement, self-evaluation, benchmarking, certification, assessment, standardization and more. All the tools stress that they should be used in the context of creating a quality culture within their institutions.

On the other hand, in particular with regards to e-learning policy and practice, institutions show a wide level of maturity levels, with some needing consultancy on how to develop a quality system for an initial foray into the field, and others seek incremental improvements to a well-developed and widely-deployed system.

Quality-service managers from several of the schemes still mention that queries from institutions that are not appropriate or not ready for the schemes remain a recurring problem.

Recommendation 6   Address unbundling and the emergence of non-traditional educational providers

Technological disruption of educational provision is rapidly leading to a situation where any institution with sufficient expertise can design and deploy a course globally at minimal cost. Coupled with the phenomenon of unbundling facilitated through regulatory changes, pedagogical innovations, and technological evolution, a multitude of specialized providers is entering the certification and assessment space. While still in their infancy, these changes promise to shake up the entire trust infrastructure that has been built around education, particularly Higher Education, in the past years.

Unbundling means that quality systems can no longer be focused exclusively on educational institutions as all-in-one learning design, teaching, testing, and certification providers. New providers are specializing in specific functions of educational provision – however, quality assurance standards and other regulatory instruments which apply to educational institutions do not necessarily apply to them directly, as they not designed to regulate such entities. On the other hand, limitations in legislation could also hinder the same companies from offering services – e.g. currently only universities can offer ECTS, even though other institutions are capable of offering individual modules at an EQF level of 5 or above.

A diversified (unbundled) landscape will likely require widely (internationally) recognized standards for provision and providers at each unbundled level, backed up by appropriate inspection and compliance bodies regulated by law.

Recommendation 7   Address quality issues around credentialisation through qualifications frameworks

Open education has recently led to an inexorable rise in ‘open’ or non-traditional qualifications. The same course might offer a learner learning badges, a certificate of attendance, a certificate of completion, a ‘verified’ certificate and a full qualification, translatable into university credit. A quality qualification should allow for the dual aim is shown below.

・certifying learning acquired (and the level of it where applicable)

・acilitating the recognition of that learning for purposes of education and employment

Recommendation 8   Support knowledge transfer from open and distance learning to traditional quality systems

Large-scale online distance learning programmes have reached significant maturity in a number of countries, has already been active for over four decades. During this time, the institutions in question have gained significant experience in utilizing technology to improve process management, course content and student outcomes. In particular, the burgeoning field of learning analytics is providing insights into processes of teaching and learning, which were never before available for purposes of quality improvement.

The experiences of open and distance learning institutions in implementing and using learning analytics, as well as other technically backed solutions for the enhancement of quality, has clear learning value for the rest of the education and quality assurance community. We, therefore, recommend that stakeholders should support a programme of best practice sharing between actors in the field, possibly through instruments such as master classes, knowledge sharing workshops and the like.

Recommendation 9    Support quality assurance audits and benchmarking exercises in the field of online, open, flexible, e-learning and distance education

There is a rapid development within the area, of open, flexible, e-learning and distance education. During this present study, stakeholders consulted have shown tremendous interested in this international quality research study. There have been suggestions that a quality assurance audit (such as this one) of the field on a 3-5 year cycle. The resulting reports could become the de facto reference point for quality assurance practices around the globe.

This recommendation can be a natural input to the suggested quality Hub, in recommendation 3. It is proposed that ICDE should take on the responsibilities to carry out this task, as with ICDE’s international membership and reach render it is well placed to coordinate international reviews and ensure regular updating. Undertaking a regular cycle of audits can thus be a task for ICDE. The leadership of quality assurance audits would position ICDE as the meta-level resource keeping abreast of improvements in the field.

Recommendation 10   Encourage, facilitate and support research and scholarship in the field of quality

This study has highlighted the urgent needs for research in the field of open and online learning, including e-learning to match the speed of technological development. In addition, there are also urgent needs for rapid dissemination and valorization of research within the area, not at least to mainstream quality in e-learning. Implementation should address not just best practice, but also next practice and the needs for innovation and sustainability for the 21st century.

Recommendation 11   Encourage, facilitate and support implementing quality assurance related to new modes of teaching

There are urgent needs for stakeholders in the field to implement quality assurance related to new modes of teaching at the governmental, institutional and quality assurance agency level.

Conclusion

Table 1 Most Used Quality Model (From Table 1 in Ossiannilsson, Williams, Camilleri and Brown 2015)

Quality Model	Certification	Benchmarking	Accreditation	Advisory Framework
ACDE (African Council for Distance Education Quality Assurance & Accreditation Agency)		√		√
ACODE (Australasian Council of Open, Distance & e-learning)		√	√
AVU (African Virtual University)		√		√
CALED (Latin American & Caribbean Institutte for Quality in DE)	√	√	√
CHEA (Council for Higher Education Accreditation), US			√
E-xcellence EADTU (European Association of Distance Teaching Universities), NL	√	√
OpenupEd EADTU (European Association of Distance Teaching Universities), NL	√	√
UNIQUe EFQUEL (European Foundation for Quality in e-learning), BE	√
ECB Check EFQUEL (European Foundation for Quality in e-learning	√
The eLearning guidelines (eLg) Tertiary Education Commission, AUT University & Massey University, NZ				√
The E-Learning Maturity Model (eMM) New Zealand Ministry of Education Tertiary E-Learning Research Fund		√		√
E-learning Quality Model (ELQ) NAHE (Swedish National Agency for Higher Education)				√
Epprobate (The Learning Agency Network, LANETO e V), DE	√			√
Khan eight-dimensional e-learning framework(Badrul Khan)				√
The OLC Quality Scoreboard (Online Learning Consortium, former Sloan-C), US	√			√
OER TIPS Commonwealth Educational Media Centre for Asia (CEMCA)				√
Pick&Mix Matic Media, SERO Consulting Ltd, UK		√