Skip to main content

Table 4 Checklist for quality practices in educational effectiveness

From: A guide for evaluation of online learning in medical education: a qualitative reflective analysis

Educational Effectiveness Checklist Level of Achievement (Fully / Partially / Not Achieved) Evidence Collection Method
Educational Program
Evaluation questions answered in advance
1. Is there an approved, updated and well-constructed online education plan?
2. Is there a matrix mapping of goals of OL against the vision and mission (at least 70% of subsets overlapping)?
3. Is data available to support decisions made in the program?
4. Is the online teaching plan aligned with the program learning outcomes?
5. Are the goals aligned and cascaded: strategic, long term, intermediate and short-term?
6. Is there a clear identification of the required resources for the program?
  1–6: document review
Course Design
Evaluation questions answered in advance:
1. Do courses have clearly stated Intended learning outcomes?
2. Is there a mapping matrix for course goals with organizational goals?
3. What is the percentage of online learning activities? (According to the nature of the course and its intended learning outcomes not more than 70% in basic courses and 50% in clinical courses).
4. What is the ratio between face-to-face to online assessment methods? (According to the nature of the course and its intended learning outcomes with consideration to the continuous, formative and summative methods)
5. What is the ratio between synchronous and asynchronous online learning activities? (1:1 is an acceptable ratio based on the requirements of the delivered material)
6. Are the selected contents up-to-date?
7. Are they related to ILOs and follow the legal requirements? (ownership, intellectual property, copyrights)
8. Do online instructional methods support the development of higher thinking skills like analysis, application, synthesis, integration, problem solving, critical thinking, clinical reasoning skills etc.?
9. Is there a faculty guide that shows how tools are applied with a specific targeted objective?
  1–9: document review
Evaluation questions answered during audit visits:
10. Are digital tools used to support students’ involvement and overall learning experience?
11. Do Instructional methods encourage student engagement?
12. Are online instructional methods variable to suit all learning styles?
  10: focus group
11: interviews / observations
12: observations
Course Delivery
Evaluation questions answered in advance:
1. Are the designed learning activities aligned with course learning outcomes as per the specifications?
2. Was the evaluation plan developed by a team of experts?
3. Was the evaluation plan revised and approved by relevant councils?
4. Are data collection methods identified in the evaluation plan? Are they concise, realistic, and aligned with the evaluation questions?
5. Is there a plan for proper evaluation?
6. Is the deviation from planned schedules kept as limited as possible?
7. Is the synchronous meeting application defined and accessible for students?
8. Is it user friendly?
9. Does the application facilitate learner engagement?
10. Is there an illustrative guide for synchronous meetings published and used by faculty and students?
  1–4, 6–9: document review / document analysis
5 and 10: expert review
Evaluation questions answered during audit visits:
11. Are learning activities purposeful and aligned with objectives?
12. Is a technology requirement document disseminated and used by faculty?
13. Are the minimum technology requirements clearly stated and instructions for use provided in a document?
14. Are the schedules present and announced?
15. Is there a complaint box/email?
  11–14: focus group / interviews
15: observations
Student Assessment
Evaluation questions answered in advance:
1. Are the used online student assessment methods clear, fair and acceptable for all students?
2. Is formative assessment used at proper intervals (at least one per module)?
3. Do students receive structured feedback after each formative assessment?
4. Is summative assessment planned with clear and transparent reporting?
5. What is the ratio between the continuous and summative assessment with consideration to use different online assessment methods?
6. Does the assessment measure the stated learning objectives/ competences?
  1–6: document analysis / interviews
Evaluation questions answered during audit visits:
7. Which digital tools are used?
8. Why were they chosen?
9. Are multiple assessment methods used to measure cognition, skills, and attitude of the students?
10. Are the assessment schedules clear, applicable and fair for all students?
11. Are the assessment schedules announced on the website for all students?
  7, 10, 11: observations
8 and 9: interviews
Evaluation
Evaluation questions answered in advance
1. Is there a documented continuous monitoring and evaluation for the online learning materials/ process by internal reviewers to collect and analyze data for continuous improvement? (about LMS, Faculty performance and satisfaction, and Students’ Engagement, Satisfaction, and Achievement)
2. Is there a documented periodic evaluation by external reviewers to validate the internal evaluation process and assess the goal achievement?
3. Are meetings held with stakeholders regularly?
4. Are evaluation results utilized for further corrective actions and planning?
  1–4: document analysis