Skip to main content

Table 2 Characteristics of the included studies: randomized controlled trials (RCTs) and cohort studies

From: Digital learning designs in physiotherapy education: a systematic review and meta-analysis

Author, year, country, study design

Population

Digital learning design, intervention, comparison

Outcome

Arroyo-Morales et al., 2012, [30] Spain, RCT

Under-graduate

Physiotherapy students second year

n = 46

Digital learning design: Blended learning

Context: Theoretical acquisition and skills training on campus

Subject/skills: Palpation and ultrasound examination of the knee joint

Duration: In-class: two 2-h sessions, traditional lectures; self-studies: 20 h

Intervention: In-class: traditional lectures; post-class: free access to the Ecofisio interactive website/app

Comparison: In-class: two 2-h sessions, traditional lectures, access to documents and books on the topic

Both groups: 3-week self-study period

MCQ: 20 MCQs (max 10 points); and assessed knowledge of ultrasound physics (5 questions), ultrasound technology (5questions), clinical applications (5 questions), and anatomy (5questions)

OSCE: Skills in palpation and ultrasound imaging of the knee; grading system: 3 = excellent, 0 = incorrect (max 15 points each)

Also measured the time taken by the student to generate a reliable ultrasound image and to localize a specific knee structure by palpation

Students’ evaluation: Quality of the educational intervention: competence of the teacher, students’ acquisition of knowledge/skills, students’ interest in participating in the study for another anatomic region and—for the experimental group—satisfaction with the Ecofisio website; also asked whether they would have preferred to be in another study group; 5-point Likert scale (5 = strongly agree, 1 = disagree)

Bartlett and Smith, 2020, [39] USA, RCT

Under-graduate

Physiotherapy students

first year

n = 20

Digital learning design: Blended learning

Context: Skills training on campus

Subject/skills: Cardiovascular and pulmonary physical therapy

Duration: 45-min laboratory session

Intervention: Mobile app only group and demonstration plus mobile app group. Mobile app only group: 5-min. Tutorial on how to navigate, no professorled demonstration of instruction of the clinical skills, then practiced the skills in a lab. Sessions. Demonstration plus mobile app group given the same demonstration and verbal information as the control group, take notes and ask questions. 5-min. Tutorial on how to navigate through the iPad and then participated in a lab. Sessions

Comparison: Demonstration-only group: demonstration and practice of the skills in a laboratory session

Practical exam: Students tested on their psychomotor skills related to their ability to perform and interpret clinical skills; assessed using a mock patient not related to the study; 1 = satisfactory or 0 = unsatisfactory (max score 18); 3 examiners

Blackstock et al., 2013, [25] Australia, 2 RCTs

Under-graduate

Physiotherapy

students first year

n = 349

Digital learning design: Blended learning

Context: Simulation training on campus and clinical placement Subject/skills: Cardiorespiratory

Duration: 4 weeks

Intervention 1: Simulated learning environment videos; 1 week in the simulated learning environment, then 3 weeks in clinical immersion

Intervention 2: 50% of day in the simulated learning environment and 50% in clinical immersion during the first 2 weeks (equal to 1 full-time simulation week), then 2 weeks in clinical immersion

Comparison: 4 weeks in clinical immersion

Practical exam: Assessment of competency to practice in the cardio-respiratory field, measured using two clinical examinations based on the Assessment of Physiotherapy Practice; 7 key standards; score range: 0 = infrequently/rarely demonstrates performance indicators, 4 = demonstrates most performance indicators to an excellent standard, N/A = not applicable and not assessed

Students’ evaluation: Scales for analysis of student’s self-rating of confidence with patients in communication, assessment and management; 13 Likert items; checked for reliability (Cronbach’s α)

Cantarero-Villanueva et al., 2012, [31] Spain, Single-blinded RCT

Under-graduate

Physical therapy students,

n = 44

Digital learning design: Blended learning

Context: Theoretical acquisition and practical training on campus

Subject/skills: Musculoskeletal palpation and ultrasound assessment of the lumbopelvic area

Duration: 1 semester

Intervention: 6 classroom hours (traditional lectures and practical training) and 20 self-study hours plus free access to an interactive website/app (Ecofisio) on musculoskeletal palpation and ultrasound assessment

Comparisons: In-class: traditional lectures and practical training; 20 self-study hours: access to documents and books on the topic

OSCE: Ultrasound imaging, two components: musculoskeletal and skills in ultrasound imaging; grading system: 3 = excellent, 0 = incorrect; maximum score: 9 (musculoskeletal) and 15 (ultrasound imaging); validated

After OSCE: Students invited to establish 2 additional measurements in the same model; graded one at a time using the same human model

Students’ evaluation: Quality of the educational programme, 5-point Likert scale (5 = strongly agree, 1 = disagree); participant assessments included teacher’s competence, participants’ own acquisition of knowledge/skills, complexity of the knowledge/skills, possibility of participation using e-learning and (for the experimental group) satisfaction with the Ecofisio website

da Costa Vieira et al., 2017, [35] Brazil, Prospective crossover RCT

Under-graduate

Physiotherapy students second to fourth year,

n = 72

Digital learning design: Blended learning

Context: Theoretical acquisition on campus

Subject/skills: Physiotherapy in oncology

Duration: 2 days and 6 modules (3 modules/day)

Intervention: Group A sequence: e-learning/traditional lectures/e-learning; had the same e-learning classroom (storage material) as Group B, 5 min given to study using the computer

Group B sequence: Traditional lectures/e-learning/traditional lectures; 5-min discussion with the teacher after the content ended; studied the slides’ content without access to professors for discussion

Same content given to Groups A and B simultaneously; after each model, students had 30 min to change to the other classroom

Written exam: 7 relevant objectives, 7 questions per module; questions had few words to minimize students’ reading time and increase the test’s reliability; 3 answer choices: true, false or do not know; 126 questions; summative evaluation at end of each module using an objective assessment with 21 questions, same answer choices

Students’ evaluation: Level of satisfaction with the different teaching methodologies and course content; open-ended questions to gather information about the course, evaluation format and suggestions/criticisms

Fernandez-Lao et al., 2016, [32] Spain, Single-blinded RCT

Under-graduate

Physiotherapy students

first semester,

n = 49

Digital learning design: Blended learning

Context: Theoretical acquisition and skills training on campus

Subject/skills: Musculoskeletal assessment competencies

Duration: 6 learning lessons and 20 self-study hours

Intervention: Free access to interactive/app (Ecofisio) as supplement to traditional lectures

Comparison: In-class, traditional lectures and access to documents and books on the topic

Written exam: 20 MCQs, maximum 10 points

OSCE: Ultrasound and palpation skills assessed; grading system: 3 = excellent, 0 = incorrect; maximum scores: 15 (ultrasound) and 12 (palpation)

Students’ evaluation: Quality of the intervention; 5-point Likert scale (5 = strongly agree, 1 = strongly disagree); 11-numeric-point rating scale (10 = totally satisfied, 0 = totally unsatisfied)

Huhn et al., 2013, [40] USA, RCT

DPT programme, first year,

n = 53

Digital learning design: Blended learning

Context: Theoretical acquisition and skills training on campus

Subject/skills: Pathology II

Duration: 1 semester

Intervention: Virtual patient simulation on clinical reasoning, knowledge acquisition, transfer of knowledge and students’ perception of their learning; 6 patient cases; worked individually in campus computer laboratory with the faculty facilitator available only to answer technical questions related to the function of the virtual reality program

Comparison: In-class, large group discussions; 6 patient cases

Written exam: 50 MCQs

Health Science Reasoning Test: Clinical reasoning prior to and after completing 6 patient cases in their respective group; 30-item test designed to assess induction, deduction, analysis, evaluation and inference skills; overall score and scores for 5 sub-scales

OSCE: Measure of transfer of learning; observed and scored by a faculty member using a tool developed by the faculty; students graded on professional behaviour and communication, safety, examination, evaluation and interventions using a 5-point scale

Hyland et al., 2010, [41] USA, RCT

Entry-level

Physical therapy students,

third year,

n = 33

Digital learning design: Distance learning

Context: Theoretical acquisition on campus

Subject/skills: Administration and management

Duration: 1 semester, 9 days

Intervention: CAI: unlimited access to the course website (Campus Pipeline); received professor’s notes online in a lecture-style format, special examples included within the notes; also received the same PowerPoint presentation, study questions and lecture online as the control group; students could ask questions and share personal experiences via email or online discussion

Comparison: In-class: PowerPoint presentations; traditional lecture instruction, 4 h per meeting

Written exam: Pre- and post-test examination: 25 and 50 MCQs, respectively; score: percentage of questions answered correctly; final course evaluative criteria: final exam (25%), final project (20%), health and wellness assignment (20%), ethics paper (15%) and 2 case studies (10% each)

Lozano-Lozano et al., 2020 [34], Spain, Double-blinded RCT

Under-graduate

Physiotherapy

Students

first and second year,

n = 110

Digital learning design: Blended learning

Context: Theoretical acquisition and skills training on campus

Subject/skills: Ultrasound imaging

Duration: In-class:4 h theoretical lessons and 4 h of practical lessons; self-studies: 2 weeks

Intervention: In-class: 4 h of theoretical lessons and 4 h of practical lessons; post-class: free access to the Ecofisio interactive website/app

Comparison: In-class: Two 2-h sessions, traditional lectures; access to books and journal papers on the topic

Both groups: 2-week self-study period

OSCE: Measured participants’ hands-on ultrasound management skills

Written exam: Evaluation of students’ theoretical knowledge; 20 MCQs; max score: 10 points

Students’ evaluation: Satisfaction survey with 5-point Likert questionnaire (1 = disagree, 5 = strongly agree); Ecofisio group also completed another satisfaction questionnaire, scores ranged from 0 = totally unsatisfied to 10 = totally satisfied

Maloney et al., 2013, [28] (pilot) Australia, RCT

Under-graduate

Physiotherapy students,

third year, 2010,

n = 49

Digital learning design: Blended learning

Context: Skills training on campus

Settings: Theoretical acquisition and skills training on campus

Subject/skills: Complex clinical skills

Duration: First half of students’ third year

Intervention: 1) 30-min pre-recorded video tutorials: demonstration of the skill, text prompts, trigger and problem solving; 2) Students produced self-video of clinical performance without tutor input or guidance

Comparison: In-class: traditional teaching with live demonstration of the entire skill; pre-recorded video also shown during practical class with no replay opportunity

OSCE: Clinical performance, written patient scenario; grades out of 50 for each performance, 10 set performance criteria: completed well (full marks), partially completed (half marks) or inadequate (zero marks); grades converted to a percentage

Students’ evaluation: 10-min group-specific survey; questionnaire: perceptions of utility and satisfaction with the teaching methods, 5-point Likert scale (1 = strongly disagree, 5 = strongly agree) and open-ended questions

Maloney et al., 2013, [28] (main) Australia, RCT

Under-graduate

Physiotherapy

Students,

third year, 2009,

n = 60

Digital learning design: Blended learning

Context: Skills training on campus

Subject/skills: Clinical skills acquisition

Duration: 2 weeks

Intervention: Students created a 5-min self-produced video recording; video reviewed by remote online tutors, often with group feedback on common strengths and weaknesses observed; students reflected on their strengths and areas for improvement; students’ own video clips and the peer benchmark ‘exemplar’ video clip remained online throughout the semester

Comparison: In-class: clinical skills with regular practical tutoring

OSCE: Two clinical skill stations, formative (quantitative and qualitative) feedback to the student on their performance

Students’ evaluation: Students’ perceptions and experiences: paper-based questionnaire; 5-point Likert scale (1 = strongly disagree, 5 = strongly agree) and open-ended questions

Moore and Smith, 2012, [42] USA, RCT

DPT, Physical therapy students, first year,

n = 33

Digital learning design: Blended learning

Context: Theoretical acquisition and skills training on campus

Subject/skills: Psychomotor skills

Duration: 3 weeks

Intervention: Video podcasting (videoclips): lecture and podcast demonstrations of transfer skills; students encouraged to review assigned readings and lecture notes and to practice podcast skills; formal class meeting: 2.5 h of lecture and laboratory; students moved directly to the laboratory component of the interaction, beginning with practice and case studies, and utilized the skills depicted in the podcasts in complex patient scenarios

Comparison: In-class: live instructor demonstration of basic psychomotor skills

Written exam: Written post-test on cognitive performance

Practical exam: Psychomotor performance using a scenario-based practical post-test, graded for safety, fluency and accuracy

Students’ evaluation: Survey of the 2 learning methods and reported study time; 7 Likert statements and 5 free-response questions

Nicklen et al., 2016, [29] Australia, RCT

Under-graduate

Physiotherapy students,

third year,

n = 38

Digital learning design: Blended learning

Context: Theoretical acquisition and skills training on campus

Subject/skills: Case: Rachel’s pregnancy, the role of the physiotherapist during stages of pregnancy

Duration: 1 week

Intervention: Remote-online CBL learning using the same case; web-conferencing with participants physically isolated from one another on campus; WebEx software (written text and audio-visual)

Comparison: In-class: same case used

Both groups: Attended the first session (30 min) that introduced key features of interacting via web-conference

Written exam: Post-intervention test after second computer session: learning and self-assessed perception of learning, satisfaction and participants’ demographics; 10 MCQs

Students’ evaluation: Perception of learning measured for each examinable learning objective; 3-point scale: superficial, moderate and in-depth; satisfaction with the remote-online CBL measured on a 5-point scale (1 = strongly disagree, 5 = strongly agree)

Noguera et al., 2013, [33] Spain, Crossover RCT

Under-graduate

Physiotherapy students,

second year,

n = 70

Digital learning design: Blended learning

Context: Skills training on campus

Subject/skills: Practical manual therapy course in a laboratory

Duration: Two 5-h practical lessons

Intervention: Anatomy-learning app for mobile devices; Group 1: mobile device used during first practical session; Group 2: mobile device used during second practical session

Comparison: Description of different manipulative techniques and a practical demonstration performed by the professor; afterwards, students practiced their manipulation technique in pairs (one of them simulating a patient)

Written exam: Post-test immediately after each practical session to assess anatomical knowledge; first test: 8 MCQs; second test: 4 open questions and 4 MCQs; score: number of correct answers out of 8

Students’ evaluation: Questions 1–17: Likert scale (range 1–5), Questions 19 and 20: Likert scale (range 1–10), Questions 21 and 22: open questions

Rocha et al., 2017, [36] Brazil, RCT

Under-graduate

Physiotherapy

Students

8th semester

n = 71

Digital learning design: Blended learning

Context: Theoretical acquisition on campus

Subject/skills: Professional Practice and Ethics in Physiotherapy discipline

Duration: Once a week for 17 weeks

Intervention: Regular classes with extra time for educational video game (quiz type); game room was available until a new room was built with new questions; four formats: the more resources students earned, the more moves they could make

Comparison: Regular in-person classes

Written exam: Specific knowledge test (final exam); 80 questions: single and multiple choice, relationships between columns and true/false

Students’ evaluation: Satisfaction with the discipline, 5-point Likert scale (1 = not at all satisfied, 5 = very satisfied); perception of learning content, 5-point Likert scale (1 = learned nothing, 5 = learned a lot)

Silva et al., 2012, [37] Brazil, RCT

Under-graduate

Physiotherapy students,

fourth year,

n = 16

Digital learning design: Blended learning

Context: Theoretical acquisition on campus

Subject/skills: Respiratory therapy field

Duration: 1 semester

Intervention: Multimodal online environment including multimedia resources (videos, animations and figures) and conventional course classes attended in person; after the end of the course, 2-week access to teachers to ask questions and to the online material to study; access to online material discontinued after 2 weeks, when all students had to take a final exam

Comparison: In-class: traditional course classes on bronchial hygiene techniques; 2-week access to teachers to ask questions and to online and conventional material to study

Knowledge test: 20 questions assessing students’ knowledge of therapeutic indications (8 questions), contraindications for the use of Bronchial Hygiene Techniques (6 questions) and concepts (6 questions); each correct answer scored 0.5 points

Ulrich et al., 2019, [38] Denmark, RCT

Under-graduate

Physiotherapy

Students,

3 groups:

1: n = 28

2: n = 26

3: n = 27

Digital learning design: Blended learning

Context: Theoretical acquisition and skills training on campus

Subject/skills: Learning practical skills

Duration: 1 month

Intervention: 360° video used as e-learning; after pre-test, Group 1 received lesson using 360° video (Samsung Gear VR), Group 2 received lesson using regular video (laptop)

Comparison: Group 3 received traditional in-class lesson from an instructor

Written exam: Pre-test: MCQs on the learning requirements for the treatments

Practical exam post-test: after treatment, tested on learning, practical setting: patient (volunteer) and a teacher in physiotherapy education recorded the results; graded: pass/fail for each question or task

Students’ evaluation: Questionnaire about students’ learning satisfaction and perception of the learning climate in each treatment group (given after final test)

Covill and Cook, 2019, [43] USA, Comparative cohort study

DPT, Physiotherapy first year, 3 classes:

A: n = 47

B: n = 54

C: n = 47

Digital learning design: Flipped classroom

Context: Theoretical acquisition

Subject/skills: Musculoskeletal content, patient management of the lower quadrant

Duration: 81 lecture hours and 79 laboratory hours

Intervention: Classes B and C: flipped classroom (alternating lecture hours); pre-class: pre-recorded lectures, readings, non-graded quizzes and discussion questions; in-class: faculty-led large group question and case discussion, small group question and case discussion, polling software and quiz discussion;

Comparison: Class A: 18 h of traditional lectures and 31 h of laboratory work

Written exam: 10 tests total, delivered every 2 weeks; 83 MCQs across all 3 cohorts specific to the content delivered

Students’ evaluation: Classes B and C (flipped classroom) received a post-course survey specific to student perceptions of the flipped method; 5-point Likert scale

Day, 2018, [44] USA, Cohort

DPT, Physical therapy first two semesters,

n = 112

Digital learning design: Flipped classroom

Context: Theoretical acquisition and skills training on campus

Subject/skills: Gross anatomy course

Duration: 15 week-long courses

Intervention: Flipped classroom; pre-class: 15-min instructor-created lecture videos prior to class (less than 60 min per week); in-class: 130 min/week, included the same activities from previous year; students also participated in a prosected cadaver laboratory 90 min per week

Comparison: Traditional in-class lectures and prosected cadaver laboratory for 90 min per week ×  15 weeks

Written exam: 120 MCQs. All final examination MCQs were divided into two levels. Lower-level MCQs (LL-MCQ) were define as “remember” and “understand” and included questions that required recall of definitions and terms. Higher-level MCQs (HL-MCQ) were defined as “apply”and “analyse.” These questions required participants to use higher-order cognitive skills to apply knowledge to new situations. No items were “create” or “evaluate,” due to the nature of the MCQ examination.

In total, 13 final examination MCQs were determined to be at a higher cognitive domain; apply or analyse. The HL-MCQs included anatomical identification on MRI images and clinical scenarios that required students to analyse the facts of the case to determine the location of an injury or possible symptoms present.

During the subsequent kinesiology course, students received 3 MCQ unit examinations that remained consistent between the two groups; traditional and flipped classroom format. Each examination was not cumulative, and no final examination was given. Student’s kinesiology grades from each of the examinations and the overall semester grade was obtained from the instructor of record.

Deprey, 2018, [45] USA, Cohort

Under-graduate

Physiotherapy students, fifth year, 3 groups:

1: n = 44

2: n = 49

3: n = 50

Digital learning design: Flipped classroom

Context: Theoretical acquisition and skills training on campus

Subject/skills: Neurological disorders

Duration: 2-h time blocks, 3 days per week

Intervention 1: Fully integrated flipped; pre-class: 5 pre-recorded lectures, in-class: worked in groups to answer instructor-posed questions and complete scenarios; internet searches or open book or note reviews; focus: student questions

Intervention 2: Partially integrated flipped; pre-class: recorded lectures, in-class: reiteration of recorded lectures and discussion without special in-class work, opportunity to ask questions or clarify concepts

Both: 2-h balance test and measures lab

Comparison: Five 2-h in-class lectures, individual homework and 2-h balance test and measures lab

Written exam: Given at completion of each of the 3 units; exams 1 and 2 included the same items for all 3 years of the study; exam scores assessed for objective change in content knowledge; primary outcome: scores on the second unit exam; changes in scores from exam 1 to exam 2 were compared

Green and Whitburn, 2016, [26] Australia, Retrospective cohort

Under-graduate

Physiotherapy students,

second year,

3 groups:

1: n = 150

2: n = 160

3: n = 151

Digital learning design: Blended learning

Context: Theoretical acquisition and skills training on campus

Subject/skills: Gross anatomy

Duration: 15 week-long courses

Intervention: Group 3: fully blended; pre- and in-class: online video clips, face-to-face lectures, practical classes, clinical anatomy classes, face-to-face tutorials

Comparison: Group 1: in-class, traditional lectures; Group 2: in-class lectures and some online content (video clips)

Practical and written exam: Aggregate practical test mark (expressed as a percentage to avoid differences in weighting between cohorts) and final written examination mark (expressed as percentage) between the cohorts

Students’ evaluation: Questionnaire, 5-point Likert scale (5 = strongly agree, 1 = strongly disagree), open-ended questions

Murray et al., 2014, [46] USA, Cohort

Under-graduate

Physiotherapy students,

third semester,

2 groups:

1: n = 43

2: n = 35

Digital learning design: Flipped classroom

Context: Theoretical acquisition and skills training on campus

Subject/skills: Pathological conditions of the extremities

Duration: 1 semester

Intervention: Flipped classroom; pre-class: 10 to 25-min asynchronous online lectures in SAKAI (course management system), students encouraged to take notes and bring questions to class for discussion; face-to-face in-class meeting: 15 min to clarify any information that was unclear from online lectures, 20 to 30-min PowerPoint presentation integrating the online lecture content into examination sequence, 120 to 240-min group discussions of cases with emphasis on clinical decision making

Comparison: Traditional face-to-face lectures

Final exam: 105 MCQs; correct answers tallied in aggregate and by cohort based on 5 areas: [1] total exam score, [2] score on examination/evaluation questions, [3] score on intervention questions, [4] score on lower-level questions and [5] score on higher-level questions

  1. RCT Randomized controlled trial, DPT Doctor of Physical Therapy, MCQ Multiple choice question, OSCE Objective structured clinical evaluation, App Application, CAI Computer-assisted instruction, CBL Case-based learning
  2. DPT Doctor of Physical Therapy; MCQ Multiple choice question