Study design
The presented randomized prospective study investigated the influence of repetitive expert feedback in skills training on the learning curve of students in the early acquisition of procedural skills. Nasogastric tube placement was selected as the clinical task as this skill represents a pivotal routine procedure in internal medicine. If it is not performed accurately, severe complications may occur, resulting in considerable costs [22,23], and it is therefore an integral part of undergraduate skills training curricula [24,25]. The study was conducted over a period of two and a half weeks alongside the regular curriculum at our faculty.
Participants
Trainees were recruited via advertisements among medical students in their first or second year of medical training. A total of N = 50 participants volunteered to participate. Only right-handed individuals were eligible for inclusion in order to standardize the camera position and facilitate raters’ assessment by not needing to change perspectives. Written consent was provided by all participants and data from all participants were treated anonymously. The trainees were informed that the purpose of the study was to assess their skill performance, but no further details were provided. All participants received a minor financial compensation for their study participation. Ethical principles according to the World Medical Association Declaration of Helsinki Ethical Principles for Medical Research Involving Human Subjects of 2008 were adhered to. Ethics approval was granted by the ethic committee of the University of Heidelberg (Nr. S-211/2009). Students with previous experience in inserting nasogastric tubes were excluded from the study. Refusal to participate had no impact on the subsequent evaluations or other assessments in the curriculum.
Assessment prior to the training (T0)
To control for potential confounding variables, each study participant provided data on their age, sex, handedness, previous clinical experience regarding clerkships, qualification as a paramedic or nurse, as well as a general self-efficacy rating using the General Self-Efficacy Scale and individual learning styles using the Kolb Learning Style Inventory (KLSI) in order to prove that conditions in the two groups were equal.
General self-efficacy scale
This questionnaire consists of ten positively worded items rated on a four-point Likert scale (4 = “I agree” to 1 = “I disagree”). It assesses perceived self-efficacy in the event of adversity and stressful life events [26].
Kolb Learning Style Inventory (KLSI)
The Kolb Learning Style Inventory (KLSI) of 2005 [27] showed an even distribution of learning styles as a potential confounder in skill acquisition although it’s immediate effect is unclear [28]. On the other hand, well-organized and strategic learning styles assessed with other inventories (which may be compared to the steps reflective observation and abstract conceptualization of the KLSI) have been shown to provide a benefit for students' later performance [29,30]. Additionally, learning settings should accommodate individual learning styles to maximize individual learning achievement [31] as learning styles not only differ but may shift across cognitive and motor settings [32].
Assignment to study groups
N = 50 participants were randomly assigned to one of the two study groups, one receiving high-frequency feedback (high-frequency feedback group, HFF group; N = 25) after each of the five repetitive practice trials before the final, sixth repetition, and one receiving low-frequency feedback only twice, i.e. after the first independent skill performance and just before the final, sixth repetition (low-frequency feedback group, LFF group; N = 25; for details see “skill training session” below; see Figure 1). After participants had been included in the study, three students opted not to participate without stating reasons or due to illness, resulting in a dropout of N = 2 in the HFF group and N = 1 in the LFF group. Thus, the final sample consisted of N = 23 in the HFF group and N = 24 in the LFF group.
Skill training session
The skill training was conducted with a student-teacher ratio of 1:1 in analogy to previous studies [24,33] and under consideration of previously published checklists for nasogastric tube placement [24,34]. Training sessions were structured as follows: Both groups received a short case history and role-play directives (i.e. including talking to the mannequin as if it were a real patient), both of which have been previously shown to enhance perceived realism in the training as well as the patient-physician communication [24,33]. Both groups were then instructed to insert a nasogastric tube in a mannequin using the four steps of Peyton’s Four-Step Approach [16,35] and subsequently performed six further repetitions of inserting a nasogastric tube. Both groups received structured feedback from the skills lab trainers [9,36] after the first independent performance of nasogastric tube placement (step 4 of Peyton’s four-step approach; T1). However, the HFF group received performance-related feedback after each of their following five repetitions, whereas the LFF group received further feedback once, after the fifth repetition only. Finally, both groups performed a final, sixth repetition of nasogastric tube placement (T2; see Figure 1). The feedback was always given immediately after the respective repetition of the skill, as proximate feedback enhances its effectiveness [37]. Feedback was positively worded and aimed at inducing an external focus, i.e. aiming at the movement effect to facilitate automaticity in motor control and promote movement efficiency (for detailed reviews, see [8,38]).
Skills lab teachers
Both the HFF and LFF group were both trained by four certified skills lab peer teachers, all of whom had at least one year of experience as skills lab trainers. Two tutors were male (both aged 22 years), and two were female (aged 21 and 22 years). The four tutors were randomly assigned to the students in the study groups. As previously shown, trained medical students as peer teachers deliver training and feedback on a par with that of faculty staff in skills training [39-41]. All trainers were blinded to the study design and received an introductory course including training in nasogastric tube placement and delivering feedback prior to the study.
Assessment of training
Assessment of the skills lab training encompassed a) acceptance ratings including value of tutor’s feedback, b) subjective skill-related demands during skill performance (cognitive workload), c) skill-specific self-efficacy ratings related to nasogastric tube placement competencies, and d) objective video expert ratings of participants’ performances by blinded independent assessors (N = 2).
Acceptance ratings and value of trainer’s feedback
For the evaluation of acceptance of the training session and the tutor’s feedback, participants completed a questionnaire with five positively worded items rated on a six-point Likert scale (6 = fully agree; 1 = completely disagree) after completing the training (after T2, see Figure 1). For the specific pre- and post-evaluation of the value of the trainer’s feedback, the participants completed an additional questionnaire with ratings on a six-point Likert scale (6 = fully agree; 1 = completely disagree) after step 4 of Peyton’s four-step approach but before beginning the repetitions (T1, see Figure 1; 5 items) and after the final, sixth repetition (T2; 12 items).
Cognitive workload/skill-related demands
We assessed the perceived overall cognitive workload using the National Aeronautics and Space Administration Task Load Index (NASA-TLX) [42] as a score of six subscales: mental, physical, and temporal demands, as well as own performance, effort and frustration. Assessment took place after step 4 of Peyton’s four-step approach (T1) and after the final, sixth repetition (T2), with ratings on Likert scales from 5 (very low demands) to 100 (very high demands), resulting in a sum score between 0 (very low demands) and 100 (very high demands).
Skill-specific self-efficacy ratings
Skill-specific self-efficacy ratings related to nasogastric tube placement competencies were assessed as in previous studies [33], with five items referring to a) knowledge of the anatomical structures required to insert a nasogastric tube, b) knowledge of the materials required to insert a nasogastric tube, c) knowledge of the steps involved in inserting a nasogastric tube, d) competence in inserting a nasogastric tube in a mannequin, and e) competence in inserting a nasogastric tube in a patient (6 = fully agree; 1 = completely disagree). Skill-specific self-efficacy ratings were assessed prior to the training (T0) as well as after step 4 of Peyton’s four-step approach (T1) and after the final, sixth repetition (T2).
Independent video assessment of performance
Participants’ performance in step 4 of Peyton’s four-step approach (T1) and in the final, sixth repetition (T2) were videotaped in both the HFF group and the LFF group using high-resolution digital cameras with optical zoom to capture all of the details required for a precise evaluation. The videos were digitally processed and were independently rated by two clinically experienced and trained video assessors who were blinded to both the aim and design of the study as well as the assignment of participants to the study groups. Raters were provided with binary checklists and global rating forms. The binary checklist consisted of 16 items reflecting the procedural steps of inserting a nasogastric tube [24,34,43]. Regarding binary checklists, video raters were asked to indicate whether single procedural steps were performed correctly or incorrectly. A global rating form, which was based on the Integrated Procedural Performance Instrument (IPPI) proposed by Kneebone et al. for the assessment of procedural skills in a clinical context, was also used [44]. The IPPI was designed to evaluate global professional and confident performance of clinical technical skills. Items of the IPPI considered relevant for the procedure were selected (items 4, 5, 9, 10, 11; six-point Likert scale; 6 = very good to 1 = unsatisfactory).
Statistical analysis
Primary endpoint was the global procedural performance. Secondary endpoints were task-specific clinical skill performance, skill-specific self-efficacy ratings, and pre- and post-assessment of trainer’s feedback (T1 and T2). Data are presented as means and standard deviation. Continuous data serving sample description were compared using a Student’s t-test (assuming equal variances), whereas ordinal data were assessed using Mann-Whitney U test (M-W-U-Test)). Differences in group characteristics pertaining to sex, previous education in a medical profession, and completed medical electives were compared using chi-square tests. For repeated measures, ordinal data (acceptance ratings, cognitive workload assessed with NASA-LTX, skills-specific self-efficacy ratings, and global skills performance assessed with the IPPI) were calculated using Wilcoxon signed-rank test or Friedman test where appropriate. Group comparisons at T1 and at T2 were calculated using M-W-U-Tests. For interval data (task-specific clinical skills performance reflected in binary checklist ratings), an ANOVA with the between-subject factor ‘Group’ (HFF vs. LFF) and the within-subject factor ‘Time’ (T1 vs. T2) was conducted. LSD-post-hoc tests were used where appropriate. A p-value < .05 was considered statistically significant. Inter-rater reliability for the two video assessors was calculated using a Pearson’s-Correlation. The software package STATISTICA 8, 2007 (Statsoft, Inc., Tulsa, OK) was used for statistical analysis.