The timing of testing influences skill retention after basic life support training: a prospective quasi-experimental study
BMC Medical Education volume 19, Article number: 452 (2019)
Proper basic life support (BLS) is key in improving the survival of out-of-hospital cardiac arrest. BLS skills deteriorate in three to 6 months after training. One method to improve skill retention may be using the “testing effect” to test skills at the end of a BLS course. The aim of our study was to investigate whether either testing or the timing of such testing after BLS training have any influence on skill retention.
This was a post-test only, partial coverage, prospective quasi-experimental study designed to evaluate a BLS training course among 464 fifth year medical students at Semmelweis University in the first semester of 2013/2014. Groups were systematically but non-randomly assigned to either a control group that took no exam or one of two experimental groups that took an exam (N = 179, NoExam group; N = 165, EndExam group – exam at the end of the BLS training; N = 120, 3mExam group – exam 3 months after the BLS training). The ability to perform ten prescribed essential BLS steps was evaluated during a skill retention assessment 2 months after the course in the NoExam, 2 months after the course (and the exam) in the EndExam and 5 months after the course (2 months after the exam) in the 3mExam group to measure skill retention and the effect of our intervention. Scores were calculated for each BLS step, and also summed up as a total score. We used Kruskal-Wallis test to assess differences in skill retention.
Overall, NoExam and EndExam groups showed similar skill retention. The mean total score (and many of the sub-scores) of students was significantly higher in the 3mExam group compared to both the NoExam and the EndExam groups, and there was no difference in the total score (and many of the sub-scores) of the latter two groups. The 3mExam group had less variability in total scores (and many of the sub-scores) than the other two groups.
Our study provides evidence that testing these skills 3 months after BLS training may be more effective than either testing immediately at the end of the course or no testing at all.
Sudden cardiac arrest is still one of the leading causes of death in Europe and the United States . Performing immediate and proper cardiopulmonary resuscitation (CPR) may increase survival . However, teaching simple and complex technical skills, which need not only theoretical knowledge but also a development in psychomotor abilities, is a challenge in several fields of medical education. One of the most investigated specialties in skill teaching is emergency medicine, because providers’ performance has influence on the patients’ outcome in this field .
The ability to perform effective basic life support (BLS) is crucial for every healthcare worker and a valuable skill for laypeople as well. After training, BLS skills deteriorate in three to 6 months if not used [4,5,6]. Numerous studies have investigated how to improve BLS skill retention and sought to identify the best educational method to achieve long term BLS skill retention. Refresher courses [7, 8], special feedback devices , the use of a virtual patient , and beginning BLS education during childhood  have all been shown to improve BLS skill retention. Simulation based learning has also been shown to be effective, but it is expensive, time consuming and needs a minimum number of human resources to secure effectiveness [12, 13].
Testing skills after BLS training might be a simple, cheap, and time-efficient method of prolonging skill retention. Testing CPR skills at the end of a CPR course increases learning outcome . Medical students who took an exam at the end of BLS training had better skill retention assessed 2 weeks after the course compared to using the same time period for practicing CPR . Skill retention after such testing may last at least for 6 months . A randomised non-inferiority trial also showed the potential of repetitive sessions of formative self-testing to refresh CPR skills . The terminology “testing effect”, which refers to the fact that repeated retrieval of memories during testing better enhances knowledge retention than repeated studying , might be the basis for this finding. Previous studies have shown that the testing effect occurs even if there is no feedback after the exam .
However, only a few studies have investigated the testing effect with respect to BLS skill retention [14,15,16,17]. To our knowledge, no studies have investigated whether the timing of testing influences learning outcome. Therefore, the aim of our study was to investigate the influence of testing and the timing of the examination on BLS skill retention.
Study design and participants
This was a post-test only, partial coverage, prospective quasi-experimental study designed to evaluate a BLS training course among 464 fifth year medical students at Semmelweis University in the first semester of academic year 2013/2014. Students were participants of a compulsory Emergency Medicine course, which contained thirteen 70-min long lecture sessions (a lecture session covered several topics) and five 90-min long practice sessions during the study period (the BLS training formed a part of this course). They supposedly had had no organized BLS training until this course, with the exception of some basic first aid training that was part of their curriculum in the first year of their medical studies.
The Semmelweis University Regional and Institutional Committee of Science and Research Ethics approved our study. The informed consent was waived due to the nature of the study based on our national regulations. Participation in the study and study results did not affect students’ grades and the results were processed anonymously.
Since the Study and Examination Policy of our university did not allow us to perform a randomized controlled study among students for this registered course, our study employed a post-test only, partial coverage, prospective quasi-experimental design. Groups were systematically but non-randomly assigned to either a control group or one of two experimental groups based on consecutive sampling according to their date of participation in the BLS practice session: the first 12 groups (N = 179 students) were assigned into the NoExam control group, the second 12 groups (N = 165 students) were assigned into the EndExam experimental group and the third 9 groups (N = 120 participants) were assigned into the 3mExam experimental group (Fig. 1). Participation in the groups was blinded in a way that all participants at the beginning of the course thought they were not going to have an exam, and those who were taking exams were told during their training session.
As part of this study, participants received a 45-min BLS lecture and a 90-min BLS training session according to the European Resuscitation Council (ERC) Guidelines 2010 . Working in pairs, the students had to solve simple BLS scenarios after a four-step presentation; one student performed BLS, and the other helped him/her. Everyone performed at least one complete BLS algorithm. Each student received the same education and was presented with the same scenario.
Peyton’s Four-step Approach  was used as the teaching method during the training sessions, which contains four steps:
Step 1: Real-time demonstration on the manikin - the instructor shows the process of BLS without any comments.
Step 2: Repeated demonstration on the manikin with explanation – the instructor displays the process again slowly and explains it in detail.
Step 3: Demonstration lead by a student on the manikin – the instructor shows the process of BLS again based on the instructions of a selected student.
Step 4: Demonstration performed by a student on the manikin – the student who gave the instructions in the previous step performs BLS under the inspection of the instructor.
The teacher to student ratio during the training sessions was 1:7. Ambu Man C Torso® (Ambu A/S, Copenhagen, Denmark) manikins were used during the BLS simulation training and exams. Students were observed and corrected by continuous assessment and finished the training only when they have performed a satisfactory BLS technique.
Intervention – practical exam (“testing effect”)
As it can further be seen in Fig. 1, the NoExam group had no practical exam after the training. Students in the EndExam group took a practical exam immediately following their BLS course, and students in the 3mExam group took the practical exam 3 months after the BLS course. The three-month period for the exam for the third group was chosen because it mirrors the official end-semester examination period. As a note, students in the 3mExam group had no organized opportunity to practice BLS between their course and their exam and were specifically asked not to train during these 3 months.
Evaluation – skill retention assessment (SRA)
The evaluation consisted of a skill retention assessment (SRA), which was identical in nature with the exam, meaning that participants knew that they were being assessed and they were assessed using identical criteria, but during the exam they were told that their assessments were scored, while during the SRA they were not told that their assessments were scored.
As part of the evaluation, students had to enact and resolve a BLS scenario, supervised by independent ERC instructors who had not been involved in the training. The following ten BLS steps were tested: 1. shouting for help, 2. examining consciousness, 3. testing vital signs, 4. call for advanced life support (ALS) team, 5. position of hands on the chest, 6. depth of chest compressions, 7. rate of chest compressions, 8. consistency of chest compressions, 9. maintaining a 30:2 compression to ventilation ratio, and 10. duty cycle.
A checklist indicating a step correct vs. incorrect was used for evaluation. The chest compression depth was measured by the built-in sensor in the Ambu Man Torso®, and chest compression frequency was determined using a stopwatch. A BLS step was considered correct if it met the ERC guidelines  described above, and if it was performed correctly at least 75% of the time during the assessment. We recorded the correctly and incorrectly performed events. The students received a score of 1 for a properly implemented step and a score of 0 for an incorrect performance, which was recorded in the evaluation sheet.
As Fig. 1 shows, the SRA took place 2 months after the BLS training for the NoExam and EndExam groups and 5 months after the BLS training (i.e., 2 months after the practical exam) for the 3mExam group. The two-month period for the follow-up assessment (SRA) was chosen because it was the longest time period in which all three groups were able to complete the evaluation of the course within the academic year.
A summary score was calculated by adding up the individual BLS scores in the SRA. The distribution of the average scores for each BLS step and for the total score was compared across the three groups using the Kruskal-Wallis test overall and the Dunn post-hoc test across groups. The level of significance was set at p < 0.05. Statistical analysis was performed using SPSS v13.0 (SPSS Inc., Chicago, IL). Figures were created using GraphPad Prism version 8.1.1. (GraphPad Software, La Jolla, CA).
The gender distribution was not significantly different (p = 0.228) between the groups (NoExam: 61%, EndExam: 52%, 3mExam: 58% females – data not shown in table or figure). As can be seen in Table 1, the SRA 2 months after the training in the NoExam, 2 months after the training and the exam in the EndExam, and 2 months after the exam in the 3mExam group showed significantly different scores across the groups regarding shouting for help, testing vital signs, position of hands, rate of chest compression, consistency of chest compression, 30:2 ratio, duty cycle, and total score. There was no significant difference in examining consciousness, calling for ALS team, and depth of chest compression. Furthermore, as can be seen in Fig. 2a and b showing the post-hoc differences across groups, students in the 3mExam group performed significantly better than students in either the NoExam or the EndExam groups in shouting for help, testing vital signs, positioning of hands, and consistency of chest compression. In addition, the 3mExam group had a significantly better performance in rate of chest compression compared to the EndExam group (which had significantly lower scores for this step than the NoExam group), as well as in keeping the 30:2 ratio. Duty cycle was retained significantly better in the 3mExam group compared to the NoExam group. The NoExam group had a higher mean score in rate and consistency of chest compressions than the EndExam group, however the EndExam group’s skill retention was significantly better in duty cycle compared to the NoExam group. Overall, the NoExam and the EndExam groups showed similar skill retention. The mean total score of students was significantly higher in the 3mExam group compared to both the NoExam and the EndExam groups, and there was no difference in the total score of the latter two groups (Fig. 3). Moreover, the 3mExam group had less variability in total scores (and many of the sub-scores) than the other two groups, and the minimum total score for the 3mExam group was only 1 point lower than the average of the other two groups.
Improving skill retention in BLS education is an important issue because it may lead to a higher success rate of resuscitation and improve outcome [3, 22]. Quality of chest compressions is one of the most important factors that determines the outcome of a cardiac arrest patient . It has been shown that formal certified courses and their periodic renewal improve the outcome of resuscitation . However, a significant degree of skill decay can occur within three to 6 months after training [4,5,6]. Several methods have been investigated as a tool to prolong skill retention [7,8,9,10,11]. Our goal was to find an effective skill retention method that is simple and time- and cost-effective. We used simulation training during our courses and exams, because it is of great benefit for the students and a proper teaching tool of BLS skills .
Our results demonstrate that testing and the timing of testing after BLS training do influence BLS skill retention among senior medical students. We found that students who took an exam 3 months after their BLS training had significantly better overall skill retention assessed during the SRA 2 months after the exam than students who either took no exam or had the exam immediately after the training.
One may wonder why we chose 3 months after training as the time point for giving the BLS skills exam to students in the third group. The timing means that for students in the 3mExam group, 5 months elapsed between the end of their BLS training and the skill retention assessment, compared with 2 months for the other two groups. We chose this time point because it mirrors the official end-semester examination period. In addition, for each group, we assessed BLS skill retention 2 months after the last educational intervention, as the exam was considered as an intervention due to the “testing effect”.
During the exams and the SRA, we evaluated ten essential BLS components, which may contribute to the detection of cardiac arrest and a successful resuscitation and therefore influence patient outcome. These steps are important in recognizing cardiac arrest (examining consciousness, testing vital parameters), calling for help, and performing correct chest compressions (position of hands, depth, frequency and consistency of chest compressions, maintaining 30:2 compression to ventilation ratio, and duty cycle). It is well known that early recognition and immediate CPR may double or triple the likelihood of survival of in- and out-of-hospital cardiac arrest with ventricular fibrillation [25, 26]. There is also evidence that the proper rate and depth of compression increase the rate of return of spontaneous circulation [27, 28].
It has already been shown that testing skills after BLS training improves skill retention more than spending the same duration with additional training at the end of a course . This finding may be the result of the “testing effect”, a phenomenon based on the fact that retrieval of memories during a test is more effective in creating long-term memory than additional study and training time [15, 16, 29]. From a psychological point of view, the stress response might play an important role in improving skill retention, and remembrance during testing acts as a stress factor. One of the neuronal changes that occurs in response to stress plays an important role in creating memories . It has also been shown that prior knowledge of testing improves sensorimotor learning . Students in the EndExam group became aware of testing only at the beginning of the training, which might have influenced their performance negatively in some sensorimotor skills. Students in the 3mExam group had more time to prepare psychologically for their exam and therefore had a longer exposure to the stress effect. However, they did not get organized re-trainings to practice BLS skills before their exam. We also need to consider that testing 3 months after training may have a more complex educational impact and it should not be taken as a single testing step.
Surprisingly we found no significant difference between NoExam and EndExam groups, which contradicts formerly published data [14,15,16]. As we mentioned previously, it may mirror the complexity of stress response. These results highlight the fact that further investigations are needed to understand the effectiveness of testing and timing of testing after BLS training.
Some limitations of our study need to be considered. Although our instructors received the same training in teaching BLS and performed the same quality teaching based on the standard ERC instructors’ guidelines, it would have been preferable if the same instructor had taught all of the students. The exams were also administered by multiple instructors. Although we tried to evaluate the students’ performances using standard guidelines, we cannot rule out teacher-related differences . We also need to highlight the lack of information about the preparation of the 3mExam group for their exam. They did not have an organized opportunity to practice after the course, but we cannot rule out that some might have practiced their skills in some other training format.
Properly taught BLS skills deteriorate in three to 6 months. One of the methods to improve skill retention is to test these skills after the training, though the proper timing of testing is unknown. We investigated in our prospective quasi-experimental study the effect of testing and the timing of the examination on BLS skill retention after a BLS course. We found that the timing of testing influences skill retention in fifth year medical students. Our study provides evidence that testing these skills 3 months after training may be more effective than either testing immediately at the end of the course or not testing at all.
Availability of data and materials
All data generated and analysed during this study are available from the corresponding author on reasonable request.
Advanced life support
Basic life support
European Resuscitation Council
Skill retention assessment
Berdowski J, Berg RA, Tijssen JG, Koster RW. Global incidences of out-of-hospital cardiac arrest and survival rates: systematic review of 67 prospective studies. Resuscitation. 2010;81:1479–87.
Nehme Z, Andrew E, Bernard S, Smith K. Comparison of out-of-hospital cardiac arrest occurring before and after paramedic arrival: epidemiology, survival to hospital discharge and 12-month functional recovery. Resuscitation. 2015;89:50–7.
Dane FC, Russell-Lindgren KS, Parish DC, Durham MD, Brown TD. Association between ACLS training and survival to discharge. Resuscitation. 2000;47:83–7.
Roppolo LP, Pepe PE, Campbell L, Ohman K, Kulkarni H, Miller R, et al. Prospective, randomized trial of the effectiveness and retention of 30-min layperson training for cardiopulmonary resuscitation and automated external defibrillators: the American airline study. Resuscitation. 2007;74:276–85.
Anderson GS, Gaetz M, Masse J. First aid skill retention of first responders within workplace. Scand J Trauma Resusc Emerg Med. 2011;19:11. https://doi.org/10.1186/1757-7241-19-11.
Riegel B, Nafziger SD, McBurnie MA, Powell J, Ledingham R, Sehra R, et al. How well are cardiopulmonary resuscitation and automated external defibrillator skills retained over time? Results from the public access defibrillation (PAD) trial. Acad Emerg Med. 2006;13:254–63.
Sutton RM, Niles D, Meaney PA, Aplenc R, French B, Abella BS, et al. Low-dose, high-frequency CPR training improves skill retention of in-hospital pediatric providers. Pediatrics. 2011;128:e145–51. https://doi.org/10.1542/peds.2010-2105.
Nori MJ, Saghafinia M, Motamedi MHK, Hosseini SMK. CPR training for nurses: how often is it necessary? Iran Red Crescent Med J. 2012;14(2):104–7.
Kim SC, Hwang SO, Cha KC, Lee KH, Kim H, Kim YK, et al. A simple audio-visual prompt device can improve CPR performance. J Emerg Med. 2013;44:128–34. https://doi.org/10.1016/j.jemermed.2011.09.033.
Kononowicz AA, Krawczyk P, Cebula G, Dembkowska M, Drab E, Fraczek B, et al. Effects of introducing a voluntary virtual patient module to a basic life support with an automated external defibrillator course: a randomised trial. BMC Med Educ. 2012;12:41. https://doi.org/10.1186/1472-6920-12-41.
Mpotos N, Iserbyt P. Children saving lives: training towards CPR excellence levels in chest compression based on age and physical characteristics. Resuscitation. 2017;121:135–40. https://doi.org/10.1016/j.resuscitation.2017.10.024.
Sorensen LJ, Ostergaard D, LeBlanc V, Ottesen B, Konge L, Dieckmann P, et al. Design of simulation-based medical education and advantages and disadvantages of in situ simulation versus off-site simulation. BMC Med Educ. 2017;17:20. https://doi.org/10.1186/s12909-016-0838-3.
So HY, Chen PP, Wong GKC, Chan TTN. Simulation in medical education. J R Coll Physicians Edinb. 2019;49:52–7. https://doi.org/10.4497/JRCPE.2019.112.
Nord A, Hult H, Kreitz-Sandberg S, Herlitz J, Svensson L, Nilsson L. Effect of two additional interventions, added to standard cardiopulmonary resuscitation training on seventh grade students’ practical skills and willingness to act: a cluster randomized trial. BMJ Open. 2017;7(6):e014230. https://doi.org/10.1136/bmjopen-2016-014230.
Kromann CB, Jensen ML, Ringsted C. The effect of testing on skills learning. Med Edu. 2009;43:21–7. https://doi.org/10.1111/j.1365-2923.2008.03245.x.
Kromann CB, Bohnstedt C, Jensen ML, Ringsted C. The testing effect on skills learning might last 6 months. Adv in Health Sci Educ. 2010;15:395–401. https://doi.org/10.1007/s10459-009-9207-x.
Mpotos N, De Wever B, Cleymans N, Raemaekers J, Loeys T, Herregods L, et al. Repetitive sessions of formative self-testing to refresh CPR skills: a randomised non-inferiority trial. Resuscitation. 2014;85:1282–6. https://doi.org/10.1016/j.resuscitation.2014.06.011.
Roediger HL, Karpicke JD. Reflections on the resurgence of interest in the testing effect. Perspect Psychol Sci. 2018;13(2):236–41. https://doi.org/10.1177/1745691617718873.
Roediger HL, Butler AC. The critical role of retrieval practice in long-term retention. Trends Cogn Sci. 2011;15:20–7. https://doi.org/10.1016/j.tics.2010.09.003.
Deakin CD, Nolan JP, Soar J, Sunde K, Koster RW, Smith GB, et al. European resuscitation council guidelines for resuscitation 2010 section 4. Adult advanced life support. Resuscitation. 2010;81:1305–52. https://doi.org/10.1016/j.resuscitation.2010.08.017.
Lake FR, Hamdorf JM. Teaching on the run tips 5: teaching a skill. Med J Aust. 2004;14:327–8.
Müller MP, Richter T, Papkalla N, Poenicke C, Herkner C, Osmers A, et al. Effects of a mandatory basic life support training programme on the no-flow fraction during in-hospital cardiac resuscitation: an observational study. Resuscitation. 2014;85:874–8. https://doi.org/10.1016/j.resuscitation.2014.03.046.
Sodhi K, Singla MK, Shrivastava A. Impact of advanced cardiac life support training program on the outcome of cardiopulmonary resuscitation in a tertiary care hospital. Indian J Crit Care Med. 2011;15:209–12. https://doi.org/10.4103/0972-5229.92070.
Nuzhat A, Salem RO, Al Shehri FN, Al HN. Role and challenges of simulation in undergraduate curriculum. Med Teach. 2014;36(Suppl 1):S69–73. https://doi.org/10.3109/0142159X.2014.886017.
Hasselquist-Ax I, Riva G, Herlitz J, Rosenqvist M, Hollenberg J, Nordberg P, et al. Early cardiopulmonary resuscitation in out-of-hospital cardiac arrest. N Engl J Med. 2015;372(24):2307–15. https://doi.org/10.1056/NEJMoa1405796.
Holmberg M, Holmberg S, Herlitz J. Factors modifying the effect of bystander cardiopulmonary resuscitation on survival in out-of-hospital cardiac arrest patients in Sweden. Eur Heart J. 2001;22:511–9.
Edelson DP, Abella BS, Kramer-Johansen J, Wik L, Myklebust H, Barry AM, et al. Effects of compression depth and pre-shock pauses predict defibrillation failure during cardiac arrest. Resuscitation. 2006;71:137–45.
Christenson J, Andrusiek D, Everson-Stewart S, Kudenchuk P, Hostler D, Powell J, et al. Chest compression fraction determines survival in patients with out-of-hospital ventricular fibrillation. Circulation. 2009;120:1241–7. https://doi.org/10.1161/CIRCULATIONAHA.109.852202.
Karpicke JD, Roediger HL. The critical importance of retrieval for learning. Science. 2008;319:966–8. https://doi.org/10.1126/science.1152408.
Joels M, Fernandez G, Roozendaal B. Stress and emotional memory: a matter of timing. Trends Cogn Sci. 2011;15:280–8. https://doi.org/10.1016/j.tics.2011.04.004.
Bastos FH, Marinovic W, de Rugy A, Tani G. Prior knowledge of final testing improves sensorimotor learning through self-scheduled practice. Hum Mov Sci. 2013;32:192–202. https://doi.org/10.1016/j.humov.2012.11.008.
Gyarmathy VA, McNutt LA, Molnár A, Morse DL, DeHovitz J, Ujhelyi E, et al. Evaluation of a comprehensive AIDS education curriculum in Hungary – the role of good educators. J Adolesc. 2002;25(5):495–508. https://doi.org/10.1006/jado.2002.0496.
The authors would like to thank Eszter Dudás, Aletta Hornyák and Balázs Udvari for their valuable help in the context of this study.
No external funding was applied to this study.
Ethics approval and consent to participate
The Semmelweis University Regional and Institutional Committee of Science and Research Ethics approved our study. The informed consent was waived by the committee due to the nature of the study based on our national regulations. Participation in the study and study results did not affect students’ grades and the results were processed anonymously.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Kovács, E., Jenei, Z., Csordás, K. et al. The timing of testing influences skill retention after basic life support training: a prospective quasi-experimental study. BMC Med Educ 19, 452 (2019). https://doi.org/10.1186/s12909-019-1881-7