This article has Open Peer Review reports available.
Lumbar puncture simulation in pediatric residency training: improving procedural competence and decreasing anxiety
© The Author(s). 2016
Received: 17 February 2016
Accepted: 29 July 2016
Published: 8 August 2016
Pediatric residents must become proficient with performing a lumbar puncture (LP) during training. Residents have traditionally acquired LP skills by observing the procedure performed by a more senior resident or staff physician and then attempting the procedure themselves. This process can result in variable procedural skill acquisition and trainee discomfort. This study assessed changes in resident procedural skill and self-reported anxiety when residents were provided with an opportunity to participate in an interactive training session and practice LPs using a simulator.
All pediatric residents at our institution were invited to participate. Residents were asked to report their post-graduate year (PGY), prior LP attempts and self-reported anxiety scores as measured by the standardized State-Trait Anxiety Inventory - State Anxiety Scale (STAI-S) prior to completing an observed pre-test using an infant-sized LP simulator. Staff physicians observed and scored each resident’s procedural skill using a previously published 21-point scoring system. Residents then participated in an interactive lecture on LP technique and were given an opportunity for staff-supervised, small group simulator-based practice within 1 month of the pre-test. Repeat post-test was performed within 4 months.
Of the pediatric residents who completed the pre-test (N = 20), 16/20 (80 %) completed both the training session and post-test. Their PGY training level was: PGY1 (38 %), PGY2 (25 %), PGY3 (25 %) or PGY4 (12 %). Procedural skill improved in 15/16 residents (paired t-test; p < 0.001), driven by a significant improvement in skill for residents in PGY1 (P = 0.015) and PGY2 (p = 0.003) but not PGY3 or PGY4. Overall anxiety scores were higher at baseline than at post testing (mean ± SD; 44.8 ± 12.1 vs 39.7 ± 9.4; NS) however only PGY1 residents experienced a significant reduction in anxiety (paired t-test, p = 0.04).
LP simulation training combined with an interactive training session may be a useful tool for improving procedural competence and decreasing anxiety levels, particularly among those at an earlier stage of residency training.
Lumbar puncture (LP) is an important diagnostic and therapeutic tool. It is performed for the purpose of obtaining a sample of cerebrospinal fluid (CSF) . In children, the most common reason for a LP is to diagnose an infection in the central nervous system (i.e. meningitis or meningoencephalitis). Lumbar puncture is also required for the diagnosis of many other non-infectious diseases in children such as inflammatory disease (e.g. transverse myelitis), hematological-oncological diseases (e.g. CNS leukemia or lymphoma) as well as subarachnoid hemorrhage or metabolic disease. In addition, LP is required to confirm CSF opening pressure to diagnose idiopathic intracranial hypertension. Less commonly, LP is required for therapeutic reasons to instill chemotherapeutic or antimicrobial agents directly into the CSF.
The Royal College of Physicians and Surgeons of Canada (RCPSC) and the Accreditation Council for Graduate Medical Education (ACGME) expect pediatric residents to become proficient at performing LP and to demonstrate effective, appropriate, safe and timely performance of this skill during their residency training [2, 3].
Residents have traditionally acquired LP skills by observing the procedure performed several times by a more senior colleague and then attempting the procedure themselves sometimes referred to as the “see one, do one, teach one” model of procedural skill teaching [3, 4]. This method of learning has been linked to variable procedural skill acquisition and resident self-reported lack of confidence and anxiety related to the supervision that they received .
Simulation-based training for procedural skills has been shown to be effective for trainees and has been widely used in many residency programs [6–9]. The aim of this study was to assess changes in pediatric resident procedural skill and self-reported anxiety using a LP simulator and an interactive training session. We hypothesized that use of a simulation-based model and an interactive training program would: 1) improve resident procedural skill as measured by a previously published 21-point scoring system  and; 2) decrease residents’ self-reported anxiety scores as measured by the standardized State-Trait Anxiety Inventory - State Anxiety Scale (STAI-S) questionnaire .
All pediatric residents (N = 54) in post-graduate year (PGY) 1 to 4 who were based at the Children’s Hospital of Eastern Ontario in Ottawa, Canada were invited to participate. Residents could be part of the University of Ottawa program (N = 41) or rotating pediatric residents from the Northern Ontario School of Medicine Program (N = 13). Residents were excluded from the study if they had started any form of fellowship training.
Residents were each assigned a unique, confidential study number and completed testing one at a time. Residents were first asked to complete a brief questionnaire stating their: 1) PGY level; 2) gender and; 3) number of prior successful LPs that they have performed. Successful LPs were defined as a procedural attempt that obtained clear cerebral spinal fluid (CSF) and/or blood-tinged CSF that cleared. An unsuccessful LP was defined as failure to obtain CSF and/or bloody CSF that did not clear. Residents were then asked to complete the STAI-S immediately prior to entering the room to perform the LP. They were asked to consider the following when completing each question: “how you feel when performing a lumbar puncture”. The completed STAI-S was placed in a sealed envelope containing only their confidential study number attached. Data were later tabulated by the study coordinator who did not know the residents and had no role in their evaluation.
Immediately after completing the STAI-S, residents entered a room used for objective structured clinical examinations (OSCEs) and were asked to: “carry out all steps you would normally perform when completing a lumbar puncture on a 10 month old infant, including a check of cerebral spinal fluid opening pressure”. In this study, we used the Pediatric Lumbar Puncture Simulator (# KKM43C; Limbs and Things, Savannah, GA) that corresponded in size and appearance to a 7–10 month old infant. Residents were told that a nurse was available to assist during the procedure. They were informed that a sterile LP kit and other equipment were available for use (i.e. sterile gloves, lidocaine, needles, cleaning solution and a garbage can). The performance of each resident was scored by direct observation by an attending pediatric neurologist or attending pediatric intensive care physician using a previously published 21-item lumbar puncture skills checklist . At the end of the LP, each resident was asked to list the tests that they would order on CSF analysis for an infant that was clinically suspected to have bacterial meningitis.
Within 1 month of completing the pre-test, the pediatric residents participated in an interactive teaching session about LPs as part of an academic half-day session. The session included a series of short videos demonstrating an LP performed by an attending physician. Residents were also provided with an opportunity to practice LP proficiency in small groups, using the pediatric lumbar puncture simulator, and to assemble the components of the LP test kit (i.e. manometer to check opening pressure, opening tubes, etc.) under the supervision of an experienced neurologist. All residents who completed the pre-test attended the teaching session.
Residents were then scheduled to complete a post-test (i.e. repeat STAI-S and observed LP) within 4 months of the teaching session.
A total of five attending physicians scored the procedural checklists. Three completed the pre-test scoring and three completed the post-test scoring. No resident had the same physician complete both his/her pre-test and post-test. All attending physicians scoring the resident LPs had more than 5 years of experience conducting and scoring resident OSCEs and all were Medical Educators and/or Program Directors at the University of Ottawa, Department of Pediatrics.
Data from the pre-tests and post-test were compared using paired t-tests with statistical significance set at p < 0.05. The IBM SPSS® Statistics 23 was used for all statistical analysis.
Twenty pediatric residents (20/54; 37 %) completed the pre-test (Sept 2014) and the interactive training session (Oct 2014). Sixteen residents (16/20; 80 %) completed the post-test (Dec 2014 to Feb 2015). Only the 16 pediatric residents who completed all three sessions, namely, the pre-test or baseline LP, teaching session and post-test or follow-up LP, were included in the analysis.
Resident baseline self-report of their prior successful lumbar punctures
Number of prior successful lumbar punctures
Procedural steps correctly performed in pre-test and post-test observation
1. Consent obtained
2. Washed hands
3. Calls “time out”
4. Positions patient
5. Anatomical location
6. Sterile gloves on
7. Equipment set up
8. Clean skin
9. Drape patient
10. Lidocaine drawn up
11. Lidocaine injected
12. Needle orientation
13. Needle bevel correct
14. Needle slowly advanced
15. Opening pressure
16. Obtains and collects fluid
17. Re-inserts stylet at end
18. Dressing applied
19. Post-op orders written
20. Correct labs ordered
21. Sterile technique maintained
In order to confirm that the PGY1 and PGY2 sample size was adequate, a post-hoc analysis was performed, calculating the mean of paired differences between pre- and post-simulation scores that would have been able to detect with our data set for the subset of participants in PGY1 and 2. Using the PASS 14 software , it was determined that our sample size of 10 PGY1 and 2 residents achieved 90 % power to detect a mean of paired differences of 4.1 with our observed standard deviation of differences of 3.6 and with a significance level (alpha) of 0.05 using a two-sided paired t-test. Given that the observed mean of paired differences for the PGY1 and 2 subsets was 6.3, with this power analysis, it was confirmed that the study was adequately powered to detect the difference that we observed in the subset of PGY1 and 2 residents.
This study demonstrates that an interactive teaching session and practice using a lumbar puncture simulator significantly improves procedural competence among junior (PGY1 and PGY2) pediatric residents and decreases self-reported anxiety among the most junior (PGY1) trainees. The improvement in resident procedural skill acquisition showed several findings. Each are of interest in the context of medical educational literature as it pertains to competency development for medical procedures.
The PGY1 residents completed their pre-test after only 2 months of training. None of the PGY1 residents reported any prior successful lumbar punctures. Given their stage of training it may not be a surprise that the most junior (PGY1) residents demonstrated the greatest improvement in procedural skill acquisition showing a mean improvement of 32.3 % in their pre-test versus post-test LP checklist scores. This finding was similar to that of Barsuk et al.  who first developed the 21-item lumbar puncture checklist. They followed 58 internal medicine residents (PGY1) using a similar pre-test versus post-test design. Their scores of the internal medicine residents showed a similar increase from 46.3 to 95.7 % after receiving a teaching session and feedback . However, our study also identified a 24.2 % improvement in the pre-test versus post-test LP checklist scores for PGY2 pediatric residents which increased to a mean 92 % in post-test scores. Unlike the PGY1 residents, the PGY2 residents had reported some previously successful procedures (Table 1). This finding indicates that there may be educational value to using simulation training with residents in other post-graduate years, particularly those that may have had non-standardized training which may result in variability and/or perpetuation of mistakes and bad habits. The most senior (PGY3 and PGY4) residents did not demonstrate a significant change in their LP checklist scores which may be attributable to the larger number of procedures done by residents in these years.
The reduction in self-reported anxiety scores was significant only for PGY1 trainees. Given that this group of junior residents reported no prior successful LPs, we interpret this as a measure of initial resident discomfort and low self-confidence as they are asked to perform a complicated, multi-step procedure with which they have little or no familiarity. A survey of practicing physicians in the United Kingdom found that 42 % recalled feeling inadequately trained to perform procedures when they first began doing so independently . These physicians cited a lack of formal and consistent training that has been a criticism of the former “see one, do one, teach one” method of acquiring procedural skills in medicine . We postulate that the reduction in post-test anxiety among PGY1 residents is attributable to their increased exposure, standardized teaching and overall improved familiarity with lumbar punctures. We would favor the use of simulation for the initial attempts at such procedures to improve resident confidence while at the same time improving patient safety and reducing potential for medical error.
Manthey and Fitch  have proposed four stages for developing competency in medical procedures. Knowledge acquisition is the first stage and is characterized by the learner creating an appropriate framework of knowledge that will complement his or her future skill acquisition. Exposure is the second stage and is acquired by indirect observation and direct participation in the procedure. Skill acquisition is the third stage and occurs when the learner performs and practices a procedure. Assessment is the fourth and final stage that occurs when the trainee demonstrates competency in their medical knowledge and independent procedural skills . We feel that our findings fit well into Manthey’s stages of developmental competency. The PGY1 residents, presumably at the earliest stage of knowledge acquisition, demonstrated the largest improvement in their pre-test versus post-test procedural ability scores as well as a reduction in self-reported anxiety. Interestingly, even though the PGY1 residents had the same interactive teaching session and equal opportunity to practice with the LP simulator, their final scores did not reach that of their PGY2 to PGY4 counterparts (Fig. 1). The PGY2 residents, having had some prior experience with LPs were likely at Manthey’s second or third stage of procedural skill acquisition. Their previous experience may have provided them with a conceptual framework to facilitate a higher overall skill acquisition and procedural competency score, as reflected in their mean post-test score of 92 % compared to their PGY1 colleagues that showed a mean post-test score of 67.8 %. We hypothesize that it is precisely the prior LP experience and familiarity that PGY2 to PGY4 residents had that accounted for their lower and stable pre and post-test anxiety scores.
Our study provides data that will be helpful for medical educators to decide how best to apply simulation-based procedural skills training across the PGY spectrum. Canadian residency training programs are in the process of moving towards a competency-based medical education (CBME) model . As such, these results may be helpful with the planning of procedural skills training under this new education model. We strongly support the use of such simulation tools for first-year or new entry pediatric residents. Our study clearly shows that lumbar puncture simulation exercises have the greatest potential benefit for improving procedural competency among PGY1 pediatric residents. There is clearly an advantage for both residents and patients alike when junior pediatric residents are required to demonstrate basic procedural competency using a simulation device before moving on to attempt LP on an infant or a child. The previously reported 21-item LP checklist is a potential tool to evaluate trainees although there is a need to ensure that this validity evidence is established for this checklist and that all items are agreed upon by multiple residency programs, post-graduate medical education committees and/or examination boards.
Our results also indicate that there may be a role for simulation training in improving procedural competency among PGY2 pediatric residents. Despite the fact that PGY2 residents reported some prior successful LPs, they demonstrated a significant improvement in procedural competency following simulation training. It is possible that the PGY2 residents may have had variable teaching that can occur with procedural skill acquisition. Moreover, PGY2 residents are adding considerably to their depth of knowledge and given the clinical experience that they have gained over their first year of residency education, they may be at an ideal stage to enhance and retain such procedural skill. As such we recommend simulation based LP teaching sessions annually for at least PGY1 and PGY2 residents. More senior residents may be excused from such simulation training if they have been formally assessed by staff physicians and found to demonstrate competency in their medical knowledge and procedural skills in both simulation and patient-procedures.
Our study re-tested residents over a short time-frame and did not assess to what degree their procedural skills were retained. It also did not examine if repeat or refresher training would be advantageous. Other studies have demonstrated that simulation training skills using central line placement by internal medicine residents are indeed retained when residents are re-tested 6 and 12 months post training .
In addition to the move towards CBME, there are additional reasons why the use of simulation-based learning for lumbar punctures may be of increasing importance. As the incidence of childhood meningitis has decreased in recent decades  as a result of conjugate vaccines there may be less opportunity for pediatric residents to perform lumbar punctures. The very appropriate increase in focus on patient care and safety has also made the traditional method of learning less acceptable. Like all procedure skills, experience is paramount for success. One adult study examined unsuccessful lumbar punctures by neurologists and found that inexperience was a greater cause of failed LP attempts as compared to all other causes .
Learning procedures through a controlled, simulated experience ensures that residents are taught the proper techniques including equipment set-up, patient positioning and assessment of risk [6, 18, 19]. Improved procedural skills have also been shown to persist among residents with continued skill improvement upon re-testing as long as 6 months after the initial simulation based training . Junior residents have a great deal to gain from simulation-based experiences and simulation-based mastery learning (or intern “boot-camp”) can have a positive impact on resident performance . Permitting residents to complete a “practice run” using a simulator provides them with an opportunity to review these aspects of the procedure in a relaxed and safe environment and better anticipate what to expect when the procedure is replicated on a patient. We recommend the use of such an itemized checklist when residents are transitioning from simulation-based practice to patient-based care to ensure that the same rigorous process is followed and patient safety is optimized.
Our study does have limitations. First, the small sample size may limit our ability to generalize our findings. It is unknown if those who participated are more or less likely to be proficient at LPs than the residents who did not participate. Second, variability can exist even among experienced OSCE examiners. Simultaneous assessment by two attending physicians and/or blinded review of video recorded sessions are two examples of ways that future projects could aim to evaluate inter-examiner variability. In addition, this study examined the value of LP simulation models and an interactive teaching session, so the improvement observed in the procedural skill and comfort of junior residents cannot be attributed to one item in isolation. Finally, we did not assess skill retention over time or LP performance on real patients in a clinical setting.
We would encourage future research projects aimed at examining whether or not repeat or refresher simulation training would be advantageous for pediatric residents. We would also encourage the collection of validity evidence for the previously reported 21-item LP checklist for use in infants and children by appropriate pediatric residency programs, post-graduate medical education committees and/or examination boards. We infer that more procedural training and experience, whether on simulated or real patients, should improve clinical outcomes. We believe that the use of simulation not only offers advantages for patient safety but also potential advantages for residents if anxiety is reduced and/or self-confidence is improved. Such answers will be important not only for integrating this into emerging CMBE curriculums but also ensuring any questions regarding the need for long-term procedural skill competency (through continuing medical education programs) are properly considered.
This study provides evidence that the combined use of a LP simulator and interactive teaching session has a positive impact on simulated procedural competence and self-reported anxiety in junior pediatric residents.
ACGME, accreditation council for graduate medical education; CBME, competency-based medical education; CSF, cerebral spinal fluid; LP, lumbar puncture; OSCE, objective structured clinical examinations; PGY, post-graduate year; RCPSC, Royal College of Physicians and Surgeons of Canada; SD, standard deviation; STAI-S, State-Trait Anxiety Inventory – State Anxiety Scale
The authors would like to thank all of the Pediatric Residents who participated in this project.
Availability of data and materials
Data can be provided upon request.
This study was funded by the University of Ottawa Faculty of Medicine and Postgraduate Medical Education, “Educational Initiatives in Residency Education” Award.
HJM: contributed to the conception and design of the study, acted as an examiner for Pediatric Residents performing LPs and drafted the manuscript. HW: contributed to the design of the study, acted as an examiner for Pediatric Residents performing LPs and provided a critical review of the manuscript. KAM: contributed to the design of the study and provided a critical review of the manuscript. KE: contributed to the design of the study and scored the STAI evaluations and provided a critical review of the manuscript. ES: contributed to the design of the study, acted as an examiner for Pediatric Residents performing LPs and provided a critical review of the manuscript. JG: compiled all confidential data, performed statistical analysis and provided a critical review of the manuscript. AD: contributed to the conception and design of the study, acted as an examiner for Pediatric Residents performing LPs and provided a critical review of the manuscript. All authors read and approved the final manuscript.
The authors declare that they have no financial or non-financial competing interests. The results of this study was presented at the Royal College of Physician and Surgeons of Canada, International Conference on Residency Education (ICRE) in Vancouver, BC (October 2015).
Consent for publication
Ethics approval and consent to participate
Children’s Hospital of Eastern Ontario (CHEO) Research Ethics Board (REB) approval (# 14/40X) was obtained prior to the start of this study. All participating pediatric residents provided informed consent to one of the study co-authors who did not have any direct supervisory role in resident evaluation or promotion.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Ellenby MS, Tegtmeyer K, Lai S, Braner DA. Videos in clinical medicine. Lumbar puncture. N Engl J Med. 2006;355(13):e12.View ArticleGoogle Scholar
- Royal College of Physicians and Surgeons of Canada. Objectives of Training in Pediatrics (2008). [homepage on the internet]. Canada [cited 10 Nov 2015]. <http://www.royalcollege.ca/cs/groups/public/documents/document/y2vk/mdaw/~edisp/tztest3rcpsced000931.pdf>. Accessed 5 Aug 2016.
- Accreditation Council for Graduate Medical Education. ACGME Program Requirements for Graduate Medical Education in Pediatrics. [homepage on the internet]. United States. [cited 16 Nov 2015]. <http://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/320_pediatrics_2016.pdf>. Accessed 5 Aug 2016.
- Nathan BR, Kincaid O. Does experience doing lumbar puncures result in expertise? Neurology. 2012;79(2):115–6.View ArticleGoogle Scholar
- Huang GC, Smith CC, Gordon CE, Feller-Kopman DJ, Davis RB, Phillips RS, et al. Beyond the comfort zone: residents assess their comfort performing inpatient medical procedures. Am J Med. 2006;119(1):71.View ArticleGoogle Scholar
- Barsuk JH, Cohen ER, Caprio T, McGaghie WC, Simuni T, Wayne DB. Simulation-based education with mastery learning improves residents’ lumbar puncture skills. Neurology. 2012;79(2):132–7.View ArticleGoogle Scholar
- Wayne DB, Barsuk JH, O’Leary KJ, Fudala MJ, McGaghie WC. Mastery learning of thoracentesis skills by internal medicine residents using simulation technology and deliberate practice. J Hosp Med. 2008;3(1):48–54.View ArticleGoogle Scholar
- Barsuk JH, McGaghie WC, Cohen ER, Balachandran JS, Wayne DB. Use of simulation-based mastery learning to improve the quality of central venous catheter placement in a medical intensive care unit. J Hosp Med. 2009;4(7):397–403.View ArticleGoogle Scholar
- Wayne DB, Didwania A, Feinglass J, Fudala MJ, Barsuk JH, McGaghie WC. Simulation-based education improves quality of care during cardiac arrest team responses at an academic teaching hospital: a case-control study. Chest. 2008;133(1):56–61.View ArticleGoogle Scholar
- Kendall PC, Finch Jr AJ, Auerbach SM, Hooke JF, Mikulka PJ. The State-Trait Anxiety Inventory: a systematic evaluation. J Consult Clin Psychol. 1976;44(3):406–12.View ArticleGoogle Scholar
- PASS 14 Power Analysis and Sample Size Software. [homepage on the internet]. USA. [cited: 15 February 2016]. http://www.ncss.com/software/pass/. Accessed 5 Aug 2016.
- Mason WT, Strike PW. See one, do one, teach one - is this still how it works? A comparison of the medical and nursing professions in the teaching of practical procedures. Med Teach. 2003;25(6):664–6.View ArticleGoogle Scholar
- Manthey D, Fitch M. Stages of comptency for medical procedures. Clin Teach. 2012;9(5):317–9.View ArticleGoogle Scholar
- Royal College of Physicians and Surgeons of Canada. Competence by Design (CBD). [homepage on the internet]. Canada [cited 16 Nov 2015]. http://www.royalcollege.ca/rcsite/competence-design-e. Accessed 5 Aug 2016.
- Barsuck JH, Cohen ER, McGaghie WC, Wayne DB. Long-term retention of central venous catheter insertion skills after simulation-based mastery learning. Acad Med. 2010;85:S9–12.View ArticleGoogle Scholar
- Dawson KG, Emerson JC, Burns JL. Fifteen years of experience with bacterial meningitis. Pediatr Infect Dis J. 1999;18(9):816–22.View ArticleGoogle Scholar
- Sahebkar-Moghaddam F, Adornato BT. The failed lumbar puncture. Neurology. 2005;64(7):E24.View ArticleGoogle Scholar
- Conroy SM, Bond WF, Pheasant KS, Ceccacci N. Competence and retention in performance of the lumbar puncture procedure in a task trainer model. Simul Healthc. 2010;5(3):133–8.View ArticleGoogle Scholar
- Lenchus J, Issenberg SB, Murphy D, Everett-Thomas R, Erben L, Arheart K, et al. A blended approach to invasive bedside procedural instruction. Med Teach. 2011;33(2):116–23.View ArticleGoogle Scholar
- Cohen ER, Barsuk JH, Moazed F, Caprio T, Didwania A, McGaghie WC, et al. Making July Safer: Simulation-Based Mastery Learning During Intern Boot Camp. Acad Med. 2013;88(2):233–9.View ArticleGoogle Scholar