This article has Open Peer Review reports available.
A simulated night shift in the emergency room increases students’ self-efficacy independent of role taking over during simulation
https://doi.org/10.1186/s12909-016-0699-9
© The Author(s). 2016
Received: 15 April 2016
Accepted: 16 June 2016
Published: 15 July 2016
Abstract
Background
Junior doctors do not feel well prepared when they start into postgraduate training. High self-efficacy however is linked to better clinical performance and may thus improve patient care. What factors affect self-efficacy is currently unknown. We conducted a simulated night shift in an emergency room (ER) with final-year medical students to identify factors contributing to their self-efficacy and thus inform simulation training in the ER.
Methods
We simulated a night in the ER using best educational practice including multi-source feedback, simulated patients and vicarious learning with 30 participants. Students underwent 7 prototypic cases in groups of 5 in different roles (leader, member and observer). Feeling of preparedness was measured at baseline and 5 days after the event. After every case students recorded their confidence dependent of their role during simulation and evaluated the case.
Results
Thirty students participated, 18 (60 %) completed all surveys. At baseline students feel unconfident (Mean −0.34). Feeling of preparedness increases significantly at follow up (Mean 0.66, p = 0.001, d = 1.86). Confidence after simulation is independent of the role during simulation (F(2,52) = 0.123, p = 0.884). Observers in a simulation can estimate leader’s confidence independent of their own (r = 0.188, p = 0.32) while team members cannot (r = 0.61, p < 0.001).
Conclusions
Simulation improves self-efficacy. The improvement of self-efficacy is independent of the role taken during simulation. As a consequence, groups can include observers as participants without impairing their increase in self-efficacy, providing a convenient way for educators to increase simulation efficiency. Different roles can furthermore be included into multi-source peer-feedback.
Keywords
Background
Junior doctors do not feel well prepared when they start into postgraduate training [1–3] independent of their objective performance [4]. Next to the accuracy of a diagnosis, adequate confidence in this diagnosis however is a necessity for safe and effective patient care. Too little confidence in an accurate diagnosis may harm patients through the delay of necessary treatment and unnecessary and potentially harmful additional investigations.
While the relationship between confidence and tendency to act applies to all of medicine, it is especially relevant to emergency medicine, where delayed action may have severe consequences.
Situational confidence (or self-efficacy) is a key factor to determine what actions one may take [5]. As an individual’s reliance on personal abilities to succeed in a given challenge, self-efficacy increases the likelihood of that individual’s actions actually occurring [6, 7]. By contrast, low self-efficacy and resulting distress is argued to contribute to mental health problems [8, 9].
Several factors have been identified to influence self-reported feelings of preparedness. The percentage of graduates not feeling well prepared for clinical work differs strongly between countries [3, 10, 11] implying a great impact of educational systems and practices. Factors known to contribute to higher feelings of preparedness include frequent and immediate feedback [12], theoretical education counterbalanced with practice training, good skills education and training in diagnostic decision-making [10].
Simulation is a teaching format that may (and should) contain all four of those elements [13, 14] and thus should affect individual feelings of preparedness besides the well-known effects on objective performance [15]. Another teaching format known to increase self-efficacy includes observational or vicarious learning which is as effective as hands-on training in the acquisition of practical skills [16].
The aim of our study was to develop a best practice simulation session and evaluate the effect of simulation on the development of students’ feelings of preparedness. We further aimed to identify factors within the simulation that affect confidence and feelings of preparedness in order to design a well-balanced simulation, budgeting both costs and educational effectiveness. To identify such factors we focused on the role students take over during the simulation and differences between self-reported confidence and confidence judged by peers.
Methods
Study design
A six-hour simulation session took place in 2013 at Charité – Universitätsmedizin Berlin as a night shift in a simulated emergency room (ER). The ER consisted of several rooms and an ambulance vehicle. Each room hosted a different simulated case of a total of seven. We invited students in their final year of medical school to participate.
Study design. Measurements in italic, roles in bold
At the beginning of the night, participants completed a questionnaire on possible confounder and self-reported feeling of preparedness in different medical specialties together with an informed consent form. Directly after each scenario, all active members recorded their confidence individually before feedback was given. Furthermore team members and observers estimated the team leader's confidence. After feedback, participants evaluated the quality of the given feedback and of the simulation overall. At the end of the night, participants filled in a final evaluation focusing on overall quality of the simulations. Participants were further asked to complete a second questionnaire of self-reported feelings of preparedness five days after the event in an online survey similar to the first questionnaire.
All evaluations (forms available upon request) were conducted using Likert scales for each item ranging from “totally agree” to “totally disagree”. We coded the responses on a numerical scale ranging from +3 to −3 with 0 equating “neutral”. All but the last evaluation forms were filled in on paper during the simulation, the final questionnaire “feeling of preparedness 2” was conducted online using lime survey software.
Participants
Medical students who had completed their fifth year of medical school (around 600 total) were invited to take part in the night shift. In Germany, five years of medical studies are conducted at university, the sixth and final year is spent in internships at different hospitals. Participants were chosen on a first come-first served basis through an online registration. 30 participants were randomized into six teams stratified by gender using a computer-generated randomization list. Participants were greeted in a general assembly and informed about the course of events of the night. After completing the written informed consent including participant information, information about opportunities to withdraw and possible consequences of withdrawl (none), they split into teams. The study was approved by the institutional office for data protection at Charité Berlin and deemed exempt from ethical review under local legislation, because it does not involve patients.
Cases
Cases were drafted following national and international guidelines and chosen based on learning objectives from a German national consensus statement [17, 18]. Each case represented a common ER patient. There were more diagnostic investigations available than necessary per case in order to ensure an uninterrupted simulation. A checklist was developed for each scenario to guide feedback by peer observers and case tutors.
Cases and simulation settings
Discipline | Diagnosis (guidelines as sources) | Mode of simulation | Anticipated course of management |
|---|---|---|---|
Pulmology | Exacerbated COPD | SP with examination possible | Chest X-Ray, blood-gas analysis, continous monitoring |
Neurology | Ischemic media-stroke | SP with examination possible | CCT, continous monitoring |
Cardiology | STEMI & non-sustained ventricular tachykardia | SP with examination possible | 12-channel ECG, enzymes, continous monitoring |
Anaesthesia | Ventricular fibrillation following STEMI | simulator-based approach | continous monitoring, ACLS |
Surgery 1 | Hemodynamic instable ruptured spleen | simulator-based approach with advanced monitoring | ATLS with FAST, continous monitoring |
Urology | Urinary tract infection & pregnancy | SP with examination and sonography possible | urin test, ultrasound and gynaecological referal |
Surgery 2 | Head laceration | SP with examination and preparation of wound possible | Stitching of the wound |
Implementation
SPs were trained for five case scenarios (pulmonology, cardiology, neurology, urology and surgery 2, see Table 1). To guarantee an appropriate level of fidelity both hybrid simulations and mechanical simulators were used [14, 19]. To represent a real time course of events, all laboratory orders and radiology inquiries had to be requested by phone and/or in written form. The operational headquarters delayed their answer depending on the requested examination. Participants finished a scenario by arranging for the patient to be transferred to a ward or to be discharged. Each scenario lasted approximately 30–45 min including feedback. Additional technical details of the simulation, a detailed description of every case and used guidelines are provided as supporting information (see Additional files 1 and 2).
Roles of participants
-
the team leader was responsible for the entire process – coordinating the team, choosing the right diagnosis and treating the patient accordingly.
-
the team member was an active part of the group and supported the team leader throughout the process of finding the right diagnosis and treating the patient.
-
the team observer observed the team using a checklist and provided feedback afterwards.
Roles within the group changed with each scenario so that at the end of the night each student had at least once taken on each role. Each group could freely develop their teamwork throughout the night shift. A peer tutor supervised and counseled the group.
Multi-source feedback
-
the SP focused on communication using the Calgary-Cambridge Observation Guide (CCOG) to guide his or her feedback [21, 22].
-
the team observer gave checklist-based feedback in order to provide the team with external observations but also to increase active monitoring of the simulation for his or her personal learning effect.
-
the case tutor focused on the decision-making process with regards to medical content using case-specific checklists.
-
the peer tutor focused on general teamwork and the development of team dynamics and gave feedback in a distinct setting to separate it from the case scenarios.
All tutors are trained in giving feedback and have extensive experience in peer teaching. Participants had experience giving and receiving feedback through curricular events. All SPs are trained regularly.
Statistical analysis
Collected data were analyzed with IBM SPSS Statistics 21.0 (SPSS Inc, Chicago, IL, USA). All data were first analyzed descriptively (mean, standard deviation). Confounder for feeling of preparedness were analyzed with Mann-Whitney-U-Tests, for differences between feeling of preparedness 1 and feeling of preparedness 2 we used a related sample Wilcoxon signed-rank test. For analysis of role and confidence we conducted a repeated measures analysis of variance (ANOVA), results of which we report as F- and p-values. Correlations between roles were analyzed with Pearson-correlations. Significance was defined as p < 0.05, Cohen’s d was calculated as effect size. We further used G*Power, version 3.1.9.2 [23] to calculate the power achieved. We determined a gain of 0.51 on the Likert scale used from before to after the simulation as the smallest meaningful difference, because such a change would imply that participants chose one point better on the scale slightly more often than expected by chance. The primary dataset is provided as supporting information (see Additional file 3).
Results
A total of 30 students (20 female) participated in the simulation. Three participants had previous medical experience as a paramedic (2) or nurse (1). All 30 available places were booked up after 30 min in the online registration.
Feeling of preparedness
Participants feel rather ill prepared to care for patients before the simulation regardless of specialty (Mean −0.34) with no significant differences between gender (p = 0.075) or age (p = 0.9).
Right after each case students feel confident in their actions and with how they handled the case (Mean 0.95). All participants completed all surveys during the event (100 % response rate), 18 of the 30 participants (60 %) completed the online survey five days after the simulation and showed a significant increase in their general feeling of preparedness compared to before the simulation (p = 0.001). Participants now report to generally feel prepared (Mean 0.66); the effect is large (d = 1.86).
Feeling of preparedness and change from before to five days after simulation
Discipline | Feeling of prepardness Baseline (Mean & SD) | Feeling of prepardness Follow Up (Mean & SD) | p-value |
|---|---|---|---|
Overall | −0.34 (0.49) | 0.66 (0.59) | 0.001** |
Taking History | 1.27 (1.02) | 1.72 (0.9) | 0.035* |
Anaesthesiology | 0.14 (1.06) | 1.17 (0.62) | <0.001*** |
Urology | −0.77 (1.25) | 0.28 (1.53) | 0.013* |
Cardiology | −0.1 (1.06) | 0.28 (1.13) | 0.145 |
Pulmonology | −0.4 (0.97) | 0.11 (1.13) | 0.07 |
Surgery | 0.13 (1.33) | 0.83 (1.3) | 0.101 |
Neurology | −0.47 (1.07) | 0.22 (1.11) | 0.1 |
Role and confidence
In a repeated measures ANOVA with case as the within subject and role as the between subject factor, the self-reported confidence of participants is independent of their role during the simulation (F(2,52) = 0.123 p = 0.884). Both, team members and observers, are equally capable of judging the team leader’s confidence independent of their own role (F(2,52) = 2.055 p = 0.138). How an active team member judges the team leader’s confidence is in part predicted by his or her own confidence (r = 0.61; p < 0.001) while the confidence of the team leader judged by the passive observers does not correlate to the observer's personal confidence (r = 0.188; p = 0.32).
General evaluation
The simulation was evaluated very positively. Students were especially satisfied with how their peer tutors cared for them (Mean 2.93), how the SPs portrayed the patients, the difficulty of the scenarios and their opportunity to apply knowledge learned in medical school (all Mean > 2.7). The quality of the simulation was judged as very good (Mean 2.58). The ratings of each scenario right after the case correspond to the overall evaluation of the night shift.
Students reported to take most out of the feedback given by the case tutors (Mean 2.5) and slightly less out of the feedback by SPs and observing team members (both Mean 2.0).
Discussion
In line with previous findings [11, 24], especially in acute care [2], this study identifies a low feeling of preparedness among medical school graduates with results comparable to previous German [10] and British [3] studies. Our results provide evidence that even a relatively short simulation lasting just one night is effective in increasing students self-efficacy significantly as we observed an overall effect size of d = 1.86. Cohen himself suggested to classify effects as small when Cohen's d > 0.2, as medium when d > 0.5 and as large when d > 0.8 [25].
Intentionally including phases with observational tasks instead of active participation into the simulation may very well explain the simulations large increase simulation efficiency. Stegmann et al. previously demonstrated that hands-on-learning is as efficient as vicarious learning in the acquisition of complex manual skills [16] and Bloch and Bloch successfully used this method in ER-training sessions [26]. Active observation however is a requirement for vicarious learning [27] and giving feedback further enhances it [28]. Our results show that the effect of vicarious learning extends beyond knowledge and skills acquisition and affects situational confidence and ultimately the feeling of preparedness which we found to be unrelated to a learner’s role during simulation. This provides a convenient opportunity for educators to increase group size in simulation with distributed, changing roles among participants and can influence the ratio of staff vs. participants to a more economic one. Furthermore, a recent study has demonstrated a large increase in diagnostic accuracy if patients are diagnosed by teams instead of individuals [29], further increasing the necessity to train medical staff in collaboration and to improve familiarity between ER-teammates, which was found to be surprisingly low in a recent observational study [30].
Training in the night may also be beneficial – nighttime hours are a neglected part of physicians training and may help to better prepare medical graduates for clinical settings [31] and reduce subjective stress of residents working on nighttime [24].
The observation that students significantly gained confidence in history taking may be explained by the facts that a) history taking was required in all cases presented during the night shift and students thus had ample opportunity to practice and b) history taking is directly observable to fellow students and tutors and participants may thus have received plenty of feedback regarding their interviewing skills. We can however only speculate as to why students feeling of preparedness improved for some (i.e. anesthesiology and urology) but not other (i.e. cardiology, pulmonology, surgery and neurology) disciplines and reasons might be discipline-specific. The change in urology may well be attributed to the fact that students hear little to nothing about this discipline during their course of studies [17, 18], while the increased feeling of preparedness in anesthesiology may be due to the high prevalence of algorithms in this discipline. However, the factors that determine changes in the feeling of preparedness warrant further study.
Beyond their implication for simulation practice, our results may also effect future studies of physician confidence. The observation that a team leaders self-reported confidence is not significantly different from his or her confidence judged by observers indicates an equivalence of self-reported and behavioral indicators of situational confidence. This finding further justifies the use of both measures in research on situational confidence, elsewhere termed self monitoring [32]. How the previous experimental finding, that discrepancies in confidence between team members is predictive of team failure [30], translates to real-world medical practice is currently explored in different studies [33]. Although we also did not find differences between team leader’s confidence and their confidence judged by team members, team members account of the leader’s confidence correlates to their own and should thus not be regarded as a valid measure.
Limitations
Because of the high personal effort and costs per participant, only a small number of students were included into the night shift simulation and this pilot study. This might be one reason for non-significant changes in feeling of preparedness in some disciplines. Achieved power however was adequate, thus implying that increasing sample size would likely only lead to the identification of irrelevant findings.
Further, one could argue that the feeling of preparedness is not necessarily linked to objective performance [34], an aspect discussed controversially [35] since self-efficacy is known to become a self-fulfilling prophecy by actually raising the chances of success on a given task [36]. In line with this model of self-efficacy, Bloch [26] and Schubert [37] both found good performance to be associated with high levels of self-reported feelings of preparedness.
Conclusion
Best-practice simulation increases the feeling of preparedness in medical students but remains expensive in the conceptual process. Assigning participants to different roles during simulation is a convenient way to increase group size. These roles have no negative influence on the increase in self-efficacy and provide an opportunity for implementing multi-source peer-feedback. The feeling of preparedness of the active team members and leader also is apparent to observers and can be used as part of a debriefing after a simulation.
Abbreviations
ANOVA, analysis of variance; CCOG, Calgary-Cambridge observation guide; ER, emergency room; SP, simulated patient
Declarations
Acknowledgments
We want to acknowledge all involved tutors of the Lernzentrum, especially Katharina Ziegeler, Leopold Vorwerk and Malte Stuhlert for participation in conception and organisation of the study. We also want to thank the Simulated Patients Program of Charité especially Bea Kampel for her support in the use of standardized patients. Furthermore we would like to acknowledge Kerry Goulston, Sydney, and Sebastian Höfer and Miriam Rüsseler, both from Frankfurt, for their constructive critique on an earlier version of the manuscript.
Funding
The study was partly funded by an unrestricted grant from Deutsche Ärztefinanz to the Skills Lab at Charite. The sponsor did not interfere with the conception and conduction of the study, analysis of data or production of the manuscript.
Availability of data and materials
The dataset supporting the conclusions of this article is included within the article (and its additional files).
Authors contributions
All authors have read and approved the manuscript. FS, KAD and WEH were responsible for the study conception and design. KAD, FS and TS performed the data collection. WEH performed the data analysis. All authors contributed to data interpretation. FS, TS and KAD were responsible for the drafting of the manuscript. All authors made critical revisions to the manuscript for important intellectual content. WEH obtained funding. WEH and AT provided administrative, technical or material support. WEH, AT and AE supervised the study.
Competing interests
The authors declare that they have no competing interests.
Consent for publication
Not applicable.
Ethics approval and consent to participate
The study was approved by the institutional office for data protection at Charité Berlin and deemed exempt from ethical review under local legislation, because it does not involve patients.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Authors’ Affiliations
References
- Morgan PJ, Cleave-Hogg D. Comparison between medical students’ experience, confidence and competence. Med Educ. 2002;36:534–9.View ArticleGoogle Scholar
- Tallentire VR, Smith SE, Skinner J, Cameron HS. The preparedness of UK graduates in acute care: a systematic literature review. Postgrad Med J. 2012;88:365–71.View ArticleGoogle Scholar
- Goldacre MJ, Taylor K, Lambert TW. Views of junior doctors about whether their medical school prepared them well for work: questionnaire surveys. BMC Med Educ. 2010;10:78.View ArticleGoogle Scholar
- Tokuda Y, Goto E, Otaki J, Jacobs J, Omata F, Obara H, Shapiro M, Soejima K, Ishida Y, Ohde S, Takahashi O, Fukui T. Undergraduate educational environment, perceived preparedness for postgraduate clinical training, and pass rate on the National Medical Licensure Examination in Japan. BMC Med Educ. 2010;10:35.View ArticleGoogle Scholar
- Woods JL, Pasold TL, Boateng BA, Hensel DJ. Adolescent health care and the trainee: roles of self-efficacy, standardized patients, and an adolescent medicine rotation. Simul Healthc. 2013;8:359–67.View ArticleGoogle Scholar
- Bandura A. Social cognitive theory: an agentic perspective. Annu Rev Psychol. 2001;52:1–26.View ArticleGoogle Scholar
- Conner M, Norman P: Predicting Health Behaviour. Volume 24. Berkshire, England: Open University Press; 2005.Google Scholar
- Tyssen R, Vaglum P. Mental health problems among young doctors: an updated review of prospective studies. Harv Rev Psychiatry. 2002;10:154–65.View ArticleGoogle Scholar
- Hsu K, Marshall V. Prevalence of Depression and Distress in a Large Sample of Canadian Residents, Interns, and Fellow. Am J Psychiatry. 1987;144:1561–6.View ArticleGoogle Scholar
- Ochsmann EB, Zier U, Drexler H, Schmid K. Well prepared for work? Junior doctors’ self-assessment after medical education. BMC Med Educ. 2011;11:99.View ArticleGoogle Scholar
- Cave J, Woolf K, Jones A, Dacre J. Easing the transition from student to doctor: how can medical schools help prepare their graduates for starting work? Med Teach. 2009;31:403–8.View ArticleGoogle Scholar
- Hattie J, Timperley H. The Power of Feedback. Rev Educ Res. 2007;77:81–112.View ArticleGoogle Scholar
- Cook DA, Hamstra SJ, Brydges R, Zendejas B, Szostek JH, Wang AT, Erwin PJ, Hatala R. Comparative effectiveness of instructional design features in simulation-based education: systematic review and meta-analysis. Med Teach. 2013;35:e867–98.View ArticleGoogle Scholar
- Issenberg SB, McGaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005;27:10–28.View ArticleGoogle Scholar
- Cortegiani A, Russotto V, Montalto F, Iozzo P, Palmeri C, Raineri SM, Giarratano A. Effect of High-Fidelity Simulation on Medical Students’ Knowledge about Advanced Life Support: A Randomized Study. PLoS One. 2015;10:e0125685.View ArticleGoogle Scholar
- Stegmann K, Pilz F, Siebeck M, Fischer F. Vicarious learning during simulations: is it more effective than hands-on training? Med Educ. 2012;46:1001–8.View ArticleGoogle Scholar
- Schnabel KP, Boldt PD, Breuer G, Fichtner A, Karsten G, Kujumdshiev S, Schmidts M, Stosch C. A consensus statement on practical skills in medical school - a position paper by the GMA Committee on Practical Skills. GMS Z Med Ausbild. 2011;28:Doc58.Google Scholar
- Blaum WE, Dannenberg KA, Friedrich T, Jarczewski A, Reinsch AK, Ahlers O. The practial use of the consensus statement on practical skills in medical school--a validation study. GMS Z Med Ausbild. 2012;29:Doc58.Google Scholar
- May W, Park J, Lee J. A ten-year review of the literature on the use of standardized patients in teaching and learning: 1996–2005. Med Teach. 2009;31:487–92.View ArticleGoogle Scholar
- Wood L, Hassell A, Whitehouse A, Bullock A, Wall D. A literature review of multi-source feedback systems within and without health services, leading to 10 tips for their successful design. Med Teach. 2006;28:e185–91.View ArticleGoogle Scholar
- Kurtz SM, Silverman JD. The Calgary-Cambridge Referenced Observation Guides: an aid to defining the curriculum and organizing the teaching in communication training programmes. Med Educ. 1996;30:83–9.View ArticleGoogle Scholar
- Kurtz S, Silverman J, Benson J, Draper J. Marrying Content and Process in Clinical Method Teaching: Enhancing the Calgary–Cambridge Guides. Acad Med. 2003;78:802–9.View ArticleGoogle Scholar
- Faul F, Erdfelder E, Lang A-G, Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods. 2007;39:175–91.View ArticleGoogle Scholar
- Brennan N, Corrigan O, Allard J, Archer J, Barnes R, Bleakley A, Collett T, de Bere SR. The transition from medical student to junior doctor: today’s experiences of Tomorrow’s Doctors. Med Educ. 2010;44:449–58.View ArticleGoogle Scholar
- Cohen J. A power primer. Psychol Bull. 1992;112:155–9.View ArticleGoogle Scholar
- Bloch SA, Bloch AJ. Simulation training based on observation with minimal participation improves paediatric emergency medicine knowledge, skills and confidence. Emerg Med J. 2015;32:195–202.View ArticleGoogle Scholar
- Chi MTH, Roy M, Hausmann RGM. Observing tutorial dialogues collaboratively: insights about human tutoring effectiveness from vicarious learning. Cogn Sci. 2008;32:301–41.View ArticleGoogle Scholar
- Grierson LEM, Barry M, Kapralos B, Carnahan H, Dubrowski A. The role of collaborative interactivity in the observational practice of clinical skills. Med Educ. 2012;46:409–16.View ArticleGoogle Scholar
- Hautz WE, Kämmer JE, Schauber S, Spieß CD, Gassmeier W. Diagnostic Performance by Medical Students Working Individually or in Teams. JAMA. 2015;313:303–4.View ArticleGoogle Scholar
- Patterson PD, Pfeiffer AJ, Lave JR, Weaver MD, Abebe K, Krackhardt D, Arnold RM, Yealy DM. How familiar are clinician teammates in the emergency department? Emerg Med J. 2015;32:258–62.View ArticleGoogle Scholar
- Hanson JT, Pierce RG, Dhaliwal G. The New Education Frontier: Clinical Teaching at Night. Acad Med. 2013;89:1–4.Google Scholar
- Eva K, Regehr G. “I’ll never play professional football” and other fallacies of self‐assessment. J Contin Educ Health Prof. 2008;28:14–9.View ArticleGoogle Scholar
- Hautz SC, Schuler L, Kämmer JE, Schauber SK, Ricklin ME, Sauter TC, Maier V, Birrenbach T Exadaktylos A, Hautz WE. Factors predicting a change in diagnosis in patients hospitalised through the emergency room: a prospective observational study. BMJ Open. 2016;6:e011585.View ArticleGoogle Scholar
- Barnsley L, Lyon PM, Ralston SJ, Hibbert EJ, Cunningham I, Gordon FC, Field MJ. Clinical skills in junior medical officers: a comparison of self-reported confidence and observed competence. Med Educ. 2004;38:358–67.View ArticleGoogle Scholar
- Duns G, Weiland T, Crotty B, Jolly B, Cuddihy H, Dent A. Self-rated preparedness of Australian prevocational hospital doctors for emergencies. Emerg Med Australas. 2008;20:144–8.View ArticleGoogle Scholar
- Eva K, Regehr G. Self-assessment in the health professions: a reformulation and research agenda. Acad Med. 2005;80(10 Suppl):S46–54.View ArticleGoogle Scholar
- Schubert A, Tetzlaff JE, Tan M, Ryckmann V, Mascha E. Consistency, Inter-rater Reliability, and Validity of 441 Consecutive Mock Oral Examinations in Anesthesiology. Anesthesiology. 1999;91:288–98.View ArticleGoogle Scholar
