Does reflection have an effect upon case-solving abilities of undergraduate medical students?
© Koole et al.; licensee BioMed Central Ltd. 2012
Received: 4 December 2011
Accepted: 3 August 2012
Published: 13 August 2012
Reflection on professional experience is increasingly accepted as a critical attribute for health care practice; however, evidence that it has a positive impact on performance remains scarce. This study investigated whether, after allowing for the effects of knowledge and consultation skills, reflection had an independent effect on students’ ability to solve problem cases.
Data was collected from 362 undergraduate medical students at Ghent University solving video cases and reflected on the experience of doing so. For knowledge and consultation skills results on a progress test and a course teaching consultation skills were used respectively. Stepwise multiple linear regression analysis was used to test the relationship between the quality of case-solving (dependent variable) and reflection skills, knowledge, and consultation skills (dependent variables).
Only students with data on all variables available (n = 270) were included for analysis. The model was significant (Anova F(3,269) = 11.00, p < 0.001, adjusted R square 0.10) with all variables significantly contributing.
Medical students’ reflection had a small but significant effect on case-solving, which supports reflection as an attribute for performance. These findings suggest that it would be worthwhile testing the effect of reflection skills training on clinical competence.
Reflection is a metacognitive process triggered by experience and characterized by three sub-processes: Awareness of self and the situation; critical analysis and understanding of self and the situation; development of new perspectives to inform future actions [1–4]. Reflection on professional experiences is considered to be an attribute that allows healthcare practitioners to cope with demanding and complex professional situations [5–8]. Accordingly, the ability to reflect is identified in many guidelines as an important learning outcome for physicians in training [9–11]. It is proposed that reflection gives a comprehensive view of contextual factors that affect clinical decisions, helps practitioners identify gaps in personal knowledge, and gives direction to their personal development [1, 5, 12, 13]. Unreflective practitioners have been reported to perpetuate routine behaviours and not open them to discussion, have narrow perspectives on their practice, find it difficult to identify learning goals and accept feedback, and find it difficult to adapt their practice [5, 13, 14]. Accordingly, systematic reflection is seen as essential for continuing professional development and lifelong learning [7, 14]. Despite this recognition, however, there is a lack of empirical evidence proving it is indeed effective [2, 15].
In the past decade, evidence has been published showing a link between personal attributes and the ability to reflect. Mamede and Schmidt  found a negative correlation between reflective practice and a physician’s age and working experience, which they attributed to older and more experienced physicians being more likely to find situations routine and use automatic reasoning based on recognition and instant retrieval of similar situations. Boenink et al.  assessed reflection by means of written answers to vignettes. Undergraduate medical students who were female, had previous health care work experience, and who were aiming for careers in general practice tended to have higher reflection scores. The authors concluded that the ability to reflect is trait-like but affected by learning processes. Results based on a self-report questionnaire developed by Sobral et al.  showed a relation between reflection and the perceived meaningfulness of learning, which is a marker of the depth of learning. Qualitative studies by Sargeant et al. [19, 20] showed that reflection helps learners to accept feedback and use it in their future clinical practice. We found only one study that demonstrated a direct link between reflection and performance. Sobral et al.  reported undergraduate students’ scores on a reflection-in-learning scale were significantly, but weakly, correlated with grade point averages, which they used as an indicator of academic achievement.
At Ghent University, undergraduate medical students follow a seven year integrated contextual curriculum, comprising patient centred, student centred, community orientated, problem based and evidence based education . The first two and a half years focus on the healthy and normal body and continue in a second cycle of two and a half years to address the body systems from a clinical perspective. Year six comprises rotational clerkships and year seven is a transitional year to postgraduate education. In the present study data was collected among students in the second cycle during year 2008–2009 (n = 362).
In line with the conceptual research framework, data were collected on the four variables shown in Figure1- the quality of case solving, reflection skills, level of knowledge and level of mastery of consultation skills. The variables “case-solving” and “reflection” were measured by presenting each student with two interactive video-cases, which confronted participants with authentic clinical problems in a standardized assessment context . They showed a simulated patient consulting a physician with a new clinical problem. Scenes were filmed from the physician’s perspective, to make the cases as realistic as possible. All consultations had the same structure: 1) reason for encounter; 2) history; 3) physical examination; 4) explanation of diagnosis, advice and treatment planning; and 5) closing the consultation. Each consultation was interrupted six times with a question (e.g. “What would you ask?”,”What physical examinations would you suggest?”, “Explain the diagnosis to the patient?”) against an otherwise blank screen. To mirror real-life consultations where there is limited time to think, a countdown timer showed students how long they had left to respond.
Quality of case solving
Students’ case-solving was measured by comparing their answers with a list of expected ones. Case scripts and evaluation forms were authored by the same two skills lab teachers to ensure coherent scoring. To test interrater reliability, three assessors (the skills lab teachers and SK) independently scored 30 student reactions per case. A Krippendorff’s alpha coefficient  above 0.97 for each case showed that interrater reliability was high so the remaining answers were single-rated. Respondents’ total score over the two cases was then the variable ‘quality of case solving’.
The referred reflection skills related to the three key elements in the six questions to structure student reflections
Awareness of the experience
1: The ability to describe an experience adequately. 2: The ability to identify essential elements and describe own thoughts and feelings.
Understanding the experience
3: The ability to pose searching questions. 4: The ability to answer searching questions and being aware of the relevant frames of reference.
Impact on future actions
5: The ability to draw conclusions. 6: The ability to describe concrete learning goals and plans for future action.
Level of knowledge
Knowledge was measured by performance in the Dutch inter-university progress test, which assesses knowledge across all medical disciplines at the level of exit from the undergraduate curriculum , during the same academic year as case solving and reflection were assessed. The progress test is a valid and reliable indicator of knowledge acquisition for undergraduate medical students in the Netherlands . It has also been validated in the context of the undergraduate medical curriculum at Ghent University .
Mastery level of consultation skills
Consultation skills are taught in a continuing strand - clinical, technical and communication skills – that runs through years 4–6 of our medical programme. Consultation skills, communication skills, and technical skills are examined using multiple tests: Clinical skills are assessed with and without simulated patients by four experienced physicians in a four station objective structured clinical examination; communication skills are assessed by two communication experts in a specific consultation setting with simulated patients; and technical skills are assessed by a written test of rational prescribing and a computer test of ordering and interpreting medical imaging. Scores from those examinations are combined into a single score, representing the generic skills needed to perform a consultation. To identify the mastery level of consultation skills at the same point in a student’s trajectory as the other variables included in this study, the single course scores of the years 2008–2009 were used for analysis.
Stepwise multiple linear regression analysis was used to determine the predictive value of reflection scores, knowledge, and consultation skills on video-case solving, which was the dependent variable. The stepwise regression procedure aimed to produce a parsimonious model, explaining the dependent variable by including or excluding predictor variables stepwise. The contribution of each variable to the model, its significance level, and the variance explained by the whole model are reported. All statistical analyses were performed using SPSS 17.0 (SPSS Inc, Chicago, IL, USA) with a pre-set significance level of p ≤ 0.05.
Descriptive statistics of all variables (highest possible score) in the multiple linear regression analysis; Mean, Standard deviation (SD), Minimum (Min) and Maximum (Max)
Case solving score (20)
Knowledge score (100)
Consultation skill score (20)
Reflection score (60)
The Beta values (B), Standard Error (SE B) and the Standardized Beta (β) of all coefficients in the linear regression analysis model, based on all students
Consultation skill score
Medical students’ ability to reflect was a significant, albeit weak, predictor of the quality of their case solving after allowing for the effects of knowledge and consultation skills. That is in line with findings of Sobral  demonstrating a significant but weak correlation (r = 0.21, p = 0.003) between undergraduate medical students’ scores on a reflection-in-learning scale and academic achievement. He explained this relationship by the underlying metacognitive skills of reflection, which also affect academic achievement through learning. A similar explanation can also be applied to the present study. Reflection includes the ability to relive an experience, analyze it critically, and come up with conclusions after careful exploration of alternatives [13, 16, 33]. Using such skills might have helped students with high reflection scores to understand the case content more profoundly and to give more carefully considered answers, which resulted in higher case solving scores.
Our results demonstrate that case solving both triggers and is affected by reflection. This relationship, however, is not as circular as it might appear. At its heart lies a distinction between the content and process of reflection. Whereas the content of reflection is context specific and influenced by its triggering experience and learners’ unique frame of reference, the process of reflection has a more generic character [34, 35]. In the present study, case solving as a triggering experience is related to the content of reflection. The effect of reflection on case-solving that we found, however, refers to the process of reflection, which is driven by more generic reflective skills.
Focus on those generic skills makes it possible to assess reflections while recognizing the uniqueness of both a learner’s frame of reference and the context in which their reflection was initiated . It also provides a counter-argument to the argument that our results can be accounted for by having measured reflection skills and the quality of case-solving in the same context whilst knowledge and consultation skills were assessed in a different context. The focus on process skills made the influence of context less important.
Although the predictive effect of reflection, knowledge and consultation skills on the quality of case solving was statistically significant, the model only explained 10 % of the total variance. From previous studies we would have expect the levels of knowledge and consultation skills to account for more variance than was demonstrated here [22, 23]. First, this inconsistency with earlier studies may be explained by the different methods used to assess case solving. As opposed to answering questions in video-cases, other studies used objective structured clinical examinations (OSCE) derived formats as clinical performance examinations (CPX) and Integrated Structured Clinical Exams (ISCE). These methods required practical knowledge and executive skills and are called performance assessment in vitro whereas video-based approach in the present study exampled a clinical context based test where students had to demonstrate theoretical knowledge by means of writing skills . Second, the specific indicators of knowledge and consultation skills may have contributed to the modest explained total variance of our model. The Dutch inter-university progress test is designed to test a greater breadth of knowledge than was needed to solve the questions in the video-cases . The scores students received in the course ‘clinical, technical and communicative skills’, used as variable for consultation skills, also included competence in radiology and pharmacology next to consultation and communication skills. Whilst these broader aspects of competence were not included in previous studies, they were clearly relevant to the diagnostic and treatment planning aspects of the video-cases.
The modest total of variance explained by our regression model suggests the set of three predictors in the model was incomplete. Factors such as case difficulty, the time of testing, and test environment were similar for all students; personal factors, however, could make cases more or less difficult for individual students and contribute to variance in the scores. Desmedt  identified motivation, beliefs, and self-efficacy as relevant factors, alongside gender, personality, intelligence and learning style. Future research could address limitations of the current study by developing a more comprehensive model to describe case-solving. It could also test the generalizability of our findings to a workplace context and from case scores to clinical practice.
Undergraduate medical students’ reflection had a small but significant effect on the quality of case solving. This empirical finding suggests that helping students develop their ability to reflect might be beneficial and it would therefore be worth testing the effect of reflection skills training on clinical competence.
- Boud D, Keogh R, Walker D: Reflection: Turning experience into learning. 1985, Kogan Page, LondonGoogle Scholar
- Atkins S, Murphy K: Reflection - A Review of the Literature. J Adv Nurs. 1993, 18 (8): 1188-1192. 10.1046/j.1365-2648.1993.18081188.x.View ArticleGoogle Scholar
- Sandars J: The use of reflection in medical education: AMEE Guide No. 44. Medical Teacher. 2009, 31 (8): 685-695. 10.1080/01421590903050374.View ArticleGoogle Scholar
- Koole S, Dornan T, Aper L, Scherpbier A, Valcke M, Cohen-Schotanus J, Cohen-Schotanus J, Derese A: Factors confounding the assessment of reflection: a critical review. BMC Med Educ. 2011, 11: 104-10.1186/1472-6920-11-104.View ArticleGoogle Scholar
- Schön DA: The reflective practitioner: How professionals think in action. 1983, Basic Books, New YorkGoogle Scholar
- Epstein RM, Hundert EM: Defining and assessing professional competence. JAMA. 2002, 287: 226-235. 10.1001/jama.287.2.226.View ArticleGoogle Scholar
- Robertson K: Reflection in professional practice and education. Aust Fam Physician. 2005, 34 (9): 781-783.Google Scholar
- Kjaer NK, Maagaard R, Wied S: Using an online portfolio in postgraduate training. Medical Teacher. 2006, 28 (8): 708-712. 10.1080/01421590601047672.View ArticleGoogle Scholar
- Scottish Deans’ Medical Curriculum Group: Learning Outcomes for the Medical Undergraduate in Scotland: A Foundation for Competent and Reflective Practitioners. 2007, http://www.scottishdoctor.org/resources/scottishdoctor3.doc [accessed January 17th 2011, 3Google Scholar
- General Medical Council: Tomorrow’s Doctors. 2009, GMC, London, http://www.gmc-uk.org/TomorrowsDoctors_2009.pdf_39260971.pdf [accessed October 28th 2010]Google Scholar
- Dutch Federation of University Medical Centres: The. 2009, http://www.nfu.nl/fileadmin/documents/Raamplan2009engelstalige_versie.pdf [accessed January 17th 2011, framework for undergraduate medical education in the NetherlandsGoogle Scholar
- Lockyer J, Gondocz ST, Thivierge RL: Knowledge translation: the role and place of practice reflection. J Contin Educ Health Prof. 2004, 24 (1): 50-56. 10.1002/chp.1340240108.View ArticleGoogle Scholar
- Plack MM, Greenberg L: The Reflective Practitioner: Reaching for Excellence in Practice. Pediatrics. 2005, 116 (6): 1546-1552. 10.1542/peds.2005-0209.View ArticleGoogle Scholar
- Evans AW, McKenna C, Oliver M: Self-assessment in medical practice. J R Soc Med. 2002, 95 (10): 511-513. 10.1258/jrsm.95.10.511.View ArticleGoogle Scholar
- Mann K, Gordon J, MacLeod A: Reflection and reflective practice in health professions education: a systematic review. Advances in Health Sciences Education. 2009, 14 (4): 595-621. 10.1007/s10459-007-9090-2.View ArticleGoogle Scholar
- Mamede S, Schmidt HG: The structure of reflective practice in medicine. Medical Education. 2004, 38 (12): 1302-1308. 10.1111/j.1365-2929.2004.01917.x.View ArticleGoogle Scholar
- Boenink AD, Oderwald AK, de Jonge P, van Tilburg W, Smal JA: Assessing student reflection in medical practice. The development of an observer-rated instrument: reliability, validity and initial experiences. Medical Education. 2004, 38 (4): 368-377. 10.1046/j.1365-2923.2004.01787.x.View ArticleGoogle Scholar
- Sobral DT: An appraisal of medical students' reflection-in-learning. Medical Education. 2000, 34 (3): 182-187. 10.1046/j.1365-2923.2000.00473.x.View ArticleGoogle Scholar
- Sargeant JM, Mann KV, van der Vleuten CP, Metsemakers JF: Reflection: a link between receiving and using assessment feedback. Advances in Health Sciences Education. 2009, 14 (3): 399-410. 10.1007/s10459-008-9124-4.View ArticleGoogle Scholar
- Sargeant J, Mann K, Sinclair D, van der Vleuten C, Metsemakers J: Understanding the influence of emotions and reflection upon multi-source feedback acceptance and use. Advances in Health Sciences Education. 2008, 13 (3): 275-288. 10.1007/s10459-006-9039-x.View ArticleGoogle Scholar
- Sobral DT: Medical students' reflection in learning in relation to approaches to study and academic achievement. Medical Teacher. 2001, 23 (5): 508-513.View ArticleGoogle Scholar
- Mattick K, Dennis I, Bradley P, Bligh J: Content specificity: is it the full story? Statistical modelling of a clinical skills examination. Medical Education. 2008, 42 (6): 589-599. 10.1111/j.1365-2923.2008.03020.x.View ArticleGoogle Scholar
- Wimmers PF, Fung CC: The impact of case specificity and generalisable skills on clinical performance: a correlated traits-correlated methods approach. Medical Education. 2008, 42 (6): 580-588. 10.1111/j.1365-2923.2008.03089.x.View ArticleGoogle Scholar
- Van der Veken J, Valcke M, De Maeseneer J, Schuwirth L, Derese A: Impact on knowledge acquisition of the transition from a conventional to an integrated contextual medical curriculum. Medical Education. 2009, 43 (7): 704-713. 10.1111/j.1365-2923.2009.03397.x.View ArticleGoogle Scholar
- de Leng BA, Dolmans DHJM, van de Wiel MWJ, Muijtjens AMM, van der Vleuten CPM: How video cases should be used as authentic stimuli in problem-based medical education. Medical Education. 2007, 41 (2): 181-188. 10.1111/j.1365-2929.2006.02671.x.View ArticleGoogle Scholar
- Hayes AF, Krippendorff K: Answering the Call for a Standard Reliability Measure for Coding Data. Communication Methods and Measures. 2007, 1 (1): 77-89. 10.1080/19312450709336664.View ArticleGoogle Scholar
- Moon JA: Reflection in learning and professional development: theory and practice. 1999, Kogan Page, LondonGoogle Scholar
- Koole S, Dornan T, Aper L, De Wever B, Scherpbier A, Valcke M, Cohen-Schotanus J, Derese A: Using video-cases to assess student reflection: Development and validation of an instrument. BMC Med Educ. 2012, 12: 22-10.1186/1472-6920-12-22.View ArticleGoogle Scholar
- Popham WJ: What's wrong–and what's right–with rubrics. Educational Leadership. 1997, 55 (2): 72-75.Google Scholar
- Andrade HG: Using rubrics to promote thinking and learning. Educational Leadership. 2000, 57 (5): 13-18.Google Scholar
- van der Vleuten CPM, Schuwirth LWT, Muijtjens AMM, Thoben AJNM, Cohen-Schotanus J, Van Boven CPA: Cross institutional collaboration in assessment: a case on progress testing. Medical Teacher. 2004, 26 (8): 719-725. 10.1080/01421590400016464.View ArticleGoogle Scholar
- Albano MG, Cavallo F, Hoogenboom R, Magni F, Majoor G, Manenti F, Schuwirth L, Stiegler I, vanderVleuten C: An international comparison of knowledge levels of medical students: The Maastricht Progress Test. Medical Education. 1996, 30 (4): 239-245. 10.1111/j.1365-2923.1996.tb00824.x.View ArticleGoogle Scholar
- Korthagen F, Vasalos A: Levels in reflection: core reflection as a means to enhance professional growth. Teachers and Teaching: theory and practice. 2005, 11 (1): 47-71. 10.1080/1354060042000337093.View ArticleGoogle Scholar
- Bourner T: Assessing reflective learning. Education + Training. 2003, 45 (5): 267-272. 10.1108/00400910310484321.View ArticleGoogle Scholar
- Mezirow J and Associates: Learning as Transformation: Critical Perspectives on a Theory in Progress. 2000, Jossey-Bass, San FranciscoGoogle Scholar
- Wass V, Van der Vleuten C, Shatzer J, Jones R: Assessment of clinical competence. Lancet. 2001, 24 (357): 945-949.View ArticleGoogle Scholar
- Desmedt E: Research into the Theoretical Base of Learning Styles in View of Educational Applications in a University Setting. 2008, PhD thesis, Ghent UniversityGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6920/12/75/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.