Research article | Open | Open Peer Review | Published:
Applications of the reflective practice questionnaire in medical education
BMC Medical Educationvolume 19, Article number: 47 (2019)
We sought to determine whether the Reflective Practice Questionnaire (RPQ) is a reliable measure of reflective capacity and related characteristics in medical students. We also planned to learn how the RPQ could be used in medical education.
The RPQ is a 40 item self-report questionnaire that includes a multi-faceted approach to measuring reflective capacity. It also includes sub-scales on several other theoretically relevant constructs such as desire for improvement, confidence, stress, and job satisfaction. The reliabilities of reflective capacity and other sub-scales were determined by calculating their Cronbach alpha reliability values. In the present study, the RPQ was answered by 98 graduating fourth-year medical students from an American University, and these RPQ scores were compared with general public and mental health practitioner samples from a prior study using ANOVA and Bonferroni adjusted comparisons.
Medical students reported a higher reflective capacity than the general public sample, but students were statistically indistinguishable from the mental health practitioner sample. For medical students, reflective capacity was associated with features of confidence, stress, and desire for improvement. Job satisfaction was positively associated with confidence in communication with patients, and negatively associated with stress when interacting with patients. A cluster analysis revealed that around 19% of the medical students exhibited a relatively high level of anxiety interacting with patients, 23% were less engaged, 5% were dissatisfied, and 7% expressed a level of over-confidence in their knowledge and skills that was concerning.
The RPQ is a reliable measure of reflective capacity (Chronbach’s alpha value = 0.84) and related characteristics (Cronbach’s alpha values from 0.75 to 0.83) in medical students. The RPQ can be used as part of pre-post evaluations of medical education initiatives, to complement student self-reflection activities in the curriculum, and to identify students who might benefit from targeted intervention.
Over the past few decades, developing reflective capacity in students has become a core element of many university courses in the health sciences broadly [1,2,3,4,5,6,7], and specifically within medical education [8,9,10,11,12,13,14,15,16,17]. Reflective capacity refers to the ability, desire, and tendency of students to engage in reflective thought during their academic studies and clinical practices. In reflective thought, individuals critically appraise assumptions and beliefs (their own, their colleagues, and their patients) and take an open-minded stance to general problem-solving and interactions with patients. The present study reports medical student responses to the newly developed reflective practice questionnaire (RPQ) .
The RPQ assesses self-reported reflective capacity and can be used to compare across every profession where interactions with clients occur . The RPQ measures multiple aspects of reflective capacity, and additional relevant dimensions such as confidence, uncertainty, stress, desire for improvement, and job satisfaction.
Measures of reflection in medical education
Several self-report questionnaires have been developed to assess reflective thought in medical students and practitioners, with some different areas of focus among the existing instruments. Some scholars focused on the extent of reflection during learning . For example, Kember et al.  designed the questionnaire for reflective thinking by asking university students to rate agreement on statements that referred to their university experience. The questionnaire asks respondents about their level of reflection on their own thoughts, actions, habits, understanding, and growth as individuals in their university courses. In a similar fashion, Sobral  constructed the reflection-in-learning questionnaire to assess self-reported reflection upon learning in a medical program. Sobral’s questionnaire differs from Kember et al.’s approach by focusing more on self-reported extent of reflection upon the meaning and purpose of the to-be-learned content, and upon study habits.
Mamede and colleagues designed a questionnaire focused primarily on reflection during medical diagnostic reasoning [21,22,23,24]. Their questionnaire covers different aspects of reasoning such as an openness towards and tendency to generate multiple alternative hypotheses, willingness to reflect on different hypotheses and question one’s assumptions, and willingness to test one’s hypotheses . Mamede and colleagues’ approach sets itself apart from other approaches by specifically focusing upon reflection during decision making.
Aukes and colleagues developed the Groningen Reflection Ability Scale (GRAS) to evaluate personal reflection in medicine . Aukes et al. describe the GRAS as a unidimensional scale yet also discuss three broad sub-types of questions within the questionnaire – self-reflection, empathetic reflection, and reflective communication . The self-reflection component of this questionnaire contains items that ask about awareness of one’s own thinking and behaviour (e.g., “I am aware of the emotions that influence my thinking”). The empathetic reflection component contains items about empathizing with others (e.g., “I can empathize with someone else’s situation”), although it contains items that do not seem to have high face validity for such a component (e.g., “I am aware of my own limitations”). The reflective communication component contains items about being responsible and accountable for one’s own communication (e.g., “I am accountable for what I say”), and also stubbornness (e.g., “I do not like to have my standpoints discussed”). Both the initial study , and a more recent evaluation , do not support separation of these components of the questionnaire on statistical grounds. Hence, the recommendation to use it as a unidimensional measure of personal reflection. Of the self-report questionnaires mentioned here, the GRAS has the most in common with the reflective practice questionnaire (RPQ) we used in the present study. However, as we will describe below, the RPQ arguably has greater clarity regarding what it measures.
Here we concentrated on self-report measures of reflection since the present study used a self-report questionnaire. However, we acknowledge that other approaches to assessment of reflective capacity have been used in the literature. A popular technique is to qualitatively rate the content of writing on aspects of reflection to assess a person’s level of reflective capacity [8,9,10,11, 27,28,29,30,31,32,33,34,35,36,37]. A wide array of criteria have been applied across studies using this technique and a complete summary is beyond the scope of the present discussion. Other researchers have taken a more structured approach by rating written responses to short written vignettes , videos , or semi-structured exercises [40, 41].
The reflective practice questionnaire (RPQ)
Beyond the use of self-report measures of reflection in medical education, there is a need to foster reflective practice in all healthcare professionals and their students including those in public health, health services management, health administration, dentistry, medicine, nursing, pharmacy, and other disciplines . In response to this shortcoming, and a need to measure reflective practice in all professionals who serve the public, Priddis and Rogers  created the RPQ to assess a wider array of psychological constructs relevant to reflective practice.
The RPQ was developed in response to this call for more comprehensive reflective practice self-report measures. The RPQ sets itself apart from other surveys in several ways. First, it was designed mindfully to use with different professions and gain/yield new insights from comparisons across contexts. Second, the RPQ items focus mostly upon interactions with clients. It is this common reference point that affords the questionnaire to have relevance across different professions. This focus also helps to set the RPQ apart from other self-report measures in the literature. The RPQ contains a 16-item reflective capacity (RC) sub-scale and six other 4-item sub-scales.
The 16-item RC sub-scale has four, 4-item sub-components that together provide a self-reported measure of a respondent’s reflective capacity. Priddis and Rogers  labeled these four sub-components reflection-in-action, reflection-on-action, reflection with others, and self-appraisal. The reflection-in-action and reflection-on-action sub-scales were inspired by the concepts of the same name in the seminal works of Schon [43, 44]. The primary distinction between the two types is that reflection-in-action refers to reflection occurring in the moment whereas reflection-on-action refers to reflection on past events. Hence in the RPQ reflection-in-action items begin “During interactions with clients…” and reflection-on-action items begin “After interaction with clients…”.
While the in-action and on-action distinctions were inspired by Schon, this is where the RPQ affiliation with any specific theorist ends. The RPQ items were designed to be broad and open to interpretation to enable wide application of the survey across different professions, and among people with varying perspectives on reflection. For example, the RPQ reflection-in-action item “During interactions with clients I consider how my personal thoughts and feelings are influencing the interaction” asks the participant to rate their extent of consideration but is not prescriptive regarding what the consideration entails. More specifically, the survey does not force upon the respondent any notions regarding how conscious (deliberate) or non-conscious (intuitive), rational or emotional, systematic or unsystematic, their reflective process should be.
This open stance is consistent with early theorists such as Schon, Dewey, and Wertheimer who were largely descriptive (rather than prescriptive) and very broad regarding what constitutes an act of reflection (for a discussion, see ). We recognize that over the years there have been numerous attempts to be more prescriptive regarding what constitutes an ideal reflective process (for some examples, see [46,47,48,49,50,51,52]). For scholars wishing to use the RPQ to assess extent of reflection from a specific viewpoint one need only introduce an introductory preamble to the survey to influence respondent interpretation of the items. Like Priddis and Rogers , we did not place any perspective on respondents in the present study and allowed them to answer the items interpreting what constitutes reflection as they wished.
The reflection with others sub-component within the RPQ is included in response to the frequent recognition in the literature that reflecting with other people can facilitate insights and understanding [53,54,55,56]. This can occur as fragmented understanding becomes solidified as a co-creation during dialogue, as people form a shared reality [57, 58]. Additionally, already solidified understandings can be modified by feedback from others that enables reconceptualization and helps one to consider ideas from different points of view [59,60,61,62]. The RPQ refers to reflection with “others” to be inclusive of reflection that might occur with peers [63,64,65,66], or in formal supervision [3, 67,68,69]. For a researcher interested in investigating differential benefits of reflection with different types of ‘others’, one need only repeat the sub-component multiple times substituting the word ‘others’ with a specific type of person (e.g., work colleagues, supervisor, friends). Like Priddis and Rogers , we left this open in the present study so that participants would interpret as best suited them.
Inclusion of the RPQ self-appraisal sub-component acknowledges that a tendency for personal reflection and reflection with others likely fosters an increased tendency to reflect upon and question one’s own capabilities for practice [70,71,72]. This is consistent with theorists that promote a cycle of learning such as Kolb , Gibbs , and Argyris and Schon  who emphasize the role of reflection for assessing one’s own strengths, weaknesses, and approaches, thus stimulating self-growth [17, 76]. Priddis and Rogers  reported strong positive correlations among the reflection-in-action, reflection-on-action, reflection with others, and self-appraisal sub-components. Hence we were expecting to observe the same pattern of results in our medical student sample. We then proceeded a step further to conduct a factor analysis on a combination of these sub-components to assess whether averaging across all items in a single RC sub-scale is appropriate.
In addition to four dimensions of reflection, the RPQ incorporates several other attributes relevant to reflective practice – general confidence, desire for improvement, uncertainty, and job satisfaction. Uncertainty is proposed to have an intimate link with reflection. For example, Clara  states that “Reflection consists of giving coherence to a situation that is initially incoherent and unclear” (p. 262). Uncertainty stimulates reflection as one reflects to try and resolve the uncertainty. It is therefore not surprising that Priddis and Rogers  reported positive associations between the RC sub-components and the uncertainty sub-scale of the RPQ. On the other hand, it has been postulated that building practitioner reflective capacity helps to build practitioner desire for continual improvement  and confidence [78, 79]. Increased confidence in one’s work has been shown to be positively related to job satisfaction . Additionally, others have noted that reflective capacity might help practitioners to be more emotionally resilient, and resistant to burnout, with related higher job satisfaction [81,82,83].
Also included in the RPQ are specific sub-scales for confidence and stress that target communication with clients in general and patients in particular. Doctor-patient communication is widely recognized as a fundamental aspect of medical practice [84,85,86,87,88,89]. A lack of patient understanding due to unclear communication can reduce the level of trust between doctor and patient, lower adherence to treatment plans, raise patient uncertainty and anxiety levels, and thus have a negative impact on patient outcomes .
The reason for including a wide array of sub-scales within the RPQ was to provide a succinct self-report instrument of use to scholars wishing to investigate current issues regarding reflective practice . Building reflective capacity is believed to lead practitioners to embrace uncertainty and build confidence. Yet a focus upon uncertainty may instead have a darker side whereby confidence is undermined, and rumination provides a fertile ground for anxiety [90,91,92]. This tension between reflection as a force for good, versus bad, has historically been over-looked within the reflective practice literature.
Additionally, a seemingly paradoxical notion is that if reflection promotes self-growth and confidence, which should reduce uncertainty, then reflection might act to reduce the need for itself over time. A basic tenant of psychological theories of skill acquisition is to emphasize how skills shift from a conscious to unconscious mode of thought as expertise develops . How reflection can be maintained as a force for good (e.g., building confidence and resilience) while maintaining a continual desire for improvement remains an area for future research. The RPQ is also designed to be used in a piecemeal fashion by scholars depending on their specific interests . For example, scholars interested solely in reflection may wish to use only the RC sub-scale of the RPQ.
The present study
Priddis and Rogers studied samples of the general public and mental health practitioners using the RPQ . The present study explores the utility of this new questionnaire with medical students. These students were about to graduate from an American medical school that states its mission to “inspire students to serve with compassion, integrity, and excellence” (http://www.rvu.edu/about/mission-and-vision/). Hence, we expected the self-reported reflective capacity of students, who attempted to live up to this mission, to be higher than the general public sample from Priddis and Rogers . We were uncertain how the medical students would compare with mental health practitioners as all individuals in the latter sample reported reflective capacity building as part of their professional training  whereas the sample of medical students did not have formal reflective capacity building. The mental health practitioner sample reported in this study includes additional responses to increase the sample size from 45 as reported in Priddis and Rogers  to 100 mental health practitioners in the current study.
Our findings with medical students show how the RPQ can be used by medical educators to assess the success of reflective practice education efforts and identify individuals/groups that may benefit from targeted educational intervention. For example, using a cluster analysis, we show that the RPQ can identify students who lack confidence, experience stress when interacting with patients, or avoid reflection.
One hundred fifty medical students from Rocky Vista University, Colorado, US were invited to complete the RPQ about two weeks before graduation and 100 students responded (67% response rate). Two of the responses were incomplete, leaving 98 anonymous responses for the present analysis. The ages and genders of the responding students (average age 28.3 years and 47% female) were similar to the entire class (average age 28.7 years and 44% female). One student’s responses constituted a multivariate outlier and were not included in the analysis. Prior to conducting this research, the Rocky Vista University Institutional Review Board approved the study (exempt category).
For comparison purposes, anonymous responses from 45 Australian mental health practitioners and 188 members of the Australian general public  were reassessed in the present study. Prior to this assessment, 55 new responses were added to the mental health practitioner sample. Members of the general public had been recruited using the survey company, Qualtrics. Mental health professionals were recruited by emailing contacts of one of us (LEP) at a range of organizations.
Participants completed the RPQ shown in section 1 of the online supplement document (Additional file 1). The RPQ was designed for flexible use across different professions by replacing the term ‘clients’ in the original survey with the normative term to describe ‘clients’ of the service . In the present study, we used the term ‘patients’. This 40-item self-report instrument provides measures for reflective capacity via the sub-components reflection-in-action, reflection-on-action, reflection with others, and self-appraisal (described in detail under Background). To improve RPQ reliability as a measure of reflection for the relatively homogeneous sample of medical students, we combined the four sub-components of reflection into a single 16-item subscale termed ‘reflective capacity’ (RC). Together, the four reflection sub-components form an overall measure of RC.
Additional sub-scales form parts of the RPQ that aim to assess a range of other theoretically relevant constructs. A desire for improvement sub-scale gauges one’s inclination to further one’s expertise. Confidence in one’s general ability is measured via a confidence (general) sub-scale, and confidence more specifically related to communication is assessed via a confidence (communication) sub-scale. Uncertainty and stress interacting with patients sub-scales are also included. Finally, the RPQ contains a general job satisfaction sub-scale. All of these sub-scales were found by Priddis and Rogers to correlate with RC . The RPQ uses the 6-point response scale; (1) Not at all, (2) Slightly, (3) Somewhat, (4) Moderately, (5) Very much, (6) Extremely. See online supplement document section 1 associated with this article for a full copy of the RPQ with scoring instructions.
Factor analysis of the RC subscale was conducted using the principal factor method in the statistical program, Stata . Using this method, the factor loadings are computed using the squared multiple correlations as estimates of the communality. Since we obtained a single factor solution, no rotation was applied.
We determined descriptive statistics for the RPQ sub-scales. For comparison purposes, we also included data previously reported by Priddis and Rogers for general public and mental health practitioner samples , and we conducted a series of one-way ANOVAs with follow-up Bonferroni adjusted comparisons to compare the RPQ sub-scales across samples. The ANOVA results are presented in online supplement section 3. For the sake of brevity, we simply report, under results, the main differences that were found. To indicate the importance of differences among sample means, Effect Size values were calculated as Cohen’s d (d = difference between means/pooled standard deviations of the means) . Values of d of about 0.2, 0.5, and 0.8 have negligible, moderate, and crucial practical importance, respectively.
Focusing solely on the medical students, we also examined the inter-relations between RPQ sub-scales and conducted an exploratory cluster analysis to uncover meaningful sub-groups. We used the RPQ scores in a hierarchical agglomerative cluster analysis using weighted average linkage method and using absolute-value distance as the dissimilarity measure. This analysis was conducted using the statistical program, Stata,1 to explore whether meaningful patterns emerged .
Factor analysis of the medical student, mental health practitioner, and general public RC sub-scales revealed single-factor solutions (Table 1). We retained all RC items for comparison purposes because our conclusions were the same regardless of whether the two items with factor loadings below 0.3 were included for medical students. While one of the latter items involves questioning one’s pre-existing beliefs, the other does not. Moreover, another item, concerning the impact of one’s personal thoughts and feelings, loaded well. For these reasons, we suggest items 15 and 16 in Table 1 both loaded poorly because of similarities in their wording rather than an inability of medical students to reflect on their own beliefs.
The RC sub-scale (total of 16 items) showed good internal consistency for the medical student sample (Cronbach’s alpha reliability value = 0.84 with two items having low correlations included, 0.85 without them). The mean medical student RC value was significantly higher than the general public value (d = 0.74, p < .001) but not higher than the mental health practitioner sample (Table 2).
The mean scores for all three samples on each of the other RPQ sub-scale are also presented in Table 2. The Cronbach’s alpha reliability values for other sub-scales of the medical student sample were acceptable to good (i.e., 0.75 to 0.83) (Table 2). The inter-correlations among items for RPQ sub-scales with four items are provided in online supplement document section 2. Overall, medical students reported a strong desire for improvement (mean sub-scale score near 5 or ‘very much’), that was greater than the other two samples (d = 0.61 and 1.48). They were also significantly higher compared to both other samples on the uncertainty sub-scale (d = 0.65 and 1.01). They reported similar levels of general confidence and communication confidence as the mental health practitioner sample but significantly less general confidence than the general public sample (d = 0.78). Their stress interacting with patients was significantly higher than the mental health practitioner sample (d = 0.49). Their job satisfaction was on par with the mental health practitioner sample, and significantly higher than the general public sample (d = 0.71).
The inter-correlations among the RPQ sub-scales are presented for the medical student sample in Table 3. Like results reported by Priddis and Rogers , the RC sub-scale was positively correlated with uncertainty and stress (r = 0.46 and 0.41). Also, RC was positively correlated with desire for improvement (r = 0.43). Uncertainty and Stress were positively associated with desire for improvement (r = 0.41 and 0.25, respectively), and negatively associated with the two confidence sub-scales (r = − 0.22 to − 0.33). Communication confidence was, however, positively associated with RC (r = 0.25). This is important to note because communication confidence has the strongest association with job satisfaction (r = 0.46), as was the case for the public sample from Priddis and Rogers . However, unlike results reported by Priddis and Rogers, we also found a moderate negative association between job satisfaction and stress interacting with patients (r = − 0.42).
To explore individual differences within our sample of medical students we conducted a cluster analysis using the RPQ sub-scale scores. Visual inspection of the dendrogram associated with the cluster analysis revealed five groups as the best classification system for the data. The dendrogram is provided in online supplement document section 4. To assist interpretation of these groups, we provide the mean values for each RPQ sub-scale in Fig. 1.
The largest number of participants (45%) were part of a typical group, with scores on the RPQ that resembled the overall statistics reported in Table 2. Of main interest were the four smaller sub-groups and how they compared with the typical student. One sub-group (23%) we refer to as less engaged. They expressed lower levels of desire for improvement, job satisfaction, and confidence than the typical group. A related sub-group (5%) seemed dissatisfied, as they had much lower job satisfaction than all other sub-groups. They also reported lower confidence in communication than typical students. A third sub-group (19%) we refer to as anxious, because they reported much higher uncertainty and stress interacting with patients than the typical students. The final group (7%) we describe as over-confident, since they reported high confidence in their knowledge, skills, and ability to communicate with patients. They also expressed low uncertainty, stress, desire for improvement, and RC. For statistical analysis of the differences between cluster groups see online supplement section 5.
This is the first study of medical students using the newly-developed RPQ . We compared the mean responses on the RPQ sub-scales from our medical students with a previous general public sample of Priddis and Rogers and an expanded sample of mental health practitioners . Since our medical student sample came from a University that prioritizes compassion and personal excellence, we anticipated their self-reported RC would be relatively high. The medical students rated themselves higher in RC than the general public sample but not higher than the mental health professional sample. Members of the latter sample reported formal reflective training and practice  whereas medical students had no such formal training.
As for findings of Priddis and Rogers , positive associations were found between the RC sub-scale and the uncertainty and stress sub-scales in medical students. Causal implications of this association are unclear. It could be that uncertainty and stress encourage greater reflection, or that greater reflection leaves one at risk of rumination that increases uncertainty and stress. Likely both processes work in concert to some degree and teasing them apart with longitudinal investigations is an intriguing avenue for future research. For example, since uncertainty and stress are both negatively associated with general confidence, perhaps self-efficacy plays a role in these relationships . While associated with stress, RC is also associated with desire for improvement. Rather than warning against reflection, we simply seek to raise awareness of potential risks for increased stress. We recommend that medical educators be mindful of this association and take steps to make their students aware of it.
The right amount of stress stimulates growth and development as professionals, while not enough or too much stress is counterproductive . Excessive stress is not only of concern for general mental health, but also for the role it plays in diminishing a positive outlook on one’s work [96,97,98]. For example, in the present study we found a negative relationship between stress and job satisfaction sub-scales. In the cases of some students, however (e.g., the over-confident ones in the present study), their very low uncertainty and stress and high job satisfaction may not have been optimal. There may be negative consequences associated with the latter combination such as increased chances of risky decision making [99,100,101,102], or an arrogant demeanor offending colleagues and patients [103,104,105].
In addition to the correlation between RC and desire for improvement, RC correlated positively with confidence in communication. The confidence in communication sub-scale in turn was found to have a positive association with the job satisfaction sub-scale, and job satisfaction correlated positively with desire for improvement. These correlations may simply show associations without causal relationships. Alternatively, RC may help a practitioner feel more confident in their interactions with patients which can help foster greater enjoyment of their medical practice. The two sub-scales to associate relatively strongly with job satisfaction were the confidence in communication sub-scale (positive association) and the stress interacting with patients sub-scale (negative association). While much of the literature in medical education acknowledges the benefit of good doctor-patient communication for the patient [84,85,86,87,88], our results also highlight how doctor-patient communication may contribute to the welfare of the doctor. We suggest that medical educators might foster better engagement with lessons on patient communication by stressing the potential benefits for the students themselves (i.e., less stress and more confidence and job satisfaction), in addition to the well-being of their patients.
As part of our analysis of results, we also conducted an exploratory cluster analysis to gain a more nuanced understanding of the cohort of participants. We were inspired by Speelman and McGann who discussed how an over-reliance on mean scores can obscure interesting individual differences present in data . We acknowledge that our cluster analysis is based on a small specific sample of medical students and that there is limited generalizability. However, our purpose was simply to try to better understand the individual differences within our sample, with potential to provide insights which might lead to further research.
Indeed, the cluster analysis revealed some interesting findings. Based on our sample, around 19% of the students fell into a category we refer to as anxious. Compared to typical students (Fig. 1), this sub-group expressed substantially higher uncertainty and stress interacting with patients. In contrast, around 5% of the sample reported much lower job satisfaction and, thus, appeared to be dissatisfied. A related group of about 23% of students were less engaged. They expressed less desire for improvement, low job satisfaction, and less confidence than the more typical group of students. Finally, a sub-group of around 7% of the sample seemed over-confident. This sub-group showed a very high level of confidence in their knowledge, skills, and ability to communicate with patients, and low levels of stress, uncertainty, desire for improvement, and RC. Thus, one use for the RPQ in medical education could be to assist in identifying students who might report concerning levels of anxiety, over-confidence, or dissatisfaction/disengagement with medical practice. This information could help educators make targeted interventions to better address the needs of individual students [107,108,109,110]. Educators must, of course, be careful not to label individual students . RPQ sub-scales are not intended to measure fixed personality characteristics but instead represent malleable capacities.
We studied a single sample of medical students so our results and interpretations should be applied cautiously to other such students. In addition, we compared American medical students to Australian general public and mental health practitioner samples. As research accumulates with the RPQ, more comparisons within and across professions in different countries should help to drive forward understanding of how different disciplines approach education for reflective practice. Another limitation is the self-report nature of the measure, and we recommend that the RPQ be used in conjunction with other reflection activities and forms of peer and teacher feedback on RC. Finally, some of our data (i.e., Table 3) show associations that may or may not reflect causal relationships. For example, perhaps medical students who are satisfied with their work (‘job satisfaction’) are more likely to want to improve to do better for their patients (‘desire for improvement’), or perhaps these are simply associated, without a causal relationship.
The RPQ can be used as a self-report instrument in medical education. It applies to program evaluations, particularly those that aim to improve reflective functioning of practitioners. In addition, the RPQ can be utilized for education where students and practitioners complete the questionnaire as part of their own self-reflection and professional development. Finally, it can be used to identify students who might benefit from targeted intervention to address issues such as anxiety, over-confidence, or dissatisfaction/disengagement with medical practice.
For a detailed description of cluster analysis options using Stata please see: https://www.stata.com/manuals13/mvcluster.pdf
Confidence - communication
Confidence – general
Desire for improvement
Reflective practice questionnaire
Stress interacting with patients
Priddis L, Rogers SL. Development of the reflective practice questionnaire: preliminary findings. Reflective Pract. 2018;19(1):89–104.
Van Winkle LJ, Schwartz BD, Michels N. A model to promote public health by adding evidence-based, empathy-enhancing programs to all undergraduate health-care curricula. Front Public Health. 2017;5:339.
Shea SE, Goldberg S, Weatherston DJ. A community mental health professional development model for the expansion of reflective practice and supervision: evaluation of a pilot training series for infant mental health professionals. Infant Ment Health J. 2016;37:653–69.
Mann K, Gordon J, MacLeod A. Reflection and reflective practice in health professions education: a systematic review. Adv Health Sci Educ. 2009;14(4):595–621.
Dube V, Ducharme F. Nursing reflective practice: an empirical literature review. J Nursing Education and Practice. 2015;5(7):91–9.
Caty M, Kinesella AK, Doyle PC. Reflective practice in speech-language pathology: a scoping review. Int J Speech-Language Pathology. 2015;17(4):411–20.
Nguyen-Truong CKY, Davis A, Spencer C, Rasmor M, Dekker L. Techniques to promote reflective practice and empowered learning. J Nurs Educ. 2018;57(2):115–20.
Van Winkle LJ. The ethics of teaching: Critical thinking and reflection to promote professionalism by mitigating biases including those against other healthcare professions. Frontiers in Pharmacology. 2016;7:56.
Van Winkle LJ, Burdick P, Bjork BC, Chandar N, Green JM, Lynch SM, La Salle S, Viselli SM, Robson C. Critical thinking and reflection on Community Service for a Medical Biochemistry Course Raise Students’ empathy, patient-centered orientation, and examination scores. Medical Science Educator. 2014;24(3):279–90.
Van Winkle LJ, Cornell S, Fjortoft N, Bjork BC, Chandar N, Green JM, La Salle S, Viselli SM, Burdick P, Lynch SM. Critical thinking and reflection exercises in a biochemistry course to improve prospective health professions students' attitudes toward physican-pharmacist collaboration. Am J Pharm Educ. 2013;77(8):169.
Van Winkle LJ, Chandar N, Green JM, Lynch SM, Viselli SM, Burdick P. Does critical reflection by biochemistry learning teams foster patient-centered beliefs among medical students? Medical Science Educator. 2011;21(2):158–68.
Tsingos C, Bosnic-Anticevich S, Smith L. Reflective practice and its implications for pharmacy education. Am J Pharmaceutical Education. 2014;78(1):18.
Aronson L. Twelve tips for teaching reflection at all levels of medical education. Medical Teacher. 2011;33(3):200–5.
Koshy K, Limb C, Gundogan B, Whitehurst K, Jafree DJ. Reflective practice in health care and how to reflect effectively. International Journal of Surgery Oncology. 2017;2(6):e20.
Duke P, Grosseman S, Novack DH, Rosenzwig S. Preserving third year medical students' empathy and enhancing self-reflection using small group "virtual hangout" technology. Medical Teacher. 2015;37(6):566–71.
Lachman N, Pawlina W. Integrating professionalism in early medical education: the theory and application of reflective practice in the anatomy curriculum. Clin Anat. 2006;19:456–60.
Butani L, Bannister SL, Rubin A, Forbes KL. How educators conceptualize and teach reflective practice: a survey of north American pediatric medical educators. Acad Pediatr. 2017;17(3):303–9.
Winkel AF, Yingling S, Jones A, Nicholson J. Reflection as a learning tool in graduate medical education: a systematic review. Journal of Graduate Medical Education. 2017;9(4):430–9.
Kember D, Leung D, Jones A, Loke A, McKay J, Sinclair K, Tse H, Webb C, Wong F, Wong M, et al. Development of a questionnaire to measure the level of reflective thinking. Assess Eval High Educ. 2000;25:381–95.
Sobral DT. Medical students' mindset for reflective learning: a revalidation study of the reflection-in-learning scale. Adv Health Sci Educ. 2005;10:303–14.
Mamede S, Schmidt HG. Correlates of reflective practice in medicine. Advances in Health Sciences Education. 2005;10:327–37.
Mamede S, Schmidt HG, Penaforte JC. Effects of reflective practice on the accuracy of medical diagnoses. Med Educ. 2008;42:468–75.
Mamede S, Schmidt HG. Reflection in medical diagnosis: a literature review. Health Professions Education. 2017;3:15–25.
Mamede S, Schmidt HG. The structure of reflective practice in medicine. Med Educ. 2004;38:1302–8.
Aukes LC, Geertsma J, Cohen-Schotanus J, Zwierstra RP, Slaets JPJ. The development of a scale to measure personal reflection in medical practice and education. Medical Teacher. 2007;29(2–3):177–82.
Anderson NB, O'Neill L, Gormsen LK, Hvidberg L, Morcke AM. A validation study of the psychometric properties of the Gronigen reflection ability scale. BMC Medical Education. 2014;14(1):214.
Wong F, Kember D, Chung L, Yan L. Assessing the level of student reflection from reflective journals. J Adv Nurs. 1995;22(1):48–57.
Kember D, Jones A, Loke A, McKay J, Sinclair K, Tse H, Webb C, Wong F, Wong M, Yeung E. Determining the level of reflective thinking from students' written journals using a coding scheme based on the work of Mezirow. International Journal of Lifelong Education. 1999;18(1):18–30.
Williams RM, Wessel J. Reflective journal writing to obtain student feedback about their learning during the study of chronic musculoskeletal conditions. J Allied Health. 2004;33(1):17–23.
Levine RB, Kern DE, Wright SM. The impact of prompted narrative writing during internship on reflective practice: a qualitative study. Adv Health Sci Educ. 2008;13(5):723–33.
Kanthan R, Senger JL. An appraisal of students' awareness of “self-reflection” in a first-year pathology course of undergraduate medical/dental education. BMC Medical Education. 2011;11:67.
Wald HS, Borkan JM, Taylor JS, Anthony D, Reis SP. Fostering and evaluating reflective capacity in medical education: developing the REFLECT rubric for assessing refelctive writing. Acad Med. 2012;87(1):41–50.
Liao HC, Wang Y. The application of heterogeneous cluster grouping to reflective writing for medical humanities literature study to enhance students' empathy, critical thinking, and reflective writing. BMC Medical Education. 2016;16:234.
Borgstrom E, Morris R, Wood D, Cohn S, Barclay S. Learning to care: medical students' reported value and evaluation of palliative care teaching involving meeting paitents and reflective writing. BMC Medical Education. 2016;16:306.
Roji R, Noguera-Tejedor A, Pikabea-Diaz F, Carrasco JM, Centeno C. Palliative care bedside teaching: a qualitative analysis of medical students' reflective writings after clinical practices. J Palliat Med. 2017;20(2):147–54.
Newcomb M, Burton J, Edwards N. Pretending to be authentic: challenges for students when reflective writing about their childhood for assessment. Reflective Pract. 2018;19(3):333–44.
Epp S. The value of reflective journaling in undergraduate nursing education: a literature review. Int J Nurs Stud. 2008;45:1379–88.
Boenink AD, Oderwald AK, De Jonge P, Van Tilburg W, Smal JA. Assessing student reflection in medical practice. The development of an observer-rated instrument: reliability, validity and initial experiences. Med Educ. 2004;38(4):368–77.
Koole S, Dorman T, Aper L, De Wever B, Scherpbier A, Valcke M, Cohen-Schotanus J, Derese A. Using video-cases to assess student reflection: development and validation of an instrument. BMC Medical Education. 2012;12:22.
Learman LA, Autry AM, O-Sullivan P: Reliability and validity of reflection exercises for obstetrics and gynecology residents. Am J of Obstetrics and Gynecology 2008, 198(4):461.e461–461.e410.
Khanum Z. Effectiveness of reflective exercises for obstetrics and Gynaecological residents. J College of Physicians and Surgeons Pakistan. 2013;23(7):468–71.
Van Winkle LJ, Schwartz A, Horst A, Michels N. An evidence-based model program to foster empathy, mitigate bias, and promote wellbeing through critical reflection on service-learning by public health/health administration and practitioner students teams. J Health Administration Education. 2018;35(4):475–90.
Schon D. The reflective practitioner: how professionals think in action. New York, NY: Basic Books; 1983.
Schon D. Educating the reflective practitioner: toward a new design for teaching and learning in the professions. San Francisco, CA: Jossey-Bass; 1987.
Clara M. What is reflection? Looking for clarity in an ambiguous notion. J Teach Educ. 2015;66(3):261–71.
Weatherston DJ, Kaplan-Estrin M, Goldberg S. Strengthening and recognizing knowledge, skills, and reflective practice: the Michigan Association for Infant Mental Health competency guidelines and endorsement process. Infant Ment Health J. 2009;30(6):648–63.
Jordan S. Learning to be surprised: how to foster reflective practice in a high-reliability context. Manag Learn. 2010;41(4):391–413.
Aronson L, Niehaus B, Hill-Sakurai L, Lai C, O'Sullivan PSO. A comparison of two methods of teaching reflective ability in year 3 medical students. Med Educ. 2012;46:807–14.
Olteanu C. Reflection and the object of learning. International Journal for Lesson and Learning Studies. 2016;5(1):60–75.
Ferreira JF, Basseches M, Vasco AB. Guidelines for reflective practice in psychotherapy: a reflection on the benefits of combining moment-by-moment and phase-by-phase mapping in clinical decision making. J Psychother Integr. 2017;27(1):35–46.
Grant A, McKimm J, Murphy F: Developing reflective practice: a guide for medical students, doctors and teachers. Oxford, UK: John Wiley & Sons; 2017.
Gray TW, Coombs CJ. Developing professonal judgment in surgical trainees: the role of critical reflection. Australasian Journal of Plastic Surgery. 2018;1(1):126-132.
Collin S, Karsenti T, Komis V. Reflective practice in initial teacher training: critiques and perspectives. Reflective Pract. 2013;14(1):104–17.
Beauchamp C. Reflection in teacher education: issues emerging from a review of current literature. Reflective Pract. 2015;16(1):123–41.
Nehring J, Laboy WT, Catarius L. Connecting reflective practice, dialogic protocols, and professional learning. Prof Dev Educ. 2010;36(3):399–420.
Dohn NB. On the epistemological presuppositions of reflective activities. Educ Theory. 2011;61(6):671–708.
Echterhoff G, Higgins ET. Creating shared reality in interpersonal and intergroup communication: the role of epistemic processes and their interplay. Eur Rev Soc Psychol. 2017;28:175–226.
Rossignac-Milon M, Higgins ET. Beyond intrapersonal cognitive consistency: shared reality and the interpersonal motivation for truth. Psychol Inq. 2018;29(2):86–93.
Goldfried MR, Davila J. The role of relationship and technique in therapeutic change. Psychother Theory Res Pract Train. 2005;42(4):421–30.
Leder G. Know thyself? Questioning the theoretical foundations of cognitive behavioral therapy. Rev Philos Psychol. 2017;8(2):391–410.
Padesky CA. Schema change processes in cognitive therapy. Clinical Psychology and Psychotherapy. 1994;1(5):267–78.
Donaghy ME, Morss K. Guided reflection: a framework to facilitate and assess reflective practice within the discipline of physiotherapy. Physiotherapy Theory and Practice. 2000;16:3–14.
Wen CC, Lin MJ, Lin CW, Chu SY. Exploratory study of the characteristics of feedback in the reflective dialogue group given to medical students in a clinical clerkship. Medical Education Online. 2015;20:25965.
Behizadeh N, Thomas C, Cross SB. Reframing for social justice: the influence of critical friendship groups on preservice teachers' reflective practice. J Teach Educ. 2017;0022487117737306. https://doi.org/10.1177/0022487117737306.
Bindels E, Verberg C, Scherpbier A, Heeneman S, Lombarts K. Reflection revisited: how physicians conceptualize and experience reflection in professional practice - a qualitative study. BMC Medical Education. 2018;18:105.
Banerjee A, Slagle JM, Mercaldo ND, Booker R, Miller A, France DJ, Rawn L, Weinger MB. A simulation-based curriculum to introduce key teamwork principles to entering medical students. BMC Medical Education. 2016;16:295.
Martin P, Kumar S, Lizarondo L. When I say... clinical supervision. Med Educ. 2017;51(9):890–1.
Gates NJ, Sendiack CI. Neuropsychology supervision: incorporating reflective practice. Aust Psychol. 2017;52:191–7.
Foong LYY, Nor MBM, Nolan A. The influence of practicum supervisors' facilitation styles on student teachers' reflective thinking during collective reflection. Reflective Pract. 2018;19(2):225–42.
Engels PT, de Gara C. Learning styles of medical students, general surgery residents, and general surgeons: implications for surgical education. BMC Medical Education. 2010;10:51.
Ganni S, Botden SMBI, Schaap DP, Verhoeven BH, Goossens RHM, Jakimowicz JJ. Reflection-before-practice improves self-assessment and end-performance in laparoscopic surgical skills training. J Surgical Education. 2018;75(2):527–33.
Muir F, Scott M, McConville K, Watson K, Behbehani K, Sukkar F. Taking the learning beyond the individual: how reflection informs change in practice. Int J Med Educ. 2014;5:24–30.
Kolb DA. Experiential learning: experience as the source of learning and development. New Jersey: Prentice Hall; 1984.
Gibbs G. Learning by doing: a guide to teaching and learning methods. Oxford Polytechnic: Oxford: Further Education Unit; 1988.
Argyris M, Schon D. Theory in practice: increasing professional effectiveness. San Francisco: Jossey-Bass; 1974.
Maudsley G, Strivens J. Promoting professional knowledge, experiential learning and critical thinking for medical students. Med Educ. 2000;34:535–44.
Frankford DM, Patterson MA, Konrad TR. Transforming practice organizations to foster lifelong learning and commitment to medical professionalism. Acad Med. 2000;75(7):708–17.
McMahon A, Hevey D. "it has taken me a long time to get to this point of quiet confidence": what contributes to therapeutic confidence for clinical psychologists? Clin Psychol. 2017;21:195–205.
Lestander O, Lehto N, Engstrom A. Nursing students' perceptions of learning after high fidelity simulation: effects of a three-step post-simulation reflection model. Nurse Educ Today. 2016;40:219–24.
Maggiori C, Johnston CS, Rossier J. Contribution of personality, job strain, and occupational self-efficacy to job satisfaction in different occupational contexts. J Career Dev. 2016;43(3):244–59.
Weng HC, Hung CM, Liu YT, Cheng YJ, Yen CY, Chang CC, Huang CK. Associations between emotional intelligence and doctor burnout, job satisfaction and patient satisfaction. Med Educ. 2011;45:835–42.
Wald HS, Anthony D, Hutchinson TA, Liben S, Smilovitch M, Donato AA. Professional identity formation in medical education for humanistic, resilient physicians: pedagogic strategies for bringing theory to practice. Acad Med. 2015;90(6):753–60.
Wald HS. Professional identity (trans)formation in medical education: reflection, relationship, resilience. Acad Med. 2015;90(6):701–6.
Ong LML, De Haes JCJM, Hoos AM, Lammes FB. Doctor-patient communication: a review of the literature. Soc Sci Med. 1995;40(7):903–18.
Von Fragstein M, Silverman J, Cushing A, Quilligan S, Salisbury H, Wiskin C. Education UCfCCSTiUM: UK consensus statement on the content of communication curricula in undergraduate medical education. Med Educ. 2008;42(11):1100–7.
Pryor E, Henry M, Christie J. Developing clinical communication skills: targeting the postgraduate years. Med Educ. 2013;47(11):1141–1.
Salmon P, Young B. Creativity in clinical communication: from communication skills to skilled communication. Med Educ. 2011;45(3):217–26.
Kaplonyi J, Bowles K-A, Nestel D, Kiegaldie D, Maloney S, Haines T, Williams C. Understanding the impact of simulated patients on health care learners' communication skills: a systematic review. Med Educ. 2017;51(12):1209–19.
Zachariae R, O'Connor M, Lassesen B, Olesen M, Kjaer LB, Thygesen M, Morcke AM. The self-efficacy in patient-centeredness questionnaire - a new measure of medical student and physician confidence in exhibiting patient-centered behaviors. BMC Medical Education. 2015;15:150.
Takano K, Tanno Y. Self-rumination, self-reflection, and depression: self-rumination counteracts the adaptive effect of self-reflection. Behav Res Ther. 2009;47(3):260–4.
Kross E. When the self becomes other: toward an integrative understanding of the processes distinguishing adaptive self-reflection from rumination. Ann N Y Acad Sci. 2009;1167(1):35–40.
Lengelle R, Luken T, Meijers F. Is self-reflection dangerous? Preventing rumination in career learning. Aust J Career Dev. 2016;25(3):99–109.
Tenison C, Fincham JM, Anderson JR. Phases of learning: how skill acquisition impacts cognitive processing. Cogn Psychol. 2016;87:1–28.
Hamilton LC. Statistics with STATA. Cengage: Boston, USA; 2013.
Sapolsky RM. Stress and the brain: individual variability and the inverted-U. Nat Neurosci. 2015;18:1344–6.
Williams ES, Konrad TR, Scheckler WE, Pathman DE, Linzer M, McMurray JE, Gerrity M, Schwartz M. Understanding physicians' intentions to withdraw from practice: the role of job satisfaction, job stress, mental and physical health. Health Care Manag Rev. 2010;35(2):105–15.
Dewa CS, Loong D, Bonato S, Thanh NX, Jacobs P. How does burnout affect physician productivity? A systematic literature review. BMC Health Serv Res. 2014;14(1):325.
Shanafelt TD, Mungo M, Schmitgen J, Storz KA, Reeves D, Hayes SN, Sloan JA, Swenseon SJ, Buskirk SJ. Longitudinal study evaluating the association between physician burnout and changes in professional work effort. Mayo Clin Proc. 2016;91(4):422–31.
Dawson NV, Connors AF, Speroff T, Kemka A, Shaw P, Arkes HR. Hemodynamic assessment in managing the critically lll: is physician confidence warranted? Med Decis Mak. 1993;13(3):258–66.
Dawson NV, Arkes HR. Systematic errors in medical decision making. J Gen Intern Med. 1987;2(3):183–7.
Arkes HR. The consequences of the hindsight bias in medical decision making. Curr Dir Psychol Sci. 2013;22(5):356–60.
Berner ES, Graber ML. Overconfidence as a cause of diagnostic error in medicine. Am J Med. 2008;121(5):S2–S23.
McKimm J, Wilkinson T. Doctors on the move: exploring professionalism in the light of cultural transitions. Medical Teacher. 2015;37(9):837–43.
Lopez A, Detz A, Ratanawongsa N, Sarkar U. What patients say about their doctors online: a qualitative content analysis. J Gen Intern Med. 2012;27(6):685–92.
Bendapudi NM, Berry LL, Frey KA, Parish JT, Rayburn WL. Patients' perspectives on ideal physician behaviors. Mayo Clin Proc. 2006;81(3):338–44.
Speelman CP, McGann M. How mean is the mean? Frontiers in Psychology. 2013;4:451.
Chen F, Lui AM, Martinelli SM. A systematic review of the effectiveness of flipped classrooms in medical education. Med Educ. 2017;51(6):585–97.
Dobkin PL, Balass S. Multiple influences contribute to medical students' well-being and identity formation. Med Educ. 2014;48(4):340–2.
Martimianakis MA, Hafferty FW. Exploring the interstitial space between the ideal and the practised: humanism and the hidden curriculum of system reform. Med Educ. 2016;50(3):278–80.
de Bruin ABH, Dunlosky J, Cavalcanti RB. Monitoring and regulation of learning in medical education: the need for predictive cues. Med Educ. 2017;51(6):575–84.
Cherry MG, Flecther I, O-Sullivan H, Dornan T. Emotional intelligence in medical education: a critical review. Med Educ. 2014;48(5):468–78.
Availability of data and materials
All data generated or analyzed during this study are included in this published article [and its supplementary information files].
Ethics approval and consent to participate
Participants were informed that their optional and anonymous survey results would be used in this study. HIPAA permits human research without obtaining informed consent (called Authorization by HIPAA). In order to do this, the research must be reviewed and approved by a duly established Institutional Review Board (IRB). This study was reviewed and approved by the Rocky Vista University IRB. The IRB found that the research would not adversely affect individual privacy rights and welfare. There is no way to associate any participant with their survey results.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Online supplement document for article: Applications of the reflective practice questionnaire in medical education. This document contains five sections: Section 1. The RPQ as used in the present study, pages 2 – 3. Section 2. Inter-correlations among items for RPQ sub-scales with four items, page 4. Section 3. Comparison across RPQ sub-scales for different groups, page 5. Section 4. Cluster analysis dendrogram, page 6. Section 5. Statistical comparison of the cluster analysis groups, page 7. (DOCX 101 kb)