Pediatric resident and faculty attitudes toward self-assessment and self-directed learning: a cross-sectional study
BMC Medical Education volume 9, Article number: 16 (2009)
The development of self-assessment and self-directed learning skills is essential to lifelong learning and becoming an effective physician. Pediatric residents in the United States are now required to use Individualized Learning Plans (ILPs) to document self-assessment and self-directed learning. A better understanding of resident and faculty attitudes and skills about self-assessment and self-directed learning will allow more successful integration of lifelong learning into residency education. The objective of this study was to compare faculty and resident attitudes, knowledge and skills about self-assessment, self-directed learning and ILPs.
Survey of pediatric residents and faculty at a single institution. Respondents rated their attitudes, knowledge, and self-perceived skills surrounding self-assessment, self-directed learning and ILPs.
Overall survey response rate was 81% (79/97); 100% (36/36) residents and 70% (43/61) faculty. Residents and faculty agreed that lifelong learning is a necessary part of being a physician. Both groups were comfortable with assessing their own strengths and weaknesses and developing specific goals to improve their own performance. However, residents were less likely than faculty to continuously assess their own performance (44% vs. 81%; p < 0.001) or continuously direct their own learning (53% vs. 86%; p < 0.001). Residents were more likely than faculty to believe that residents should be primarily responsible for directing their own learning (64% vs. 19%; p < 0.0001), but at the same time, more residents believed that assigned clinical (31% vs. 0%; p < 0.0001) or curricular (31% vs. 0%; p < 0.0001) experiences were sufficient to make them competent physicians. Interns were less likely than senior residents to have a good understanding of how to assess their own skills (8% vs. 58%; p = 0.004) or what it means to be a self-directed learner (50% vs. 83%; p = 0.04).
Qualitative comments indicated that while ILPs have the potential to help learners develop individualized, goal-directed learning plans based on strengths and weaknesses, successful implementation will require dedicated time and resident and faculty development.
These findings suggest that training and experience are necessary for physicians to understand the role of self-directed learning in education. Deliberate practice, for example by requiring residents to use ILPs, may facilitate self-directed, lifelong learning.
There is broad consensus that the development of self-assessment and self-directed learning skills is essential to lifelong learning and a critical step towards becoming an effective physician. Lifelong learning is considered by many to be an integral component of professionalism [2–4]. In 2002, the American Board of Internal Medicine Foundation, the American College of Physicians Foundation, and the European Federation of Internal Medicine published a physician charter defining medical professionalism for the new millennium, which included the statement that "physicians must be committed to lifelong learning"[2, 3]. This charter was endorsed by over 120 national and international organizations across the world and was translated into 10 languages[4, 5]. Documentation of lifelong learning is now required for residency training, board certification and maintenance of certification in the United States, Canada, Australia, New Zealand and many European countries [6–8]. Furthermore, in the United States, the Review Committee for pediatrics of the Accreditation Council for Graduate Medical Education now requires all training programs to use Individualized Learning Plans (ILPs) to document resident self-assessment and self-directed learning as part of the practice-based learning and improvement competency.
Despite the importance of self-assessment to lifelong learning, previous studies have shown that physicians have limited ability to self-assess. In addition, those who were the least skilled and the most overconfident were also the least accurate when self-assessment was compared with external assessment (e.g. faculty evaluations, multisource feedback, observed interactions, written examinations) [10–12]. Physicians' limited ability to self-assess makes development of these skills during training important. However, relatively little is known about how best to teach the components of lifelong learning: self-assessment and self-directed learning. Even less is known about how ILPs, now required for pediatric residents in the United States, can help in that process.
Developing a better understanding of how to teach and learn these self-assessment and self-directed learning skills is important, because producing physicians that are better lifelong learners may improve the quality of care that patients receive. To begin to address this issue, our study sought to understand resident and faculty attitudes, knowledge, and skills surrounding self-assessment, self-directed learning and ILPs prior to implementation of ILPs in our pediatric residency program. This is important because successful implementation of ILPs depends on (1) understanding baseline attitudes, knowledge, and skills of residents and faculty toward self-assessment and self-directed learning and (2) identifying potential resident and faculty biases toward the overall ILP process. We hypothesized that residents and faculty have different attitudes and skills related to self-assessment and self-directed learning. Developing a better understanding of these differences may allow for more effective integration of lifelong learning strategies into residency education and ultimately result in improved lifelong learning in practicing physicians.
This study was performed prior to the introduction of ILPs in our program. A paper-based survey was distributed to all pediatric residents in mid-July 2007. A web-based survey was sent to all pediatric faculty in late July 2007. Reminder emails were periodically sent out until the survey closed in August 2007.
The survey asked respondents to rate their attitudes, knowledge, and self-perceived skills surrounding self-assessment, self-directed learning, and ILPs on a 6-point Likert scale, with 1 indicating strongly disagree and 6 indicating strongly agree. In addition, residents were asked their gender, year of residency, area of pediatrics they were interested in (general pediatrics, subspecialty, undecided), and future intended practice setting (academic, community, undecided). Faculty was asked whether they were currently an advisor for a resident, their gender, their area of pediatrics (general pediatrics, subspecialty), number of years ago they completed fellowship/residency training, and distribution of their time (clinical, teaching (not clinical), research, administration).
The University of California, Davis Institutional Review Board approved this study.
In all survey questions, the 6 point Likert scale was dichotomized by collapsing the strongly agree and agree responses into one category and all other responses into a second category. Univariate analysis was used to describe the self-perceived attitudes, knowledge, and skills about self-assessment and self-directed learning. Bivariate analyses were performed using Pearson's chi-square. Our primary analysis compared resident and faculty agreement (strongly agree, agree) with the statements in the survey. In order to explore differences based on level of training, a sub analysis was performed comparing first-year resident responses to senior (second- and third-year) resident responses.
All statistical analysis was conducted using STATA 8.0 statistical software.
Two authors (STL and MAF) reviewed the free text responses on the survey and independently identified themes. Data were collected as part of a purposeful sampling and manually coded into categories using inductive analyses. Coding discrepancies of the identified themes were resolved by consensus discussion.
The overall response rate was 81% (79/98), 100% (36/36) of residents and 70% (43/61) of faculty. A total of 12 first-year residents and 24 senior residents (12 second-year and 12 third-year residents) responded. Table 1 displays the demographic data of the respondents and all faculty. Faculty respondents and non-respondents were similar in gender, specialty area, and whether or not they were currently faculty advisors. Compared with residents, faculty was more likely to be male, a subspecialist (or planning subspecialty training), and practice in an academic setting (or planning to practice in an academic setting).
Table 2 compares resident and faculty responses. Residents and faculty responses were similar in many areas. Both groups agreed that lifelong learning is necessary to being a physician. They also agreed that self-assessment and self-directed learning improves patient care. Residents and faculty were equally comfortable assessing their own areas of strengths and areas for improvement and developing specific goals to improve their own performance. Residents were less likely than faculty to continuously assess their own performance (44% vs. 81%; p < 0.001) or continuously direct their own learning (53% vs. 86%; p < 0.001).
Residents were more likely than faculty to feel that residents should be primarily responsible for directing their own learning (64% vs. 19%; p < 0.0001). Residents were also more likely to believe that assigned clinical experiences (rotations, clinics) (31% vs. 0%; p < 0.0001) or assigned curricular experiences (lectures, didactics, readings) (31% vs. 0%; p < 0.0001) were sufficient to make residents competent physicians.
Residents and faculty were more confident in their ability to identify their individual strengths and weaknesses (44–65%) and write specific goals to improve their own performance (47–51%), but were less confident in their ability to write an effective ILP to improve their own performance (25–28%).
First-year vs. senior residents
When first-year residents (n = 12) were compared to senior (second- and third-year) residents (n = 24), both groups believed that lifelong learning is necessary to being a physician. In addition, both first-year and senior residents believed that residents should be primarily responsible for directing their own learning and that assigned clinical (33% vs. 29%; p = 0.80) and curricular (42% vs. 25%; p = 0.31) experiences were sufficient to make them competent physicians. However, first-year residents were less likely to have a good understanding of how to assess their own skills (8% vs. 43%; p = 0.004) or what it means to be a self-directed learner (50% vs. 83%; p = 0.0.04). In addition, first-year residents were less confident in their ability to assess their own performance (8% vs. 54%; p = 0.008). While interns trended toward being less confident in their ability to write specific goals to improve their performance (25% vs. 58%; p = 0.20), the difference was not statistically significant. First-year residents were less likely than senior residents to continuously assess their own performance (17% vs. 53%; p = 0.02), but were equally likely to continuously direct their own learning (42% vs. 53%; p = 0.35).
Table 3 shows that residents and faculty identified several similar advantages to developing an ILP. These included: focused, goal-directed learning plan, "focus, will actually have goals," learning plan based on improvement through identification of learner strengths and weaknesses, "it allows you to pinpoint your areas of weakness and strength and ... create a plan to address your weaknesses," individualized learning, "it will be targeted to the individual's style of learning ..." and accountability, "more accountability." Some faculty additionally commented on the potential benefit of ILPs to faculty through: "focused expectations for resident and instructor," and "identifying areas of strength and of goals for improvement in collaboration between residents and faculty."
In addition, Table 4 shows that both residents and faculty identified similar areas of challenge. Both groups expressed concern over the amount of time involved to "develop" and "implement" the plan. Residents expressed concern over "finding time in our schedule to follow our plan" and how this time could be better utilized, as it might "take time away from reading." Faculty expressed concern over faculty time to "develop and monitor" the ILP. Another challenge that was identified by both groups involved insufficient understanding of how to construct an effective ILP. This was evidenced by commentary noting a lack of "knowing exactly what to include," and lack of "adequate understanding of strengths/weaknesses and goals that should be achieved." Both groups also remarked on "follow through" as an area of challenge. In addition, residents expressed concern about "maintain [ing] the same goals/plans no matter the rotation," while faculty commented on resident's "motivation to develop it [ILP] and desire to make changes."
We found that residents and faculty agreed that lifelong learning is necessary to being an effective physician. However, a majority of residents, but a minority of faculty, believed that residents should be primarily responsible for directing their own learning. At the same time, a third of residents, but no faculty, thought that assigned clinical and curricular experiences were sufficient to make an individual a competent physician. Because these views are inconsistent, and the fact that residents believed this and faculty did not, suggests the possibility that the training received and experience gained during residency training and beyond are important in developing a full understanding of how self-directed learning fits into practice. This finding also underscores the importance of fostering the ongoing development of self-assessment and self-directed learning skills throughout the continuum of medical education.
Only half the residents and faculty had a good understanding of how to assess their own skills or were confident in their ability to do so. In addition, first-year residents were less likely than senior residents to have a good understanding of how to assess their own skills and were less confident in their ability to do so. The challenge of "determining one's own strengths and weaknesses" was also voiced in the qualitative comments, and is consistent with previous studies that have shown that physicians have limited ability to self-assess. Fortunately, previous studies have shown that physician self-assessment skills can be improved through performance feedback. When fourth-year medical students were shown their standardized patient ratings and how their performance compared with that of other students, their self-assessment of their clinical skills improved. In addition, studies of portfolios have indicated that reflection is enhanced with adequate faculty mentorship[15, 16]. Our findings, and those of these other studies suggest it would be useful to design ILPs in a way that requires residents to pair their own self-evaluation with external evaluations (such as multisource feedback) with the help of a faculty advisor in order to further develop self-assessment skills.
Our study also showed that residents were less confident in their ability to develop effective ILPs than they were in their ability to assess their own skills and develop specific goals to improve performance. This discrepancy is highlighted in our qualitative data, in which residents describe challenges to developing an ILP that include "creating a realistic plan" to accomplish their specific goals, "adequate time and motivation to really focus on it," and "follow [ing] through with the plan." These comments suggest that residents acknowledge that the challenge of self-directed learning is not just assessing weaknesses and setting specific goals to improve those weaknesses, but developing and implementing realistic plans to accomplish these goals. This commentary suggests that in order for ILPs to be effective, faculty should try to assist learners with the development and implementation of realistic plans.
Residents were less likely than faculty to continuously assess their own performance or continuously direct their own learning even though survey results demonstrated that they felt primarily responsible for their own learning. Our findings support previous reports that suggest that physicians-in-training may not be well-prepared to self-direct their learning[17, 18]. For example, when third-year medical students in a family medicine clerkship were given the opportunity to choose their own learning goals, most selected from prewritten learning goals rather than develop their own. In a pilot study of ILPs in a pediatric continuity clinic, most (90%) residents were able to develop learning goals, but few (25%) documented progress toward achieving their goals. Our results and the results of these previous studies suggest the need to develop tools to improve self-directed learning skills. Exposure to problem-based learning curricula[19, 20] has been shown to improve self-directed learning in pediatric residents, but other tools to promote self-directed learning, such as ILPs, have not been well studied.
Our study has several limitations and identified several potential barriers to implementation of ILPs that must be considered. The survey was administered at a single institution; therefore, generalizability is limited because it remains possible that residents and faculty at other institutions and other specialties may have different attitudes toward physician self-assessment and self-directed learning. However, our institution is similar to other academic medical centers in that faculty have trained in a broad representation of training programs and residents come from a wide range of medical schools throughout the United States. The demographic differences we found between residents and faculty (practice setting, subspecialty, gender) in our study may be a confounding factor, but our study had insufficient power to test this possibility. By definition, all faculty practice in an academic institution, most academic pediatricians are subspecialists, and until recently, there were fewer women in pediatrics (and medicine, in general)[22, 23]. A larger study will need to be done in order to more closely examine these factors. Faculty who responded to the survey may be different from faculty who did not respond to the survey. However, demographics of faculty respondents and all faculty were similar. Finally, in our survey we discovered that some faculty and residents did not have a good understanding of what an ILP was. In addition, many residents were unsure how to write an effective ILP and faculty were unsure of their ability to help them. While these findings may be due to the fact that the survey was distributed prior to initiation of ILPs, they underscore the need to provide appropriate training for faculty and residents prior to implementation of an ILP.
Our paper describes initial attitudes, knowledge and skills surrounding self-assessment, self-directed learning and ILPs prior to the implementation of ILPs in our program. Our study suggests that self-assessment and self-directed learning are not innate skills and need to be developed over the course of training. While ILPs may be a tool to improve self-assessment and self-directed learning skills through deliberate practice, our study suggests that several challenges will need to be addressed before ILPs can be successfully implemented. These include: assisting residents and faculty in viewing ILPs as a valuable tool to improve lifelong learning, not merely a "time-consuming" exercise; building flexibility into ILPs in order to meet the evolving goals of learners with different learning styles; and better integrating ILPs into residency training. Ideally, dedicated time would be provided to develop and implement ILPs. Resident and faculty development is needed on how to construct an effective ILP. Such training would most likely involve how to (1) compare external assessment with self-assessment to identify strengths and weaknesses, (2) write specific goals focused on individual improvement, and (3) construct realistic plans to achieve those goals. ILPs could be actively integrated into clinical rotations by having residents share their learning goals with attendings at the beginning of each clinical rotation, so that learning opportunities surrounding specific learning goals could be optimized and learning would become a more "collaborative" partnership.
Further research is needed to better understand how best to integrate ILPs into graduate medical education and whether ILPs help residents develop self-assessment and self-directed learning skills. A repeat survey after implementation of ILPs may help determine whether actual implementation of ILPs affects resident attitudes, knowledge or skills surrounding self-assessment, self-directed learning and ILPs. A national survey of residents at multiple training programs could validate the findings from our study and improve its generalizability. Research is needed on the efficacy of ILPs to improve lifelong learning and best implementation strategies if ILPs are to be most effective.
In conclusion, our study provides additional evidence that self-assessment and self-directed learning skills are not innately present in physicians; rather these skills are developed over the course of training and development of experience. Incorporation of curricular activities to support development of self-assessment and self-directed learning skills into both undergraduate and graduate medical education is important to the development of lifelong learners. Further study of the efficacy of ILPs and other tools to improve self-assessment and self-directed learning skills is warranted. Effective implementation of ILPs involves faculty development that is focused on fostering and sustaining self-directed learning, and determining the most effective manner of utilizing ILPs. Because lifelong learning is essential to the practice of medicine, medical education curricula must promote the development of self-assessment and self-directed learning skills by integrating the deliberate practice of self-assessment and self-directed learning into residency training.
Individualized Learning Plan
Miller SH: American Board of Medical Specialties and Repositioning for Excellence in Lifelong Learning: Maintenance of Certification. Journal of Continuing Education in the Health Professions. 2005, 25 (3): 151-156. 10.1002/chp.22.
Medical professionalism in the new millennium: a physician charter. Ann Intern Med. 2002, 136 (3): 243-246.
Medical professionalism in the new millennium: a physicians' charter. Lancet. 2002, 359 (9305): 520-522. 10.1016/S0140-6736(02)07684-5.
Blank L, Kimball H, McDonald W, Merino J: Medical professionalism in the new millennium: a physician charter 15 months later. Ann Intern Med. 2003, 138 (10): 839-841.
Physician charter on medical professionalism endorsed by Chinese Doctors Medical Association. [http://www.abimfoundation.org/pressroom/pdf_press_release/CMDA_062105.pdf]
ACGME Common Program Requirements, effective July 1, 2007. [http://www.acgme.org/acWebsite/dutyHours/dh_dutyhoursCommonPR07012007.pdf]
American Board of Medical Specialties' Maintenance of Certification. [http://www.abms.org/Maintenance_of_Certification/MOC_competencies.aspx]
Merkur S, Mladovsky P, Mossialos E, McKee M: Do lifelong learning and revalidation ensure that physicians are fit to practise?. World Health Organization European Ministerial Conference on Health Systems: 25–27 June 2008 2008; Tallin, Estonia. 2008, World Health Organization, [http://www.euro.who.int/document/hsm/9_hsc08_epb_12.pdf]
ACGME Program Requirements for Graduate Medical Education in Pediatrics, effective July 1, 2007. [http://www.acgme.org/acWebsite/downloads/RRC_progReq/320_pediatrics_07012007.pdf]
Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L: Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. Jama. 2006, 296 (9): 1094-1102. 10.1001/jama.296.9.1094.
Violato C, Lockyer J: Self and peer assessment of pediatricians, psychiatrists and medicine specialists: Implications for self-directed learning. Adv Health Sci Educ Theory Pract. 2006, 11 (3): 235-244. 10.1007/s10459-005-5639-0.
Langendyk V: Not knowing that they do not know: self-assessment accuracy of third-year medical students. Med Educ. 2006, 40 (2): 173-179. 10.1111/j.1365-2929.2005.02372.x.
Statacorp: Stata Statistical Software: Release 8.0. 2003, College Station, TX: Stata Corporation
Srinivasan M, Hauer KE, Der-Martirosian C, Wilkes M, Gesundheit N: Does feedback matter? Practice-based learning for medical students after a multi-institutional clinical performance examination. Med Educ. 2007, 41 (9): 857-865. 10.1111/j.1365-2923.2007.02818.x.
Driessen E, van Tartwijk J, Vleuten van der C, Wass V: Portfolios in medical education: why do they meet with mixed success? A systematic review. Med Educ. 2007, 41 (12): 1224-1233.
Driessen EW, van Tartwijk J, Overeem K, Vermunt JD, van der Vleuten CP: Conditions for successful reflective use of portfolios in undergraduate medical education. Med Educ. 2005, 39 (12): 1230-1235. 10.1111/j.1365-2929.2005.02337.x.
Schwiebert LP, Crandall SJS, Brown DS: Incorporating goal setting into a third-year family medicine clerkship: a pilot project. Teach Learn Med. 1991, 3 (4): 239-244.
Stuart E, Sectish TC, Huffman LC: Are residents ready for self-directed learning? A pilot program of individualized learning plans in continuity clinic. Ambul Pediatr. 2005, 5 (5): 298-301. 10.1367/A04-091R.1.
Shokar GS, Shokar NK, Romero CM, Bulik RJ: Self-directed learning: looking at outcomes with medical students. Fam Med. 2002, 34 (3): 197-200.
Ozuah PO, Curtis J, Stein RE: Impact of problem-based learning on residents' self-directed learning. Arch Pediatr Adolesc Med. 2001, 155 (6): 669-672.
Bravata DM, Huot SJ, Abernathy HS, Skeff KM, Bravata DM: The development and implementation of a curriculum to improve clinicians' self-directed learning skills: a pilot project. BMC Med Educ. 2003, 3: 7-10.1186/1472-6920-3-7.
Emans SJ, Goldberg CT, Milstein ME, Dobriner J: Creating a faculty development office in an academic pediatric hospital: challenges and successes. Pediatrics. 2008, 121 (2): 390-401. 10.1542/peds.2007-1176.
Mayer ML, Preisser JS: The changing composition of the pediatric medical subspecialty workforce. Pediatrics. 2005, 116 (4): 833-840. 10.1542/peds.2004-2446.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6920/9/16/prepub
The authors declare that they have no competing interests.
STL conceived of and designed the study, acquired the data, analyzed and interpreted the data, and drafted the manuscript. MAF participated in the study design, helped analyze and interpret the data, and helped draft the manuscript. DCW participated in interpreting the data and critically revised the manuscript. All authors read and approved the final manuscript.
About this article
Cite this article
Li, ST.T., Favreau, M.A. & West, D.C. Pediatric resident and faculty attitudes toward self-assessment and self-directed learning: a cross-sectional study. BMC Med Educ 9, 16 (2009). https://doi.org/10.1186/1472-6920-9-16