The development and implementation of a curriculum to improve clinicians' self-directed learning skills: a pilot project

Background Clinicians need self-directed learning skills to maintain competency. The objective of this study was to develop and implement a curriculum to teach physicians self-directed learning skills during inpatient ward rotations. Methods Residents and attendings from an internal medicine residency were assigned to intervention or control groups; intervention physicians completed self-directed learning curricular exercises. Results Among the 43 intervention physicians, 21 (49%) completed pre- and post-curriculum tests; and 10 (23%) completed the one-year test. Immediately after exposure to the curriculum, the proportion of physicians defining short- and long-term learning goals increased [short-term: 1/21 (5%) to 11/21 (52%), p = 0.001; long-term: 2/21 (10%) to 15/21 (71%), p = 0.001]. There were no significant changes post-curriculum in the quantity or quality of clinical question asking. The physicians' mean self-efficacy (on a 100-point scale) improved for their abilities to develop a plan to keep up with the medical literature (59 vs. 72, p = 0.04). The effects of the curriculum on self-reported learning behaviors was maintained from the immediate post-curriculum test to the one-year post curriculum test: [short-term learning goals: 1/21 (5%) pre-, 11/21 (52%) immediately post-, and 5/10 (50%) one-year after the curriculum (p = 0.0075 for the pre- vs one-year comparison); long-term learning goals: 2/21 (10%) pre-, 15/21 (71%) immediately post-, and 7/10 (70%) one-year (p = 0.0013 for the pre- vs one-year comparison). At one-year, half of the participants reported changed learning behaviors. Conclusions A four-week curriculum may improve self-directed learning skills.


Background
Clinicians are expected to remain current with the medical literature to maintain their clinical competence. With over 2 million biomedical articles published annually and a medical literature that has grown increasingly complex, physicians struggle to remain informed of the many new therapies and diagnostic tools that relate to their practices. [1][2][3] Despite investing $3 billion a year, the most widely used continuing medical education (CME) programs such as lectures, short conferences, and written information have rarely been shown to change clinical practices or improve patient outcomes. [4][5][6][7][8][9][10][11] The CME programs that are most likely to promote the adoption of new behaviors are those that use multiple educational strategies including physician-specific feedback, practice-based educational programs, physician participation in the design of the educational interventions, contact with local opinion leaders, and self-directed programs (problembased learning directed by the physician, in which the patient serves as the impetus for the learning experience). [4,[12][13][14][15][16][17] Adult learning theory posits that adults learn best when they are required to address problems (problem-based as opposed to subject-based learning). [18,19] Furthermore, learning is maximized when it is self-directed so that adults study material that is most relevant to them. [18,19] Clinicians engage in self-directed learning by first identifying a clinical problem, then pursuing the learning task, next acquiring the new knowledge or skill, and finally practicing the new knowledge or skill. [20] A recent survey of housestaff practices demonstrated that residents engage in self-directed learning activities less than eight hours per week, an amount considered inadequate to prepare housestaff for self-directed learning after training. [21] Although medical educators have described the importance of improving house officers' self-directed learning skills, no consensus exists regarding the best methods for providing such training. [22] In this paper, we describe a curricular intervention that employs several educational and administrative modalities to teach attending and resident physicians the skills necessary for self-directed learning. The curriculum was designed to incorporate principles of adult-learning theory. The curriculum contains exercises to enhance skills for assessing learning needs, developing flexible shortand long-term learning plans, and asking and answering clinical questions efficiently. We sought to characterize the effects of this teaching program on physicians' (1) selfreported learning behaviors, (2) capacity for asking and answering well-constructed clinical questions, and (3) self-efficacy for performing essential self-directed learning behaviors.

Design
This study was designed as a prospective cohort trial of the effects of a self-directed learning curriculum on physicians' learning behaviors. The study protocol received Human Investigations Committee approval, and all participants provided written informed consent.

Subjects and Setting
Resident and attending physicians from a university internal medicine residency program assigned to one-month ward rotations at a Connecticut community teaching hospital between September 1997 and April 1998 were enrolled in the study. Physicians assigned to one of the two internal medicine ward teams at the hospital were designated as the intervention team. Interns and residents were assigned randomly to either the intervention or nonintervention team. In general, attendings physicians were assigned so that the general internal medicine attendings were assigned to the intervention team. Ward teams consisted of one attending physician and two resident-intern pairs. Each resident-intern pair admitted new patients every fourth day. Attending physicians made morning work rounds with the housestaff five to seven days per week and conducted formal attending rounds three times a week. Participation in the study was voluntary.

Curriculum
The curriculum consisted of five components: (1) performing a learning needs assessment, (2) using appropriate learning resources, (3) developing efficiency in reading medical journals, (4) developing and supporting a learning plan, and (5) asking clinical questions. In addition, each participant was asked to maintain a clinicalquestion diary. The curriculum physically consisted of a bound volume containing: an introduction, a sample calendar for the ward rotation, a consent form, a pre-test, components 1 through 5, a post-test, an evaluation form, and selected references. Accompanying the bound curriculum was a pocket-sized clinical-question diary. A brief description of each component follows.

Learning-Needs Assessment
The goal of this component of the curriculum was to guide participants in identifying specific learning needs. Based on published information about learning contracts and self-formulated learning plans, [18,19] we developed exercises to help participants identify areas of medical knowledge or clinical skills in which they needed improvement or practice [23,24] Participants were asked to articulate specific short-and long-term learning needs based on questions from their practices, board certification or in-service examinations, comments from their colleagues, and their general interest. The physicians were asked to define specific time frames for their various learning goals.

Learning-Resources Exercise
The goal of this component of the curriculum was to have participants identify optimal resources for answering clinical questions. Participants examined the usefulness of common learning resources and were asked to expand the types of learning resources that they employed in answering clinical questions. They were asked to list all possible sources of medical information, identify the sources that they used most commonly, and rate each source in terms of relevance and convenience (using a 10-point scale, where 10 was the most convenient or most relevant). They then graphed each source of information on a grid, with relevance on the x-axis and convenience on the y-axis to determine whether sources clustered (e.g., low relevance clustered with high convenience).
Prior research has demonstrated that physicians use a variety of learning resources; [25,26] that convenience drives many physicians' choice of learning resources; [9] and that as learners become more skillful in using a wider variety of resources, they find that some resources are more convenient to use than they had previously thought. [10] Participants were encouraged to maximize both convenience and relevance in their choice of learning resources, and to also consider situations where they might emphasize relevance over convenience.

Journal-Reading Exercise
The goal of this component of the curriculum was to help the clinicians optimize their journal-reading time and habits. Surveys have indicated that practitioners prefer journal-reading to all other continuing-education activities. [9] Research has shown that physicians can benefit more from their medical reading if they learn skills and habits that permit them to select the most clinically useful articles. [27] This exercise used a group-learning format where members of the team were asked to teach their colleagues techniques to identify appropriate journals, to scan potentially relevant articles, to set up and maintain a filing system of critical references, and to set reasonable goals for keeping up with the medical literature. Participants were asked to consider the problems associated with using journals as the primary source for clinical information, the circumstances under which sources other than original journal articles (e.g., textbooks or colleagues) might serve their learning needs, the difference between their goals for library-reading and for home-reading, and the criteria by which they should include or exclude particular journals from routine reading. This curricular component was developed based on the hypothesis that by explicitly stating ideal journal-reading habits participants would become more aware of their own behaviors and improve the efficiency of their journal-reading.

Learning-Plan Development and Support
The goal for this component of the curriculum was to have participants complete the learning contract to formulate a realistic and specific personal plan for meeting the learning goals that they identified in the needs assessment. The participants were asked to define both short-term (e.g., during this ward rotation) and longer-term (e.g., during this academic year) learning plans, including: group learning activities (e.g., attending rounds), self-directed learning activities (e.g., using the learning portfolio), teaching activities (e.g., work rounds), and scanning activities (e.g., text-book reading). Participants shared their goals with the other members of the team to facilitate planning of group-learning experiences that maximized the learning of their colleagues. The whole team worked together to help each individual articulate learning goals and devise learning plans. For example, if a resident stated a plan to focus on the cardiovascular physical examination during the coming month, then the attending and other resident physicians agreed to watch for interesting cardiovascular physical examination findings in the patients cared for by the team, to emphasize such findings during rounds, and to encourage that resident to teach the other physicians about such signs. During this curricular component the participating physicians were practicing two important self-directed learning skills: identifying a specific learning task (usually based on a clinical question) and completing the task (usually obtaining the answer to the question). [20] Asking Clinical-Questions The goal of this component of the curriculum was to have participants practice constructing questions that incorporate features of effective clinical questioning: (1) the specific patient or problem being addressed, (2) the intervention or exposure being considered, (3) a comparison to another intervention or exposure, and (4) the clinical outcome of interest. [2] Participants were asked to use this format (e.g., "Do patients with acute stroke who receive thrombolytic therapy have an increased rate of inhospital mortality compared to similar patients who do not receive such therapy?") on work rounds, during attending rounds, and when using their question diaries.

Clinical Question Diary
The goal of this component of the curriculum was to have participants generate and record in their individual pocket-size diary at least one clinical question for each patient they admitted. They were also asked to record the learning resources used to answer the question. We modeled the question diary after the paper version of the question portfolio of the Maintenance of Competence Program (MOCOMP) of the Royal College of Physicians and Surgeons of Canada. MOCOMP is a self-directed learning program in which participating physicians maintain a personal portfolio of the questions that they would like to answer, the situations that stimulated the questions, and the educational resources that they used to answer the questions. [27][28][29] A random survey of physician users of MOCOMP indicated that 49% of questions that participants entered into their log led them to learn something that resulted in a change in their clinical practice. [28] Other authors have demonstrated that learning is more durable when clinical questions and subsequent learning is linked to the patient who stimulated the question. [13] The use of learning portfolios has been shown to improve clinicians identification and meeting of their own learning needs. [30]

Data Collection
Participating physicians were given test instruments (described below) on the first day of their ward rotation (pre-test) and on the last day (post-test). One-year followup test instruments were given in April 1999 to those intervention physicians who were rotating on the Internal Medicine wards at that time. At the end of each ward month rotation the clinical question diaries were collected from the participants.

Curriculum Implementation
Prior to beginning their ward rotation, the attending physicians on the intervention teams were given a 20-minute orientation that reviewed the objectives of the study, introduced the curriculum, provided specific suggestions for the implementation of the curriculum into the ward rotation, reviewed the importance of completing all the evaluation instruments, and provided an opportunity to ask questions. Self-directed learning curricular exercises occurred during the initial 10 to 15 minutes of the first six attending rounds of each rotation. Participants used their remaining attending rounds to practice these skills by addressing specific patient-derived questions.
Specific educational and organizational methods were used to facilitate implementation and use of the curriculum. The educational methods employed to teach these self-directed learning skills included: individual study (e.g., physicians read individually on topics related to patients on the service), group study (e.g., the team performed the learning-resource exercise together), and the use of attending physicians as role models. In addition to these educational methods, the curriculum provided an organizational structure for the ward-month experience, as well as administrative tools that facilitated review and evaluation of learning experiences (e.g., the learning plan). The learning diary served as a record of all patients admitted during the month, obviating the need for other redundant record keeping methods that had historically been used.
In addition, participating physicians were asked to adopt several important attitudes: to think of themselves as learners; to consider their ongoing professional development as a responsibility to themselves, their patients, and their colleagues; to discard the notion that physicians should have all the answers and to embrace situations where gaps in their medical knowledge generate learning opportunities; and to value helping colleagues achieve their learning goals.

Curriculum Evaluation
The five components of the curriculum were evaluated with 30 item pre-, post-, and 1-year follow-up test instruments to assess the participants': (1) self-reported learning behaviors, (2) abilities to generate clinical questions from a clinical scenario and to describe a strategy for answering those questions, and (3) self-efficacy to perform specific self-directed learning behaviors. The test instruments were identical with the exception of the clinical scenarios. They were developed specifically for the current study and were pilot tested on five physicians who were not participating in the current study for face validity and clarity. Two of the authors, blinded to the physicians' group assignment and other data, rated the questions that the physicians generated in response to the clinical scenarios according to the four criteria of good clinical questions defined above. The raters were trained during the pilot phase to achieve excellent inter-rater reliability. We assessed self-reported learning behaviors with eight short-answer questions and selfefficacy for self-directed learning behaviors with 12 questions rated on a scale of 0 to 100, where 0 indicates no confidence and 100 indicates extreme confidence. [31] The clinical question diaries were collected from all participants to evaluate if they had been used.

Statistical Analysis
Microsoft Excel 5.0 (Redmond, Oregon 1995) was used for database management and statistical analyses. Means and standard deviations were calculated for continuous variables. Differences in responses of participants on pretests, post-tests, and 1-year follow-up tests were assessed with non-paired two-tailed Student's t-tests, Chi-square tests, or Fisher's Exact test; where statistical significance was accepted for p-values <0.05.

Subjects and Response Rates
The intervention group consisted of a total of 43 physicians; 37 residents and six attendings. All of the eligible residents and attendings in the intervention group participated in the curriculum. Of the 43 participants, 21 (49%) completed pre-tests, 21 (49%) completed post-tests. The physicians who used the curriculum but did not submit pre-or post-tests reported participating fully in the curriculum. Ten of the 43 (23%) were present on the wards oneyear after the curriculum and all 10 completed the 1-year follow-up test.

Short-Term Effects of the Curriculum: Pre-Test vs. Post-Test Results
To evaluate the short-term effects of the curriculum, participants' pre-curriculum test results were compared to participants' post-curriculum test results.

Clinical Questions
There were no significant changes in either the quantity or quality of clinical questions that participants generated in response to the clinical cases (mean number of questions per subject: pre-curriculum 3.6 versus 3.1 post-curriculum, p = 0.05). There were no changes post-curriculum in the proportions of questions that made a comparison to another intervention, that identified a clinical outcome of interest, or that failed to meet any of criteria of a well-constructed clinical question.

Self-Efficacy for Self-directed Learning Activities
The physicians' mean self-efficacy improved from pre-to post-curriculum for their abilities to develop a plan to keep up with the medical literature (59 pre-versus 72 post-curriculum, p = 0.04) but was not statistically significant for the other behaviors (data not shown).

Clinical Question Diary
All of the participants in the intervention group used the clinical question diaries. All of the diaries demonstrated that participants asked clinical questions and identified the learning resource used to answer the questions. The most positive feedback about the clinical question diaries came from first-year residents, who found them to be helpful in remembering the many clinical questions that arose during the day, and useful in organizing and focusing their medical reading. One attending commented that the diary was a helpful tool for self-reflection. The most negative comments came from the senior residents, who stated that the diaries increased the documentation burden of their busy ward-rotation (despite the fact that the diaries replaced the previously used method of recording all patients admitted during the month, and therefore should not have increased the need for documentation).

Durability of Curriculum Effects: Post-Test vs. One-Year Post-Curriculum Results
The effects of the curriculum on self-reported learning behaviors appeared to be maintained from the immediate post-curriculum test to the one-year post curriculum test (Table 1). For example, the proportion of physicians with short-term learning goals started at 1/21 (5%) pre-curriculum, rose to 11/21 (52%) immediately post-curriculum, and remained at 5/10 (50%) one-year after the curriculum (p = 0.0075 for the pre-vs one-year comparison). Similarly, the proportion of physicians with long-term learning goals was 2/21 (10%) before the curriculum, increased to 15/21 (71%) post-curriculum, and remained at 7/10 (70%) one-year later (p = 0.0013 for the pre-vs one-year comparison).

Overall Change In Learning Behaviors
One-half of the participants surveyed at 1-year reported that they had changed their learning behaviors as a result of their participation in the curriculum. Comments included, "It increased my motivation to attempt answering clinical questions," "I research clinical questions more frequently," and "I am better at goal setting." The half who said they had not changed their learning practices wrote, for example: "I have continued to use the same resources as prior to participation," "Time on [the] rotation was limited, [I was] unable to fully participate, time constraints prevented full attention to curriculum goals," and "having [had] several mentors of evidence-based medicine prior to going through the curriculum, I already had a handle on how to answer questions, the challenge has been actually doing it."

Discussion
We developed a curriculum comprising multiple educational and structural components to teach physicians selfdirected learning skills. We found that the curriculum was effectively increased the number of short-and long-term learning goals that the participating physicians articulated and improved participants' self-efficacy for their abilities to develop a plan to keep up with the medical literature.
A major limitation of this study was the low response rate. As a result, the final sample size was small, limiting our ability to detect statistically significant differences. We have no information to suggest that physicians who completed their evaluation materials were different from those who did not with regard to their participation in the curriculum.
An additional important limitation of this study is the lack of comparison to a control group. Although this study was originally designed to include a comparison between physicians on the intervention team and nonintervention (control) teams, we were not able to complete this comparison because too few of the control physicians returned their test instruments to make the analysis valid.
Our study demonstrated the difficulties inherent in conducting educational research during residency training. The magnitude of these difficulties was surprising given the academic nature of the residency program, the university appointments of the attending physicians, and the general environment that has historically emphasized the need for scholarship and has encouraged participation in clinical research on the part of the faculty and the housestaff. The attending physicians were encouraged to take ownership of this curriculum and this study, and their ideas and feedback were solicited. The curriculum contained all the necessary materials and extra copies were made readily available. Participants were given numerous personal reminders about the need to complete the evaluation materials. As mentioned above, the learning diaries were accepted as replacements for documentation that was required of the residents. Given the reality of the busy pace of inpatient medicine and the many competing needs of physicians on ward teaching services, a suggestion to improve response rates for future studies is to have study authors administer each of the surveys at assigned times in lieu of regularly scheduled educational activities such as noon conference.
Another limitation of the current study is that we relied upon physician self-reports. While self-efficacy requires physician surveys, future studies should employ direct observation of self-directed learning behaviors.
Incorporating self-directed learning techniques as described in this report, may enhance the effectiveness of existing residency program curricula. As an educational technique, self-directed learning should be explicitly integrated into residency program curricula. Self-directed learning is described in adult learning theory which suggests that adult learners can identify their learning needs, find solutions to problems, base learning on experience, and self-direct their education. Self-directed programs for physicians use problem-based learning, in which the patient becomes the impetus for the learning experience, and require the learner to reflect on his or her own clinical practice. [15,16] Shin and colleagues reported that a medical school that employed a self-directed, problem-based curriculum produced graduates who adhered more closely to published clinical practice guidelines than did graduates of a traditional medical school. [4,15] Given the evidence that practice patterns that have been established during residency persist after graduation, and other research demonstrating that learning behaviors taught during medical school affect practice, medical educators should emphasize self-directed learning skills during residency training.
The first steps in the self-directed learning process are the definition of learning needs and the articulation of learning goals. Research has shown that physicians consistently overestimate their knowledge and underestimate gaps in that knowledge. [12] Recognition of these gaps tends to elicit a variety of responses: anxiety, guilt, and, under the best circumstances, a desire to fill them in. [19] Our curriculum helped clinicians to create personalized learning plans and encouraged them to see every patient encounter as a potential learning opportunity. We found that the proportion of physicians with defined learning goals increased after participating in the curriculum.
The most important facets of the curriculum are its strong foundation in adult learning theory, the use of multiple educational methods, incorporating non-educational components, and linking the curriculum to direct patientcare activities. Important topics for future research include evaluations of similar curricula over longer periods, in other medical arenas (e.g., ambulatory rotations), in different medical settings (e.g., non-academic environments, where there are fewer competing educational opportunities and where the burden on physicians to maintain their competence is likely to be greater), and in other medical specialties (e.g., psychiatry or surgery).

Conclusions
Our research findings demonstrate that it is feasible to implement a four-week curriculum in self-directed learning during a four-week internal medicine ward rotation, and that this curriculum may improve some components of self-directed learning.
Publish with Bio Med Central and every scientist can read your work free of charge