This article has Open Peer Review reports available.
Cross-sectional longitudinal study of the academic half-day format in a hematology-oncology fellowship training program
© Eid et al. 2015
Received: 1 February 2015
Accepted: 29 July 2015
Published: 25 August 2015
Few published studies have evaluated the effectiveness of changing the traditional curriculum of several hourly educational sessions per week to an academic half-day (AHD) educational format. This study describes our experience with implementing an AHD format in the Hematology-Oncology Fellowship Program at The University of Texas MD Anderson Cancer Center and evaluates the perceptions that learners had for this format.
Using a mixed quantitative and qualitative approach, we evaluated our AHD program using four yearly fellows’ surveys to assess the Hematology-Oncology Fellows’ perceptions of the effectiveness of the AHD format. We analyzed the fellows’ perceptions using descriptive statistics and thematic analysis of the qualitative data collected from the surveys. We used a quality improvement approach by implementing and testing changes to the AHD over 4 years on the basis of the data collected from the yearly fellows’ surveys. We also collected third-year fellows’ Oncology In Training Exam (ITE) scores from 2008 to 2014.
We found that the fellows perceived the AHD format favorably; fellows agreed that they had more motivation to attend AHD, more concentration during the sessions, more effective weekly work organization, and increased knowledge retention. We established the reliability of our survey tool (Cronbach’s alpha = 0.83) as well as content and construct validity. We saw an increasing trend in ITE scores since the AHD was implemented.
Our results contribute to further understanding the effect of the AHD format on trainees. Using a continuous evaluation and an educational quality improvement strategy, we found that the AHD curriculum was associated with a rising trend in learners’ exam scores and increased learner satisfaction.
To our knowledge, no studies have examined the effectiveness of the academic half-day (AHD) format in internal medicine subspecialty fellowship programs. To the best of our knowledge and according to recent publications , no available validated survey exists to evaluate trainees’ perceptions about their AHD experience. To address this gap in the literature, we present our experience and evaluate the evolution of the AHD format at The University of Texas MD Anderson Cancer Center Hematology-Oncology Fellowship Program and the validation of our survey tool.
The Accreditation Council for Graduate Medical Education mandates that internal medicine graduate medical programs must schedule regular didactic sessions that include lectures and other media that communicate the core principles of internal medicine . Traditionally, many programs have fulfilled this requirement with noon conferences or other 1-h sessions several days a week. Evidence about whether these didactic lectures in internal medicine residencies lead to long-term knowledge retention varies [3–5], and some studies suggest that resident performance is not necessarily enhanced by this educational structure . Currently, some training programs have altered the format and combined multiple sessions per week into one AHD session per week. This shift away from the traditional model could enhance knowledge acquisition and influence clinical performance, according to previous research on the AHD format .
However, few residency programs have provided descriptions of their AHD-format curricula [7–13] or evidence of effectiveness [1, 14, 15]. Publications about the implications of the AHD format and the opinions of the trainees in these programs as well as validated methods to measure the effectiveness and usefulness of the AHD format are lacking. Such measurement is critical for program evaluation, improvement, and reliable comparison among the approaches employed by various institutions or by the same institution over time.
This study describes the implementation and evolution of our AHD program, the perceptions of the fellows as the program evolved, and the improvement strategies developed on the basis of fellows’ evaluations and feedback.
In May of each academic year from 2011 through 2014, fellows enrolled in the program were asked to complete a paper-based survey about their perception of the AHD format. For the first year of the study (2011), in addition to the currently enrolled fellows, the fellows who graduated in 2010 were also asked to complete a similar survey anonymously. Thus, a total of seven classes participated in our surveys: the graduating class of 2010 through the class of 2016. The fellows who graduated in 2010, 2011, and 2016 only filled out one survey. The graduating classes of 2012, 2013, 2014, and 2015 participated by filling out the survey more than once.
The surveys consisted of two sections (Additional file 1: Appendix 1). The first section of the survey covered the fellows’ perceptions of their ability to concentrate during the AHD sessions, their motivation to attend the AHD sessions, their retention of information presented during the AHD sessions, and their ability to organize their work schedules around the AHD sessions. The perception survey was a 5-point Likert-like scale asking fellows to rate their perceptions from favorable (anchored at 5) to unfavorable (anchored at 0). In addition to these original four categories, starting in 2013, we included four new items: distraction, usefulness, influence, and refresh (Additional file 1: Appendix 2). The second section of the survey for all four years requested input on the services added to enhance the AHD experience as well as suggestions for improvement.
We collected third-year fellows’ oncology In Training Exam (ITE) scores from 2008 to 2014 to explore the effect of the AHD on the fellows’ cognitive knowledge acquisition since the program was implemented. The oncology ITE is a validated, standardized examination aimed at assessing oncology fellows’ medical knowledge and identifying areas of deficiencies that require further learning intervention.
We conducted all analyses using Statistical Package for the Social Sciences software, version 21. We ran descriptive analysis to describe the sample distribution over the 4 survey years. We calculated means and standard deviations of the core perception survey items. We carried out multivariate analysis of variance (MANOVA) to examine group differences over the 4 years for four perception items that measured motivation, concentration, organization, and retention. The independent variable was the survey year, and we treated the four perception items as dependent variables.
The University of Texas at MD Anderson Cancer Center Institutional Review Board approved the study (Protocol DR11-0150). Because information was collected anonymously and because participants could not be linked to the data, the study was approved as exempt research, and written consent was not required. All participants read and acknowledged a consent statement explaining the goals and methods of the study and how their privacy and confidentiality would be protected.
Hematology-Oncology Fellows who participated in the surveys (2011–2014)
Survey Participants (total enrolled)
Graduation year (cohort)
Survey Participants (total enrolled)
Hematology-Oncology Fellows’ perception of the academic half-day format by survey year (2011–2014)
P = 0.06
P < 0.01
P < 0.05
P < 0.001
Means (standard deviation) of fellows’ perception of improvements added to the academic half-day format based on 2013 survey results
Clickers by lecture
Clickers for attendance
Arranging lectures by themes
“This academic year is much better than last year. I really enjoy the protected time. If these were lunch-hour lectures, you would have inevitably been late or missed lectures due to patient care. Lectures are now clinically relevant and geared toward the boards!” [2012 survey]
“Some lectures can be more board-oriented. Pharmacology lectures are an excellent addition! [This program is a] great way to learn oncology, and it has improved significantly since last year!” [2012 survey]
“Sometimes time is not managed well, and gaps are present between lectures. However, there has been a substantial positive improvement since last year.” [2012 survey]
“[The AHD] has gotten better over the years.” [2013 survey]
“It is so much better compared to last year.” [2014 survey]
“I think the 12-month curriculum was a great change. Continue.” [2014 survey]
“Certain services and rotations make it difficult to learn and benefit from academic Tuesday.” [2011 survey]
“I feel that some attendings were upset that we were missing so much clinic time that Tuesday morning.” [2011 survey]
“Nurses call all the time, and I cannot sit through an entire lecture without being interrupted.” [2013 survey]
“Most of lectures are didactic and not interactive. It was very difficult to concentrate because of the didactic format.” [2011 survey]
“We have to stay focused for a couple of hours, which is also challenging since completion of medical school.” [2013 survey]
“Please provide caffeinated soda at lunch. It would make focusing on the noon content easier.” [2014 survey]
Thematic summary of Hematology-Oncology Fellows’ suggestions for improving the academic half-day format (2011–2014)
• Curriculum for sessions
• Curriculum for sessions
• Decrease length of curriculum (18 months to ≤ 12 months)a
• Decrease duration from 5 to 4 h
• Online repository for the lectures
• Website where lectures are helda
• More breaks
• Schedule lectures in blocks (GI, GU, breast, etc.)a
• Record Lecturesa
• Easier access for lectures at home
• Pre and posttests each month
• References for articles
• Schedule main blocks earlier in year
• Sign in sheets instead of clickersa
• More board review sessionsa
• Continue board review and pharmacology sessionsa
• More case-based teaching, perhaps a morning report in residencies to help fellows think more critically
• More clinical lectures
• Make lectures more interactive
• Make lectures more interactive
• More clinical based talks and board review series
• More interactive sessions
• More case-based formata
• Add different formats: small group, problem-based learninga
• More interactive sessions, more varied types of sessions (not just PowerPoint)
• More review questions, more group exercises
• More pharmacology sessionsa
• More benign hematology lecturesa
• More Q&A sessions
• More case-based and problem-based sessions
• More pharmacology talks
• More faculty involvementa
• More involvementa
• Ensure that best faculty are involved
• Academic leaders participate in lectures
• Statistics faculty at journal clubsa
• Snacks, coffee breaksa
• Consider a room with table set up
• No disturbance from calls and pagers from service
• Reduce the calls from chemo suites during that period
• More interactive style of learning
To our knowledge, this is the first study to examine the effectiveness of the AHD format in a fellowship program using an internally validated survey and a quality improvement approach. Although some descriptive articles have been published , our study focuses on measuring the fellows’ perceptions of their learning experience. We also showed that the AHD format implemented in our program is well received by the fellows and offers them the opportunity to make the most of this learning experience. The conclusions presented here are based on a 4-year experience with our AHD format and thus offer a more reliable evaluation of the feasibility of this format.
One of the strengths of our study is that the fellows benefited from the improvements that they suggested on our surveys and thus benefited from the study. Our continuous evaluation and improvement approach supports the call for using quality improvement methods in medical education .
Another strength of the survey is that we did not restrict it to questions related to how satisfied the fellows were with the AHD format; we also included the effects of the AHD format on secondary and tertiary levels, as defined in the Kirkpatrick model . The survey included questions about organization, motivation, and concentration during the sessions that addressed the first level of the Kirkpatrick model (learner satisfaction). The survey also included questions about how the AHD format helped to refresh fellows’ knowledge and affected their retention of knowledge; these items correlate with the second level of the Kirkpatrick model (learner knowledge). The increasing trend in fellows’ ITE scores since the AHD was implemented (Fig. 2) confirms the fellows’ perception about the improvement in knowledge acquisition after the AHD was implemented. Finally, the survey included questions about how the AHD format influenced decision making in the clinic, which correlates with the third level of the Kirkpatrick model (transfer of knowledge). We hope that future studies will evaluate these questions not only by the fellows’ perceptions but also by other objective and feasible matrices. Our current study serves as a baseline for future research.
In a recently published description of the AHD format in three internal medicine residency programs, the percentage of residents answering “yes” to the Accreditation Council for Graduate Medical Education annual survey question, “Are the core conference series educationally valuable?” was higher in programs with the AHD format than in those with the noon lecture format . Our study’s results support these findings by showing that the AHD format provides an educationally valuable experience for trainees in graduate medical education.
One study that compared knowledge retention differences in the traditional noon or 1-h conference format and the AHD format suggested that a “dispersed delivery” method or the traditional noon conference format was more beneficial for long-term knowledge retention . Another study showed that family practice residents attending didactic lectures in a block conference format did not show improvement in their knowledge over the long-term . However, other studies have found that the AHD format possibly helped improve In-Training Exam scores and had a positive effect on the overall intellectual climate within a residency program . Though we cannot claim causality, ITE scores have improved since the implementation of the AHD, suggesting that the new learning program may have had a positive effect on learning outcomes. The lack of access to individual fellows’ ITE scores prevented us from testing if this trend was statistically significant. Finding a statistically significant difference is also unlikely due to the small sample size of the fellows taking the exam each year.
Therefore, it seems important to continue investigating the AHD format in graduate medical education particularly because our study shows that the fellows perceive this format favorably. Hopefully, further studies will illuminate the effect of the AHD format on knowledge retention and clinical performance.
Our study had several limitations, some of which were inherent to the cross-sectional survey study design. Ensuring that the fellows evaluated the program honestly while in the program could be an inherent conflict of interest for the study. On the one hand, the fellows are the beneficiaries of the teaching program, and their evaluation of the program is thus the most direct measurement of the usefulness and efficacy of the program. On the other hand, they are a vulnerable population, and our institutional review board has strict guidelines to protect the fellows’ interest. To ensure we accounted for both these factors, we decided to conduct this study in a de-identified and anonymous manner.
Another limitation to our study is the lack of information on the fellows’ baseline characteristics and fund of knowledge assessment such as ITE scores during their residency training or at the beginning of the fellowship. This information is important to make sure the increased trend in fellows' ITE scores over the years of the AHD implementation is related to the AHD effect and not due to other variables. Unfortunately this information is not available since our program does not collect residency ITE scores and does not offer the ITE exam to the fellows during the first year of their fellowship training. Although a variation in the fellows’ baseline characteristics is possible, it is less likely that this difference will be significant enough to account for the rise in ITE scores since our program’s selection criteria for new fellows has not changed and the vast majority of our fellows are selected based on their excellent academic track records.
Furthermore, participation was voluntary, and no record was kept of who returned the survey. No identifying questions were asked. Thus, the fellows felt more assured about giving their honest opinion. However, we could not link the data to any other outside variable or even to responses from the previous year(s). This posed the most serious limitation to our study, as we could not corroborate our findings with any data collected outside of the survey. However, we believe that the results we obtained were meaningful and reliable because of the trend among fellows toward more favorable perceptions of the ADH format as we implemented more of their suggestions each year.
It would be worthwhile to evaluate the effect of this new teaching format through more objective outcomes, such as improvement in clinical decision making, and/or improvements in patient care outcomes. However, designing such a study would entail accounting for all the confounding factors related to feasibility and the privacy of the fellows and their patients. Taking that into account, we thought that a voluntary and anonymous estimation of the fellows’ perception was the most direct, unbiased, and feasible method to evaluate the AHD format. Our study adds to the literature on the AHD format, which currently lacks information about the AHD format’s effect on the trainees, training programs, and institutions. We internally validated the core items of the satisfaction survey. Future external validation of the survey in other training programs who have adopted the AHD format is required.
Using a mixed quantitative and qualitative approach, we used four yearly surveys to assess the Hematology-Oncology Fellows’ perceptions of the effectiveness of the AHD format. We found that fellows perceived the AHD format favorably; fellows agreed that they had more motivation to attend AHD, more concentration during the sessions, more effective weekly work organization, and more knowledge retention. We established the reliability of the survey tool as well as content and construct validity. This study has contributed to understanding the effects of the AHD format on trainees. Using a continuous evaluation and an educational quality improvement strategy, we found that the fellows’ perceptions of the AHD format improved. Though we cannot claim causality, ITE scores have improved since the implementation of the AHD in our program, suggesting that the new learning program may have had a positive effect on learning outcomes. It is important to continue investigating the effect of the AHD format on knowledge retention, concentration, and clinical performance to ensure that this format is beneficial to the learners.
We wish to thank the Hematology Oncology fellows and graduates who volunteered to participate in the study’s surveys. We also wish to acknowledge the contributions of Heli Bhatt, Patricia Lenihan, Karin Hahn, Martin Raber, Kathryn Gold, Margaret Van Meter, Carlos Barcenas, Jeffrey Yorio, Sharon Shaw, the scientific reviewers, and the members of the fellowship coordination team especially Camillia Moses who helped with distributing and collecting the survey data.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Ha D, Faulx M, Isada C, Kattan M, Yu C, Olender J, et al. Transitioning From a Noon Conference to an Academic Half-Day Curriculum Model: Effect on Medical Knowledge Acquisition and Learning Satisfaction. J Grad Med Educ. 2014;6(1):93–9.View ArticleGoogle Scholar
- ACGME Common Program Requirements. [http://acgme.org/acgmeweb/portals/0/PFassets/2013-PR-FAQ-PIF/140_internal_medicine_07012013.pdf]
- Cacamese SM, Eubank KJ, Hebert RS, Wright SM. Conference attendance and performance on the in-training examination in internal medicine. Med Teach. 2004;26(7):640–4.View ArticleGoogle Scholar
- McDonald FS, Zeger SL, Kolars JC. Factors associated with medical knowledge acquisition during internal medicine residency. J Gen Intern Med. 2007;22(7):962–8.View ArticleGoogle Scholar
- McDonald FS, Zeger SL, Kolars JC. Associations of conference attendance with internal medicine in-training examination scores. Mayo Clin Proc. 2008;83(4):449–53.View ArticleGoogle Scholar
- Davis D, O'Brien MA, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA. 1999;282(9):867–74.View ArticleGoogle Scholar
- Batalden MK, Warm EJ, Logio LS. Beyond a Curricular Design of Convenience: Replacing the Noon Conference With an Academic Half Day in Three Internal Medicine Residency Programs. Acad Med. 2013;88(5):644–51.View ArticleGoogle Scholar
- Nigwekar SU, Morse DS. Educational half day: an innovative way to incorporate substance abuse curriculum into residency training. Subst Abus. 2006;27(3):1–3.View ArticleGoogle Scholar
- Chalk C. The academic half-day in Canadian neurology residency programs. Can J Neurol Sci. 2004;31(4):511–3.View ArticleGoogle Scholar
- Tannenbaum DW. New “horizontal” curriculum in family medicine residency. Can Fam Physician. 1998;44:1669–75.Google Scholar
- Zweifler J, Ringel M, Maudlin RK, Blossom HJ. Extended educational sessions at three family medicine residency programs. Acad Med. 1996;71(10):1059–63.View ArticleGoogle Scholar
- Klein D, Schipper S. Family medicine curriculum Improving the quality of academic sessions. Can Fam Physician. 2008;54(2):214–8.Google Scholar
- Wong RY, Saber SS, Ma I, Roberts JM. Using television shows to teach communication skills in internal medicine residency. BMC Med Educ. 2009;9(1):9.View ArticleGoogle Scholar
- Academic Half-Day. An Initial Survey of Residents in the Harvard Combined Orthopaedic Residency Program. [http://orthojournal.wordpress.com/2011/12/26/academic-half-day/]
- Winter RO, Picciano A, Birnberg B, Chae M, Chae S, Jacks M, et al. Resident knowledge acquisition during a block conference series. Family medicine-Kansas city. 2007;39(7):498.Google Scholar
- Cleghorn GD, Headrick LA. The PDSA cycle at the core of learning in health professions education. Jt Comm J Qual Improv. 1996;22(3):206–12.Google Scholar
- Kirkpatrick DL, Kirkpatrick JD. Evaluating Training Programs, 3rd edn. San Francisco, California, USA: Berrett-Koehler Publishers; 2009Google Scholar
- Raman M, Mclaughlin K, Violato C, Rostom A, Allard J, Coderre Y. Teaching in small portions dispersed over time enhances long-term knowledge retention. Med Teach. 2010;32:250–5.View ArticleGoogle Scholar