Skip to main content

Advertisement

First year medical student performance on weekly team-based learning exercises in an infectious diseases course: insights from top performers and struggling students

Article metrics

Abstract

Background

In Team-Based Learning (TBL) preparation of relevant coursework during self-directed learning time is evaluated by the individual readiness assurance test (iRAT). We recently reported that student performance on iRATs is strongly correlated with final examination scores in an infectious diseases (ID) course. We now investigated how student preparation for each individual iRAT exercise relates to course performance.

Methods

Two-hundred and sixty medical students were enrolled in this three-year study. Student TBL iRAT scores were collected and correlated with final examination scores using Kruskal-Wallis One-Way ANOVA and Newman-Keul’s statistical methods.

Results

Students performing in the upper and middle 33rd percentile on the final examination showed highly significant (p < 0.01) weekly improvements in their iRAT scores. However, students performing in the lower 33rd percentile did not show improvement in their iRAT scores until the last week of the course. Although there was a highly significant correlation between final examination and iRAT scores amongst all students participating in the study, this correlation was stronger in students performing in the lower 33rd percentile.

Conclusions

Our data suggest that students who do not consistently prepare for TBL, as evidenced by low iRAT scores, exhibit poorer performance on the final examination. This lack of preparation likely interferes with the efficacy of this learning method. iRAT scores can also be used for early identification of struggling students in need of additional supports. Additionally, changes in TBL incentive structure may provide more tangible rewards for pre-class preparation in particular for struggling students.

Background

Educational strategies that promote active learning, such as Problem-Based Learning (PBL), Case-Based Learning (CBL), and Team-Based Learning (TBL), are essential to affording students the necessary tools for lifelong learning. Several studies have provided strong evidence that TBL exercises improve educational outcomes for students in multiple disciplines [1,2,3] and settings [4, 5]. We and others have also reported the beneficial effects of TBL in medical education [1, 6,7,8,9], where it improves the learning experience [10] and helps students to better understand complex subjects [1]. L. Michelsen, who developed TBL to address concerns regarding student engagement in a large class of undergraduate students, has suggested that properly formed teams, student accountability, frequent and timely feedback to students, and team assignments that promote learning and team development, are key to the success of the TBL experience [11, 12].

In the first phase of TBL [13], learners study independently outside of class to master identified objectives. In the second phase, mastery of relevant course material prepared during self-directed learning time is evaluated at the beginning of the TBL activity in a quantitative fashion using a graded multiple-choice quiz, the individual readiness assessment test (iRAT). In this phase, learners also work together in teams of 5–6 students to complete the group readiness assurance test (gRAT), which is composed of the same quiz questions as the iRAT. At the end of the gRAT, it is expected that students will achieve mastery of the target learning objectives through peer-to-peer teaching [11, 12]. We recently reported that implementation of TBL exercises in an Infectious Disease (ID) course at our institution resulted in improved performance on final examinations, in particular for more poorly performing students [6,7,8].

Several groups have studied the effects of incentive structure and grading of TBL exercises on the efficacy of this method [14,15,16]. Some studies have suggested that ungraded TBLs would be more student friendly and would support the active learning environment while reinforcing teamwork skills [15]. However, we found that at our institution, iRAT scores are higher and are also better predictors of course success when TBL exercises are graded as opposed to ungraded [7]. Furthermore, we recently reported that in graded TBL exercises, iRAT performance, a surrogate for individual student preparation for TBL exercises, correlated with final examination scores [6]. Accordingly, a recent study suggests that medical students see graded TBL exercises as the optimal teaching strategy over other active learning activities, such as PBL, because the structure and format of TBL allows better engagement and participation, which is conducive to more productive learning [17].

As a continuation of our previous studies, we are now investigating how the degree of preparation for each individual TBL relates to student course performance in an ID course for medical students, the only course in our curriculum where students engage in one TBL exercise each week of the course, for a total of four TBL exercises. Our analysis seeks to evaluate how weekly, pre-class preparation, as assessed by iRAT scores, relates to final examination performance amongst strongly performing and struggling students. Specifically, we hypothesize that students with better performance on the course final examination will show weekly improvements in their iRAT performance. The results of this study could provide key insights into the relationship between iRAT-TBL performance and student success in undergraduate medical education.

Methods

The ID course is a first year medical school course, which reviews microbiology and virology as they relate to infectious diseases and their treatment. In the ID course, there are four, weekly TBL exercises, which cover all material taught in the course. Additionally, a final examination of medical knowledge composed of multiple-choice, factual questions requiring recall of facts and application of knowledge from the ID course is administered at the end of the course.

Each TBL exercise takes place at the beginning of each of week of the course, with the last exercise administered just a few days before the final examination. The TBL exercise is composed of an iRAT, a gRAT, and two application questions. Grading for each TBL exercise is based on an assignment of 33.3% to the iRAT and 66.7% to the gRAT. Each TBL accounts for 3% of the final course grade. To investigate factors that contribute to successful academic performance, we studied iRAT performance in students performing in the upper, middle and lower 33rd percentile on the final examination. These studies were approved by the Rowan Institutional Review Board (IRB).

Mean averages and standard deviations for iRAT and final examination scores amongst the various study subgroups were calculated. Changes in student iRAT scores in the four TBLs were evaluated using Kruskal-Wallis One-Way Analysis of Variance (ANOVA) and Newman-Keul’s statistical methods. We also used a Pearson correlation coefficient (r) to assess the strength and direction of the linear association between final examination and iRAT scores. For all analyses, p values < 0.05 were considered statistically significant and p values < 0.01 were considered highly significant. All data were analyzed using IBM SPSS 24 (Armonk, NY).

Results

Data for this study were collected from three consecutive cohorts (Table 1), whose enrollment was 79, 86 and 95 students for years 1, 2, and 3, respectively. Overall mean average final examination and iRAT scores for the full class and students performing in the upper, middle and lower 33rd percentiles on final examination were calculated (Table 1). Pearson correlation coefficients (r) were also determined.

Table 1 Analysis of final examination and iRAT scores in an Infectious Disease Course for the full class and students performing in the upper, middle, and lower 33rd percentiles on the final examination

We found a positive correlation between final examination and iRAT scores for the full class in each year of this study (0.2674, p < 0.01, 0.3289 p < 0.01, 0.3994 p < 0.01 for years 1, 2, and 3, respectively). We found that this correlation was stronger for students performing in the lower 33rd percentile as compared to students in the upper or middle 33rd percentiles in every year of the study (Table 1). Indeed, in students performing in the upper and middle 33rd percentiles, we only found a correlation between final examination and iRAT scores in year 2 of the study (0.2236, p < 0.05 for the upper 33rd percentile) with no statistically significant correlation between final examination and iRAT scores in these groups in years 1 and 3.

Mean average iRAT scores for each of the four weekly TBL exercises were calculated and were compared to iRAT performance in the first week (Table 1). We detected significant increases between the first week iRAT scores and some of the iRAT scores from the second, third, and fourth week amongst students performing in the upper and middle 33rd percentiles. In students performing in the lower 33rd percentile, we only detected significant improvement between iRAT scores from the first and fourth week of the course, suggesting that their iRAT performance only improved in the final week of the course. This finding was consistent in all years of the study.

Discussion

In this study, we provide a detailed examination of iRAT scores as they relate to final examination performance in an ID course at our institution. We found that: (1) final examination grades are most strongly correlated with iRAT scores in students performing in the lower 33rd percentile on the final examination; and (2) more consistent improvement in weekly iRAT scores was seen in students performing in the upper and middle 33rd percentiles as compared to students performing in the lower 33rd percentile, which only showed improved iRAT scores in the last TBL exercise that takes place a few days before the final examination.

As described above, iRAT scores reflect pre-class preparation, or Phase 1 of the TBL exercise [13]. Accordingly, a recent study suggests that medical students with better preparation prior to TBL, assessed by significantly more downloading of TBL-required pre-class materials, performed significantly better on graded iRAT exercises than their classmates, even after controlling for cohort effects [18]. In our ID course, students use self-directed learning time to review lecture and laboratory content in preparation for TBL exercises. Our results show that students who will ultimately struggle with mastery of course content, as assessed by the final examination, can be identified early in the course based on iRAT scores. The poor correlation between iRAT scores and final examination grades in the upper and middle 33rd percentile indicates that the ability of iRAT scores to predict final examination scores in these groups is limited. However, these students likely benefit from the preparation necessary for TBL exercises and, importantly, can provide support to struggling students in the mastery of course content during the gRAT exercise.

We also found that weekly iRAT scores consistently improve in students in the upper and middle 33rd percentiles, while students in the lower 33rd percentile only show improvement in the last TBL exercise, a few days prior to the final examination. For instance, in year 1, students in the upper 33rd percentile improved their iRAT scores from 7.42(1.79) in the first week to 8.88(1.07) p < 0.01 and 8.38(1.06) p < 0.05 in weeks two and four. Students participating in the ID course have participated of several TBL exercises prior to this course; therefore we do not believe that our data reflects adaptation of student behavior to this learning activity. The pre-class assignments are clearly identified in the course syllabus and orientation as well as in communications provided weekly prior to each TBL. An alternative explanation is that students that perform better in the course improve their pre-class preparation each week because they study more consistently and therefore better achieve the self-directed learning habits necessary for mastery of course content. On the other hand, students that perform more poorly in the course only show improved iRAT scores in the final week, a time where they are likely attempting to master all of the course content in preparation for the final examination. Of note, in some academic years we did not detect significant (p > 0.05) differences in iRAT scores between the first and third weeks, a phenomenon that could be explained by final examinations in other first year courses, which take place during that week.

Finally, the results presented in this manuscript are supported by the validity and reliability of ID course final examination. The office of assessment at our institution analyzes all summative assessments for various forms of validity and reliability in conjunction with course directors. As such, each examination question is linked to individual educational sessions ensuring the assessment tool has equal representation across sessions. Additionally, educational session goals and objectives are compared to question content to ensure content validity. An examination review committee, comprised of faculty members with various areas of expertise and the office of assessment, review psychometrics related to each examination question annually to look for poorly performing questions and evaluate for potential confounding variables which can affect question reliability over time. This committee is also tasked with ensuring construct validity. The examination review committee reviews thousands of questions with virtually the same group members for multiple academic years to ensure inter-rater reliability. The office of assessment also compares student written examination performance with other national test performance standards to look for relationships between internal assessment performance and national assessment tool performance. This office also conducts a detailed question item analysis over time to ensure that examinations questions are meeting internal reliability standards.

Conclusions

Overall, this study shows that consistent review of course content, as encouraged by weekly TBL exercises, is correlated with successful course mastery in undergraduate medical education. We initially hypothesized that students with higher performance on the course final examination would show weekly improvements in their iRAT performance, likely reflecting consistent review of course materials. Our data suggest that while better performers in the course show improved iRAT performance each week, poorly performing students do not show this improvement, possibly due to failure to improve preparation for TBL exercises in the first three weeks of the course. Poorer preparation likely interferes with the efficacy of this learning method for these students and potentially their TBL teammates. As such, changes in incentives for preparation for TBL exercises could provide more tangible rewards for pre-class preparation, in particular for these students. Our study also shows that poor performance in TBL may identify a population of students that will perform poorly on the final examination. These students may need additional interventions to improve time management for self-directed learning [19] and coping skills to address the stresses inherent to medical education [20].

Abbreviations

ANOVA:

Analysis of variance

gRAT:

group readiness assurance test

ID course:

infectious diseases course

iRAT:

individual readiness assurance test

r:

Pearson correlation coefficient

TBL:

Team-based learning

References

  1. 1.

    Zgheib NK, Simaan JA, Sabra R. Using team-based learning to teach pharmacology to second year medical students improves student performance. Med Teach. 2010;32(2):130–5.

  2. 2.

    Sparrow SM, McCabee MS. Team-based learning in law. Social Sc Res Network. 2012;158:153–208.

  3. 3.

    Thistlethwaite JE, Davies D, Ekeocha S, Kidd JM, MacDougall C, Matthews P, Purkis J, Clay D. The effectiveness of case-based learning in health professional education. A BEME systematic review: BEME guide no. 23. Med Teach. 2012;34(6):e421–44.

  4. 4.

    Wanzek J, Kent SC, Vaughn S, Swanson EA. Implementing team-based learning in middle school social studies classes. J Educ Res. 2015;108(4):331–44.

  5. 5.

    Behling KC, Murphy MM, Mitchell-Williams J, Rogers-McQuade H, Lopez OJ. Team-based learning in a pipeline course in medical microbiology for under-represented student populations in medicine improves learning of microbiology concepts. J Microbiol Biol Educ. 2016;17(3):370–9.

  6. 6.

    Carrasco GA, Behling KC, Lopez OJ. Evaluation of the role of incentive structure on student participation and performance in active learning strategies: a comparison of case-based and team-based learning. Med Teach. 2018;40(4):379–86.

  7. 7.

    Behling KC, Gentile M, Lopez OJ. The effect of graded assessment on medical student performance in TBL exercises. Med Sc Educator. 2017;27(3):451–5.

  8. 8.

    Behling KC, Kim R, Gentile M, Lopez O. Does team-based learning improve performance in an infectious diseases course in a preclinical curriculum? Int J Med Educ. 2017;8(8):39–44.

  9. 9.

    Thompson BM, Haidet P, Borges NJ, Carchedi LR, Roman BJ, Townsend MH, Butler AP, Swanson DB, Anderson MP, Levine RE. Team cohesiveness, team size and team performance in team-based learning teams. Med Educ. 2015;49(4):379–85.

  10. 10.

    Koles PG, Stolfi A, Borges NJ, Nelson S, Parmelee DX. The impact of team-based learning on medical students' academic performance. Acad Med. 2010;85(11):1739–45.

  11. 11.

    Parmelee D, Michaelsen LK. Team-based learning: it's here and it WORKS. Acad Med. 2010;85(11):1658 author reply 1658-59.

  12. 12.

    Michaelsen L, Sweet M. New Directions for Teaching and Learning. In: MLS M, editor. The Essential Elements of Team Based Learning: Wylie; 2008. p. 7–27.

  13. 13.

    Parmelee D, Michaelsen LK, Cook S, Hudes PD. Team-based learning: a practical guide: AMEE guide no. 65. Med Teach. 2012;34(5):e275–87.

  14. 14.

    Wahawisan J, Salazar M, Walters R, Alkhateeb FM, Attarabeen O. Reliability assessment of a peer evaluation instrument in a team-based learning course. Pharm Pract (Granada). 2016;14(1):676.

  15. 15.

    Deardorff AS, Moore JA, McCormick C, Koles PG, Borges NJ. Incentive structure in team-based learning: graded versus ungraded group application exercises. J Educ Eval Health Prof. 2014;11:6.

  16. 16.

    Fete MG, Haight RC, Clapp P, McCollum M. Peer evaluation instrument development, administration, and assessment in a team-based learning curriculum. Am J Pharm Educ. 2017;81(4):68.

  17. 17.

    Burgess A, Bleasel J, Haq I, Roberts C, Garsia R, Robertson T, Mellis C. Team-based learning (TBL) in the medical curriculum: better than PBL. BMC Med Educ. 2017;17(1):243.

  18. 18.

    Koh YYJ, Rotgans JI, Rajalingam P, Gagnon P, Low-Beer N, Schmidt HG. Effects of graded versus ungraded individual readiness assurance scores in team-based learning: a quasi-experimental study. Adv Health Sci Educ Theory Pract. 2019. https://doi.org/10.1007/s10459-019-09878-5.

  19. 19.

    Pulfrey C, Butera F. Why neoliberal values of self-enhancement lead to cheating in higher education: a motivational account. Psychol Sci. 2013;24(11):2153–62.

  20. 20.

    Smith CK, Peterson DF, Degenhardt BF, Johnson JC. Depression, anxiety, and perceived hassles among entering medical students. Psychol Health Med. 2007;12(1):31–9.

Download references

Acknowledgements

Not applicable.

Funding

These studies were funded by the International Association of Medical Science Educators (IAMSE) Educational Scholarship Grant.

Availability of data and materials

The deidentified datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request. The questionnaire used in final examinations and TBL exercises are part of our database and are not to be released.

Author information

OJL was course director of the ID Course during the first two years of this study, and GC was course director during the last year. GAC, KCB and OJL analyzed the data and wrote and edited the current manuscript. All authors read and approved the final manuscript.

Correspondence to Gonzalo A. Carrasco or Osvaldo J. Lopez.

Ethics declarations

Ethics approval and consent to participate

These studies were approved by the Rowan Institutional Review Board (IRB). Consent to Participate was not required and/or obtained from the students as the need for consent was waived by the Rowan University IRB.

Consent for publication

Not applicable.

Competing interests

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Keywords

  • Active learning
  • Team-based learning
  • Individual readiness assurance test
  • Performance
  • Self-directed learning
  • Pre-class preparation