Impact of student choice on academic performance: cross-sectional and longitudinal observations of a student cohort

Background Student choice plays a prominent role in the undergraduate curriculum in many contemporary medical schools. A key unanswered question relates to its impact on academic performance. Methods We studied 301 students who were in years 2 and 3 of their medical studies in 2005/06. We investigated the relationship between SSC grade and allocated preference. Separately, we examined the impact of ‘self-proposing’ (students designing and completing their own SSC) on academic performance in other, standard-set, summative assessments throughout the curriculum. The chi-squared test was used to compare academic performance in SSC according to allocated preference. Generalised estimating equations were used to investigate the effect of self-proposing on performance in standard-set examinations. Results (1) Performance in staff-designed SSC was not related to allocated preference. (2) Performance in year 1 main examination was one of the key predictors of performance in written and OSCE examinations in years 2, 3 and 4 (p<0.001). (3) The higher the score in the year 1 examination, the more likely a student was to self-propose in subsequent years (OR [CI] 1.07 [1.03-1.11], p<0.001). (4) Academic performance of students who self-proposed at least once in years 2 and/or 3 varied according to gender and year of course. Conclusion In this study, no association was observed between allocated preference and SSC grade. The effect of self-proposing on academic performance in standard-set examinations was small. Our findings suggest instead that academically brighter students are more likely to design their own modules. Although student choice may have educational benefits, this report does not provide convincing evidence that it improves academic performance.


Conclusion:
In this study, no association was observed between allocated preference and SSC grade. The effect of self-proposing on academic performance in standard-set examinations was small. Our findings suggest instead that academically brighter students are more likely to design their own modules. Although student choice may have educational benefits, this report does not provide convincing evidence that it improves academic performance.

Background
General Student choice has long featured in undergraduate learning, but recently has played a more prominent role than previously within the curriculum in British medical schools. This followed recommendations made by the General Medical Council in Tomorrow's Doctors, first published in 1993 [1]. In that document, allowing students to express their individuality and to explore areas of particular interest to them was seen as crucial to harness their engagement with the process of reform.
Intercalated degree courses, and electives, whilst laudable, were deemed insufficient on their own to provide the degree of choice envisagedstudent-chosen modules would have to be embedded throughout the entire curriculum, forming "a thread running throughout the course rather than confined to a discrete period" [1]. In subsequent editions of Tomorrow's Doctors, the proportion of curriculum time that must be devoted to student-chosen modules has been progressively reduced, but their stated purpose remains "the intellectual development of the student through exploring in depth a subject of their choice" [2].
Student-selected components (SSC), as they are now known, provide students with experience of learning in small groups, and opportunities to develop self-directed learning skills. Some medical schools allow students to design their own modules (within the boundaries of the curriculum), permitting an unrivalled degree of student ownership over learning. However, their accommodation within the undergraduate curriculum has required substantial changes [3]. SSC programmes are administratively complex, and standardisation of assessment is problematic [4]. Moreover, providing students with genuine choice from diverse programmes of high-quality modules presents an ongoing challenge in many institutions [5]. Evidence of added value is therefore important.
A key unanswered question relates to the impact of student choice on academic performance. If it could be shown that student choice resulted in better academic performance, this would be important evidence of benefit. In the current report, we posed the question in cross-sectional and longitudinal ways: (1) Is the performance of students in SSC affected by the allocated preference (i.e. whether the allocated module is their first or other choice)? (2) Does the experience of designing and completing their own modules affect the performance of students in other summative assessments?

Student-selected components in Dundee
The student-selected 'thread' of the Dundee undergraduate curriculum is outlined in Figure 1. Students perform longitudinal exercises in years 1 and 4 of a 5-year curriculum, e.g. literature reviews, data analyses, etc. In years 2, 3 and 5, SSC are undertaken in discrete blocks, which occupy approximately one-third of curriculum time. The undergraduate curriculum was administratively divided into three phases: Phase 1 (year 1), Phase 2 (years 2 and 3) and Phase 3 (years 4 and 5). In Phase 2, sixteen weeks were devoted to SSC, all of which were either two or four weeks long. Students were allowed to choose from a menu of staff-designed SSC, and/or to design ('selfpropose') their own modules. No restriction or obligation was placed on them to self-propose; many completed both staff-designed and student-designed modules. However, it was made clear to them that self-proposing was the only way of guaranteeing allocation to a preferred topic or unit. Allocation to staff-designed SSC was based on iterative computer-based matching of ranked student preferences with available places.

Student-designed modules ('self-proposed' SSC)
The self-proposal process has been described previously [6]. Briefly, self-proposing students were required initially to contact potential supervisors and, after consultation with them, to submit a written proposal detailing the educational objectives and learning outcomes of the proposed module. The detailed substance of individual self-proposals was arrived at independently and the course-work was not linked to the work of previous students. Proposals were subsequently modified as appropriate after discussion with the SSC Convenor and their supervisor, in a series of iterations of educational objectives in terms of detail and focus. Thus, selfproposing students had, before they started, invested substantial time and effort in establishing realistic, feasible objectives and, more generically, in defining the educational content of their module.

Students
Students who were in years 2 and 3 of the curriculum in 2005/06 were studied. 301 students undertook 1551 SSC modules (including SPSSC modules). 188 (62.5%) were female; 113 (37.5%) male. Students who had returned to year 4 of their medical studies after completing an intercalated degree were excluded from the analysis of performance in year 4 summative examinations (although their data were included for the analysis of performance in earlier years of the medical curriculum). Graduate entrants were also excluded from analysis. The skills gained during completion of undergraduate degrees may have confounded our analyses, because they might affect (a) the academic performance of graduate students in assessments [7] and (b) their threshold for self-proposing.
For the analysis of academic performance in standardset examinations, students were categorised by whether or not they had, at any time during years 2 and 3 of their medical studies, designed their own SSC module ('selfproposed'). Thus, students were categorised as either 'ever self-proposed' if they had, or 'never self-proposed' if they had not. Assessment SSC assessment was summative and separate from other parts of the curriculum. Formal SSC assessment was divided into three sections, with corresponding weightings: 70% of the marks were allocated to written course-work; 15% to interest/motivation; 15% to reliability. Grade descriptors were generic, reflecting the heterogeneous nature of the SSC programme. Standard-set examinations consisted of written or online examinations and, from year 2, objective structured clinical examinations (OSCE). A modified Angoff approach was used to standard-set these examinations. Briefly, this is a test-centred, criterionreferenced approach that relies on expert judgements of how borderline candidates will perform [8].

Statistical methods
The chi-squared goodness-of-fit test was used to compare academic performance in staff-designed SSC according to allocated preference.
Generalised estimating equations (GEE) were used to investigate the effect of undertaking a self-proposed SSC on performance in standard-set examinations throughout the curriculum. Examination scores in adjacent years were more highly correlated than those more than one year apart, so the AR (1) correlation matrix structure was used. The best-fit model was identified using the Quasi Likelihood under Independence Model Criterion (QIC). The score attained in the year 1 main examination was the first available measure for each student, and was used as a covariate in GEE analyses. It was compared in the relevant self-proposal groups ('ever' and 'never') in order to investigate base-level differences in performance in standard-set examinations. Only data from first attempts were included in the analyses.
The data reported here were collected as part of routine quality assurance of undergraduate medical training in our institution. Ethical principles were adhered to in the retrieval and analysis of the data, and a waiver obtained from the University of Dundee Research Ethics Committee.

Results
Phase 2 SSC grades according to preference SSC grades awarded to students allocated to staffdesigned SSC were compared according to preference (first-, second-, third-, and other choice allocations). Performance was not related to allocated preference, either in females (p=0.79) or males (p=0.49) ( Table 1).

Factors affecting performance in standard-set examinations including self-proposal status
Student numbers varied from one year to the next, due, for example, to termination of studies, withdrawal for health reasons, etc. Of the 205 students for whom pro-gression information was available, 101 (49.3%) selfproposed at some stage during Phase 2, i.e. in year 2 and/or in year 3. 65 of these (64.4%) were female. The higher the score in the year 1 main examination, the more likely a student was to self-propose: OR [CI], P 1.07 [1.03-1.11, p<0.001].
The main predictors of performance in written and OSCE examinations in years 2, 3 and 4 were performance in the year 1 main examination, and year of course (p≤0.001 for these associations, Table 2). Gender was also a significant factor in OSCE examinations; females scored more highly than males. Self-proposal status had no impact on examination performance as a main effect, although an interaction with gender was observed in written (p=0.006) and OSCE examinations (p=0.015) ( Table 2). Table 3 summarises student performance in standardset examinations from years 1 through 4, by gender and self-proposal status. (In year 1 there was no OSCE examination). The only comparisons significant at the 5% level were in OSCE exams at years 3 and 4: in both cases, males who self-proposed scored more highly than those who did not. There were no similar findings in females.

Discussion
Given the importance afforded to student choice, and the scale of the task of trying to provide genuine choice to all students throughout the entire undergraduate curriculum [5], it would be helpful if it could be shown that student choice results in better academic performance. However, the heterogeneous nature of most SSC programmes means that differences between modules in educational content and in the assessment of students confound interpretation of academic performance in SSC. For these reasons, in the current report, we combined cross-sectional and longitudinal approaches to the analysis of student performance, in SSC, and in standard-set examinations throughout the curriculum. Our cross-sectional analysis did not show an association between allocated preference and performance in SSC modules. However, firm conclusions cannot be drawn from this about the relationship between student choice and performance in SSC. Our longitudinal analysis suggests that the experience of designing and completing their own SSC modules had only a slight impact overall on student performance in standard-set examinations. Self-proposal status had no impact on performance as a main effect (Table 2); the interaction with gender, although statistically significant, reflected small gender differences in performance between ever and never self-proposers (Table 3). By contrast, the score attained in the first year main standard-set examination was one of the main predictors of subsequent academic performance, and also predicted subsequent self-proposal. Taken together these findings suggest that better-  performing students are more likely to self-propose rather than the other way round. Our study has several strengths. First, our longitudinal analysis allowed us better to assess the causality of observed associations between student choice and academic performance. Second, we assessed performance in standard-set examinations, thus removing a potential confounder. Third, we took into account some of the factors known to affect academic performance, such as gender [9] and graduate status [7]. Fourth, the amount of curriculum time devoted to SSC (one-third) was in line with the recommendations of the original Tomorrow's Doctors [1]; if it had followed the most recent recommendation (10%) [2], a potential criticism may have been that the amount of time was insufficient to allow us to observe an association. Weaknesses include the fact that our study pertains to 2005/06, although the most likely relevant intervening change in undergraduate curriculum planning will have been a reduction in time devoted to SSC (see previous point). Finally, we did not take ethnicity into account in our analyses.

Conclusion
This is the first study to examine the association between student choice and academic performance. We studied a large, well-characterized cohort of students, and used cross-sectional and longitudinal approaches. Our findings provide little convincing evidence that student choice affects academic performance in our undergraduate curriculum. Rather, they suggest that academically brighter students are more likely to design their own modules.