Skip to main content

Table 1 Outcome measures

From: A randomised controlled trial of extended immersion in multi-method continuing simulation to prepare senior medical students for practice as junior doctors

Point in study

Outcome measure name

Details

Outcomes compared

End of in-school week in Year 3

MCQ3

20x ‘choose one from five’ Multiple Choice Questions based on the content of the seminars and workshops undertaken during the Year 3 in-school week by participants in both study groups.

Number correct out of 20, for each individual participant.

SCQ3

50x Script Concordance Questions, formulated and scored using the method described by Fournier’s group [28], to assess clinical reasoning in relation to conditions covered in the seminars and workshops undertaken during the Year 3 in-school week by participants in both study groups.

Total score out of 50, for each individual participant.

PS3

A structured prescribing exercise comprising a printed scenario describing the circumstances of a typical hospital medical admission (a 64 year old woman with an infective exacerbation of COPD), including a GP referral letter and ‘registrar directions’, with the following request to the participant: ‘Please fill out a standard hospital drug chart to prescribe her usual medications and follow your registrar’s directions. You may add any other therapy you think is clinically appropriate at this time.’ Scored according to a structured score matrix by an expert (blinded to participants’ study group) in relation to therapeutic decisions, medication safety and technical aspects of chart completion. All of these issues were covered in workshops undertaken during the Year 3 in-school week by participants in both study groups.

Total score out of 100, for each individual participant.

RS3

Team resuscitation exercise in which each participant team is distracted by being led to believe that they are going to be tested on urinary catheterisation, then a ‘nurse’ in the adjoining room bursts in seeking their help, saying that her patient has ‘stopped breathing’. The team is then required to assess the patient (a simulation mannequin) and initiate cardiopulmonary resuscitation. Whole exercise is video recorded and timed from the recording post hoc.

Time from call for help to initiation of cardiac compressions, for each team.

End of Year 3 academic year

SDP3

Final summative assessment scores for the Doctor and the Patient (clinical skills) theme for Year 3 of the Medical Program (comprised of Mini-CEX assessments from each clinical rotation, multiple choice and ‘mini-case’ short answer questions).

Final score out of 269 marks for each individual participant.

SOV3

Final summative assessment scores for Year 3 of the entire Medical Program (comprised of SDP3 plus multiple choice, ‘mini-case’ short answer questions and assignments in the other themes).

Final score out of 672 marks, for each individual participant.

Beginning of in-school week in Year 4 (‘retention’ measures from previous year)

MCQR

Identical to MCQ3 but administered an average of nine months later to measure ‘retention’ of knowledge and understanding acquired in Year 3 in-school week.

Number correct out of 20, for each individual participant.

SCQR

Identical to SCQ3 but administered an average of nine months later to measure ‘retention’ of knowledge and understanding acquired in Year 3 in-school week.

Total score out of 50, for each individual participant.

PSR

Identical to PS3 but administered an average of nine months later to measure ‘retention’ of knowledge, understanding and skills acquired in Year 3 in-school week.

Total score out of 100, for each individual participant.

RSR

Identical to exercise to RS3 but undertaken in Year 4 allocation teams, an average of nine months later to measure ‘retention’ of knowledge, understanding and skills acquired in Year 3 in-school week and its transferability to new team settings.

Time from call for help to initiation of cardiac compressions, for each Year 4 team.

End of in-school week in Year 4

MCQ4

25x ‘choose one from five’ Multiple Choice Questions based on the content of the seminars and workshops undertaken during the Year 4 in-school week by participants in both study groups.

Number correct out of 25, for each individual participant.

SCQ4

30x Script Concordance Questions, formulated and scored using the method described by Fournier’s group [28], to assess clinical reasoning in relation to conditions covered in the seminars and workshops undertaken during the Year 4 in-school week by participants in both study groups.

Total score out of 30, for each individual participant.

PS4

A similar exercise to PS3 but based on a more complex patient scenario involving anticoagulant usage, cardiac and respiratory disease. Scored by same method as PS3.

Total score out of 100, for each individual participant.

RS4

Team resuscitation exercise in which each participant team plays the part of an emergency department team receiving a seriously ill child who has collapsed following prolonged vomiting and diarrhoea at home and has been brought in by ambulance. The team is required to receive hand over from the ‘paramedic’, assess the patient (a simulation mannequin) who was pulseless and apnoeic, then initiate appropriate urgent treatment. Whole exercise is video recorded and timed from the recording post hoc.

Time from entering room to initiation of fluid resuscitation, for each team.

Summative assessment in Year 4 (after conclusion of all in-school weeks)

SDP4

Final summative assessment scores for the Doctor and the Patient (clinical skills) theme for Year 4 of the Medical Program (comprised of Mini-CEX assessments from each clinical rotation and multiple stations in an Objective Structured Clinical Examination [OSCE]).

Final score out of 204.5 marks for each individual participant.

SOS4

Total score on all stations in the final OSCE for the Medical Program (including stations from all three themes using the OSCE as an assessment tool).

Final score out of 211.5 marks for each individual participant.

SOV4

Final summative assessment scores for Year 4 of the entire Medical Program (comprised of SDP4 plus multiple choice, ‘mini-case’ short answer questions, OSCE stations and assignments in the other themes).

Final score out of 516.5 marks for each individual participant.