Skip to main content

Table 3 Summary of studies included in the review

From: The role of feedback in improving the effectiveness of workplace based assessments: a systematic review

Author Study design Data collection methods Number Country
of origin
Profession Aim of study (implied/stated) Type of WBA Grade of Strength of findings and main findings
Archer et al [7] Survey Analysis of MSF data 4770 United Kingdom Paediatrics To report the evidence for and challenges to the validity of Sheffield Peer Review Assessment Tool (SPRAT) with paediatric specialist trainees across the UK as part of Royal College of Paediatrics and Child Health workplace based assessment programme. SPRAT (MSF) Grade 3. Assessor seniority is important. Free text boxes allow feedback for personal development
Bullock et al [8] Survey Analysis of MSF data 1928 United Kingdom Junior doctors To address differences in staff groups in their assessment of junior doctors’ professional attitudes and behaviour. TAB (MSF) Grade 3. Peers and administrators were less likely to indicate concern compared to consultants and senior nurses.
Burford et al [9] Survey Questionnaire 901 United Kingdom Junior doctors and trainers To compare perceptions of two tools for giving MSF to UK junior doctors, based on usability, usefulness and validity. MSF Grade 3. Trainees were asked in detail whether they would change their behaviour. Attitudes towards MSF in principle were positive and tools felt to be usable. Text-oriented tool rated more useful for giving feedback on communication and attitude
Canavan et al [10] Survey Questionnaire 970 United States Five medical and one surgical specialty To assess qualitatively written comments on multisource assessments based on psychological feedback theory for professional development MSF Grade 3. Quality of written feedback varies; a substantial portion of comments were useless and at worst detrimental to progress
Violato et al [11] Longitudinal comparative study Forms analysed on two occasions, 5 years apart 250 Canada General Practice Examining the validity and reliability of MSF for general practice and whether it has led to change in performance when reassessed in 5 years MSF Grade 4. There is evidence for the construct validity of the instruments and stability over time
Sargeant et al [12] Qualitative study Interviews 28 Canada General Practice To increase understanding of the consequential validity of MSF by exploring how doctors used their feedback and the conditions influencing this use. MSF Grade 3.Feedback usefulness enhanced by increasing its specificity. Strong influence of direct patient feedback on doctors’ performance
Sargeant et al [13] Observational study Focus group 15 Canada General Practice Exploration of physicians’ reactions to MSF, perceptions influencing these and the acceptance and use of feedback MSF Grade 3. Physicians’ perceptions of the MSF process and feedback can influence how and if they use the feedback for practice improvement.
Wellor et al [14] Observational study Questionnaire based ratings and written answers 92 New Zealand Anaesthetics To evaluate mini-CEX for both summative and formative assessment for anaesthetics training Mini CEX Grade 3. Factors that facilitated or hindered implementation or limited effective feedback were identified
Wellor et al [15] Survey Analysis of mini-CEX forms 331 New Zealand Anaesthetics Psychometric characteristics, logistics of application, and impact on the quality of supervision of the mini- CEX Mini CEX Grade 3. The positive effect of the mini CEX on feedback, its relative feasibility, and acceptance as a potential assessment tool was demonstrated
Holmboe et al [16] Observational study Videotaping of feedback sessions 107 United States Primary care and internal medicine To examine how often faculty provided recommendations and used interactive techniques when providing feedback as part of a mini CEX. Mini CEX Programs should consider both specific training in feedback and changes to the miniCEX form to facilitate interactive feedback.
James et al [17] Observational study Times taken to complete the consenting and operative components of the forms were recorded. 22 United Kingdom Surgery Assessing the time required to complete PBA forms and ease of use in the surgical workplace. PBA Grade 3. PBAs are feasible in clinical practice and are valued by trainees as a means of enabling focused feedback and targeted training.
Marriot et al [18] Prospective observational study Direct observation using the PBA. 749 United Kingdom Surgery The aims were to evaluate the validity, reliability and acceptability of PBA. PBA Grade 3. PBA demonstrated good overall validity and acceptability, and exceptionally high reliability.
Murphy et al [19] Prospective study Questionnaire 171 United Kingdom General Practice To investigate the reliability and feasibility of six potential workplace-based assessment methods MSF, criterion audit, patient feedback, referral letters, significant event analysis, and video analysis of consultations. Grade 3. Two WBA tools involving patient and colleague feedback have high reliability suitable for high stakes WBA in the general practice setting.
Cohen et al [20] Survey Questionnaire 138 United Kingdom Dermatology To collate the experience and views on three workplace assessments DOPS, mini- CEX, MSF Grade 3.Trainees appreciate the formative benefits which derive from the assessments, namely feedback and reassurance of
satisfactory performance.
Johnson et al [21] Observational study Questionnaires and focus groups 120 United Kingdom Medicine To gain feedback from trainees and supervisors in relation to components of core medical training including workplace- based assessments, All WBA Grade 4.WPBA assessments were well received as means of evidencing achievement and for learning development The majority of trainees felt that in particular the feedback following WBA assessments had been useful.