Table 3 Summary of studies included in the review

Archer et al [7]

Survey

Analysis of MSF data

4770

United Kingdom

Paediatrics

To report the evidence for and challenges to the validity of Sheffield Peer Review Assessment Tool (SPRAT) with paediatric specialist trainees across the UK as part of Royal College of Paediatrics and Child Health workplace based assessment programme.

SPRAT (MSF)

Grade 3. Assessor seniority is important. Free text boxes allow feedback for personal development

Bullock et al [8]

Survey

Analysis of MSF data

1928

United Kingdom

Junior doctors

To address differences in staff groups in their assessment of junior doctors’ professional attitudes and behaviour.

TAB (MSF)

Grade 3. Peers and administrators were less likely to indicate concern compared to consultants and senior nurses.

Burford et al [9]

Survey

Questionnaire

901

United Kingdom

Junior doctors and trainers

To compare perceptions of two tools for giving MSF to UK junior doctors, based on usability, usefulness and validity.

MSF

Grade 3. Trainees were asked in detail whether they would change their behaviour. Attitudes towards MSF in principle were positive and tools felt to be usable. Text-oriented tool rated more useful for giving feedback on communication and attitude

Canavan et al [10]

Survey

Questionnaire

970

United States

Five medical and one surgical specialty

To assess qualitatively written comments on multisource assessments based on psychological feedback theory for professional development

MSF

Grade 3. Quality of written feedback varies; a substantial portion of comments were useless and at worst detrimental to progress

Violato et al [11]

Longitudinal comparative study

Forms analysed on two occasions, 5 years apart

250

Canada

General Practice

Examining the validity and reliability of MSF for general practice and whether it has led to change in performance when reassessed in 5 years

MSF

Grade 4. There is evidence for the construct validity of the instruments and stability over time

Sargeant et al [12]

Qualitative study

Interviews

28

Canada

General Practice

To increase understanding of the consequential validity of MSF by exploring how doctors used their feedback and the conditions influencing this use.

MSF

Grade 3.Feedback usefulness enhanced by increasing its specificity. Strong influence of direct patient feedback on doctors’ performance

Sargeant et al [13]

Observational study

Focus group

15

Canada

General Practice

Exploration of physicians’ reactions to MSF, perceptions influencing these and the acceptance and use of feedback

MSF

Grade 3. Physicians’ perceptions of the MSF process and feedback can influence how and if they use the feedback for practice improvement.

Wellor et al [14]

Observational study

Questionnaire based ratings and written answers

92

New Zealand

Anaesthetics

To evaluate mini-CEX for both summative and formative assessment for anaesthetics training

Mini CEX

Grade 3. Factors that facilitated or hindered implementation or limited effective feedback were identified

Wellor et al [15]

Survey

Analysis of mini-CEX forms

331

New Zealand

Anaesthetics

Psychometric characteristics, logistics of application, and impact on the quality of supervision of the mini- CEX

Mini CEX

Grade 3. The positive effect of the mini CEX on feedback, its relative feasibility, and acceptance as a potential assessment tool was demonstrated

Holmboe et al [16]

Observational study

Videotaping of feedback sessions

107

United States

Primary care and internal medicine

To examine how often faculty provided recommendations and used interactive techniques when providing feedback as part of a mini CEX.

Mini CEX

Programs should consider both specific training in feedback and changes to the miniCEX form to facilitate interactive feedback.

James et al [17]

Observational study

Times taken to complete the consenting and operative components of the forms were recorded.

22

United Kingdom

Surgery

Assessing the time required to complete PBA forms and ease of use in the surgical workplace.

PBA

Grade 3. PBAs are feasible in clinical practice and are valued by trainees as a means of enabling focused feedback and targeted training.

Marriot et al [18]

Prospective observational study

Direct observation using the PBA.

749

United Kingdom

Surgery

The aims were to evaluate the validity, reliability and acceptability of PBA.

PBA

Grade 3. PBA demonstrated good overall validity and acceptability, and exceptionally high reliability.

Murphy et al [19]

Prospective study

Questionnaire

171

United Kingdom

General Practice

To investigate the reliability and feasibility of six potential workplace-based assessment methods

MSF, criterion audit, patient feedback, referral letters, significant event analysis, and video analysis of consultations.

Grade 3. Two WBA tools involving patient and colleague feedback have high reliability suitable for high stakes WBA in the general practice setting.

Cohen et al [20]

Survey

Questionnaire

138

United Kingdom

Dermatology

To collate the experience and views on three workplace assessments

DOPS, mini- CEX, MSF

Grade 3.Trainees appreciate the formative benefits which derive from the assessments, namely feedback and reassurance of

satisfactory performance.

Johnson et al [21]

Observational study

Questionnaires and focus groups

120

United Kingdom

Medicine

To gain feedback from trainees and supervisors in relation to components of core medical training including workplace- based assessments,

All WBA

Grade 4.WPBA assessments were well received as means of evidencing achievement and for learning development The majority of trainees felt that in particular the feedback following WBA assessments had been useful.