Author | Study design | Data collection methods | Number | Country of origin | Profession | Aim of study (implied/stated) | Type of WBA | Grade of Strength of findings and main findings |
---|---|---|---|---|---|---|---|---|
Archer et al [7] | Survey | Analysis of MSF data | 4770 | United Kingdom | Paediatrics | To report the evidence for and challenges to the validity of Sheffield Peer Review Assessment Tool (SPRAT) with paediatric specialist trainees across the UK as part of Royal College of Paediatrics and Child Health workplace based assessment programme. | SPRAT (MSF) | Grade 3. Assessor seniority is important. Free text boxes allow feedback for personal development |
Bullock et al [8] | Survey | Analysis of MSF data | 1928 | United Kingdom | Junior doctors | To address differences in staff groups in their assessment of junior doctors’ professional attitudes and behaviour. | TAB (MSF) | Grade 3. Peers and administrators were less likely to indicate concern compared to consultants and senior nurses. |
Burford et al [9] | Survey | Questionnaire | 901 | United Kingdom | Junior doctors and trainers | To compare perceptions of two tools for giving MSF to UK junior doctors, based on usability, usefulness and validity. | MSF | Grade 3. Trainees were asked in detail whether they would change their behaviour. Attitudes towards MSF in principle were positive and tools felt to be usable. Text-oriented tool rated more useful for giving feedback on communication and attitude |
Canavan et al [10] | Survey | Questionnaire | 970 | United States | Five medical and one surgical specialty | To assess qualitatively written comments on multisource assessments based on psychological feedback theory for professional development | MSF | Grade 3. Quality of written feedback varies; a substantial portion of comments were useless and at worst detrimental to progress |
Violato et al [11] | Longitudinal comparative study | Forms analysed on two occasions, 5 years apart | 250 | Canada | General Practice | Examining the validity and reliability of MSF for general practice and whether it has led to change in performance when reassessed in 5 years | MSF | Grade 4. There is evidence for the construct validity of the instruments and stability over time |
Sargeant et al [12] | Qualitative study | Interviews | 28 | Canada | General Practice | To increase understanding of the consequential validity of MSF by exploring how doctors used their feedback and the conditions influencing this use. | MSF | Grade 3.Feedback usefulness enhanced by increasing its specificity. Strong influence of direct patient feedback on doctors’ performance |
Sargeant et al [13] | Observational study | Focus group | 15 | Canada | General Practice | Exploration of physicians’ reactions to MSF, perceptions influencing these and the acceptance and use of feedback | MSF | Grade 3. Physicians’ perceptions of the MSF process and feedback can influence how and if they use the feedback for practice improvement. |
Wellor et al [14] | Observational study | Questionnaire based ratings and written answers | 92 | New Zealand | Anaesthetics | To evaluate mini-CEX for both summative and formative assessment for anaesthetics training | Mini CEX | Grade 3. Factors that facilitated or hindered implementation or limited effective feedback were identified |
Wellor et al [15] | Survey | Analysis of mini-CEX forms | 331 | New Zealand | Anaesthetics | Psychometric characteristics, logistics of application, and impact on the quality of supervision of the mini- CEX | Mini CEX | Grade 3. The positive effect of the mini CEX on feedback, its relative feasibility, and acceptance as a potential assessment tool was demonstrated |
Holmboe et al [16] | Observational study | Videotaping of feedback sessions | 107 | United States | Primary care and internal medicine | To examine how often faculty provided recommendations and used interactive techniques when providing feedback as part of a mini CEX. | Mini CEX | Programs should consider both specific training in feedback and changes to the miniCEX form to facilitate interactive feedback. |
James et al [17] | Observational study | Times taken to complete the consenting and operative components of the forms were recorded. | 22 | United Kingdom | Surgery | Assessing the time required to complete PBA forms and ease of use in the surgical workplace. | PBA | Grade 3. PBAs are feasible in clinical practice and are valued by trainees as a means of enabling focused feedback and targeted training. |
Marriot et al [18] | Prospective observational study | Direct observation using the PBA. | 749 | United Kingdom | Surgery | The aims were to evaluate the validity, reliability and acceptability of PBA. | PBA | Grade 3. PBA demonstrated good overall validity and acceptability, and exceptionally high reliability. |
Murphy et al [19] | Prospective study | Questionnaire | 171 | United Kingdom | General Practice | To investigate the reliability and feasibility of six potential workplace-based assessment methods | MSF, criterion audit, patient feedback, referral letters, significant event analysis, and video analysis of consultations. | Grade 3. Two WBA tools involving patient and colleague feedback have high reliability suitable for high stakes WBA in the general practice setting. |
Cohen et al [20] | Survey | Questionnaire | 138 | United Kingdom | Dermatology | To collate the experience and views on three workplace assessments | DOPS, mini- CEX, MSF | Grade 3.Trainees appreciate the formative benefits which derive from the assessments, namely feedback and reassurance of satisfactory performance. |
Johnson et al [21] | Observational study | Questionnaires and focus groups | 120 | United Kingdom | Medicine | To gain feedback from trainees and supervisors in relation to components of core medical training including workplace- based assessments, | All WBA | Grade 4.WPBA assessments were well received as means of evidencing achievement and for learning development The majority of trainees felt that in particular the feedback following WBA assessments had been useful. |