From: The social validity of a national assessment centre for selection into general practice training
1. Provision of relevant information regarding the job and the organisation. | |||||
---|---|---|---|---|---|
Positive Features | Negative Features | ||||
71 (8%) | Candidates’ experience of the NAC provided insight and understanding into the requirements of general practice training. | “I received a wider knowledge of common issues in GP practice in Australia”. | 58 (7%) | Candidates perceived there to be inadequate information provided prior to attendance at the NAC. | “There was a lack of references available or a guide to the type of preparation required”. |
“Little information beforehand re. format of the day”. | |||||
“Very real scenarios allowing me to know what to expect from GP. I now have an understanding that some scenarios require quick decision making skills”. | “Little more material for preparation especially SJT as quite a specific MCQ style not everyone would be familiar with”. | ||||
2. Opportunity to participate and display relevant knowledge and skills. | |||||
Positive Features | Negative Features | ||||
384 (43%) | Candidates valued the face to face interaction with interviewers during the MMI and the opportunity to demonstrate their experience and skills. | “Face to face MMI was good as it provides an opportunity to explain answers and discuss”. | 380 (43%) | Candidates expressed concern about inadequate time to complete the SJT and difficulty in interpreting questions. | “English is not my first language so there was not enough time in the SJT to read lengthy paragraphs”. |
“Able to highlight past skills and experience”. | SJT questions were “incredibly vague”, “ambiguous”, and “confusing”. | ||||
“….allows me to show clear clinical judgement and reasoning”. | “SJT – difficult because no opportunity to justify answers”. | ||||
62 (7%) | Candidates valued the opportunity for assessment of areas other than knowledge and clinical skills. | “It assessed me on different skills both personal and professional”. | 83 (9%) | Candidates found it difficult to respond to the MMI questions due to lack of relevant experience. | “Very GP focused in some ways. Not all of us have had GP rotations yet!” |
“Allows me to speak for myself and show my personality traits, not just on paper”. | “Not yet having experience in GP and many of the MMI questions referred to a GP scenario”. | ||||
3. Transparency of the selection process and the selection tools. | |||||
Positive Features | Negative Features | ||||
186 (21%) | Generally, the candidates found the NAC selection process to be fair and equitable. | “Fair across all criteria and all states”. | 189 (21%) | Candidates had difficulty in interpreting the MMI questions – or the style of the interviewer. They were unsure what some questions were asking, and needed more guidance. | “I had difficulty with clarification of questions during the MMI”. |
“It did not appear to matter the level at which one is working ie. Intern vs more senior”. | “For one of the MMI stations the examiner started to use closed questions almost immediately without letting me explore the situation. Given that, I felt thrown off guard”. | ||||
“SJT is anonymous - takes out role of interviewer bias when it comes to candidates”. | “Some questions were not very clear, so it’s hard to address solutions”. | ||||
4. Provision of feedback regarding results | |||||
Positive Features | Negative Features | ||||
32 (4%) | Although applicants felt that the process provided a suitable ranking method to inform RTPs, they reflected on expectations about feedback on their performance. | “Allows ranking for RTP's and finds suitability”. | 52 (6%) | Candidates found it difficult to gauge their own performance. They expressed a desire for some kind of immediate, formative individual feedback. | “The MMI stations were vague with no immediate feedback.” |
“Allows differentiation between candidates”. | “Interviews are always stressful - hard when you don't get any feedback.” | ||||
“Get an idea about the issues faced at the GP practices. Also get feedback for further improvements in areas which you are not familiar”. | “No feedback regarding our performance (often subtle feedback is given even at the most formal college interview processes)”. | ||||
“Face to face examination allows assessment of communication skills and immediate feedback from examiner, to ensure complete answering of questions” |