Passing MRCP (UK) PACES: a cross-sectional study examining the performance of doctors by sex and country

Study design, setting and source of data

This cross-sectional study was conducted using an international database from the Federation of Royal Colleges of Physicians in the UK, which organises the MRCP (UK) internal medicine specialty exams. The data and permission to use the data for research purposes were obtained from MRCP (UK).

Membership of the Royal Colleges of physicians (UK) diploma

In the UK, doctors who wish to enter into higher specialist training in internal medicine are required to complete a three-part examination known as the Membership of the Royal College of Physicians United Kingdom (MRCP (UK)) Diploma, which aims to test the knowledge, skills, and behaviour of doctors in training [19]. The MRCP (UK) Diploma consists of three parts: MRCP (UK) Part 1; MRCP (UK) Part 2 Written; and MRCP (UK) Part 2 Practical Assessment of Clinical Examination Skills (PACES). Candidates are required to successfully complete all three parts of the exam before they are able to start specialist internal medicine training in the UK [19].

Doctors in many countries outside the UK sit the examination. In some countries (e.g. India) the MRCP (UK) qualification has status similar to local graduate training programmes. In others (e.g. Hong Kong) it forms part of a conjoint qualification. In countries where doctors may have difficulty accessing any formal training programmes, the MRCP (UK) qualification is used to benchmark a doctor’s knowledge and clinical skills against an internationally recognised standard. Completion of the MRCP (UK) diploma can improve a doctor’s chances of getting GMC registration and a license to practise in the UK, but anecdotal evidence suggests the majority of doctors sitting internationally do not come to the UK to work. MRCP (UK) does not collect employment status from candidates in the UK or internationally. It records registration with the GMC but not with any other regulatory bodies. GMC registration can be used as a proxy for current or previous employment in the UK.

The current study focuses on the performance of candidates at the clinical assessment component of the Diploma, PACES. PACES is a structured standardised assessment that was first introduced in 2001, with the aim of providing a valid and reliable assessment of physical examination and communication skills [20]. PACES is run at clinical centres across the UK and in 14 other countries [21]. In 2009, the format of the exam was revised considerably with an aim of ensuring that successful candidates were competent across the range of clinical skills assessed, and that they possessed the attitudes and behaviour required of a specialist trainee in internal medicine [18]. Following a transitional phase between October 2009 and July 2010, the new format of the PACES assessment was introduced for all candidates from October 2010 [18]. Today PACES consists of five stations and a total of eight patient encounters, during which seven core clinical skills relating to communication, physical examination and diagnostic reasoning are assessed (see Fig. 1). Each station has two examiners who independently judge the candidate’s performance (10 examiners per PACES examination). Different skills are assessed at different stations i.e. not all stations test all skills. At each station, each examiner scores the relevant skills as 2 = satisfactory, 1 = borderline, 0 = unsatisfactory. A candidate must achieve a passing score in each skill to pass the examination overall. See https://www.mrcpuk.org/mrcpuk-examinations/paces/paces-examination-format for more details of the examination format and scoring.

PACES examiners must be registered with the General Medical Council (or regulatory equivalent in country of practice), be registered with a licence to practise, and be in good standing. They must also be a Collegiate Member or Fellow in good standing of one of the Royal College of Physicians of the UK. Collegiate Members must have achieved the Certificate of Completion of Training (CCT), or be on the specialist register, and be in a substantive consultant post. Physicians who are resident outside the UK and who wish to examine must hold Fellowship of one of the UK colleges of physicians and be in good standing. When the examination is taken outside of the UK, one of the examiners must be from the UK, and one must be from the host country. New examiners complete 3 days of training, and once trained, they must commit to examine at least six cycles (30 candidates) per annum averaged over 2 years. If they cannot do that, they must refresh their examiner training.

Population and primary outcome

The study population included all doctors attempting PACES for the first time, at any of the official MRCP (UK) examination centres in the UK or internationally between October 2010 and May 2013. The pass standard did not change over the study period. We chose to restrict the study population to those attempting PACES for the first time, first attempt score being good predictor of score at subsequent attempts [22]. The predictor variable of interest was the candidates’ sex, as declared by the candidate to MRCP (UK).

Selection of variables

The selection of co-variates was constrained data routinely collected by MRCP (UK). Co-variates were selected prior to any statistical analysis and selection was based on published findings that suggested they may influence and confound the association between candidates’ sex and performance at examinations [7, 9–13]. Variables with multiple categories were collapsed to create meaningful binary categories in order to increase statistical power. Included covariates were: ethnicity (white vs BME); country of PMQ (UK vs non-UK); GMC registration (registered vs not registered); country of PACES centre (UK vs non-UK).

Statistical methods

We first performed bivariate analyses, examining the association between candidates’ sex and the other categorical variables; and between PACES pass rates and other categorical variables. Then we completed multivariate analyses using binary logistic regression models to examine the association between candidates’ sex and PACES pass rates, controlling for the other variables.

Subgroup analyses

Statistical analyses were conducted using the software Stata V.12/SE.

The study followed guidelines set out by the STROBE statement [23].

Descriptive analyses

Bivariate associations between candidate sex and other variables

Table 1 shows the distribution of each variable by sex of the candidates. Male candidates were statistically significantly more likely to be from a BME background, more likely to be a non-UK medical graduate, less likely to be registered with the GMC, and more likely to sit PACES at a non-UK centre (all P < 0.001).

Bivariate associations between PACES pass rates and other variables

For the remainder of the descriptive analyses, candidates with one or more variable missing were excluded, leaving a total of 6820 candidates. Of those, just under half (49%) passed PACES at their first attempt. Passing candidates were statistically more likely to be female rather than male [56% vs. 42%; χ ² (1) = 144, P < 0.001]; white rather than BME [65% vs. 38%; χ ² (1) = 463, P < 0.001]; be a UK rather than a non-UK medical graduate [64% vs 32%; χ ² (1) = 695, P < 0.001]; to be registered with the GMC rather than not [55% vs. 35%; χ ² (1) = 232, P < 0.001]; and to have completed the examination at a UK examination centre rather than a centre outside the UK [51% vs. 40%; χ ² (1) = 65, P < 0.001].

In summary, ethnicity, PMQ world region, GMC registration, and location of the examination centre, were associated with passing PACES at first attempt and candidate sex. We therefore considered these four variables as confounders of the association between sex and passing PACES at first attempt.

Candidates with missing data

We compared the PACES performance of the 851 candidates (11% of the study population) who were missing data for at least one variable, to the PACES performance of the 6820 candidates with no missing data. We found no evidence of a statistically significant difference between these two groups in terms of the outcome of interest (P = 0.6). Given these findings we felt that candidates with missing data could be removed from the regression analyses.

Regression analyses

Comparison with other studies

The finding that female doctors are more likely to pass PACES mirrors the findings of Dewhurst and colleagues [9], who analysed data from the pre-2009 version of PACES in UK graduates only. They found that, after controlling for ethnicity, women had 1.69 the odds of passing (95% CI = 1.42–2.02), a significantly larger order of magnitude to the finding from our logistic regression examining UK graduates only (OR = 1.18; 95% CI = 1.03-1.35).

Sex differences in performance at clinical assessments in the UK in other specialties has also been shown, with women performing better than men at the General Practice specialty examination in both the written Applied Knowledge Test and the practical Clinical Skills Assessment [10, 26]. Women also performed better in the intercollegiate specialty board examinations for surgical training [27]. Indeed, a recent unpublished meta-analysis of UK-based studies has also demonstrated that female doctors perform better than male doctor at postgraduate medical examinations of a clinical nature, although sex differences are generally less pronounced in written examinations [28]. Interestingly the effect of sex is not so clear outside of the UK. In the US, the USMLE Step 3 is the final and clinical component of the medical licensing examination. Two studies have examined the sex difference in performance at this examination in US medical graduates, with one study finding women outperforming men [29], but the other study demonstrated no sex difference [30].

Our finding that female international medical graduates outperform male international medical graduates is also seen in other UK and US postgraduate examinations [26, 31–33]. To our knowledge, there are no studies looking at sex differences in the examination performance of international medical graduates with and without registration to practise medicine in the country in which the examination is set.

Limitations of the study

The data were collected for routine administrative purposes, which limited our ability to gather potentially relevant data such as doctor’s age; however this also meant that the data on many variables were complete. The variable that contributed to the majority of the missing data was ethnicity, which was self-declared; however, a comparison of candidates with and without missing data showed no evidence of a difference in terms of passing PACES.

Unanswered questions

It is not clear why candidate demographics relate to PACES outcome. Female doctors may be better at performing the skills tested in clinical assessments [28, 32]. It has been demonstrated that, during one clinical assessment, women ask more relevant history taking items and perform correctly more physical examination manoeuvres [33]. It could be that women are better at retaining and appropriately applying theoretical scientific and medical knowledge to clinical encounters in examinations; although scientific theoretical medical knowledge is formally assessed through written assessments [34] and a recent unpublished meta-analysis showed that sex differences were smaller or not present on written assessments. Further, there was no evidence for a sex difference in the written components of the MRCP (UK) Diploma when examined by Dewhurst and colleagues in 2003/04 [9], although it would be of interest to analyse sex difference on the written MRCP (UK) components during the time of period of the current study.

A popular hypothesis is that sex differences in performance are due to differences in communication styles. A meta-analytic review of medical consultations found that female doctors are more likely to adopt a patient-centred communication style [35]. It has also been demonstrated that women doctors have greater interpersonal skills, which lead to empathic relationships [36–38]. These interpersonal skills may encourage the patient to be more forthcoming with regards to salient clinical information, enabling the doctor to correctly diagnose and manage the presenting ailment. However, this hypothesis would not completely explain the sex difference seen in performance at PACES, because there are stations where there is no meaningful verbal interaction with the patient.

A further hypothesis is that male and female doctors differ in values, and that these values lead to different motivations, which in turn influence achievement. Female doctors have been shown to have higher person-related values [37, 39, 40], and one study found that performance in a clinical setting was predicted by person-related values held by the doctor [39]. It could also be that the design of the exam favours female candidates, perhaps examiners of clinical assessments may be unfairly biased towards female candidates or against male candidates. However there is no evidence to suggest a sex bias in clinical examiners in this current format of the PACES examination when assessed between 2009 and 2011 [41].

We did not examine the individual countries from which candidates had obtained their PMQ, but it is possible that the female candidates were more likely to have graduated from English-speaking countries when compared to male international medical graduates. Native English speakers perform better at clinical assessments conducted in English [10], and if women international medical graduates are found to be more likely from a country where English is the dominant language, or where communication skills and cultural values are more similar to the UK when compared to men, this could go toward explaining the sex difference in performance seen in international medical graduates. It may also reflect differences in the selection and training of female doctors in countries around the world [42]. It is also possible that access to PACES and medical education in general may be biased outside of the UK. This may plausibly result in female candidates needing to be higher performers and to be more motivated than their male counterparts, to overcome obstacles that may limit their access to the examination.

Candidates’ age was not examined in this study, but it is likely that non-UK graduates were older and a previous study has demonstrated that older candidates do not perform as well as younger candidates in clinical assessments [33]. The variation between the sex difference in non-UK graduates with and without GMC registration may reflect gendered migration patterns. For example, Lebanese medical graduates practising in the US are significantly less likely to be female than graduates of other countries [43], and a study of Lebanese medical students by the same authors found that female students had less intention of working abroad after graduation than male students [44]. It would be of interest to explore whether the difference in PACES performance has varied year upon year, or whether the sex difference is stable. This study captures just under 3 years worth of data and therefore it is unlikely that any meaningful conclusions will able to be drawn with regards to performance over time.

It is likely that performance in large, high-stakes clinical examinations that have demonstrated good validity reflects performance in actual clinical practice. A study comparing the MRCP (UK) scores of doctors who had and had not had their registration subject to action by the GMC found that those whose license had been acted upon had lower PACES scores [45]. It may be that the factors underlying sex differences in performance in clinical examinations also contribute to female doctors being less subject to medico-legal action [6].