This article has Open Peer Review reports available.
A preliminary investigation to explore the cognitive resources of physicians experiencing difficulty in training
© The Author(s). 2017
Received: 9 February 2016
Accepted: 26 April 2017
Published: 15 May 2017
Treating patients is complex, and research shows that there are differences in cognitive resources between physicians who experience difficulties, and those who do not. It is possible that differences in some cognitive resources could explain the difficulties faced by some physicians. In this study, we explore differences in cognitive resources between different groups of physicians (that is, between native (UK) physicians and International Medical Graduates (IMG); those who continue with training versus those who were subsequently removed from the training programme); and also between physicians experiencing difficulties compared with the general population.
A secondary evaluation was conducted on an anonymised dataset provided by the East Midlands Professional Support Unit (PSU). One hundred and twenty one postgraduate trainee physicians took part in an Educational Psychology assessment through PSU. Referrals to the PSU were mainly on the basis of problems with exam progression and difficulties in communication skills, organisation and confidence. Cognitive resources were assessed using the Wechsler Adult Intelligence Scale (WAIS-IV). Physicians were categorised into three PSU outcomes: ‘Continued in training’, ‘Removed from training’ and ‘Active’ (currently accessing the PSU).
Using a one-sample Z test, we compared the referred physician sample to a UK general population sample on the WAIS-IV and found the referred sample significantly higher in Verbal Comprehension (VCI; z = 8.78) and significantly lower in Working Memory (WMI; z = −4.59). In addition, the native sample were significantly higher in Verbal Comprehension than the UK general population sample (VCI; native physicians: z = 9.95, p < .001, d = 1.25), whilst there was a lesser effect for the difference between the IMG sample and the UK general population (z = 2.13, p = .03, d = 0.29). Findings also showed a significant difference in VCI scores between those physicians who were ‘Removed from training’ and those who ‘Continued in training’.
Our results suggest it is important to understand the cognitive resources of physicians to provide a more focussed explanation of those who experience difficulties in training. This will help to implement more targeted interventions to help physicians develop compensatory strategies.
During medical education, training and clinical practice, some physicians experience difficulty . It is important to understand potential causes of difficulty, implement interventions, and support physicians through this process , since if they are not identified and supported, there may be negative consequences for their practice, patient care and safety. Across a range of medical specialties, physicians’ competence, and thus quality of patient care, is related to cognitive resources [3–6]. In essence the brain has finite processing power for various cognitive resources such as memory, attention and recognition and therefore as tasks become harder, performance can degrade . In medicine, substantial cognitive resources are needed to learn the knowledge required to complete medical training and then apply this knowledge into clinical practice . Similarly, once training has been completed, the rapid pace of development in medicine means that qualified doctors must constantly update their knowledge.
Previous research shows there are differences in cognitive resources between physicians who experience difficulties in training and clinical practice, and those who do not [9–11]. For example, Perry  found that physicians referred for remedial action scored significantly lower than a comparison group on a wide variety of cognitive ability tests; including picture arrangement, numerical attention and complex figure learning. Similarly, Yao and Wright  found that the three most common physician deficiencies related to cognitive resources (e.g. poor clinical judgement, insufficient medical knowledge, and inefficient use of time), leading to significant gaps in basic clinical knowledge, diagnoses and management of patients. There have been several different explanatory frameworks put forward to explain difficulties faced by both physicians [2, 14, 15] and medical students [16–19], and these have included not only cognitive resources, but also factors such as personal circumstance, learning style, personality and experience. Existing research suggests that differences in cognitive resources may provide an explanation for some of the difficulties faced by physicians. Some frameworks attempt to classify cognitive resource difficulties, with an aim to understand causality and guide trainers in effectively supporting physicians [20–22]. However, researchers have not yet converged on a set of cognitive resources that relate specifically to experiencing difficulty in medical training or clinical practice.
The Wechsler Adult Intelligence Scale (WAIS-IV) is one way of understanding a person’s cognitive resources. Intelligence as originally defined by Wechsler  is the “capacity to act purposefully, to think rationally, and to deal effectively with his [or her] environment”. The WAIS was designed based on intelligence theory, cognitive development and neuroscience and measures four key cognitive resources including verbal comprehension, perceptual reasoning, working memory and processing speed. Verbal Comprehension represents the ability to express thoughts verbally; to understand, analyse and interpret written information ; Perceptual Reasoning includes nonverbal reasoning, spatial processing and visual perception; Working Memory represents the ability to hold several pieces of information in the mind at once, and to manipulate and use this information , and Processing Speed relates to visual perception, scanning and hand-eye co-ordination .
In practice, treating patients uses many cognitive resources: accurate and efficient information processing is required with sustained attention, rapid decision making, and skilled execution of relevant actions . Physicians must do this whilst facing competing priorities, managing distractions and interruptions; adding to already challenging tasks . It is plausible therefore that physicians in difficulty may differ on some of these important cognitive resources; for example, the demand placed on a physician’s working memory (WM) is likely to be high; where WM provides temporary storage and manipulation of several pieces of information at once and so WM is needed for dealing competently with complex tasks . A further consideration may be the additional cognitive resource demands required for physicians practicing in a second language . Large-scale, meta-analytic studies have found differences in training success between native and International Medical Graduate (IMG) physicians (as measured by postgraduate examinations), with IMGs performing less well in comparison to native graduates [27–29]. As yet, there is no theoretical explanation for this phenomenon [28, 30] however, in developed countries there is continued dependence on IMGs to meet workforce needs [31, 32]. Therefore, it is important for countries internationally to better understand barriers that IMGs may face in completing medical training and practice; for example, the extent to which this is due to differences in specific abilities related to IMGs working in a second language.
Indeed, research relating to cognitive resource allocation theories  would imply that IMGs practicing in a second language are likely to have significantly increased working memory (WM) demands; for example, needing to correctly translate when communicating with others. This effect may become more pronounced when working under stress and in environments with high demands for cognitive resources, as in a medical context . High demands are placed on the WM of medical physicians practicing in their own language [35–39], and those practicing in a second language may have the added burden of using WM capacity for translation processes. This in turn reduces the available WM capacity allocated to medical tasks and problem solving. Arguably, there may be other explanations for poorer clinical performance of IMGs related to the passivity in learning style in some non-Western cultures (rather than the critical analysis and challenge-based approach to learning strategies in Western learning strategies, [40, 41]). This may make the problem-solving nature of clinical practice more challenging for IMGs practising in a foreign country than for native-trained doctors.
Are there differences in cognitive resources between physicians experiencing difficulties compared with the general population?
Are there differences in cognitive resources between native (UK) physicians and IMGs?
Are there differences in cognitive resources between physicians who continued with training and those who were subsequently removed from the training programme?
A secondary evaluation was conducted on an anonymised dataset provided by the East Midlands Training Support Service (TSS, as it was originally known at the time of the study) in the UK. TSS was a service provided by Health Education East Midlands for physicians having trouble with performance and progression in training. The dataset spanned from 2005 (when the TSS was established) to the end of May 2013. The TSS is now called the Professional Support Unit (PSU) so we will use this terminology throughout the rest of the paper. The PSU informs all users of the service that information about progression in training is gathered and that their data may be used for evaluation purposes. All data are stored in accordance with the Data Protection Act and were anonymised to remove identifying information.
Participants and procedure
Data was available for 138 physicians who took part in an Educational Psychology assessment through PSU. Cognitive resources were assessed using the Wechsler Adult Intelligence Scale (WAIS). However, both WAIS-III and WAIS-IV were used during the period and given the changes between WAIS-III and WAIS-IV in calculating cognitive ability Index scores  we used only data available from the WAIS-IV. Therefore 15 people were omitted from the dataset. This resulted in a sample of 123 participants, spanning the different years of postgraduate training, with the most common grade at the time of referral being specialty training year three (n = 44), followed by specialty training year two (n = 23), specialty training year four (n = 10) and core training year two (n = 10), the rest of the sample (n = 36) included participants from foundation training, core training and specialty training. The sample included a mix of those who completed their medical qualification in the UK (N = 63; 33 males, 30 females), and those who were IMGs (N = 60; 39 males, 20 females); data was missing from one participant. For the IMG sample, countries where qualifications were obtained included India, Pakistan, Poland and Nigeria where English is likely to have been a second language. However, two participants were removed from the IMG sample since their places of qualification were New Zealand and Australia (meaning that English was unlikely to be a second language) resulting in N = 58 participants; 37 males, 20 females, and one participant for which this data was missing. The final sample was 121 participants, comprising 70 males and 50 females (data was missing from n = 1), with a mean age of 33.9 years (S.D. = 5.82 years). Referrals to the PSU were mainly based on problems with exam progression, difficulties in communication skills, organisation and confidence.
To compare our physician sample with a general population sample, we used the WAIS-IV UK validation sample, details of which can be obtained in the WAIS-IV manual . This sample included 270 people (52.5% were female and 47.5% were male), their mean age was 44.33 years (S.D. = 19.14 years). The WAIS-IV manual highlights that this sample closely matches the adult population relating to distribution of demographic variables in terms of age, gender, ethnicity .
To compare cognitive resource scores across successful/unsuccessful training groups, physicians were categorised into groups of three possible PSU outcomes. ‘Continued in training’ (n = 57) represents those physicians who achieved Certificate of Completion of Training (CCT) and/or remained in the programme after receiving remedial support. ‘Removed from training’ (n = 15) represents those physicians who had their National Training Number removed, voluntarily left the programme, or were suspended by the UK’s regulator (General Medical Council). Finally, ‘Active’ (n = 49) represents physicians currently in the system and accessing support through the PSU at the time the study was conducted.
WAIS-IV index scores: capacities assessed and relevance to physicians’ job rolea
WAIS Index Primary Capacities Targeted for Assessment
Relevance to Physicians’ Job Role
Verbal Comprehension Index
Verbal reasoning and conceptualising ability – synthesising and modelling ideas. Lexical knowledge – receptive and expressive vocabulary. General societal knowledge.
Retrieval of verbal information from long-term storage
Reasoning with verbal information
For example: being able to explain information clearly to patients and colleagues; being able to remember the diagnostic criteria for different medical conditions; orally presenting cases and justifying decisions made in relation to care.
Perceptual Reasoning Index
Interpreting and reasoning with pictorial, diagrammatic, schematic and graphical information – problems of omission and commission in pattern recognition and matching. Integration of two and three dimensional data. Hypothetico-deductive reasoning.
Reasoning with nonverbal visual stimuli
For example: being able to interpret nonverbal information on a patient, e.g. an x-ray or scan; being able to interpret and use presenting visual symptoms to contribute towards making an informed diagnosis.
Working Memory Index
Short-term auditory sequential memory for holding complex patient and colleague information in mind. Central executive working memory for mental computation, whilst drawing on information in long term memory, and then executing conclusion in ordered and logical manner. Personal organisation.
Initial registration and holding of information (sometimes referred to as short term memory)
The mental manipulation of information that is being held in mind (often referred to as working memory)
For example: being able to listen to a patient tell you their symptoms, hold this information in mind, whilst combining it with other information from alternative sources, e.g. past medical history, information from colleagues or family members, whilst also bringing to mind the diagnostic criteria for different possible conditions.
Processing Speed Index
Proofing and clerical checking. Working against the clock in time pressured situations without error. Transcribing. Fine motor precision and accuracy. Visual scanning and tracking. Holding data in the visual-spatial sketchpad aspect of working memory. Dealing with cognitive noise (Stroop).
Processing speed with nonverbal, visual stimuli
For example: being able to quickly scan information about a patient to identify relevant details; proofing reports or clinical notes; analysing nonverbal information (e.g. figures, pictures, images) during a time-pressured exam.
Means, SDs and one-sample Z statistics for the current physician sample and the UK WAIS-IV general population sample
UK General Population Sample
Current sample (IMG & UK Combined)
UK Physician Sample
IMG Physician Sample
The WAIS-IV index scores are standardised, with a mean of 100 and standard deviation of 15. This is the standard format for tests of intelligence such as the WAIS-IV [23, 25] and is generally conducted in this way so that scores can be more readily compared to those from other tests and to aid interpretation by lay-users. The UK general population (WAIS-IV UK norm group) sample parameters are known (accessible from the manual , and outlined in Table 2). A one sample z-test was used to compare the physician sample to the UK general population sample on the WAIS-IV. Given an alpha level of.05 and using a two-tailed design, if z is +/−1.96, the referred physician sample is significantly different from the UK general population sample. We also explored Cohen’s d statistic to examine the size of the difference, where as a rule of thumb, 0.2 is a small effect size; 0.5 is medium; 0.8 is large and 1.2 is a very large effect .
Results in Table 2 show that the physician sample does not differ significantly from the UK general population sample in Perceptual Reasoning (PRI) or Processing Speed (PSI). However, the referred sample is significantly higher in Verbal Comprehension (VCI; z = 8.78, p < .001, d = 1.25) and significantly lower in Working Memory (WMI; z = −4.59, p = .01, d = .29), than the UK general population sample.
To explore the second research question are there differences in cognitive resources between native (UK) physicians and IMGs? we split the sample by place of medical qualification (native versus IMG). Findings show that the native physician sample is significantly higher in Verbal Comprehension than the UK general population sample (with a very large effect). The IMG sample is also significantly higher than the UK general population sample, but to a lesser degree, showing a small effect size (VCI; native physicians: z = 9.95 p < .001, d = 1.25; IMGs: z = 2.13 p = .03, d = .29). This implies that any difference between VCI scores in the UK general population sample and the physician sample are explained to a greater degree by the native physicians.
In addition, the native physician sample is significantly higher on Perceptual Reasoning (PRI; z = 2.47, p = .01, d = .32), whereas the IMG physician sample is significantly lower (z = −2.53, p = .01, d = −.35) than the UK general population sample. Both the native and IMG physician samples were significantly lower on Working Memory than the UK general population sample (WMI; native: z = −3.03, p = .002, d = −.38; IMG: z = −3.47, p < .001, d = −.46). Neither the native nor the IMG physicians had significantly different Processing Speed (PSI) scores to the UK general population sample.
Means and SDs for WAIS-IV index scores for PSU outcome and place of qualification
Continued in training
Removed in training
Results showed that there was no significant PSU outcome X place of qualification interaction where Pillai’s trace V = 0.53, F(8, 186) = 0.64, p = .74, partial ETA2 = .03. However, we explored each factor separately, the PSU outcome and the place of qualification, to identify whether the WAIS-IV scores were differentially influenced by these factors.
For PSU outcome, Pillai’s trace approached significance on WAIS-IV index scores, V = 0.16, F(8, 186) = 1.95, p = .06, partial ETA2 = .08, although for the same effect, Roy’s largest route indicated a significant effect Θ = 0.12, F(4, 93) = 2.92, p = .03. Separate univariate ANOVAs on index scores revealed a significant effect only on Verbal Comprehension (VCI); F(2, 113) = 4.42, p = .01, partial ETA2 = .07. Contrasts using the ‘continued in training’ group as the referent category highlighted that the significant differences in Verbal Comprehension scores are between those ‘removed from training’ and those who ‘continued in training’ (mean difference = 11.90, p = .004, 95% CIs 3.96 to 19.84).
When considering place of qualification, Pillai’s trace showed a significant effect on WAIS-IV index scores, V = 0.20, F(4, 92) = 5.80, p < .001, partial ETA2 = .20. Separate univariate ANOVAs revealed significant effects on Verbal Comprehension (VCI, F(1, 115) = 34.39, p < .001, partial ETA2 = .23) and on Perceptual Reasoning (PRI, F(1, 112) = 14.72, p < .001, partial ETA2 = .12). In both cases IMGs scored on average lower than those who trained in the UK.
This study is a first step towards understanding the cognitive resources of physicians referred for support due to difficulties experienced during training. Overall, our results indicate that the physicians in our sample had similar Perceptual Reasoning and Processing Speed to the general UK population sample; however, they were significantly higher in Verbal Comprehension and significantly lower in Working Memory. Considering the profile of the sample of referred physicians, they can be described as ‘Average’ in Perceptual Reasoning, Processing Speed and Working Memory, and ‘Higher than Average’ in Verbal Comprehension . However, research elsewhere suggests that most physicians typically perform within the above average to superior range in these cognitive resources [12, 44]. Therefore, our findings may suggest that this sample of physicians is performing at a lower level of cognitive functioning than expected. IMG physicians were significantly lower in their Verbal Comprehension and Perceptual Reasoning than those trained in the UK. Furthermore, physicians who continued in training after receiving support scored significantly higher in Verbal Comprehension than those who were removed from training. Since there was no training outcome by place of qualification interaction, it is not the case that IMGs who were removed from training are significantly lower in Verbal Comprehension than other groups.
Working memory ability
Findings showed that WM was the lowest of all index scores in the physician sample. This does not mean that the physicians have significant problems in WM, since they are performing within the average band. Rather, because WM represents the ability to hold several pieces of information in the mind at once, and to manipulate and use this information , lower scores may represent limitations in a physician’s clinical practice. Since a physician’s performance requires rapid processing of patient information and decision making, lower WM could mean that physicians may experience difficulties in identifying the correct decisions.
This latter notion could be explained with reference to dual-processing cognitive theories, which suggests two types of cognitive processes: one that is automatic and unconscious and the other is controlled and conscious. WM capacity is known to moderate the impact of automatic and controlled processes on self-regulatory behaviour . Controlled processing is cognitively demanding and, in attention-demanding circumstances (such as during clinical practice), high WM capacity promotes the effortful and intentional behaviours of controlled processing, and also inhibits effortless, impulsive and potentially less desired behaviours seen in automatic processing. Experienced physicians generally formulate diagnostic decisions using automatic processing and only in novel/challenging situations would they access controlled processing . Since the cognitive load for physicians is often high, and they operate in challenging situations, lower WM resources means less access to controlled processing and more likelihood that decisions will be based on automatic processing. This may lead to inappropriate or inaccurate decisions or behaviours. Further, the high cognitive demands of the physician’s role may be intensified for IMGs who are less familiar with the local health service. However, with appropriate training interventions WM could be improved, for example WM training has been conducted successfully in older adults where significant improvements in WM are seen even 8 months after the intervention . Practically, this implies that earlier identification of struggling physicians would significantly benefit trainees and relevant educational interventions could support them appropriately.
Verbal comprehension ability
Our findings showed that Verbal Comprehension (the ability to express thoughts verbally; to understand, analyse and interpret written information ) differentiates those who continued in, and those who were removed from, training; as well as native and IMG physicians. A physician lower in Verbal Comprehension may experience problems explaining information clearly; when verbally presenting cases; and when justifying decisions, especially under pressure. This may attenuate a physician’s clinical performance and progression (e.g. clinical skills, communication skills, exam performance and patient interaction). This is supported by the fact that most physicians were referred because of problems in postgraduate examination performance (which requires a high level of Verbal Comprehension), and difficulties in communication skills and organisation.
Lower scores for IMGs in Verbal Comprehension than native physicians may imply that performance issues are the result of language/accent difficulties. However, as there were no interaction effects, Verbal Comprehension in IMGs does not fully explain progression in training. Individuals whose first language is not English may initially experience problems when expressing thoughts and explaining information clearly  and may also over-estimate the language capabilities of others or medical understanding amongst patients (since many semi-technical terms are used in general discourse, such as “migraine” ); and inappropriately use overly-technical language.
Indeed, even IMGs whose first language is English may struggle to understand accent and colloquial language [48, 49]. In our sample, data were not available for participants’ native language so we could not substantiate this. Nevertheless, practising medicine in the UK requires an understanding of the nuances of British language (including humour and irony), which may differ across English-speaking nations. Consider for example the different terminology and phrasing used by Britons compared to Americans; therefore we may infer that some conversation may be challenging for IMGs, even if their first language is English.
Perceptual reasoning ability
A possible explanation for the significant differences in PRI for the native physician sample (higher than the UK general population sample) and IMGs (lower than the UK general population sample) is the difference in learning methods between UK- and non-UK-trained doctors. Problem-Based Learning (PBL) is the prevalent approach to teaching in Western cultures , which enhances learning by focusing on problem solving in a self-directed and reflective way . PBL positively impacts students’ abilities to apply science knowledge and transfer problem-solving skills to ‘real-world’ situations, across both medical and non-medical contexts , and has been found to prepare graduates with the knowledge and skills needed to practice in a complex healthcare system . However, the PBL approach to learning is rooted in Western culture [40, 54], and other cultures tend to be focussed around teacher-led learning. Therefore, students from non-Western cultures may feel uncertain about the independence required for self-directed and problem-based learning  and these cultural differences in teaching and learning style may explain the observed differences between IMGs and UK-trained doctors in Perceptual Reasoning. The PRI includes problem solving, placing missing parts into an uncompleted picture, and the ability to differentiate between essential and non-essential details. These are abilities that PBL aims to promote, whereas in teacher-led learning, students learn what is taught to them by ‘experts’, rather than learning in a self-directed way .
Research by Frambach and colleagues  found that students across different cultures increasingly internalised the principle of self-directed learning as they progressed through their education and training, suggesting that Perceptual Reasoning skills may develop over time with more PBL-based approaches. Therefore, appropriately designed interventions may increase the Perceptual Reasoning ability of physicians experiencing difficulties, via increased PBL-style learning. Future research should investigate the suitability of PBL and other interventions for increasing Perceptual Reasoning ability in physicians experiencing difficulty.
Our results are a first step towards understanding the cognitive resources of physicians experiencing difficulty and this may provide a more focussed explanation of difficulties in training, and help to implement targeted interventions for physicians to develop compensatory strategies. The recent literature on cognitive resources training (CT, also known as ‘brain training’) may provide some insights into possible interventions for physicians facing difficulties. CT is defined as “an intervention providing structured practice on tasks relevant to aspects of cognitive functioning” using standardized tasks and is “intended to address cognitive function and/or cognitive impairment directly” ( p.3). Although the majority of research in this area has focused on elderly participants, the medical context with healthy and comparatively young adults may be an area for exploration. CT approaches in a medical context may include: early testing for cognitive ability, rehabilitation for minor cognitive resource problems , development of critical thinking skills  and WM training . Ultimately it is important to develop interventions for physicians because if left to struggle they may develop further problems which inhibit their progress as effective clinicians. Early interventions would prevent significant costs to the physician, patients and the health service as a whole.
As with any research, there were a number of limitations of the current study that should be noted. First, we were unable to compare WAIS-IV scores with those of a control group of non-referred physicians. However, the UK working population norm group was considered a good comparison group in this context. Second, we did not have data on why physicians were removed from training. This data was not available for data protection reasons. It could be that a subset of individuals within this group left the training programme due to reasons other than those relating to cognitive functioning (e.g. for social, personal or financial reasons). Future research could expand upon our preliminary findings to separate out the causes of physicians being removed from training. Finally, we did not have information regarding whether English was a first language for the participants in our sample. To further substantiate our findings, future research is needed to compare the cognitive resources and clinical performance of IMGs whose first language is English, and those for whom English is a second language. Nevertheless, we believe that this paper highlights some key areas to be considered for physicians in difficulty.
This preliminary study extended empirical research relating to physicians experiencing difficulties in training. Our findings may go towards explaining some of the difficulties physicians faced. Our sample was lower than may be expected in WM, which could lead to limitations in physicians’ performance when making decisions in a fast-paced environment. Verbal Comprehension ability was significantly different between those who continued in, and those who were removed from, training; as well as between native and IMG physicians. Furthermore, while native graduates were significantly higher in Perceptual Reasoning than the UK general population sample, IMGs were significantly lower, which may be attributable to differences in learning styles across countries and cultures. Future research is necessary to expand upon these initial findings, to assess whether the cognitive resources in this study represent a potential taxonomy to explain difficulties faced by physicians in wider samples.
The authors would like to express sincere thanks to Michael Lock of MLCP who provided the data and aided the interpretation. Our thanks also to Caroline McCarthy from HEEM who facilitated data access and support in the original report on which this paper was based. Finally, we gratefully acknowledge Anna Rosselli at Work Psychology Group for her input into the original report and earlier iterations of the manuscript.
Funding for the study was from HEE East Midlands.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
FP, JJ and SM conceived of the original study, the study design and methodology; JJ and SM organised data collection. IC, FC and LZ analysed and interpreted the data, and wrote the paper along with FP. All authors commented on the original and final versions of the paper. All authors read and approved the final manuscript.
Professor Fiona Patterson is a Professor of Organisational Psychology with expertise in selection and assessment. She is a Visiting Researcher at the University of Cambridge and founding Director for the Work Psychology Group, an international research-led organisational psychology consulting practice.
Fran Cousans is an Associate Research Psychologist at Work Psychology Group and teaching fellow in the Department of Neuroscience, Psychology and Behaviour, University of Leicester. Her research interests include the application of occupational psychology to the healthcare context, and she has published and presented her work in this area. She also works with organisations to design, deliver and evaluate interventions including selection systems.
Dr. Iain Coyne is Senior Lecturer in Organisational Psychology in the School of Business and Economics, Loughborough University. Iain is a Registered Occupational Psychologist, an Associate Fellow of the British Psychological Society (BPS), past Senior Editor for the BPS Test Reviews and a previous Council member of the International Test Commission.
Dr. Jo Jones FRCPsych is an Associate Postgraduate Dean with a lead role for HEEM Training Support Service. Outside of this role she works clinically as Consultant in the Psychiatry of Intellectual Disabilities in the Nottinghamshire Healthcare NHS Trust.
Professor Sheona Macleod is the Postgraduate Dean in Health Education England (HEE), East Midlands, Chair of HEE Deans, HEE Recruitment Advisor and Honorary Professor at the Universities of Nottingham and Leicester. She was a GP in Ashbourne for 26 years.
Dr. Lara Zibarras is a Senior Lecturer at City, University of London and Associate Director for Research at Work Psychology Group. Lara is a Registered Occupational Psychologist and an Associate Fellow of the British Psychological Society (BPS). Her research areas of interest are employee assessment and selection, and innovation; she has published widely in academic journals and presented her research both nationally and internationally.
FP, FC and LZ provide advice to Health Education East Midlands (HEEM) on workplace-based assessments through Work Psychology Group, however they do not receive royalties for any method used. IC is an associate of Work Psychology Group. SM is the Postgraduate Dean at HEEM, and JJ is an Associate Postgraduate Dean at HEEM.
Consent for publication
Ethics approval and consent to participate
This work was carried out in accordance with the Declaration of Helsinki. With regards to ethics, guidance from the then chair of the local ethics committee (East Midlands Strategic Health Authority) confirmed that since this was a secondary evaluation of a fully anonymised dataset which was already in the public domain, further ethical review and approval was not required.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Mitchell C, Bhat S, Herbert A, Baker P. Workplace-based assessments of junior doctors: do scores predict training difficulties? Med Educ. 2011;45(12):1190–8.View ArticleGoogle Scholar
- Patterson F, Knight A, Stewart F, Mac Leod S. How best to assist struggling trainees? Developing an evidence-based framework to guide support interventions. Educ Prim Care. Sep. 2013;24(5):330–9.View ArticleGoogle Scholar
- Holmboe ES, Wang Y, Meehan TP, Tate JP, Ho SY, Starkey KS, Lipner RS. Association between maintenance of certification examination scores and quality of care for Medicare beneficiaries. Arch Intern Med. 2008;168(13):1396–403.View ArticleGoogle Scholar
- Norcini JJ, Lipner RS, Kimball HR. Certifying examination performance and patient outcomes following acute myocardial infarction. Med Educ. 2002;36(9):853–9.View ArticleGoogle Scholar
- Tamblyn R, Abrahamowicz M, Dauphinee WD, Hanley JA, Norcini J, Girard N, Grand-Maison P, Brailovsky C. Association between licensure examination scores and practice in primary care. J Am Med Assoc. 2002;288(23):3019–26.View ArticleGoogle Scholar
- Wenghofer E, Klass D, Abrahamowicz M, Dauphinee D, Jacques A, Smee S, Blackmore D, Winslade N, Reidel K, Bartman I, T. R. Doctor scores on national qualifying examinations predict quality of care in future practice. Med Educ. 2009;43(12):1166–73.View ArticleGoogle Scholar
- Lachman R, Lachman J, Butterfield E. Cognitive psychology and information processing: an introduction. New York: Psychology Press; 2015.Google Scholar
- Jarodzka H, Boshuizen H, Kirschner P. Cognitive skills in catheter-based cardiovascular interventions. In: Lanzer P, editor. Catheter-based cardiovascular interventions; 2012. p. 69–86.Google Scholar
- Korinek LL, Thompson LL, McRae C, Korinek E. Do physicians referred for competency evaluations have underlying cognitive problems? Acad Med. 2009;84(8):1015–21.View ArticleGoogle Scholar
- Turnbull J, Carbotte R, Hanna E, Norman G, Cunnington J, Ferguson B, Kaigas T. Cognitive difficulty in physicians. Acad Med. 2000;75:177–81.View ArticleGoogle Scholar
- Turnbull J, Cunnington J, Unsal A, Norman G, Ferguson B. Competence and cognitive difficulty in physicians: a follow-up study. Acad Med. 2006;81(10):915–8.View ArticleGoogle Scholar
- Perry W, Crean RD. A retrospective review of the neuropsychological test performance of physicians referred for medical infractions. Arch Clin Neuropsychol. 2005;20:161–70.View ArticleGoogle Scholar
- Yao DC, Wright SM. National survey of internal medicine residency program directors regarding problem residents. J Am Med Assoc. 2000;284(9):1099–104.View ArticleGoogle Scholar
- Smith C, Stevens N, Servis M. A general framework for approaching residents in difficulty. Fam Med. 2007;39(5):331–6.Google Scholar
- Patel R, Sandars J, Carr S. Clinical diagnostic decision-making in real life contexts: a trans-theoretical approach for teaching: AMEE guide no. 95. Med Teach. 2015;37(3):211–27.View ArticleGoogle Scholar
- Ferguson E, Semper H, Yates J, Fitzgerald JE, Skatova A, James D. The ‘dark side’ and ‘bright side’ of personality: when too much conscientiousness and too little anxiety are detrimental with respect to the acquisition of medical knowledge and skill. PLoS One. 2014;9(2):e88606.View ArticleGoogle Scholar
- E. Ferguson, D. James, and L. Madeley, “Factors associated with success in medical school: systematic review of the literature,” BMJ, vol. 324, no. April, pp. 952–957, 2002.Google Scholar
- Sandars J, Patel R, Steele H, McAreavey M. Developmental student support in undergraduate medical education: AMEE guide no. 92. Med. Teach. 2014;36(12):1015–26.View ArticleGoogle Scholar
- Patel R, Tarrant C, Bonas S, Yates J, Sandars J. The struggling student: a thematic analysis from the self-regulated learning perspective. Med Educ. 2015;49(4):417–26.View ArticleGoogle Scholar
- Patterson F, Tavabie A, Denney M, Kerrin M, Ashworth V, Koczwara A, MacLeod S. A new competency model for general practice: implications for selection, training, and careers. Br J Gen Pract. 2013;63(610):e331–8.View ArticleGoogle Scholar
- Hays RB, Lawson M, Gray C. Problems presented by medical students seeking support: a possible intervention framework. Med. Teach. 2011;33(2):161–4.View ArticleGoogle Scholar
- Hicks PJ, Cox SM, Espey EL, Goepfert AR, Bienstock JL, Erickson SS, Hammoud MM, Katz NT, Krueger PM, Neutens JJ, Peskin E, Puscheck EE. To the point: medical education reviews - dealing with student difficulties in the clinical setting. Am J Obstet Gynecol. 2005;193(6):1915–22.View ArticleGoogle Scholar
- Lichtenberger EO, Kaufman AS. Essentials of WAIS-IV Assessment. 2nd ed. Hoboken: John Wiley & Sons, Inc.; 2013.Google Scholar
- Baddeley A. Working memory. Science (80-). 1992;255(5044):556–9.View ArticleGoogle Scholar
- Wechsler D. Wechsler Adult Intelligence Scale (WAIS-IV-UK). 4th ed. Administration and Scoring Manual London: Pearson; 2010.Google Scholar
- Laxmisan A, Hakimzada F, Sayan OR, Green RA, Zhang J, Patel VL. The multitasking clinician: decision-making and cognitive demand during and after team handoffs in emergency care. Int J Med Inform. 2007;76(11):801–11.View ArticleGoogle Scholar
- Bates J, Andrew R. Untangling the roots of some IMG’s poor academic performance. Acad Med. 2001;76(1):43–6.View ArticleGoogle Scholar
- Boulet JR, Swanson DB, Cooper RA, Norcini JJ, McKinley DW. A comparison of the characteristics and examination performances of US and non-US citizen international medical graduates who sought educational Commission for Foreign Medical Graduates certification: 1995–2004. Acad Med. 2006;81(10):116–9.View ArticleGoogle Scholar
- Spike NA, Hays RB. Analysis by training status of performance in the certification examination for Australian family doctors. Med Educ. 1999;33(8):612–5.View ArticleGoogle Scholar
- Patterson F, Denney M, Wakeford R, Good D. Making assessment fair and equal in postgraduate training: an interdisciplinary perspective of a future research agenda. Br J Gen Pract. 2011:712–713.Google Scholar
- Esmail A, Roberts C. Academic performance of ethnic minority candidates and discrimination in the MRCGP examinations between 2010 and 2012: analysis of data. BMJ. 2013;347(sep26_2):f5662.View ArticleGoogle Scholar
- P. C. Alguire, The International Medical Graduate’s Guide to US Medicine & Residency Training. Philadelphia: ACP Press; 2009. Google Scholar
- Kanfer R, Ackerman PL. Motivation and cognitive abilities: an integrative/aptitude-treatment interaction approach to skill acquisition. J Appl Psychol. 1989;74(4):657–90.View ArticleGoogle Scholar
- Hockey GRJ. Compensatory control in the regulation of human performance under stress and high workload: a cognitive energetical framework. Biol Psychol. 1997;45:73–93.View ArticleGoogle Scholar
- Bowen JL. Educational strategies to promote clinical diagnostic reasoning. N Engl J Med. 2006;355(21):2217–25.View ArticleGoogle Scholar
- Croskerry P. Perspectives on diagnostic failure and patient safety. Healthc Q. 2012;15:50–6.View ArticleGoogle Scholar
- Fuks A, Boudreau JD, Cassell EJ. Teaching clinical thinking to first-year medical students. Med Teach. 2009;31:106–11.View ArticleGoogle Scholar
- Kee F, Bickle I. Critical thinking and critical appraisal: the chicken and the egg. QJM. 2004;97(9):609–14.View ArticleGoogle Scholar
- Krupat E, Sprague JM, Wolpaw D, Haidet P, Hatem D, O’Brien B. Thinking critically about critical thinking: ability, disposition or both? Med Educ. 2011;45(6):625–35.View ArticleGoogle Scholar
- Frambach JM, Driessen EW, Chan LC, van der Vleuten CP. Rethinking the globalisation of problem-based learning: how culture challenges self-directed learning. Med Educ. 2012;46(8):738–47.View ArticleGoogle Scholar
- Waller D. Teaching marketing to Asian students: are they missing the message? J Mark Educ. 1993;1:47–55.View ArticleGoogle Scholar
- Horne J, Carstairs K, Baron H, Coyne I. “Test Review Wechsler Adult Intelligence Scale − Fourth UK edition (WAIS−IV UK).” Leicester: Br. Psychol. Soc; 2012.Google Scholar
- Cohen J. Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum Associates; 1988.Google Scholar
- Matarazzo JD, Goldstein SG. The intellectual calibre of medical students. J Med Educ. 1972;47(2):102–11.Google Scholar
- Hofmann W, Gschwendner T, Friese M, Wiers RW, S. M. Working memory capacity and self-regulatory behavior. Toward an individual differences perspective on behavior determination by automatic versus controlled processes. J Pers Soc Psychol. 2008;95(4):962–77.View ArticleGoogle Scholar
- Borella E, Carretti B, Riboldi F, De Beni R. Working memory training in older adults: evidence of transfer and maintenance effects. Psychol Aging. 2010;25(4):767.View ArticleGoogle Scholar
- Dahm M. Coming to terms with medical terms: exploring insights from native and non-native English speakers in patient-physician communication. J Lang Commun Bus. 2012;49:79–98.Google Scholar
- Malau-Aduli BS. Exploring the experiences and coping strategies of international medical students. BMC Med Educ. 2011;11(1):40.View ArticleGoogle Scholar
- Parker V, McMillan M. Challenges facing internationalisation of nursing practice, nurse education and nursing workforce in Australia. Contemp Nurse. 2007;24(2):128–36.View ArticleGoogle Scholar
- Kinkade S. A snapshot of the status of problem-based learning in U.S. medical schools, 2003–04. Acad Med. 2005;80:300–1.View ArticleGoogle Scholar
- Hung W, Jonassen DH, Liu R. Problem-based learning. Handb Res Educ Commun Technol. 2008:485–506.Google Scholar
- Woods DR. Problem-based learning for large classes in chemical engineering. New Dir Teach Learn. 1996;1996(68):91–9.View ArticleGoogle Scholar
- Hoffman K, Hosokawa M, Blake R, Headrick L, Johnson G. Problem-based learning outcomes: ten years of experience at the University of Missouri-Columbia School of Medicine. Acad Med. 2006;81(7):617–25.View ArticleGoogle Scholar
- Ladyshewsky R. East meets west: the influence of language and culture in clinical education. Aust J Physiother. 1996;42(4):287–94.View ArticleGoogle Scholar
- Biggs J. Approaches to learning in secondary and tertiary students in ong Kong: some comparative studies. Educ Res J. 1991;6:27–39.Google Scholar
- M. Martin, L. Clare, A. M. Altgassen, M. H. Cameron, and F. Zehnder, Cognition-based interventions for healthy older people and people with mild cognitive impairment. 2011.View ArticleGoogle Scholar
- Pitkanen M, Hurn J, Kopelman MD. Doctors’ health and fitness to practise: performance problems in doctors and cognitive impairments. Occup Med (Chic Ill). 2008;58(5):328–33.View ArticleGoogle Scholar
- Harasym PH, Tsai T-C, Hemmati P. Current trends in developing medical students’ critical thinking abilities. Kaohsiung J Med Sci. 2008;24(7):341–55.View ArticleGoogle Scholar
- Olesen PJ, Westerberg H, Klingberg T. Increased prefrontal and parietal activity after training of working memory. Nat Neurosci. 2004;7(1):75–9.View ArticleGoogle Scholar