Using entrustable professional activities to guide curriculum development in psychiatry training

Background Clinical activities that trainees can be trusted to perform with minimal or no supervision have been labelled as Entrustable Professional Activities (EPAs). We sought to examine what activities could be entrusted to psychiatry trainees in their first year of specialist training. Methods We conducted an online survey of Fellows of the Royal Australian and New Zealand College of Psychiatrists (RANZCP). Results The majority of respondents considered initiating patients with the common medications, discharging patient suffering from schizophrenia, bipolar disorder or following a crisis admission, conducting risk assessments and managing psychiatric emergencies were activities that trainees could be entrusted with by the end of the first stage of training. Conclusions Four activities were identified that trainees should be entrusted with by the end of their first year of training. Each of these activities comprises a set of competencies in each of the CanMEDS roles. When a trainee is unable to satisfactorily perform an EPA, deficits in the underpinning competencies can be a focus for remediation. Further EPAs are being identified in areas of more specialised practice for use within more advanced training.


Background
Over the past 5-10 years there has been a change in which postgraduate specialist training takes place with a move towards competency based curricula rather than the more traditional time-based apprentice approaches of old [1][2][3]. The Royal Australian and New Zealand College of Psychiatrists (RANZCP), in line with many specialist colleges, is moving to implement a competency based training program. Its current training program is a timebased curriculum with 3 years of basic training and 2 years of advanced training. A summative assessment (comprising a written assessment, an Observed Clinical Interview and an OSCE) takes place at the end of basic training prior to moving into advanced training.
The move to implement a competency-based curriculum has come about for several reasons. First, in line with contemporary educational thinking, we recognise that the outcome of a training program is to produce specialists that demonstrate that they are competent, across a range of roles, in dealing with contemporary medical problems [4]. In order to ensure that the College's training program fulfils its social obligation in producing such specialists, training has to be aligned with this outcome; this can only be adequately achieved through the use of a competency-based program.
Second, within the RANZCP, we had identified issues of concern within the extant training program. The time to complete training was considerably longer than the five years with the mean time to complete training being 7.4 years. This issue is of particular concern when there are considerable workforce shortages along with increasing demand for specialist psychiatrists. Third, we have had a poor pass rate in our summative assessments suggesting that the training program did not prepare the trainees for these assessments. Fourth, there was considerable trainee dissatisfaction, particularly in the tension between service commitments and time spent in training activities. Finally, we recognised that our training program was not adequately equipping trainees for their future work as consultant psychiatrists, suggesting poor alignment in the training program.
The RANZCP will be implementing a competencybased program from 2013, with some pilot studies being carried out in 2012. In moving towards a competencybased program we elected to adopt the CanMEDS framework [5,6], following an extensive literature review [7] as the most appropriate framework for our purposes. A dedicated committee of the College's Board of Education was established to oversee the development of the new curriculum. The members of this committee had expertise in various aspects of postgraduate education. It included Directors of Training, supervisors and psychiatrists with expertise in training and assessment. Consumers and carers and a trainee representative were also included within this committee. This committee could be considered as an 'expert' panel overseeing the project. As part of this committee's work, we consulted with international educationalists to gauge their opinions regarding the development of our new program.
As part of the process we held extensive consultations with Directors of Training across Australia and New Zealand. Feedback from these groups was provided to the overseeing committee. Whilst these consultations were not formally established as focus groups, the discussions that took place in them were fed back to the committee effectively fulfilling the role of focus group work.
Our new training program will be in three stages; general adult psychiatry for one year to acquire basic competencies; two years spent acquiring competencies in a range of areas of practice (with mandated rotations in Child and Adolescent psychiatry, Consultation and Liaison psychiatry) and finally the acquisition of advanced competencies in a specific area or areas of practice. A purposeful and rigorous program of Workplace-Based Assessments (WBA) -Case-based Discussions, Mini-CEX, Observed Clinical Activities -and summative assessments will support this program.
While workplace-based assessments are recognised as being reasonably valid assessments of competencies, they are generally designed to assess a specific competency. They do not assess across the range of roles expected of a competent psychiatrist. Assessing competencies in this way runs the risk of 'atomising' them rather than bringing them together. This could end up in the situation where a trainee may have a set of competencies (adequately demonstrated in WBAs) but is unable to integrate them across roles that are necessary to demonstrate competent performance.
WBAs do not adequately assess the actual clinical duties and responsibilities that trainees have. They are often required to deal with emergency clinical situations at times when a supervisor is not available and there are currently no formalised mechanisms of determining what level of responsibility should be accorded to a trainee at different levels of training.
To overcome this problem we plan to introduce the concept of Entrustable Professional Activities (EPA) to give a more wholistic picture of competence and define what a trainee is entrusted to do with differing levels of supervision.
Ten Cate has introduced the concept of EPAs [8,9], as activities, or units of work, that a trainee can be trusted to perform competently. He outlined a set of criteria for these activities: they are essential professional work that can only be carried out by qualified staff, they specify knowledge, skill and attitudes that can be acquired through training, they lead to recognised output of professional labour, can be independently executable within a specific timeframe, they are observable and measurable in process and outcome and lead to a conclusion (done well, or not well). Importantly, they encompass a set of competencies across different roles.
An EPA will enable a supervisor to know when a trainee can be trusted to carry out specific tasks with minimal or no supervision. They can be mapped to the CanMEDS roles and aligned with the learning objectives of a clinical rotation. Entrustment would reflect not only confidence in a trainee being able to competently complete an activity, but also contribute to the trainee being able to recognise when they need additional supervision In order to determine what would be appropriate activities that could be entrusted within the first stage of our new training program the curriculum committee elected to survey College Fellows (psychiatrists who have successfully completed their training with the RANZCP) to ascertain their opinions regarding the potential value and use of such activities as EPAs and to identify if the proposed activities could be classed as EPAs for the first year of training, when the trainees are in the earliest developmental stage in their trajectory to acquire Fellowship.

Methods
All RANZCP Fellows were invited to complete an anonymous online survey which was constructed using Survey-Monkey. The curriculum committee, had previously, through its development work, identified a set of activities that trainees would be expected to carry out in the early stages of training with varying levels of supervision. The committee defined an EPA for the purposes of the survey and included the description in the survey introduction. The identified activities had good 'face validity' as they were determined by expert psychiatrists with extensive experience in psychiatry and postgraduate psychiatry training. Furthermore the face validity was evidenced in that forms of the identified EPAs were already present in the training and assessment program. The survey asked College Fellows to consider whether a trainee should be entrusted with the specific activity by the end of the first stage of training prior to moving on to rotations in specific areas of practice. Fellows were asked to rate whether competence in the proposed areas should be mandatory, very important, somewhat important or of little importance.
The activities that were asked about included initiating various treatments, discharging patients, engaging in family education sessions, conducting risk assessments, admitting patients without supervision, crisis assessments, referring for laboratory investigations, performing neurological examinations and formulating a case.
The survey invited the Fellows to identify other potential EPAs. The survey included an opportunity to collect qualitative data through free text feedback.
The survey was posted on the RANZCP website and an invitation to respond to the survey was sent out in two ways; first, in the College's regular online newsletter (psych-e-bulletin) and then followed up with a group email to all 2736 College Fellows. No further reminders were sent out as in our experience that has not substantially increased the response rate.

Results
Four hundred and eighty college Fellows (n = 2736) responded to the online survey, a response rate of 17.5%. Table 1 shows the proportion of respondents rating the activities as 'mandatory', suggesting that the activity should be 'entrusted' by the end of the first stage of training.
The majority of respondents (over 70%) considered initiating patients with the common medications should be an activity that trainees could be entrusted with by the end of the first stage of training. By contrast, they did not consider initiating a course of cognitive behaviour therapy, brief psychotherapy or long-term psychotherapy as activities that should be entrusted to trainees in this early phase of their training as shown in Figure 1.
There seemed to be broad agreement that trainees could be entrusted to discharge patients suffering from schizophrenia or bipolar disorder from hospital as well as discharging patients following a crisis admission after an episode of self-harm. However, there was lower endorsement for discharging patients suffering from an adjustment disorder.
There was strong agreement that trainees at this stage in their career could be entrusted to conduct a risk assessment, conduct a crisis assessment with handover and report to the duty consultants, and be involved in the acute assessment and immediate management of psychiatric emergencies. The ratings were lower with regard to performing appropriate laboratory investigations to exclude medical complications and in formulating a case or completing a discharge summary.
The majority of respondents did not consider conducting family education sessions for the major psychiatric illnesses, managing a patient with a personality disorder without the use of medication, performing ECT or the nonpharmacological and pharmacological management of anxiety disorders should be entrusted in the first year of training. Some activities were considered mandatory by over 50% (but less than 70%) of respondents, including admitting a patient without supervision, performing a neurological examination, designing a biopsychosocial care plan and consenting patients ECT.
Respondents identified further activities that they considered could be included as EPAs, however many of the activities they identified were specific competencies or areas of knowledge, rather than clinical activities that could be described as 'entrustable' based on our acceptance of ten Cate's (2007) definition.

Discussion
While online surveys are now widely used in survey research, there remains considerable debate about the best strategies to assure response rates that will allow one to generalise from the data. Historically, low response rates characterise online surveys conducted by our College; given this, we were satisfied that the available data as 'exploratory data' could justifiably inform our deliberations.
In developing a new curriculum for the RANZCP's training program we have elected to use Entrustable Professional Activities as an overarching assessment of competence. These EPAs provide a content focus for WBA, with assessments for each of the EPAs being undertaken early in a rotation; it is intended that these will inform the development of a learning plan, devised by the supervisor and trainee. Before embarking on this we conducted a survey of College Fellows to ascertain their views about activities that could be 'entrusted' to trainees by the end of the first stage of training prior to moving on to train in specific areas of practice. The activities they endorsed are the same activities that trainees often carry out with minimal or no supervision in their day-to-day clinical work. Activities that required more expertise or would not be expected of trainees in this early stage of training did not gain endorsement. This suggests that there is good understanding of EPAs as we presented the concept, particularly the developmental nature of entrustment among College Fellows who responded to our survey, many of whom are engaged in the supervision and teaching of trainees.
Based on the results of this survey and what we considered should be expected of our trainees, the curriculum development group decided on a set of four EPAs that trainees will be required to have assessed during Stage 1 of their training. The survey results were discussed by the curriculum committee at length, especially with regard to how many EPAs should be selected and which one would cover the main areas of competence for this particular stage of training. The committee considered that discussing with a family a diagnosis of schizophrenia was an important EPA as it covered basic knowledge, communication skills, dealt with a disorder commonly seen by our trainees at this stage of their training and is something that trainees are expected to do. The four selected are; 1. completing a discharge summary; 2. initiating antipsychotic medication in a patient with schizophrenia; 3. leading the multidisciplinary team discussion regarding the care of a patient; and 4. carrying out a diagnostic explanation to a family about a young adult's psychiatric illness.
The decision to recommend four EPAs in the first year of training was a pragmatic one based on the survey responses as outlined above. This is a new concept and we wish to conduct more pilot work testing their acceptability, utility and feasibility. Second, we did not want to overburden trainees and supervisors with too Table 1 Proportion of participants that rated the activity as mandatory prior to progressing to the next stage of training. many EPAs and their associated WBAs in the initial implementation of this competency-based training program and risk the process being perceived as too burdensome. Assessment of the EPAs will be by observation of the clinical activity, review of the case notes and discussion with the trainee about the activity. The EPA is a global assessment of competence, as such trainees will be observed in real practice and will either be able to be entrusted (pass) or not entrusted (fail) in the particular activity. For those who are not entrusted it will be necessary to "unpack" the activity (into the relevant CanMEDS roles, as shown in Table 2) to identify the areas of weakness so there can be further remediation supported by specific workplace-based assessment to ensure they have improved that specific skill.
We are now planning an expanded set of EPAs for specific rotations, such as Child and Adolescent psychiatry, Consultation and Liaison psychiatry, Psychiatry of Old Age, Addiction Psychiatry, Rural Psychiatry and other areas of practice. It is intended that these will become one of the major ways in which competent performance is assessed in the later stages of training.

Conclusion
The survey data as we have described it gives us confidence that there are clear advantages in using EPAs for our training program. The limitations of the survey are restricted to the low response rate, however as an exploratory study and given the RANZCP's history of low response rates to online surveys, we see the data and our analysis as useful in the broader context of this specific curriculum redevelopment.
Each EPA comprises a set of underpinning competencies across a range of roles (for our purposes the Can-MEDS roles) as shown in Figure 2. This allows the activity to be "unpacked" to identify the role specific competencies as shown in Table 2 for the EPA, initiating an antipsychotic medication to a patient with schizophrenia. This will aid trainees in developing their learning plans so that they have the necessary competencies for the specific EPA. Second, it will guide assessors in what they need to be checking when they come to 'entrust' a trainee for that particular activity (in Table 2 we provide indicative questions that an assessor could think about when assessing a trainee). Third, when a trainee is not performing at a level where they can be entrusted, the supervisor will be better able to guide the trainee in which competencies they need to develop and improve as part of their learning plan. Finally, based on anecdotal feedback received during consultation initiatives as part of the curriculum development, it is anticipated that the use of EPAs will be of great value for clinical directors and supervisors to know what level of responsibility can be accorded to a trainee.
We believe that the use of EPAs in conjunction with a more transparent outcome-oriented training program informed by CanMEDS, will allow us to better prepare Figure 1 Responses to question asking whether initiating treatments should be an activity a trainee could be expected to perform by the end of the first stage of training.
trainees and to give supervisors and others engaged in the training program greater confidence in assessing performance. We anticipate that our pilot studies will assist us in developing a more comprehensive understanding of performance based assessment and the complex nature of learning in the clinical setting of postgraduate medical education. Moreover, it offers new opportunities to develop collaborative research across training programs nationally and internationally as medical education continues to investigate the efficacy of competency-based training models and their impact on both learning and patient safety.

Medical expert
The trainee demonstrates the ability to make an accurate diagnosis, has conducted the appropriate assessments, can describe the evidence for the use of the medication, its dosage, interactions and side effects.