- Research article
- Open Access
- Open Peer Review
Exploring taught masters education for healthcare practitioners: a systematic review of literature
BMC Medical Educationvolume 19, Article number: 340 (2019)
Masters-level education is a key pathway of professional development for healthcare practitioners. Whilst there is evidence that Masters-level education leads to career enhancement, it is unclear how the programme pedagogy contributes to this. The objective was to: (1) examine the programme pedagogies and context that supports learning, and (2) synthesise the outputs, outcomes and impact of Masters-level healthcare programmes.
A systematic review was conducted according to the Cochrane Collaboration handbook and is reported in line with PRISMA. Using pre-defined key terms and eligibility criteria, two reviewers independently searched Medline, ERIC, Web of Science, ProQuest, and CINAHL Plus databases from inception to 14th November 2016, reference lists of retrieved articles and selected websites. Data were extracted independently. The Mixed Methods Appraisal Tool was used to assess methodological quality. A Weight of Evidence Framework enabled evaluation of the overall quality of evidence. Data were synthesised using thematic qualitative analysis.
Thirty-five studies were included. All studies were retrospective, evaluated programmes in nursing (n = 19), physiotherapy (n = 6), general and family medicine (n = 4), public health (n = 3), dentistry (n = 1), interdisciplinary (n = 1), and occupational therapy (n = 1). Most studies were rated low in methodological quality, with an overall low to moderate weight of evidence for programmes’ outcomes and impact. Pedagogies that promote social participation and knowledge co-construction, reflection, learner-centred approach, relevance and authenticity influenced outcomes and impact.
Notwithstanding the low to moderate weight of evidence, the review identified multiple positive outcomes of Master-level education for healthcare practitioners. Whilst the pedagogies that contributed to such positive outcomes were examined in some studies, there is a need to further explore links between programme pedagogy, outputs, outcomes and impact. A cultural approach to evaluation may capture how M-level education drives changes.
Preparing expert healthcare practitioners with a high level of competencies is an urgent global need to ensure the delivery of advanced level of patient care [1, 2]. To enable this, healthcare practitioners, those involved in the delivery of patient care, are required to engage in professional development activities  with many undertaking Masters’ level (M-level) education . M-level education is defined for the purpose of this review as postgraduate formal and structured education based in higher education institution leading to either a Postgraduate Diploma or a Master of Science qualification. This level of education has been associated with an increase in the number of practitioners seeking senior or advanced practice roles [5, 6]. In the context of specialist M-level physiotherapy education, authors suggest that M-level education forms the basis for developing clinical expertise [7,8,9,10,11], although there is limited understanding of how the learning culture contributes to this; that is an understanding how interactions at the learners, programme and workplace levels supports change.
The learning culture of M-level education involves structured engagement with multiple pedagogies and contexts over a period of 1–2 years of full time study . Understanding how this learning culture supports professional learning requires the use of logic model that maps out the programme’s outputs, outcomes, impact and the context of change [13,14,15,16]. This is rooted in a cultural view of learning which rejects the premise that learning is a process of acquiring and transferring knowledge . Instead it is suggested that professional learning extends beyond the learning site and involves sociocultural contextual factors that modulate and shape a practitioner’s learning experience by influencing the interaction between a programme’s output, outcome, and impact [14,15,16]. Although such contextual factors are arguably external to programme activities , they are integral to a programme’s learning culture by virtue of the learners’ biographies . It is less clear, however, how much M-level pedagogies draw on this cultural dimension of professional learning.
Three low to medium quality systematic reviews [19,20,21], rated using AMSTAR , have explored the influence of M-Level education in a healthcare context, with the most recent one including studies up to November 2011. Whilst these reviews explored the outcomes and impact of M-level programmes, there was limited exploration of programme pedagogies and contexts that drove changes. Moreover M-level education was not clearly defined which affected confidence in findings due to the inclusion of studies that evaluated pre-registration entry-level Masters and combined M-Level/PhD programmes. Therefore, conclusions were not specific to M-level education resulting in a need for a robust current systematic review that examines how M-level education in healthcare supports the professional development of practitioners.
To undertake an evidence synthesis to explore how M-level education culture supports professional development of healthcare practitioners. In particular, to:
Understand the programme pedagogy and context that supports learning.
Identify the outputs, outcomes and impact of M-level education in supporting healthcare practitioners.
A systematic review designed using the guidelines of the Cochrane Collaboration handbook was conducted . The review is reported in line with Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) guidelines .
Was informed by the adapted search concept tool ‘PICOS’ to ensure the objectives of the review could be achieved. In this instance Population referred to participants/graduates of M-level education, Intervention was M-level education, Outcome referred to programme outputs and impact and all forms of study design considered.
Studies that evaluated M-level healthcare programmes were included, specifically including Postgraduate diploma and Master of Science programmes.
Qualitative, quantitative and mixed-methods research designs.
Studies that were published in the English language.
Theoretical studies with no data collection
Evaluated online, long distance M-level courses.
Residency and Fellowship programmes that are not based in higher education settings
Information sources and search strategy
Two independent reviewers (MM/HH) searched:
Medline (Ovid), ERIC, Web of Science, ProQuest, and CINAHL Plus databases from inception to 14th November 2016.
Reference lists of retrieved articles, websites (Google scholar, science direct, and Taylor and Francis) and grey literature (dissertations and theses)
The following is the search strategy used in Medline (Ovid) from 1946 until 14th November 2016:
Master’s level education.mp.
1 or 2 or 3 or 4 or 5
Evaluation.mp. or Evaluation Studies as Topic/
7 or 8 or 9 or 10
6 and 11
Reviewers independently evaluated retrieved studies against the pre-specified eligibility criteria, rating each study as ‘eligible’, ‘not eligible’ or ‘might be eligible’. A third reviewer was available to mediate disagreements.
Following piloting, one reviewer (MM) extracted data using an adapted Cochrane Collaboration’s data extraction form . The second reviewer (HH) checked extracted data for accuracy. Data that answer the main review questions or aid assessing the quality of individual studies were extracted. These included: reference details, country, funding source, conflicts of interest, programme level, programme title, programme aims, study design, study aims, outcome measures, participants, method/s of recruitment, response rate, obtained consent, ethical approval, programme activities and pedagogy, point of approaching graduates, evaluation model used, programme outputs, outcomes, and impact. Authors of eligible studies were contacted to retrieve missing or clarify ambiguous data.
To ensure precise and consistent data extraction, the Logic Model  definitions of programme output, outcome, and impact were used. The Logic Model has previously been used to map programme design, structure, output, outcome, and impact . Where output is used to refer to the direct products of programme activities which may include types, levels and targets of services to be delivered by the programme; outcomes describe the specific changes in participants’ behaviour, knowledge, skills, status and level of functioning; and impact refers to the fundamental intended or unintended change occurring in organizations, communities or systems as a result of programme activities within 7 to 10 years [25, 27]. Thus, the use of these terms offered a unified lens to review M-level programme evaluation literature.
Methodological quality assessment
The two reviewers (MM/HH) independently assessed the methodological quality of included studies. The Mixed Method Appraisal Tool (MMAT) was used . MMAT is a valid (based on experts judgment), reliable (ICC = 0.80) and efficient tool to critically appraise the methodological quality of studies of differing designs [28,29,30]. Additionally, a Weight of Evidence (WoE) Framework  was used to appraise the quality and relevance of evidence using pre-specified criteria of three domains: A) soundness of studies; B) appropriateness of study design for answering the review questions; and C) relevance of the study focus to the review. High WoE was defined as A) scoring more than 50% using MMAT, B) Drawing on multiple cohorts of students using a longitudinal pre-post study design, and C) defining primary outcomes with clear description of specific programme pedagogy. Medium WoE was defined as A) scoring 50% using MMAT, B) drawing on one cohort of learners using a longitudinal pre-post design, and C) defining primary outcomes with no specific description of programme pedagogy. Finally, low WoE was defined as A) scoring less than 50% using MMAT, B) a post hoc programme evaluation, and C) the lack of defined outcomes as well as proper description of programme pedagogy.
Synthesis of results
The extracted data were tabulated and synthesised using thematic qualitative analysis . Data were deductively coded using the logic model terminologies under the themes of programme pedagogy, contextual factors, outputs, outcomes, and impact. With iterative examination of data, each category under these main themes was clearly defined to establish its properties (Table 1). This was followed by capturing the relationships between coded data of various programmes in a way that illuminates the chain of reasoning of how and why they work. This analytical approach allowed mapping of M-level programme Logic Model across multiple healthcare disciplines.
Summary and characteristics of included studies
Thirty-five eligible studies that drew on the accounts of 2834 graduates and a total of 87 programme educators, clinical managers and workplace colleagues were included. See Fig. 1.
The characteristics of these studies are shown in Table 2. A list of excluded studies and reasons for exclusion are included in Additional file 1. Included studies explored M-level programmes originated from the UK (n = 16), USA (n = 3), Australia (n = 4), Ireland (n = 3), New Zealand (n = 3), Canada (n = 2), Jordan (n = 1), and Vietnam (n = 1). Two studies [60, 66] explored M-level programmes originated from multiple countries. All studies were retrospective in nature, and the exact timing of approaching graduates was unclear in 18 studies (Table 2).
These studies drew on qualitative (n = 15), quantitative (n = 14) and combined qualitative and quantitative methods of data collection (n = 6) (Table 3). Most quantitative studies were descriptive, with only two cross-sectional analytical studies [51, 63] that compared M-level participant-students’ cohorts with other cohorts. Three studies used large scale alumni surveys to study career pathways in nursing , physiotherapy , and public health .
Details of methodological quality of individual studies are described in Table 4. The methodological quality ranged from low to medium for most studies. Only four studies [40, 50, 54, 58] were deemed to be of high quality.
Logic model synthesis
The synthesis of the findings into a completed programme theory Logic Model is illustrated in Fig. 2. This synthesis of M-level programme theory across several healthcare professions offers a pathway that represents how the programme pedagogical activities and context interacted to produce outputs, outcomes and impact. The inclusion of programme activities and context by some studies facilitated the collective synthesis of this model.
Three studies reported a programme’s modules with no further exploration of its pedagogy [36, 53, 64]. The pedagogies of nine programme evaluations were described (Additional file 2). These pedagogical approaches were synthesised into five categories. They were generally informed by social constructivism, adult learning, and reflective learning theories, with an overall structure that promoted a learner centred approach. The participant-students’ perceptions of a programme’s pedagogy were not explored.
The role of theoretical content
Graduates in five studies [38, 40, 42, 53, 58] explicitly referred to the role of a programme’s theoretical component as a source of advancing practice. They believed that the specialist theoretical knowledge they received positively influenced their professional development, particularly where the theoretical aspect of education was not covered in undergraduate education, leading to a belief that incorporating, or lack of, propositional knowledge in M-level education can directly impact the overall advancement of graduate’s skills.
Social participation and knowledge co-construction
Across physiotherapy, nursing, general practice, and public health programmes, social learning supported peer-peer communication and co-construction of knowledge and experiences [34, 35, 38], especially when done in small groups . The collaborative and problem-solving environments were valued by graduates when compared to their pre-masters rote-learning educational experiences [44, 47]. Social learning was thought to promote integration of the shared knowledge and experiences in clinical practice, and adoption of a biopsychosocial model of practice .
Environment for reflection
A few researchers found that reflection on experience helped the most in advancing clinical reasoning skills [35, 38, 58]; especially when students documented processes of reflection . However, no study reported the processes of reflection experienced by graduates. In contrast, learning transition occurred when educators facilitated students’ critical reflections and provided feedback on performance .
Graduates of multiple programmes believed that adopting a learner-centred pedagogy contributed to positive outcomes [35, 47, 48]. The learner-centred pedagogy included setting out learning and development needs ; flexibility of the programme delivery ; analysing progress throughout the programme ; and encouraging students to speak their mind during interactive discussions .
Drawing on adult learning theory
Three studies indicated that programmes drew on principles of adult learning theory [35, 61, 63]. Promoting self-directed learning was the most prominent feature, in terms of graduates assuming responsibility for identifying personal and professional development needs .
Programme outputs component of the logic model
Programme outputs are the direct product of programme activities which facilitate achieving programme outcomes ; and include participant-students’ reactions to programme activities . Three main outputs were sporadically documented in 10 studies of low to medium quality (Table 1). These outputs highlight the importance of changing learner’s attitudes in order to drive transformative changes.
Successful collaborative work
While successful formation and support of learning groups, including small class size and promoting diversity of opinions drove learning engagement in some cohorts [34, 36, 44], poor attendance for some students created a sense of frustration which affected group dynamics .
Relevance of programme activities
Positive reactions to the learning contradictions
Achieving learning outcomes was found to be contingent on the students’ positive reactions to the learning contradictions that characterise M-level education [36, 39, 41, 44, 50, 58]. Graduates of these programmes suggested that achieving programme outcomes was associated with questioning the effectiveness of their previous practice, which lead to a process of reconstruction of their knowledge and skills. This process was described as ‘shrugging off the old’ and ‘assuming the new’ .
Programme outcomes component of the logic model
Seven main outcomes of M-level education were reported in 22 studies (Table 1). These outcomes reflected changes in graduates’ behaviour, knowledge, skills, status and level of functioning. They ranged from advancement of critical thinking and clinical reasoning skills to an enhanced career progression for most graduates.
Critical thinking skills and analysis
Advancement of critical thinking skills was identified across all healthcare disciplines. Significant differences in critical thinking ability between graduates and freshmen of six nursing programmes in Ireland were identified using the Watson–Glaser Critical Thinking Appraisal tool . The remaining evidence of advancement in critical thinking resulted from qualitative research. Graduates demonstrated a transformation from non-critical, routine and therapist-centred practice to a more critical and patient-centred one . Participating in M-level education advanced graduates’ abilities to critically discuss research evidence [34, 36, 47] which enabled them to justify their own practices [43, 58]. In one study of acute nurse graduates, their perceptions of advancement in critical thinking skills was not associated with a tangible effect on patient care such as length of hospital stay . Moreover, some students questioned their ability to continue at this level of high criticality upon returning to their workplace environment .
Clinical reasoning skills
M-level education advanced graduates’ clinical reasoning skills during both assessment and treatment phases of patient management . This was associated with open mindedness in selecting alternative management options . Likewise, graduates became more attentive to details, able to interpret patient data, and articulate diagnostic and treatment decisions . While there was limited exploration of how programme pedagogy supported change, some studies attributed it to the theoretical aspect of the curriculum. For example, graduates of some programmes demonstrated an advanced understanding of ethical reasoning because of a module or content that is related to ethical issues [50, 52, 53]. Also, a cohort of physiotherapy graduates suggested that the limited psychosocial content of their curriculum adversely affected their ability to manage patients with complex psychosocial issues .
Confidence and motivation to practice
Increased confidence and motivation in clinical practice was described by graduates of several programmes (Table 1). In particular, changes involved increased credibility in front of others ; increased willingness to engage in critical debates ; enhanced ability to conduct and publish scholarly research ; and enhanced capabilities to meet the requirements of extended scope practice, clinical specialist, and consultant roles [6, 43]. The evidence suggests that confidence improved as a result of the specific professional knowledge provided at this level of education [38, 58]. The perceived sense of efficacy allowed graduates to advocate practice and policy changes and to support professional learning of junior colleagues [41, 44, 52].
Enhanced career progression
Some studies reported, but poorly defined, a career progression as an outcome of M-level education. For example, 84% of nurse graduates agreed or strongly agreed that their programme promoted career progression through promotion, increased payment, and change of job description or specialisation . For some graduates, career changes occurred either during or upon completion of the programme, indicating a high demand on M-level qualified practitioners [6, 38, 39]. What is less clear, however, is the impact of career progression on direct patient care since graduates from some programmes assumed management, research and education duties [6, 47].
Becoming a lifelong learner
M-level education supported engagement in a lifelong learning process [36, 39, 43, 47]. It provided graduates with the tools to “learn how to learn” , and become an adult learner . However, such tools and processes were poorly defined. On the other hand, graduates were thought to be able to learn from their practice though processes of reflection . Therefore, becoming a lifelong learner was not only limited to locating sources of knowledge, but also extended to synthesising practice-based knowledge, which was linked to patient-centred practice, ongoing introspection and self-critique [48, 53].
Advanced communication skills
Graduates in public health reported a 78% increase in communication competencies . Multiple areas of advancement were identified in nursing cohorts including: oral communication, written communication, working and coping with team conflicts, understanding team members’ feelings, listening to others, and communication with colleagues . Three studies [33, 50, 52] reported advancement in communication with patients, however two studies [33, 50] reported that this was the least developed outcome.
Enhanced sense of autonomy
Whilst an increase in graduates’ autonomy was reported in multiple studies [36, 43, 50, 52], the authors did not to evaluate what this meant or entailed. Seventy-five percent of M-level nurse graduates (N = 236) reported an increase in ability to assume work roles independently . The authors attributed this to the enhanced level of self-confidence [36, 52]. However, they argued that 22% of graduates who did not perceive an increase in autonomy were working within a healthcare system that does not support autonomous practice [36, 52], something that was also identified in another study  leading the authors to believe that workplace context could have an adverse effect on M-level graduates’ professional development.
Programme impact component of the logic model
Programme impact is conceptualised as the fundamental intended or unintended change occurring in organizations, communities or systems within 7 to 10 years after the programme . Due to the variability between the time of graduation and the time of approaching the graduates (Table 2), the synthesis of the evidence of programme impact is limited. One study  clearly stated that they approached participant 7–10 years after the programme, although for many this was unclear. Impact was self-reported in most studies (Table 1), with two studies [49, 65] drawing on stakeholders such as managers and workplace colleagues. The main domains of M-level programme impact were: management of complexities; assuming research, leadership and management positions; assuming teaching roles; increased retention rate of practitioners; and enhanced patient care.
Management of complexities
M-level education enhanced graduates’ abilities to understand the healthcare system [56, 63], demonstrate flexibility in management decisions [58, 60], and demonstrate creative practice . Moreover, graduates were able to manage complex patient presentations . They also showed attitudes of appreciating others’ perspectives, thinking analytically, defining problems, and resolving conflicts .
Assuming research, leadership and management positions
Data suggests that M-level education enhanced graduates’ research, leadership and management skills putting them in a position to drive change in practice and service delivery [56, 57]. M-level public health graduates were able to evaluate service delivery and recommend development needs , however, it was not clear whether these changes were at a local level i.e. graduate’s workplace or at a national level.
Assuming teaching roles
Engagement in collegial teaching duties, supporting a peer’s learning, and involvement in university education were reported in multiple evaluations (Table 1). Involvement in teaching activities was not only an opportunity for M-level graduates to give back to society, but also an opportunity for them to engage in a lifelong learning process [36, 40, 48, 52].
Increased retention rate of practitioners
In one study, graduates expressed a tendency to remain in clinical practice because of increased motivation and confidence . This was evident in the cases of practitioners with more experience because of the fresh perspectives and insights offered by M-level education , which led some graduates to express their desires to implement knowledge and skills within the clinical context . In the UK context, graduate retention was associated with the presence of clear National Health Service (NHS) scope of practice that acknowledges and rewards M-level graduates .
The data demonstrated limited exploration of the impact of M-level education in terms of capturing tangible changes to patient care. For example, duration of recovery. It was implicitly demonstrated that indirect improvement of the quality of patient care is plausible through advancement in knowledge, cognitive and clinical reasoning skills , particularly in terms of embracing patient-centred practice [50, 53]. Nonetheless, some managers or colleagues of nursing programme graduates did not perceive any change in direct patient care [49, 65]. In one study, this was attributed to workplace restrictions or practice policies that do not differentiate between M-level graduates and less qualified practitioners .
Multiple factors at the level of the individual learner, the programme of study and workplace environment were identified as potential facilitators or barriers for achieving positive M-level programme outcomes and impact.
Stress and anxiety, time management, and meeting the demand of assignments were all described as barriers for successful engagement during M-level education [39,40,41]. Student’s avoidance of group discussion and collaborative peers learning indicated that positive changes are less likely to occur . Moreover, integrating M-level knowledge and skills were contingent on a student’s psychological preparedness for advanced level of practice .
Students’ acceptance of scrutinising their practice was associated with supportive learning environment that offered constructive feedback . These learner-centred environments were augmented by having approachable educators for a personalised feedback and support [47, 48]. However, the nature and frequency of feedback was not further examined. Moreover, consistent with principles of adult learning , promoting authenticity and relevance to learner’s clinical environment were seen as important to drive positive outcomes . However, there were limited details of what constitutes an authentic learning environment.
It was suggested that workplace structure could limit the full integration of knowledge and skills [6, 43, 44, 50]. Whilst some graduates moved towards senior positions and advanced practice roles, others expressed a lack of enhancement because of pay and reward systems . A few graduates described a lack of time, a large caseload, an uncooperative employer’s attitude, a lack of autonomous practice and a ‘poor vision’ of the NHS in accommodating their skills as barriers for continued learning [6, 41]. Having less clear job descriptions or career prospects were also documented as barriers to integration, which brought graduates into conflict with managers and colleagues .
Modelling for learner’s transformation
Three studies [41, 58, 63] developed explanatory models that linked M-level programme activities and outcomes. Perry et al.’s  ‘knowledge acquisition’ model, which consisted of five phases, attempted to explain learner’s transformation through changing expectations and deconstructing professional knowledge followed by ‘reconstruction’ and ‘actualisation’ of M-level knowledge and skills in practice. While this model did not capture the context of transformation, Petty et al.’s  ‘Learning Transition Model’ highlighted the role of learner’s biography and expectations in mediating transition, in particular their reaction to the critical nature of M-level education. On the other hand, Cragg and Andrassy’s  evaluation of a nursing programme demonstrated an ‘evolutionary’ type of learner transformation; adding new knowledge to what graduates already knew from their undergraduate programmes.
This is the first methodologically rigorous review of evidence that explored taught M-level education in the healthcare context. The following is a discussion of the key findings as well as the weight of this evidence in the context of the review aims.
The review identified evidence that learners’ reactions to programme activities determined the extent of transformation. For example, whilst engagement in critical reflection drove transformative changes in practice , such scrutiny to one’s practice generated reactions that ranged from being defensive of their experience to being receptive to new knowledge. This gap between students’ learning dispositions and the intended outcomes of M-level education can be a source of conflict that potentially interrupts the learning process. The evidence suggests that acceptance of such scrutiny of one’s practice is associated with a supportive learner-centred environment that offers constructive feedback . Such learner-centred environments were found elsewhere to promote learners’ autonomy [68,69,70,71] because of engagement in transactional relationships with their peers and educators. Moreover, learner-centred environments have the potentials to alleviate learners’ anxieties that impact learning engagement [72, 73] and therefore, achieving successful learning outcomes.
While learners’ confidence and motivation were identified as a ‘catalyst for personal growth’ , there is a need to further explore how extrinsic motivation at the learning site contributed to personal and professional development . In accordance with Hager and Hodkinson , who emphasised the role of workplace structure in supporting practitioners’ learning, this review identified that learners’ motivation to maintain an advanced level of practice was dependent on workplace environment [6, 44, 50]. Graduates from several programmes expressed a sense of frustration because M-level advanced skills were not welcome within the healthcare system [6, 41, 43, 44, 65]. This potentially limits the full integration of knowledge and skills in practice, and brings graduates into conflict with managers and colleagues. For example, drawing on the experience of nursing, occupational therapy and physiotherapy educators, Gerrish et al.  likened graduates of nursing programmes to mavericks who did not fit comfortably into workplace cultures. They further suggested that learners’ empowerment and awakening led to conflict with managers and colleagues . On the other hand, the current NHS’s post structure supports UK practitioners to working towards M-level qualification . Similarly, in the context of musculoskeletal physiotherapy, Haywood et al.  demonstrated the positive role that physiotherapy professional bodies and employers play in supporting practitioners’ professional learning when compared with other healthcare practitioners who manage musculoskeletal conditions. Therefore, understanding motivators for professional learning would better inform M-level educators in designing effective learning environments that can cultivate learning engagement and even augment motivation. For Ryan and Deci , this involves understanding how the interaction between psychological and sociological aspects of motivation modify learners’ actions.
Moreover, modelling learners’ transition was described in three medium quality studies. Perry’s et al.’s  and Petty’s et al.’s  models drew on physiotherapy population and are consistent with Mezirow’s  stages of adult learners’ ‘revolutionary’ transition where learner’s professional identity is transformed (Fig. 3). In contrast, Cragg and Andrassy’s  model drew on nursing population and indicated an ‘evolutionary’ nature of transition, where learners develop their existing professional identity. This potentially explains why graduates from several nursing programmes highlighted the positive impact of their programme’s theoretical content [38, 50]. Therefore, it appears that learners’ transition is discipline-specific and influenced by workplace context. Such an interpretation is limited considering the absence of comprehensive examination of learners’ biographies.
Weight of evidence
The overall quality of synthesised evidence demonstrated low to medium evidence across M-level pedagogy, outcomes, and impact (Table 4). This was mainly due to low to medium quality of evidence, and the inappropriate and limited relevance of included studies. With regard to methodological quality, Drennan  was the only researcher to use a validated assessment tool to evaluate differences in critical thinking of two nursing cohorts. While six studies [6, 38, 42, 50, 53, 62] used combined qualitative and quantitative methods of data collection, the value of this design was not clear, nor did it appear to impact the overall study conclusion. It was not clear if qualitative data were used to interpret quantitative survey; or if qualitative data facilitated the design of postal questionnaire . Programme document analysis was not reported in any studies, hence losing a rich source of data related to programme structure and delivery . Moreover, while qualitative-based research identified programme outcomes and impact inductively, some themes lacked theoretical saturation because of underreporting of causes, conditions, context, contingencies, consequences, and covariances  that would modulate changes. On the other hand, while most studies drew on graduates’ accounts, five programme evaluations drew on stakeholder data i.e. educators, managers and colleagues, adding more credibility to findings [38, 45, 49, 61, 65].
Whilst Logic Model terminologies were clearly defined and used in one study , terms were often ill-defined and used interchangeably across other studies; for example, ‘impact’ was used in most studies to describe the programme outcomes. Constantine and Carpenter , who evaluated graduates’ experience of M-level programme in musculoskeletal physiotherapy, repeatedly used terms interchangeably. Moreover, Pelletier et al. , who set out to evaluate the outcomes of M-level nursing programme, only reported the immediate outputs of the programme.
Where studies were retrospective in nature, the synthesised evidence offered few details linking outcomes and impact to programmes’ pedagogies, learners’ biographies and the wider context. Investigating prospective longitudinal studies would be useful to examine programme’s pedagogy and the learning context that drives change , capturing the frequency, type and duration of programme activities as well as making an informed judgment whether the programmes delivered learning activities as planned or not. This would usefully include a comprehensive understanding of student’s biography and learning dispositions , particularly in terms of understanding how learners’ biography, pre-programme clinical experience, in-service training and peer learning might contribute to outcomes and impact . Moreover, it could have provided an account for how spatial (place-related) and temporal (time-related) dimensions of M-level education can influence learners’ dispositions and identity development [83, 84]. Therefore, because of the low to medium quality of included studies, the synthesised Logic Model needs to be interpreted and used with caution, and requiring further testing.
Strengths and limitations of the review
It is suggested that a realist type of review has the potential to synthesise a programme theory that contextualises what programme components are most effective in achieving desired outcomes [85, 86]. However, since the aim of this review was to synthesise the effectiveness of a single programme in comparative contexts, the outputs of realist review may not be transferable to other programmes. A realist review would be limited since the aim is to understand how various programmes that are heterogeneous in nature (i.e. M-level health care programmes) lead to similar outcomes and impact. In this review, building an M-level programme logic model provided a more nuanced and comprehensive understanding of the complex pathways, i.e. the nuts and bolts, from conceptualisation of the programme to the application of ‘learned skills’ in the practitioners’ own environment [25, 27]. This generative theoretical explanation of change has the potential to inform planning and evaluation of M-level programmes .
The review has fulfilled the methodological quality criteria of evaluating and conducting systematic reviews AMSTAR . In contrast to existing reviews, this review consistently used a universally-accepted Logic Model terminology of ‘output’, ‘outcome’ and ‘impact’  offering a unified lens for the purpose of the synthesis. Derived conclusions are affected by the inclusion of low-medium quality studies, where aims are not fully aligned to the objectives of this study. Studies involving ‘residency’ and ‘fellowship programmes’ were not included. These programmes can be equivalent to M-level education, yet they are not based in higher education settings.
Findings from this methodologically rigorous review underpin two key points. Firstly, multiple positive outcomes and areas of impact are reported in M-level healthcare education. This synthesised evidence was derived from retrospective studies in which a single method of data collection was used, thus, limiting the generalisability and transferability of findings. Secondly, although the link between programme pedagogy, context and outcomes was underreported, drawing on contemporary learning theories such as participatory learning [13, 14] is believed to produce the intended learning outcomes. Future research need to examine, through a longitudinal empirical study, learners’ dispositions prior, during and after engagement in M-level healthcare education as well as the influence of various contexts to bridge this gap in evidence.
Availability of data and materials
The datasets analysed in this review are available from the corresponding author on reasonable request.
World Health Organization. Transforming and scaling up health professionals’ education and training: World Health Organization guidelines; 2013.
Department of Health. Liberating the NHS: developing the healthcare workforce: From design to delivery. London: Department of Health; 2012. Available from: https://www.gov.uk/government/publications/developing-the-healthcare-workforce-from-design-to-delivery.
Health and Care Professions Council. Your guide to our standards of continuing professional development. 2012.
House G. Postgraduate education in the United Kingdom: higher education policy institute and the British library; 2010.
Department of Health. Meeting the Challenge: A Strategy for the Allied Health Professions: Arts Therapists, Chiropodists and Podiatrists, Dietitians, Occupational Therapists, Orthoptists, Paramedics, Physiotherapists. In: Prosthetists and Orthotists, Diagnostic Radiographers, Theraputic Radiographers, Speech and Language Therapists. London: Department of Health; 2000.
Green A, Perry J, Harrison K. The influence of a postgraduate clinical master’s qualification in manual therapy on the careers of physiotherapists in the United Kingdom. Man Ther. 2008;13:39–147.
Beeston S, Rastall M, Hoare C. Factors influencing the uptake of taught master’s programmes among physiotherapists. Physiotherapy. 1998;84:480–6.
Gosling S. Physiotherapy and postgraduate study: a follow-up discussion paper. Physiotherapy. 1999;85:117–21.
Milidonis MK, Godges JJ, Jensen GM. Nature of clinical practice for specialists in orthopaedic physical therapy. J Orthop Sports Phys Ther. 1999;29(4):240.
Petty NJ. Becoming an expert: a masterclass in developing clinical expertise. Int J Osteopath Med. 2015;18:207–18.
Rushton A, Lindsay G. Defining the construct of masters level clinical practice in manipulative physiotherapy. Man Ther. 2010;15:93–9.
The Quality Assurance Agency for Higher Education. Master’s degree characteristics. Gloucester: The Quality Assurance Agency for Higher Education; 2010.
Hager P, Hodkinson P. Moving beyond the metaphor of transfer of learning. Br Educ Res J. 2009;35:619–38.
Hodkinson P, Biesta G, James D. Understanding learning culturally: overcoming the dualism between social and individual views of learning. Vocat Learn. 2008;1:27–47.
Leithwood K, Levin B. Assessing school leader and leadership programme effects on pupil learning: conceptual and methodological challenges (Report No. RR662). London: Department for eduction and skills (DfES) Publications; 2005.
Coldwell M, Simkins T. Level models of continuing professional development evaluation: a grounded review and critique. Prof Dev Educ. 2010;37:143–57.
Marchal B, van Belle S, van Olmen J, Hoerée T, Kegels G. Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research. Evaluation. 2012;18:192–212.
Peim N, Hodkinson P. Contexts, cultures, learning: contemporary understandings. Educ Rev. 2007;59:387–97.
Cotterill-Walker SM. Where is the evidence that master's level nursing education makes a difference to patient care? A literature review. Nurse Educ Today. 2012;32:57–64.
Gijbels H, O’Connell R, Dalton-O’Connor C, O’Donovan M. A systematic review evaluating the impact of post-registration nursing and midwifery education on practice. Nurse Educ Pract. 2010;10:64–9.
Zwanikken PA, Dieleman M, Samaranayake D, Akwataghibe N, Scherpbier A. A systematic review of outcome and impact of master’s in health and health care. BMC Med Educ. 2013;13:18.
Shea B, Grimshaw J, Wells G, Boers M, Andersson N, Hamel C, Porter A, Tugwell P, Moher D, Bouter L. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7:10.
Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration; 2011. Available from: http://handbook-5-1.cochrane.org/.
Moher D, Liberati A, Tetzlaff J, Altman DG, The PG. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med. 2009;6:e1000097.
Kellogg WK. Using Logic Models to Bring Together Planning, Evaluation, and Action: Logic Model Development Guide. Michigan: W.K. Kellogg Foundation; 2004.
Armour KM, Makopoulou K. Great expectations: teacher learning in a national professional development programme. Teach Teach Educ. 2012;28:336–46.
Baxter SK, Blank L, Woods HB, Payne N, Rimmer M, Goyder E. Using logic model methods in systematic review synthesis: describing complex pathways in referral management interventions. BMC Med Res Methodol. 2014;14:62.
Pluye P, Robert E, Cargo M, Bartlett G, O’Cathain A, Griffiths F, Boardman F, Gagnon MP, Rousseau MC. Proposal: A mixed methods appraisal tool for systematic mixed studies reviews. Retrieved on [10/09/2014] from http://mixedmethodsappraisaltoolpublicpbworkscom Archived by WebCite® at http://wwwwebcitationorg/5tTRTc9yJ. 2011.
Pace R, Pluye P, Bartlett G, Macaulay AC, Salsberg J, Jagosh J, Seller R. Testing the reliability and efficiency of the pilot mixed methods appraisal tool (MMAT) for systematic mixed studies review. Int J Nurs Stud. 2012;49:47–53.
Pluye P, Hong QN. Combining the power of stories and the power of numbers: mixed methods research and mixed studies reviews. Annu Rev Public Health. 2014;35:29–45.
Gough D. Weight of evidence: a framework for the appraisal of the quality and relevance of evidence. Res Pap Educ. 2007;22:213–28.
Bazeley P. Qualitative data analysis: practical strategies. London: Sage publications Ltd; 2013.
Calvert G, Britten N. The UMDS MSc in general practice: attainment of intended outcomes. Br J Gen Pract. 1998;48:1765–8.
Calvert G, Britten N. The united medical and dental School of Guy's and St Thomas' Hospitals' MSc in general practice: graduates' perspectives. Med Educ. 1999;33:130–5.
Baron R, Mckinlay D, Martin J, Ward B. Master’s degree-accredited professional education and development courses for general practitioners in the North Western deanery 1997–2002: impact on effectiveness, retention and recruitment to additional roles. Educ Prim Care. 2006;17:147–54.
Constantine M, Carpenter C. Bringing Masters’ level skills to the clinical setting: what is the experience like for graduates of the master of science in manual therapy programme? Physiother Theory Pract. 2012;28:595–603.
Bearn DR, Chadwick SM. Problem-based learning in postgraduate dental education: a qualitative evaluation of students' experience of an orthodontic problem-based postgraduate programme. Eur J Dent Educ. 2010;14:26–34.
Chaboyer W, Retsas A. Critical care graduate diploma: nursing students’ needs identified in evaluation. Aust Crit Care. 1996;9:10–3.
Conneeley AL. Study at master’s level: a qualitative study exploring the experience of students. Br J Occup Ther. 2005;68:104–9.
Petty NJ, Scholes J, Ellis L. The impact of a musculoskeletal masters course: developing clinical expertise. Man Ther. 2011;16:590–5.
Perry J, Green A, Harrison K. The impact of masters education in manual and manipulative therapy and the ‘knowledge acquisition model’. Man Ther. 2011;16:285–90.
Nicolson P, Burr J, Powell J. Becoming an advanced practitioner in neonatal nursing: a psycho-social study of the relationship between educational preparation and role development. J Clin Nurs. 2005;14:727–38.
Spencer RL. Nurses’, midwives’ and health visitors’ perceptions of the impact of higher education on professional practice. Nurse Educ Today. 2006;26:45–53.
Stathopoulos I, Harrison K. Study at Master’s level by practising physiotherapists. Physiotherapy. 2003;89:158–69.
Spence D. Advancing nursing practice through postgraduate education (part one). Nurs Prax N Z. 2004;20:46–55.
Spence D. Advancing nursing practice through postgraduate education (part two). Nurs Prax N Z. 2004;20:21–30.
Whyte DA, Lugton J, Fawcett TN. Fit for purpose: the relevance of masters preparation for the professional practice of nursing. A 10-year follow-up study of postgraduate nursing courses in the University of Edinburgh. J Adv Nurs. 2000;31:1072–80.
Drennan J. Critical thinking as an outcome of a Master’s degree in nursing programme. J Adv Nurs. 2010;66:422–31.
Barnhill D, McKillop A, Aspinall C. The impact of postgraduate education on registered nurses working in acute care. Nurs Prax N Z. 2012;28:27–36.
Tsimtsiou Z, Sidhu K, Jones R. The benefits and costs of a master’s programme in primary health care: a cross-sectional postal survey. Br J Gen Pract. 2010;60:e434–9.
LeCount J. Education, empowerment, and elderly adults--enhancing nursing expertise in the long-term care setting. J Gerontol Nurs. 2004;30:6–13.
Pelletier D, Donoghue J, Duffield C. Australian Nurses’ perception of the impact of their postgraduate studies on their patient care activities. Nurse Educ Today. 2003;23:434–42.
Wildman S, Weale A, Rodney C, Pritchard J. The impact of higher education for post-registration nurses on their subsequent clinical practice: an exploration of students’ views. J Adv Nurs. 1999;29:246–53.
Drennan J. Professional and academic destination of masters in nursing graduates: a national survey. Nurse Educ Today. 2008;28:751–9.
Pelletier D, Duffield C, Gallagher R, Soars L, Donoghue J, Adams A. The effects of graduate nurse education on clinical practice and career paths: a pilot study. Nurse Educ Today. 1994;14:314–21.
Drennan J. Masters in nursing degrees: an evaluation of management and leadership outcomes using a retrospective pre-test design. J Nurs Manag. 2012;20:102–12.
Zwanikken PA, Huong NT, Ying XH, Alexander L, Wadidi MS, Magana-Valladares L, Gonzalez-Robledo MC, Qian X, Linh NN, Tahir H, et al. Outcome and impact of master of public health programs across six countries: education for change. Hum Resour Health. 2014;12:40.
Petty NJ, Scholes J, Ellis L. Master’s level study: learning transitions towards clinical expertise in physiotherapy. Physiotherapy. 2011;97:218–25.
Cragg CB, Andrusyszyn M-A. The process of Master’s education in nursing: evolution or revolution? Int J Nurs Educ Scholarsh. 2005;2:21.
Stark SW. The effects of master’s degree education on the role choices, role flexibility, and practice settings of clinical nurse specialists and nurse practitioners. J Nurs Educ. 2006;45:7–15.
Murray C, Judd D, Snyder P. Evaluation of a post-professional master’s program in allied health. J Allied Health. 2001;30:223–8.
Le LC, Bui Q, Nguyen HT, Rotem A. Alumni survey of masters of public health (MPH) training at the Hanoi School of Public Health. Hum Resour Health. 2007;5:24.
Cragg CB, Andrusyszyn M-A. Outcomes of master’s education in nursing. Int J Nurs Educ Scholarsh. 2004;1:18.
Gerstel L, Zwanikken PA, Hoffman A, Diederichs C, Borchert M, Peterhans B. Fifteen years of the tropEd masters in international health programme: what has it delivered? Results of an alumni survey of masters students in international health. Trop Med Int Health. 2013;18:377–84.
Zahran Z. Master’s level education in Jordan: a qualitative study of key motivational factors and perceived impact on practice. Nurse Educ Today. 2013;33:1051–6.
Pelletier D, Donoghue J, Duffield C. Understanding the nursing workforce: a longitudinal study of Australian nurses six years after graduate study. Aust J Adv Nurs. 2005;23:37.
Knowles MS, Holton III EF, Swanson RA. The adult learner: The definitive classic in adult education and human resource development. London: Routledge; 2014.
Nissilä SP. Individual and collective reflection: how to meet the needs of development in teaching. Eur J Teach Educ. 2005;28:209–19.
Harrison J. Professional learning and the reflective practitioner. In: Dymoke S, editor. Reflective Teaching and Learning: A guide to professional issues for beginning secondary teachers. 2nd ed. London: Sage; 2012.
Trede F, Smith M. Teaching reflective practice in practice settings: students’ perceptions of their clinical educators. Teach High Educ. 2012;17:615–27.
Hughes G, Smith H, Creese B. Not seeing the wood for the trees: developing a feedback analysis tool to explore feed forward in modularised programmes. Assess Eval High Educ. 2015;40:1079–94.
Glover P, Bulley C, Howden S. Influences on physiotherapists when deciding to study at masters level: an exploratory study. Adv Physiother. 2008;10:14–20.
Rao N, Sayed Y, Morris P. Knowledges, identities and aspirations: implications for learning. J Comp Int Educ. 2014;44:849–51.
Gerrish K, Ashworth PD, McManus M. Some dilemmas of master's level nurse education. J Adv Nurs. 2000;32:834–41.
National Health Service. National Profiles for Physiotherapy. 2005. http://www.nhsemployers.org/PayAndContracts/AgendaForChange/NationalJobProfiles/Documents/Physiotherapy.pdf. Accessed 1 Apr 2014.
Haywood H, Pain H, Ryan S, et al. The continuing professional development for nurses and allied health professionals working within musculoskeletal services: a national UK survey. Musculoskeletal Care. 2013;11:63–70.
Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol. 2000;55:68.
Mezirow J. Understanding Transformation Theory. Adult Educ Q. 1994;44:222–32.
Teddlie C, Tashakkori A. Foundations of mixed methods research: integrating quantitative and qualitative approaches in the social and behavioral sciences. Los Angeles: Sage Publications Inc.; 2009.
Creswell JW, Plano-Clark VL. Designing and conducting mixed methods research. 2nd ed. Los Angeles: Sage Publications Inc.; 2011.
Glaser BG. Theoretical sensitivity: advances in the methodology of grounded theory. Mill Valley: Sociology Press; 1978.
Huber SG. The impact of professional development: a theoretical model for empirical research, evaluation, planning and conducting training and development programmes. Prof Dev Educ. 2011;37:837–53.
Bloomer M, Hodkinson P. Learning careers: continuity and change in young people's dispositions to learning. Br Educ Res J. 2000;26:583–97.
Hall JN. Pragmatism, evidence, and mixed methods evaluation. N Dir Eval. 2013;138:15–26.
Whitehurst GJ. The value of experiments in education. Education. 2012;7(2):107–23.
Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review-a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10(1_suppl):21–34.
Blamey A, Mackenzie M. Theories of change and realistic evaluation peas in a pod or apples and oranges? Evaluation. 2007;13(4):439–55.
Rotem A, Zinovieff M, Goubarev A. A framework for evaluating the impact of the United Nations fellowship programmes. Hum Resour Health. 2010;8:1–8.
No funding was received.
Ethics approval and consent to participate
This is a systematic review of studies, and therefore, neither ethical approval nor participants’ consent were required.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.