Evaluating an evidence-based curriculum in undergraduate palliative care education: piloting a phase II exploratory trial for a complex intervention
© Schulz et al.; licensee BioMed Central Ltd. 2013
Received: 8 February 2012
Accepted: 11 December 2012
Published: 4 January 2013
By 2013 Palliative Care will become a mandatory examination subject in the medical curriculum in Germany. There is a pressing need for effective and well-designed curricula and assessment methods. Debates are on going as how Undergraduate Palliative Care Education (UPCE) should be taught and how knowledge and skills should be assessed. It is evident by this time that the development process of early curricula in the US and UK has led to a plethora of diverse curricula which seem to be partly ineffective in improving the care for the seriously ill and dying offered by newly qualified doctors, as is demonstrated in controlled evaluations. The goals of this study were to demonstrate an evidence-based approach towards developing UPCE curricula and investigate the change in medical students’ self-perceived readiness to deal with palliative care patients and their families.
To evaluate the effects of the UPCE curriculum we chose a prospective, controlled, quasi-experimental, pre, retrospective-pre, post study design. A total of n = 37 3rd and 4th –year medical students were assigned to the intervention group (n = 15; 4th -year) and to the control group (n = 22; 3rd-year). Resting on the self-efficacy concept of Bandura the measurement was conducted by a refined test-battery based on two independent measurements (the revised Collet-Lester-Fear-of-Death-Scale and the instrument of the “Program in Palliative Care Education and Practice” at Harvard Medical School) including 68 items altogether in a five-point Likert-scale. These items were designed to test elementary skills in caring for the dying and their relatives as perceived by medical undergraduates. Datasets from both groups were analysed by paired and independent two-sample t-test. The TREND statement for reporting non-randomized evaluations was applied for reporting on this quasi-experimental study.
Three constructs showed statistically significant differences comparing the intervention group before and after. Willingness to accompany a dying patient increased from 21.40 to 37.30 (p < .001). Self-estimation of competence in communication with dying patients and their relatives increased from 12.00 to 23.60 (p = .001). Finally, self-estimation of knowledge and skills in Palliative Care increased from 8.30 to 13.20 (p = .001).
This study is a small but systematic step towards rigorous curricular development in palliative care. Our manualised curriculum is available for scrutiny and scientific feedback to support an open and constructive process of best-practice comparison in palliative care.
The need for Palliative Care (PC) was acknowledged by the European Council at the beginning of the century . The task had been set up to establish PC systematically in the European Union. Later, the European Parliament, in order to build-up PC in healthcare-systems, emphasized the specific training needs of professionals and the demand for systematic training programmes . In 2009, supported by a broad civil movement, the German legislation passed a law that introduced an obligation to teach undergraduate palliative care education (UPCE) within the medical curriculum. German universities were obliged to develop and implement curricula by 2013. A German survey conducted at the same time showed that by then only six out of 36 medical schools offered mandatory UPCE, but most faculties offered some kind of teaching in PC . Although there are recommendations available from the European Association of Palliative Care and national organisations for curriculum planning, debates are ongoing as to how UPCE should be taught and how knowledge and skills should be assessed . In comparison the movement to improve UPCE in medical schools in the US and UK started earlier in the late 1980s resulting in the implementation of UPCE, e.g. in the UK in 1993 [5, 6]. It is evident by this time that the development process of those early curricula has led to a plethora of diverse curricula which, most importantly, seem to be partly ineffective in improving the care for the seriously ill and dying offered by newly qualified doctors, as demonstrated in controlled evaluations [7–9]. In 2004 a systematic review found a lack of consistency in analysed UPCE curricula focussing on knowledge and skills teaching rather than on attitude. Furthermore they recognised rare formal assessment . This has been confirmed by a more recent systematic review of the US situation . We do see parallels between the implementation process of UPCE in the US/UK and today’s situation in Germany where recent findings demonstrate that final year medical students show only limited confidence and knowledge towards palliative care . Therefore we hypothesise that the efforts taken in developing UPCE curricula in Germany need to be strongly supported by medical education expertise and evidence-based approaches.
This report is part of a series of systematic investigations demonstrating an evidence-based approach towards developing UPCE curricula . The goal of this study was to investigate the change in medical students’ self-perceived readiness to deal with palliative care patients and their families.
T2 (response rate)
Eligibility criteria were: (1) medical undergraduates at Witten/Herdecke, (2) students have to be in their 3rd or 4th year of study and (3) students in the intervention group (IG) must agree to complete the whole intervention. Two incentive study credits were rewarded to participants in the IG. No incentives were offered to the control group (CG) whose participants were confronted with considerably less workload during participation. In general, the new UPCE curriculum was accessible for all students, including those who did not consent to participate in the evaluation study. To assure psychological support during potential stressful confrontations with palliative patients one psychologist trained in psycho-oncology was introduced prior to the commencement of the intervention and offered immediate help to the participants by phone during the whole study.
(1) Attitude towards palliative care
(2) Willingness to accompany dying patients
(3) Self- estimation of competence in communication with dying patients and their relatives; (4) self-estimation of knowledge and skills in PC
(5.1) (5.2) Attitude towards death and dying of self
(5.3) (5.4) Attitude towards death and dying of others
Unfortunately, the maximum study size was not modifiable due to the naturalistic design of the study, which took place in a medical curriculum with pre-existing student cohorts.
Randomisation or matching was not possible due to the naturalistic design of the study. 4th -year medical students were assigned to the IG to graduate the new UPCE curriculum, while 3rd -year students were assigned to the CG only, passing the standard medical curriculum. We could not exclude selection bias because of the missing randomisation. One can hypothesize that participants with a higher interest in PC were more likely to participate than unmotivated students (in both groups).
There was no possibility of blinding in this setting.
Unit of analysis
Analyses were performed at the individual level of each student participant.
The data from the new instrument was collected anonymously via a password-secured web portal from the IG and CG each at T1 and T2/T3 (http://www.palliative-research.de). Datasets from both groups at T1, T2 and T3 were processed with SPSS® 19.0 and analysed by dependent t-test for paired samples (e.g. T1 IG vs. T2 IG) and independent two-sample t-test (e.g. T1 IG vs. T1 CG). Levene’s test was applied to prove equality of variances. P-value for two-tailed statistical significance was predefined to p < .05. Only retrospective-pre data of previously significant constructs in pre-post comparison was tested.
A total of 87 students met the inclusion criteria and were invited to participate in the study. In T1 n = 37 students completed the questionnaire (IG n = 15; CG n = 22). In T2 n = 26 students remained (IG n = 10; CG n = 16). Total response rate in T1 was 0.43 (IG 0.35; CG 0.50) compared to the total response rate in T2 of 0.30 (IG 0.23; CG 0.36). We had n = 11 dropouts representing a dropout rate of 0.30 in total (IC 0.33; CG 0.27). We did not find a difference in compliance between IG and CG (Fisher’s exact test .73). Six dropouts resulted from lost contact and five students had to abort the curriculum due to electives in foreign countries. An overview of participant flow can be seen in Table 1.
Recruitment started in September 2006 and ended in October with T1 and the beginning of the UPCE curriculum. Follow-up in T2 was performed in February 2007 at the end of the intervention.
Contact with dying patients before trial participation
IG (n = 15)
CG (n = 22)
total (n = 37)
(n = 14)
(n = 17)
(n = 31)
Baseline intergroup-comparison at T1
T1 M ± SD
(2) Willingness to accompany dying patients
24.7 ± 7.17
23.91 ± 7.17
(3) competence in communication with dying patients and their relatives
14.53 ± 5.4
13.59 ± 4.83
(4) self-estimation of knowledge and skills in PC
8.27 ± 1.87
7.55 ± 1.99
(5.1) Attitude towards death of self
22.2 ± 8.71
(5.2) Attitude towards dying of self
12.33 ± 2.41
15.23 ± 5.07
(5.3) Attitude towards death of others
15.13 ± 3.46
17.95 ± 4.04
(5.4) Attitude towards dying of others
18.6 ± 5.36
19.0 ± 4.43
In T1 data was obtained from n = 37 participants (IG n = 15; CG n = 22). In T2 the number of data sets was n = 26 (IG n = 10; CG n = 16). We had to note n = 11 dropouts, which were excluded from analysis.
Outcomes and estimation
Intragroup-comparison between T1 and T2
T1 M ± SD
T2 M ± SD
21.4 ± 6.82
37.3 ± 6.65
22.56 ± 7.05
24.19 ± 7.53
12.0 ± 4.32
23.6 ± 5.58
14 ± 5.02
13.69 ± 6.12
8.3 ± 2.11
13.2 ± 2.7
7.88 ± 2.19
8.75 ± 2.89
18.8 ± 5.43
20.0 ± 4.19
23.88 ± 6.52
23.75 ± 6.28
11.3 ± 2.0
14 ± 4.32
14.94 ± 5.18
15.63 ± 6.12
13.6 ± 2.95
15.9 ± 3.96
18.06 ± 4.45
17.88 ± 5.11
17.7 ± 6.07
21.8 ± 7.01
19.25 ± 4.82
20.25 ± 5.64
We measured changes in emotional involvement of the participants while attending the seminar “Communication with the dying patient” which fosters death-awareness. Therefore the IG was assessed by a modified version of Izard’s Differential Emotions Scale (mDES) pre and post . The data is provided in Additional file 4.
There was no need for psychological intervention as reported by the accompanying psychologist. The participants did not demand for psychological help throughout the study.
The presented pilot study is the first one to measure and report effects of a manualised and systematically implemented UPCE curriculum in Germany. Systematic implementation and detailed evaluation are most important for high quality medical education in palliative care. The quasi-experimental design of this study does not allow proofing for causal relations. However, our statistical results can inform further research by demonstrating associations, e.g. between an evidence-based curriculum in UPCE adopting modern teaching methods and a rise in self-efficacy in elementary skills in caring for the dying and their relatives as perceived by medical undergraduates. The UPCE curriculum seems to have the potential to foster self-efficacy in the following domains: willingness to accompany a dying patient, competence in communication with dying patients and their relatives, and knowledge and skills in PC. The self-efficacy concept was chosen for several reasons: firstly, this concept allows testing for a central paradigm in palliative care which values the importance of patient-physician-relationship depending on personal qualities of engagement, self-confidence, self-reflection and patient-centred perspective. Secondly, this concept has already been used in various settings investigating the effects of UPCE [24–31]. And thirdly, from a research ethics point of view this model was economical to use and its measurement easy to apply.
In our ancillary analyses we found hints that palliative care education may have an emotional impact on medical students. Furthermore, these results indicate that those emotional states, which block effective learning, particularly shame, decreased during a seminar about “Communication with the dying patient”.
Unfortunately, some sources of bias could not be avoided in this approach. Randomisation was not possible because of the small study-population and therefore we have to expect self-selection bias. The use of statistical tests on a small sample implies it’s own risks to overestimate the results as well. In this context we can only speak of “association”, not “effect”. Furthermore, self-selection could have brought more motivated students into the IG, leading to over-estimation of the effect-size in the results. Such critical appraisal aspects need to be taken in consideration when applying the study design to a larger sample of medical students.
This UPCE curriculum consists of 31 teaching units with different didactic methods and approaches. It therefore forms a complex intervention. According to the Medical Research Council framework for complex interventions, research at a high level of complexity should follow a step-wise approach . To gain a systematic insight into this UPCE curriculum we chose an exploratory phase (II) trial design following the MRC recommendations. Several limitations applied to this trial. The study used the implementing of a new curriculum as an intervention (quasi-experimental) and therefore we had to accept the low target population and the resulting low number of participants, which was limited by the given number of eligible students. Calculation of statistical power was not possible in this scenario, reducing the generalisation of the results. We offered incentives only in the IG but surprisingly participation was higher in the CG (.35% vs. 50%), which hints at the perceived workload that medical students might have associated with undergoing this curriculum. This is further strengthened by the high dropout-rate in the intervention group. This highlights the question of mandatory undergraduate training in palliative care, an ongoing debate in Germany . We did measure some baseline differences in one construct concerning fear of death but this did not have any recognizable effect on the interpretation of the results in the other constructs.
Complex interventions such as the UPCE curriculum pose particular problems for evaluation trials and this has led to a debate as to which method should be used. On the one hand, randomised controlled trials are widely accepted as the most reliable (“gold standard”) method of determining effectiveness [39, 40]. On the other hand, the role of observational methods  and the value of qualitative research  in evaluation of palliative care services has been underlined by others. Consequently, methodological frameworks have been developed to integrate different research paradigms into a sound research strategy . Being conscious of the lacking possibility of generalisation from a phase (II) study, we do recommend conducting phase (III) or (IV) trials with higher study populations and a more robust design to provide more precise study results, and to encourage research in this field. However, we believe we are giving an example of a step-wise and systematic approach towards palliative care education development and evaluation. In a period of time where constituting PC in medical faculties in Germany is still ongoing, we want to emphasize the importance of a systematic approach towards curriculum development, implementation and evaluation following established principles from medical education expertise. In Germany, promising initiatives have begun to form, which are intended to connect and interlink curricular developers and leading experts in palliative care education .
By 2013 Palliative Care will become a mandatory examination subject in the medical curriculum in Germany. There is a pressing need for effective and well-designed curricula and assessment methods. It would be wise to learn from international experience and take established didactic evidence into account. Development, implementation and evaluation of evidence-based curricula should be fostered by drawing expertise from medical education specialists and intense interfaculty collaboration. This study is a small but systematic step towards rigorous curricular development in palliative care. By making our manualised curriculum readily available for scrutiny and scientific feedback we want to support an open and constructive process of best-practice comparison and high-quality education in palliative care.
Palliative care education and practice
Undergraduate palliative care education.
We thank all participating patients for their openness, while approaching the end of their lives. Further, we want to thank all staff involved in conducting the curriculum and the support of Dr. Oliver Schmalz, HELIOS-Klinikum (Wuppertal, Germany).
- Assembly P: Protection of the human rights and dignity of the terminally ill and the dying. Recommendation 1418 (1999). 1999, European Union: Official Gazette of the Council of Europe
- Martin-Moreno J, Harris M, Gorgojo L, Clark D, Normand C, Centeno C: Palliative Care in the European Union. 2008, European Parliament: Policy Department EaSP
- Laske A, Dietz I, Ilse B, Nauck F, Elsner F: Palliativmedizinische Lehre in Deutschland: Bestandsaufnahme an den medizinischen Fakultäten 2009. Z Palliativmed. 2010, 11: 18-25. 10.1055/s-0029-1223482.View Article
- Elsner F, Fittkau-Tönnesmann B, Schiessel C: Curriculum: Grundlagen der Palliativmedizin. 2009, Berlin: Deutsche Gesellschaft für Palliativmedizin
- Field D, Wee B: Preparation for palliative care: teaching about death, dying and bereavement in UK medical schools 2000–2001. Med Educ. 2002, 36 (6): 561-567. 10.1046/j.1365-2923.2002.01232.x.View Article
- Council GM: Tomorrow's Doctors: Recommendations on Undergraduate Medical Education. 1993, London: GMC
- The SUPPORT Principal Investigators: A controlled trial to improve care for seriously ill hospitalized patients. The study to understand prognoses and preferences for outcomes and risks of treatments (SUPPORT). JAMA. 1995, 274 (20): 1591-1598. 10.1001/jama.1995.03530200027032.View Article
- Oddi LF, Cassidy VR: The message of SUPPORT: study to understand prognosis andpreferences for outcomes and risks of treatment. Change is long overdue. J Prof Nurs. 1998, 14 (3): 165-174. 10.1016/S8755-7223(98)80092-6.View Article
- Gibbins J, McCoubrie RaF K: Why are newly qualified doctors unprepared to care for patients at the end of life?. Medical Education. 2011, 45: 389-399. 10.1111/j.1365-2923.2010.03873.x.View Article
- Lloyd-Williams M, MacLeod RD: A systematic review of teaching and learning in palliative care within the medical undergraduate curriculum. Med Teach. 2004, 26 (8): 683-690. 10.1080/01421590400019575.View Article
- Bickel-Swenson D: End-of-life training in U.S. medical schools: a systematic literature review. J Palliat Med. 2007, 10 (1): 229-235. 10.1089/jpm.2006.0102.R1.View Article
- Weber M, Schmiedel S, Nauck F, Alt-Epping B: Knowledge and attitude of final - year medical students in Germany towards palliative care - an interinstitutional questionnaire-based study. BMC Palliat Care. 2011, 10: 19-10.1186/1472-684X-10-19.View Article
- Just JM, Schulz C, Bongartz M, Schnell MW: Palliative care for the elderly-developing a curriculum for nursing and medical students. BMC Geriatr. 2010, 10: 66-10.1186/1471-2318-10-66.View Article
- Palliative Research: Palliative Research. http://www.palliative-research.de.
- Kern DE, Thomas PA, Howard DA: Curriculum Development for MedicalEducation - A Six-Step Approach. 1998, Baltimore, Maryland: John Hopkins University Press, http://www.amazon.com/Curriculum-Development-Medical-Education-Six-Step/dp/0801893674.
- Moher D, Liberati A, Tetzlaff J, Altman DG: Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009, 339: b2535-10.1136/bmj.b2535.View Article
- Just JM, Schnell MW, Bongartz M, Schulz C: Exploring effects of interprofessional education on undergraduate students behaviour: a randomized controlled trial. Journal of Research in Interprofessional Practice and Education. 2010, 1 (3): 182-199.
- Schulz C, Möller MF, Schmincke-Blau I, Schnell MW: Communication with the dying patient – Results of a controlled intervention study on communication skills in undergraduates. European Journal of Palliative Care. Volume 11. Edited by: Nauck F. 2009, Vienna: Hayward Medical Communications, 152.
- Des Jarlais DC, Lyles C, Crepaz N: Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: the TREND statement. Am J Public Health. 2004, 94 (3): 361-366. 10.2105/AJPH.94.3.361.View Article
- Pratt CC, McGuigan WM, Katzev AR: Measuring program outcomes: using retrospective pretest methodology. Am J Eval. 2000, 21 (3): 341-349.View Article
- Sullivan AM, Lakoma MD, Billings JA, Peters AS, Block SD: Teaching and learning end-of-life care: evaluation of a faculty development program in palliative care. Acad Med. 2005, 80 (7): 657-668. 10.1097/00001888-200507000-00008.View Article
- Bandura A: Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev. 1977, 84 (2): 191-215.View Article
- Adriaansen MJ, van Achterberg T: A test instrument for palliative care. Int J Nurs Stud. 2004, 41 (1): 107-117. 10.1016/S0020-7489(03)00073-7.View Article
- Fineberg IC, Wenger NS, Forrow L: Interdisciplinary education: evaluation of a palliative care training intervention for pre-professionals. Acad Med. 2004, 79 (8): 769-776. 10.1097/00001888-200408000-00012.View Article
- Fraser HC, Kutner JS, Pfeifer MP: Senior medical students' perceptions of the adequacy of education on end-of-life issues. J Palliat Med. 2001, 4 (3): 337-343. 10.1089/109662101753123959.View Article
- Kaye JM, Loscalzo G: Learning to care for dying patients: a controlled longitudinal study of a death education course. J Cancer Educ. 1998, 13 (1): 52-57.
- Mason S, Ellershaw J: Assessing undergraduate palliative care education: validity and reliability of two scales examining perceived efficacy and outcome expectancies in palliative care. Med Educ. 2004, 38 (10): 1103-1110. 10.1111/j.1365-2929.2004.01960.x.View Article
- Ross DD, Shpritz D, Hull MM, Goloubeva O: Long-term evaluation of required coursework in palliative and end-of-life care for medical students. J Palliat Med. 2005, 8 (5): 962-974. 10.1089/jpm.2005.8.962.View Article
- Schwartz CE, Clive DM, Mazor KM, Ma Y, Reed G, Clay M: Detecting attitudinal changes about death and dying as a result of end-of-life care curricula for medical undergraduates. J Palliat Med. 2005, 8 (5): 975-986. 10.1089/jpm.2005.8.975.View Article
- Sullivan AM, Lakoma MD, Block SD: The status of medical education in end-of-life care: a national report. J Gen Intern Med. 2003, 18 (9): 685-695. 10.1046/j.1525-1497.2003.21215.x.View Article
- Lester D: The factorial structure of the revised Collett-Lester fear of death scale. Death Stud. 2004, 28 (8): 795-798. 10.1080/07481180490483472.View Article
- Hegedus K, Zana A, Szabo G: Effect of end of life education on medical students' and health care workers' death attitude. Palliat Med. 2008, 22 (3): 264-269. 10.1177/0269216307086520.View Article
- Durlak JA, Reisenberg LA: The impact of death education. Death Stud. 1991, 15 (1): 39-58. 10.1080/07481189108252408.View Article
- Neimeyer RA: Death Anxiety Handbook: Research, Instrumentation, and Application. 1994, London: Taylor & Francis
- Hasson F, Keeney S, McKenna H: Research guidelines for the Delphi survey technique. J Adv Nurs. 2000, 32 (4): 1008-1015.
- Boyle GJ: Reliability and validity of Izard's differential emotions scale. Pers Indiv Differ. 1984, 5 (6): 747-750. 10.1016/0191-8869(84)90124-7.View Article
- Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, Tyrer P: Framework for design and evaluation of complex interventions to improve health. BMJ. 2000, 321 (7262): 694-696. 10.1136/bmj.321.7262.694.View Article
- Weber M, Schmiedel S, Nauck F, Alt-Epping B: Knowledge and attitude of final - year medical students in Germany towards palliative care - an interinstitutional questionnaire-based study. BMC Palliat Care. 2011, 10: 19-10.1186/1472-684X-10-19. http://www.biomedcentral.com/1472-684X/10/19.View Article
- Stephenson J, Imrie J: Why do we need randomised controlled trials to assess behavioural interventions?. BMJ. 1998, 316 (7131): 611-613. 10.1136/bmj.316.7131.611.View Article
- Simon S, Higginson IJ: Evaluation of hospital palliative care teams: strengths and weaknesses of the before-after study design and strategies to improve it. Palliat Med. 2009, 23 (1): 23-28.View Article
- Black N: Why we need observational studies to evaluate the effectiveness of health care. BMJ. 1996, 312 (7040): 1215-1218. 10.1136/bmj.312.7040.1215.View Article
- Flemming K, Adamson J, Atkin K: Improving the effectiveness of interventions in palliative care: the potential role of qualitative research in enhancing evidence from randomized controlled trials. Palliat Med. 2008, 22 (2): 123-131. 10.1177/0269216307087319.View Article
- QB 13 - Palliativmedizin als Pflichtfach im Medizinstudium - Rückblick auf den 2. Dozentenworkshop. http://www.cio-koeln-bonn.de/mediziner/neuigkeiten-aktuelles/news-details/detail/qb-13-palliativmedizin-als-pflichtfach-im-medizinstudium-rueckblick-auf-den-2-dozentenworkshop/?tx_ttnews[backPid]=147&cHash=aa76a4a2f9B.
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6920/13/1/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.