- Research article
- Open Access
- Open Peer Review
Incorporating prognosis in the care of older adults with multimorbidity: description and evaluation of a novel curriculum
© Schoenborn et al. 2015
- Received: 21 July 2015
- Accepted: 18 November 2015
- Published: 1 December 2015
Prognosis is a critical consideration in caring for older adults with multiple chronic conditions, or “multimorbidity”. Clinicians are not adequately trained in this area. We describe an innovative curriculum that teaches internal medicine residents how to incorporate prognosis in the care of older adults with multimorbidity.
The curriculum includes three small-group sessions and a clinical exercise; it focuses on the assessment, communication, and application of prognosis to inform clinical decisions. The curriculum was implemented with 20 first-year residents at one university-based residency (intervention group). Fifty-two first-year residents from a separate residency affiliated with the same university served as controls.
Evaluation included three components. A survey assessed acceptability. A pre/post survey assessed attitude, knowledge, and self-reported skills (Impact survey). Comparison of baseline and follow-up results used paired t-test and McNemar test; comparison of inter-group differences used t-test and Fisher’s exact test. A retrospective, blinded pre/post chart review assessed documentation behavior; abstracted outcomes were analyzed using Fisher’s exact test.
The curriculum was highly rated (4.5 on 5-point scale). Eighteen intervention group residents (90 %) and 29 control group residents (56 %) responded to the Impact survey. At baseline, there were no significant inter-group differences in any of the responses. The intervention group improved significantly in prognosis communication skills (5.2 to 6.6 on 9-point scale, p < 0.001), usage of evidence-based prognostic tools (1/18 to 14/18 responses, p < 0.001), and prognostic accuracy (1/18 to 9/18 responses, p = 0.005). These responses were significantly different from the control group at follow-up.
Of 71 charts reviewed in each group, prognosis documentation in the intervention group increased from 1/25 charts (4 %) at baseline to 8/46 charts (17 %) at follow-up (p = 0.15). No prognosis documentation was identified in the control group at either time point. Inter-group difference was significant at follow-up (p = 0.006).
We developed and implemented a novel prognosis curriculum that had significant short-term impact on the residents’ knowledge and communication skills as compared to a control group. This innovative curriculum addresses an important educational gap in incorporating prognosis in the care of older adults with multimorbidity.
- Older adults
- Resident education
- Primary care
Older adults often have multiple chronic conditions, or multimorbidity, which is associated with increased morbidity and mortality [1–3]. In this heterogeneous population, prognostication that takes into account comorbidities and functional status helps differentiate older adults in relatively good health and those with more limited prognosis. Research and clinical practice recommendations increasingly advocate for incorporating prognosis to individualize care decisions that have long lag-time to benefit but potential short-term harms, such as cancer screening and diabetes glycemic control, in the care of older adults [1, 4–13].
Prognosis is not often incorporated in clinical decisions; [14–19] a major contributor may be inadequate clinician training . Existing curricula on prognosis are limited to palliative care or oncology; [21–26] we have not found any described curriculum that teaches assessing, discussing, and incorporating prognosis to frame clinical decisions in the care of older adults with multimorbidity.
Internal medicine residents are a key learner group because residency is a formative time and many older adults receive care from clinicians trained in internal medicine . Assessing and incorporating prognosis to inform care are part of key competencies for internal medicine residents . We describe the development and evaluation of a novel curriculum that teaches internal medicine residents how to incorporate prognosis to inform clinical decisions in primary care of older adults with multimorbidity.
Learner objectives and corresponding evaluation strategies
- Post-only acceptability survey of the Intervention group
- Rate the curriculum as acceptable and relevant.
- Pre/post survey (Impact Survey) of Intervention and Control groups
- Rate incorporating prognosis to inform clinical decisions as important in the care of older adults with multimorbidity.
- Pre/post survey (Impact Survey) of Intervention and Control groups
- Demonstrate assessment of prognosis using evidence-based tools.
- Demonstrate application of prognosis to inform clinical decisions.
- Be able to communicate prognosis a) as incorporated into the benefits/harms discussion related to a clinical decision, and b) explicitly in a discussion
- Pre/post chart review of Intervention and Control groups for prognosis documentation
- Routinely incorporate prognosis to inform clinical decisions in the care of older adults with multimorbidity.
There are two internal medicine residency programs at our university; neither program had any pre-existing curricula on prognosis. The prognosis curriculum was first implemented in January 2014 in one residency program with 20 first-year residents (intervention group) while the other program’s 52 first-year residents served as controls. The curriculum was implemented in an existing, required rotation that consisted of two 2-week outpatient blocks.
Evaluation was informed by the Miller’s pyramid of clinical assessment;  it consisted of three strategies: a survey to assess acceptability, a pre/post survey to assess impact on attitude, knowledge, and self-reported skills, and a pre/post chart review to assess impact on documentation behavior (Table 1). This study was approved by an institutional review board.
Evaluation of acceptability
A survey on curriculum acceptability was administered immediately after the curriculum to the intervention group, asking the residents to rate on a 5-point Likert scale (1 = poor, 5 = excellent) the curriculum’s quality and relevance. The results are presented using descriptive statistics.
Evaluation of attitude, knowledge, skills
A second survey (Impact Survey) was administered at baseline and at follow-up after the curriculum to both groups to assess the curriculum’s impact on attitude, knowledge, and self-reported skills. We did not find any existing validated assessment tools and developed our own instrument; we piloted it for clarity, face and content validity with six geriatric medicine fellows at our institution. We used the same questions at baseline and at follow-up.
Questions on attitude and self-reported skills used 9-point Likert scales (1 = “not at all important”/ “not at all prepared”, 9 = “extremely important”/ “extremely prepared”). Questions on knowledge used two hypothetical patient cases involving older adults with multimorbidity and functional dependence. The respondents were asked to estimate the patients’ 4- or 5-year mortality risk (in percentages) based on the provided clinical information; this was then compared to validated prognostic tools to determine prognostic accuracy [32, 33]. The respondents were also asked to report how they derived the estimate (using a prognostic tool or citing literature versus using clinical experience alone) and apply the estimated prognosis to answer multiple-choice questions about clinical decisions such as screening colonoscopy and glycemic control in older adults with diabetes mellitus.
Comparisons of baseline and follow-up results used paired t-test and McNemar test. Comparisons between intervention and control groups used t-test and Fisher’s exact test. Statistical analyses were performed using STATA 12.0 (StataCorp LP, College Station, TX) and Microsoft Excel 2010 (Microsoft, Redmond, WA).
Evaluation of behavior
To assess the curriculum’s impact on clinical practice, we conducted a retrospective, controlled, blinded chart review at baseline before the curriculum (September-December 2013) and at follow-up after the curriculum (late February-June 2014). A “chart” is defined as the documentation of a single clinic visit. Charts were eligible if the visit was between a resident in either study group and the resident’s primary care patient who was at least 60 year old and had at least two of 30 conditions in the Elixhauser comorbidity index .
In the baseline period, we randomly selected 25 out of 43 eligible charts in the intervention group and 25 out of 128 eligible charts in the control group for analysis. In the follow-up period, we included all 46 eligible charts in the intervention group and randomly selected 46 out of 203 eligible charts in the control group for analysis. A chart abstraction guide assessed for documentation of overall prognosis, which was defined to include: “life expectancy”, “prognosis”, “mortality”, “mortality rate”, “likelihood to live/likelihood to die”, “survival”, “life span”, and “long-term outcome”. If overall prognosis documentation was present, we examined if the prognosis estimate was qualitative (e.g. “good”, “poor”) or quantitative (e.g. “10-year mortality risk is 93 %”), whether any literature or evidence-based tool was cited in deriving the prognosis, whether the prognosis was communicated, and whether it affected a clinical decision. We also examined whether overall prognosis was documented when there was documentation of cancer screening because clinical practice recommendations increasingly advocate for incorporating life expectancy in cancer screening [10–13].
Patient, resident, clinic identifiers and date were removed from charts. Two investigators (NS, KN), blinded to the timing of the visit and the resident’s group designation, reviewed the de-identified charts. We piloted the abstraction guide with 12 charts for clarity, face and content validity, and inter-rater reliability. We revised the abstraction guide with input from three investigators (NS, KN, CB), and used the revised guide to review the remaining charts. Inter-rater agreement was 100 % regarding whether prognosis was documented, whether the estimate was qualitative or quantitative, whether prognostic tool was cited, and whether prognosis was communicated. Inter-rater agreement was 89.9 % regarding whether prognosis affected a clinical decision. Abstracted outcomes were analyzed using Fisher’s exact test. Patient characteristics including demographic information, number of chronic conditions, and number of medications were also abstracted and compared between the intervention and control groups using t-test and Fisher’s exact test. All analyses were conducted using STATA 12.0 (StataCorp LP, College Station, TX) and Microsoft Excel 2010 (Microsoft, Redmond, WA).
The intervention group included 20 first-year residents (12 women, eight men). The control group included 52 first-year residents (24 women, 28 men).
Evaluation of acceptability
All 20 intervention group residents completed the survey on curriculum acceptability. On a 5-point Likert scale (1 = poor, 5 = excellent), the residents rated the curriculum highly (mean 4.5, SD 0.5) and found it highly relevant (mean 4.5, SD 0.7).
Evaluation of attitude, knowledge, skills
There were no significant differences between the two groups’ responses to any of the survey questions at baseline. The intervention group respondents rated incorporating prognosis as highly important at baseline and at follow-up (7.6 to 7.9 on a 9-point scale, p = 0.15). Two aspects of prognosis communication were assessed; self-reported skill levels in both aspects increased significantly from 5.3 to 6.9 and 5.2 to 6.6 on a 9-point scale (both p < 0.001); these responses were significantly different from the control group at follow-up.
In knowledge assessment, the intervention group respondents improved in the accuracy of prognosis estimates (i.e. same percentage as predicted by validated tools) in both hypothetical patient cases, reaching statistical significance in one of the two cases (1/18 to 9/18 correct responses, p < 0.001). The intervention group respondents also significantly increased in usage of evidence-based prognostication methods in both cases (1/18 to 14/18 and 2/18 to 14/18, both p < 0.001); these were significantly different from the control group at follow-up. In both cases, the intervention group respondents improved in correctly answering clinical decision questions that involved applying prognosis information, reaching statistical significance in one of two cases (9/18 to 17/18, p = 0.005) but inter-group difference was not significant at follow-up (p = 0.07).
Evaluation of behavior
Chart review results-- patient characteristics and prognosis documentation in the intervention and control groups
Number of patients involved in reviewed charts
N = 49
N = 63
Patient age, mean (SD), year
Female patient, number (%)
23 (47 %)
42 (67 %)
Patient race, number (%)
31 (63 %)
2 (3 %)
- African American
12 (24 %)
56 (89 %)
6 (12 %)
5 (8 %)
Number of chronic conditions a, mean (SD)
Number of medications, mean (SD)
Number of charts reviewed
N = 71 (baseline 25, follow-up 46)
N = 71 (baseline 25, follow-up 46)
Overall prognosis documentation among charts that documented cancer screening
In the intervention group, overall prognosis was documented in 1/25 charts (4 %) at baseline and 8/46 charts (17 %) at follow-up (p = 0.15). Five charts included quantitative estimates of prognosis; two cited a prognostic tool. Two charts documented prognosis communication: one as part of a goals of care discussion, another discussed how patient’s life expectancy might vary according to smoking status as part of smoking cessation counseling. Four charts documented prognosis affecting a clinical decision (e.g. “recommendation is 7.5–8 % goal hemoglobin A1C for older adults with life expectancy <10 years”). We found no documentation of overall prognosis in the control group at either time period. The difference between the two groups at follow-up (8/46 vs. 0/46) was significant (p = 0.006). We found cancer screening documentation in 32 charts in the intervention group (15 baseline, 17 follow-up) and 39 charts in the control group (13 baseline, 26 follow-up). Only three charts in the intervention group at follow-up documented both cancer screening and overall prognosis.
We describe the development and evaluation of an innovative curriculum that teaches internal medicine residents how to assess, communicate and apply prognosis to inform clinical decisions in the care of older adults with multimorbidity. This addresses a significant gap in the literature as we find no previously described curriculum that teaches about incorporating prognosis in the care of this patient population. While existing curricula teach communication or assessment of prognosis in oncology or palliative care [21–26], none addresses the unique challenges of incorporating prognosis in the care of older adults with multimorbidity. Without a predominant terminal illness such as cancer or being at the very end of life, older adults with multimorbidity may be less likely to consider prognosis relevant . On the other hand, a growing literature suggests that life expectancy in the range of a few years impacts a number of clinical decisions [1, 4–9]. Our curriculum is innovative not only in the patient population it targets but also in teaching about a longer time frame of prognostication.
Although our curriculum uses many of the same strategies, such as case discussions, reflection, and role play, as other curricula that teach prognosis communication [21, 24–26], it is innovative in combining role play with subsequent guided experiential learning with actual patients in clinical practice. Experiential learning theory suggests that knowledge and skill gained via this form of learning are more substantive and long-lasting .
We found significant improvement in the intervention group’s knowledge and ability to use evidence-based tools to estimate prognosis. Although there is no study directly comparing the predictive accuracy of validated prognostic tools with clinical intuition, there are suggestions that clinicians’ intuition may be inaccurate and these tools may be helpful in clinical practice [20, 36–38].
We found an increase in documentation of overall prognosis. This did not reach statistical significance although Kolb’ model of experiential learning that was used in the curriculum was suggested to support long-lasting learning. Several factors may contribute to why rates of prognosis documentation only increased moderately in the intervention group. First, documentation may not accurately capture what occurred during a visit . The resident may have considered the patient’s prognosis in clinical decisions without documenting it. Second, prognosis may not be discussed or documented in every visit; it may have been documented in a previous visit and be missed in the chart review. Without a gold standard to assess all the specific clinical scenarios in which prognosis should inform clinical decisions, we could not assess if prognosis should have been incorporated and documented in all of the visits included in chart review; prognosis may not have been relevant to the clinical decisions during a particular visit. Third, as our institution’s electronic medical record moves towards more information-sharing with patients, the residents may not have felt comfortable documenting prognosis without first discussing prognosis with the patient; and the residents may not have discussed prognosis with patients due to time constraints or that the patient was not ready to discuss prognosis . Literature describes numerous barriers in effecting behavior change in clinicians . In addition to this educational intervention, a multi-faceted approach that also targets the clinical practice environment and patients may be needed to achieve more significant changes.
This project has several limitations. As a novel intervention, this curriculum was implemented at one institution; the results may not reflect residents or patients elsewhere. Evaluation focused on short-term outcomes and we cannot draw conclusions about long-term impact of the curriculum. Additional evaluation in larger and different populations and of long-term outcomes are needed after curriculum dissemination. Second, although the two residency programs involved in the study are both affiliated with the same academic institution, the residents may not be completely comparable in relevant characteristics such as baseline geriatric training. However, both programs are highly selective academic residency programs and both groups of residents are first year trainees. There were no significant differences in the groups’ survey responses or chart review results at baseline. Third, there are no existing validated instruments to assess knowledge or behavior in incorporating prognosis. We developed our own survey instrument and chart review abstraction guide and tested both for face and content validity. Inter-rater agreement in chart review was high. However, further validity, reliability and psychometric testing of the instruments are needed. The multiple-choice question format used in part of knowledge assessment may have been susceptible to repeated testing bias and random error. Fourth, self-reported skill levels and documentation may not accurately reflect actual skill and behavior, respectively. We plan in a future study to evaluate residents’ communication skills through audio-recording the clinic visits, triangulate the information with resident survey and chart review to better capture practice patterns. Lastly, in the chart review, the patient populations served by the two study groups are different in gender and race. Limited by the low frequency of positive outcomes, we were not able to stratify the results by gender and race or adequately adjust for clustering of residents and patients, which may affect the accuracy of results.
Older adults with multimorbidity are a heterogeneous population in which applying prognosis to inform clinical decisions is critical. Clinicians currently lack adequate training in this area. We describe a novel curriculum with significant short-term impact on prognosis knowledge and communication skills. This curriculum addresses an important educational gap in the care of older adults with multimorbidity.
The authors wish to thank Drs. Belinda Chen, David Kern, and Henry Taylor for their assistance in the curriculum development process.
This project was made possible with a grant from the Arnold P. Gold Foundation (The Picker Institute / Gold Foundation Graduate Medical Education Challenge Grant); it was also supported by The John A. Hartford Foundation Geriatric Center of Excellence. Dr. Schoenborn was supported by the Donald W. Reynolds Consortium Faculty Development to Advance Geriatric Education Program: Next Steps and the Daniel and Jeannette Hendin Schapiro Geriatric Medical Education Center. Dr. Boyd was supported by the Paul Beeson Career Development Award (NIA K23 AG032910), the John A. Hartford Foundation, Atlantic Philanthropies, and the Starr Foundation. Dr. McNabney was supported by the Fellowship Training Program in Geriatric Medicine and Gerontology at Johns Hopkins, and Dr. Cayea was supported by the Daniel and Jeannette Hendin Schapiro Geriatric Medical Education Center.
Dr. Cynthia Boyd received a royalty from UptoDate for having co-authored a chapter on Multimorbidity, however we do not believe this has resulted in any conflict with the design, methodology, or results presented in this manuscript.
This study was approved by a Johns Hopkins School of Medicine institutional review board. (Approval date: 10/10/2013, Application number: NA_00087265).
This manuscript has not been previously published and is not under consideration in the same or substantially similar form in any other peer-reviewed media. A description of the development process of the curriculum is being included as an example of curriculum development in the appendix to the 3rd edition of the book “Curriculum Development in Medical Education: A Six Step Approach”. The appendix chapter is entitled: “Teaching internal medicine residents to incorporate prognosis in the care of older patients with multimorbidity”. It focuses on the process of developing the curriculum and does not include any of the evaluation results or any of the tables and figure included in this manuscript. We presented an earlier version of the manuscript as a poster at the Society of General Internal Medicine national meeting in Denver, CO in 2013, and the American Geriatrics Society national meeting in Grapevine, TX in 2013.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Guiding principles for the care of older adults with multimorbidity: an approach for clinicians. Guiding principles for the care of older adults with multimorbidity: an approach for clinicians. American geriatrics society expert panel on the care of older adults with multimorbidity. J Am Geriatr Soc. 2012;60:E1–E25.View ArticleGoogle Scholar
- Fortin M, Hudon C, Haggerty J, Akker M, Almirall J. Prevalence estimates of multimorbidity: A comparative study of two sources. BMC Health Serv Res. 2010;10:111.View ArticleGoogle Scholar
- Boyd CM, Fortin M. Future of multimorbidity research: How should understanding of multimorbidity inform health system design. Public Health Rev. 2010;32:451–74.Google Scholar
- Cho H, Klabunde CN, Yabroff KR, Wang Z, Meekins A, Lansdorp-Vogelaar I, et al. Comorbidity-adjusted life expectancy: A new tool to inform recommendations for optimal screening strategies. Ann Intern Med. 2013;159:667–76.Google Scholar
- Taksler GB, Keshner M, Fagerlin A, Hajizadeh N, Braithwaite RS. Personalized estimates of benefit from preventive care guidelines: A proof of concept. Ann Intern Med. 2013;159(3):161–8.View ArticleGoogle Scholar
- American Geriatrics Society Expert Panel on the Care of Older Adults with Diabetes Mellitus. Guidelines abstracted from the american geriatrics society guidelines for improving the care of older adults with diabetes mellitus: 2013 update. J Am Geriatr Soc. 2013;61(11):2020–6.View ArticleGoogle Scholar
- Walter LC, Covinsky KE. Cancer screening in elderly patients: A framework for individualized decision making. JAMA. 2001;285(21):2750–6.View ArticleGoogle Scholar
- Yourman LC, Lee SJ, Schonberg MA, Widera EW, Smith AK. Prognostic indices for older adults: A systematic review. JAMA. 2012;307(2):182–92.View ArticleGoogle Scholar
- Lee SJ, Leipzig RM, Walter LC. Incorporating lag time to benefit into prevention decisions for older adults. JAMA. 2013;310:2609–10.View ArticleGoogle Scholar
- Schnipper LE, Lyman GH, Blayney DW, Hoverman JR, Raghavan D, Wollins DS, et al. American society of clinical oncology 2013 top five list in oncology. J Clin Oncol. 2013;31(34):4362–70.Google Scholar
- Choosing Wisely Workgroup AGS. American geriatrics society identifies another five things that healthcare providers and patients should question. J Am Geriatr Soc. 2014;62(5):950–60.View ArticleGoogle Scholar
- Williams AW, Dwyer AC, Eddy AA, Fink JC, Jaber BL, Linas SL, et al. Critical and honest conversations: The evidence behind the “choosing wisely” campaign recommendations by the american society of nephrology. Clin J Am Soc Nephrol. 2012;7(10):1664–72.Google Scholar
- Society of General Internal Medicine. Five things physicians and patients should question - the choosing wisely campaign. 2013. http://www.choosingwisely.org/wp-content/uploads/2013/09/SGIM-5things-List_091213.pdf, accessed Sept. 2014.
- Casarett D, Karlawish J, Morales K, Crowley R, Mirsch T, Asch DA. Improving the use of hospice services in nursing homes: A randomized controlled trial. JAMA. 2005;294:211–7.View ArticleGoogle Scholar
- Sima CS, Panageas KS, Schrag D. Cancer screening among patients with advanced cancer. JAMA. 2010;304:1584–91.View ArticleGoogle Scholar
- Schonberg MA, Breslau ES, McCarthy EP. Targeting of mammography screening according to life expectancy in women aged 75 and older. J Am Geriatr Soc. 2013;61(3):388–95.View ArticleGoogle Scholar
- Walter LC, Lindquist K, Nugent S, Schult T, Lee SJ, Casadei MA, et al. Impact of age and comorbidity on colorectal cancer screening among older veterans. Ann Intern Med. 2009;150(7):465–73.Google Scholar
- Royce TJ, Hendrix LH, Stokes WA, Allen IM, Chen RC. Cancer screening rates in individuals with different life expectancies. JAMA Intern Med. 2014;174:1558–65.View ArticleGoogle Scholar
- Drazer MW, Prasad SM, Huo D, Schonberg MA, Dale W, Szmulewitz RZ, et al. National trends in prostate cancer screening among older american men with limited 9-year life expectancies: Evidence of an increased need for shared decision making. Cancer. 2014;120(10):1491–8.Google Scholar
- Christakis NA, Iwashyna TJ. Attitude and self-reported practice regarding prognostication in a national sample of internists. Arch Intern Med. 1998;158:2389–95.View ArticleGoogle Scholar
- Kissane DW, Bylund CL, Banerjee SC, Bialer PA, Levin TT, Maloney EK, et al. Communication skills training for oncology professionals. J Clin Oncol. 2012;30:1242–7.Google Scholar
- Lim LS, Kandavelou K, Khan N. Palliative care teaching in medical residency: A review of two POGO-e teaching products. J Am Geriatr Soc. 2012;60:1141–4.View ArticleGoogle Scholar
- Montagnini M, Varkey B, Duthie Jr E. Palliative care education integrated into a geriatrics rotation for resident physicians. J Palliat Med. 2004;7:652–9.View ArticleGoogle Scholar
- Smith L, O’Sullivan P, Lo B, Chen H. An educational intervention to improve resident comfort with communication at the end of life. J Palliat Med. 2013;16:54–9.Google Scholar
- Tchorz KM, Binder SB, White MT, Lawhorne LW, Bentley DM, Delaney EA, et al. Palliative and end-of-life care training during the surgical clerkship. J Surg Res. 2013;185:97–101.Google Scholar
- Epner DE, Baile WF. Difficult conversations: Teaching medical oncology trainees communication skills one hour at a time. Acad Med. 2014;89:578–84.View ArticleGoogle Scholar
- Reuben DB, Zwanziger J, Bradley TB, Fink A, Hirsch SH, Williams AP, et al. How many physicians will be needed to provide medical care for older persons? physician manpower needs for the twenty-first century. J Am Geriatr Soc. 1993;41:444–53.Google Scholar
- Williams BC, Warshaw G, Fabiny AR, Lundebjerg Mpa N, Medina-Walpole A, Sauvigne K, et al. Medicine in the 21st century: Recommended essential geriatrics competencies for internal medicine and family medicine residents. J Grad Med Educ. 2010;2:373–83.Google Scholar
- Kern D, Thomas P, Hughes M. Curriculum Development for Medical Education - A Six-Step Approach. Second Edition. Baltimore, Maryland: The Johns Hopkins University Press, 2009.Google Scholar
- Kolb DA, Boyatzis RE, Mainemelis C. Experiential learning theory: Previous research and new directions. Perspectives on thinking, learning, and cognitive styles. 2001;1:227–47.Google Scholar
- Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65:S63–7.View ArticleGoogle Scholar
- Cruz M, Covinsky K, Widera EW, Stijacic-Cenzer I, Lee SJ. Predicting 10-year mortality for older adults. JAMA. 2013;309:874–6.View ArticleGoogle Scholar
- Schonberg MA, Davis RB, McCarthy EP, Marcantonio ER. External validation of an index to predict up to 9-year mortality of community-dwelling adults aged 65 and older. J Am Geriatr Soc. 2011;59(8):1444–51.View ArticleGoogle Scholar
- Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998;36:8–27.View ArticleGoogle Scholar
- Morris DA, Johnson KS, Ammarell N, Arnold RM, Tulsky JA, Steinhauser KE. What is your understanding of your illness? A communication tool to explore patients’ perspectives of living with advanced illness. J Gen Intern Med. 2012;27:1460–6.View ArticleGoogle Scholar
- Christakis NA, Lamont EB. Extent and determinants of error in physicians’ prognoses in terminally ill patients: Prospective cohort study. West J Med. 2000;172(5):310–3.View ArticleGoogle Scholar
- Gripp S, Moeller S, Bolke E, Schmitt G, Matuschek C, Asgari S, et al. Survival prediction in terminally ill cancer patients by clinical estimates, laboratory tests, and self-rated anxiety and depression. J Clin Oncol. 2007;25(22):3313–20.Google Scholar
- Brickner L, Scannell K, Marquet S, Ackerson L. Barriers to hospice care and referrals: Survey of physicians’ knowledge, attitudes, and perceptions in a health maintenance organization. J Palliat Med. 2004;7(3):411–8.View ArticleGoogle Scholar
- Luck J, Peabody JW, Dresselhaus TR, Lee M, Glassman P. How well does chart abstraction measure quality? A prospective comparison of standardized patients with the medical record. Am J Med. 2000;108(8):642–9.View ArticleGoogle Scholar
- Curtis JR, Patrick DL, Caldwell ES, Collier AC. Why don’t patients and physicians talk about end-of-life care? Barriers to communication for patients with acquired immunodeficiency syndrome and their primary care clinicians. Arch Intern Med. 2000;160(11):1690–6.View ArticleGoogle Scholar
- Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PA, et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282(15):1458–65.Google Scholar