Locally adapting generic rubrics for the implementation of outcome-based medical education: a mixed-methods approach

Background Rubrics are frequently used to assess competencies in outcome-based medical education (OBE). The implementation of assessment systems using rubrics is usually realised through years of involvement in projects with various stakeholders. However, for countries or specialities new to OBE, faster and more simplified processes are required. In March 2019, Japan introduced nine competencies and generic rubrics of competencies for medical residents. We explored the local adaptation of these generic rubrics and its consequences for assessors. Methods The study followed three steps. First, we locally adapted the generic rubrics. This was followed by conducting mixed-method research to explore the effect of the local adaptation. In step two, we examined the correlations between the scores in the locally adapted assessment sheets for supervising doctors and generic rubrics. In step three, we conducted interviews with supervising doctors. The study was conducted in the General Internal Medicine Department of Nagoya University, Japan. In the first step, doctors in the Medical Education Center and other medical departments, clerks, and residents participated. Supervising doctors in the General Internal Medicine Department participated in the second and third steps. Results A locally adapted assessment system was developed and implemented in seven months. The scores of the generic rubrics and the adapted assessment tool completed by the supervising doctors showed good correlations in some items as opposed to others, assessed mainly with other tools. Participant interviews revealed that local adaptation decreased their cognitive load leading to consistent ratings, increased writing of comments, and promoting reflection on instruction. Conclusions This adaptation process is a feasible way to begin the implementation of OBE. Local adaptation has advantages over direct use of generic rubrics. Supplementary Information The online version contains supplementary material available at 10.1186/s12909-022-03352-4.


Level 2
Level 3 Expected level at the end of residency Level4 ■Be able to outline the historical flow of medicine and medical treatment, clinical ethics, ethical issues related to life and death, and norms related to various ethics. ■To be able to explain the significance and necessity of patients' fundamental rights, the significance of the right to self-determination, patients' values, informed consent and informed assent. ■To understand the significance and necessity of the right to self-determination, patient values, informed consent and informed assent, etc. To consider patient privacy and understand the importance of confidentiality and handle patients appropriately.
Demonstrate respect for the dignity of the human person and the inviolability of life.
Protect the dignity of the human person and respect the inviolability of life.
Demonstrate role models to others.
Give minimum consideration to patient privacy and maintain confidentiality.
Respect patient privacy and maintain confidentiality.
Demonstrate role models to others.
Recognize the existence of ethical dilemmas.
Recognize ethical dilemmas and respond based on mutual respect.
Recognize ethical dilemmas and make multifaceted decisions and responses based on mutual respect.
Recognize the existence of conflicts of interest.
Recognize conflicts of interest and deal with them in accordance with the management policy.
Demonstrate role models to others.
Recognize the need to ensure transparency and prevent misconduct in medical treatment, research, and education. Level4 ■Identify problems, rank them in terms of importance and necessity, and work with other learners and faculty to find better and more specific ways to solve them. Be able to conduct appropriate self-evaluation and develop List the basic differential diagnosis and plan the initial response for the common syndromes.
Differential diagnosis and initial response to common syndromes through an appropriate clinical reasoning process.
Provide adequate differential diagnosis and initial response for major syndromes. Securely collect the minimum necessary information on the patient's health status, including psychological and social aspects.
Collect information about the patient's health status, including psychological and social aspects, in an effective and safe manner.
Collect information about the patient's health, including psychological and social aspects, in an effective and safe manner in complex cases.
Safely implement optimal treatment of basic diseases.
Safely implement the most appropriate treatment for the patient's condition.
Optimal treatment of complex diseases is safely implemented according to the patient's condition. and chronic diseases.
Appropriate medical records and documentation of medical treatment and its rationale, including at least the minimum necessary information.
Prepare medical records and documents related to medical treatment and its rationale appropriately and without delay. Treat patients and families with a minimum of wording, attitude, and manner.
Treat patients and families with appropriate wording, politeness, and appearance.
Treat patients and their families with appropriate wording, courtesy, and manner, and in a manner that is appropriate to the situation and the patient's family.
Organize and explain the minimum necessary information for the patient and family. Support the patient's independent decision-making with the supervising physician.
Organize necessary information for patients and their families, explain it in easy-to-understand terms, and support their independent decision-making.
Organize necessary and sufficient information for patients and their families in an appropriate manner, explain it in easy-to-understand terms, and support patients' independent decision-making after taking medical judgment into account.
Identify key needs of patients and families.
Identify the needs of patients and their families in terms of physical, psychological, and social aspects. ■To be able to explain the significance of team medicine and to participate in medical treatment as a member of the team (as a student). ■To be able to recognize one's own limitations and seek assistance from other healthcare professionals. ■To be able to explain the role of the physician in team medicine.
Understand the purpose of the organization and team that provides healthcare in a simple case.
Understand the purpose of a healthcare organization or team and the role of each member of the team.
In complex cases, practice with an understanding of the purpose of the organization or team providing health care and the purpose of the team, etc.
In simple cases, share information and collaborate with each member of the team.
Share information and collaborate with each member of the team.
Proactively share information with each member of the team and collaborate with them to practice the best team medicine. To be able to explain medical planning, regional medical concepts, regional comprehensive care, and regional health. ■To be able to explain disaster medicine. ■Actively participate in and contribute to Understand the laws, regulations, and systems related to health care.
Understand the purpose and structure of laws and systems related to health care.
Understand the purpose and structure of laws and systems related to health care and apply them to actual clinical practice.
Understand the system of health insurance and publicly funded medical care.
Appropriately utilize health insurance and publicly funded medical care, while giving consideration to the patient's burden of medical costs.
Determine whether or not health insurance and publicly funded medical care are applicable and utilize them appropriately.
Understand the importance of understanding the health issues and needs of the community.
Identify local health problems and needs, and propose necessary measures.
Identify community health issues and needs, and propose and implement necessary measures.
Understand the necessity of preventive medicine, health care, and health promotion.
Strive for preventive medicine, health care, and health promotion.
Provide concrete suggestions for improvement in preventive medicine, health care, and health promotion.
Understand the comprehensive community care system.
Understand the comprehensive community care system and contribute to its promotion.
Understand the comprehensive community care system and actively participate in its promotion.
Understand that extraordinary medical needs, such as disasters and infectious disease pandemics, can occur.
Prepare for extraordinary medical demands, such as disasters and infectious disease pandemics.
Assume extraordinary medical demands, such as disasters and infectious disease pandemics, and respond to them in practice leading an organized response. Recognize the need to absorb rapidly changing and developing medical knowledge and technology.
Strive to absorb rapidly changing and developing medical knowledge and technology.
To constantly self-reflect and strive for self-improvement in order to absorb rapidly changing and developing medical knowledge and technology.
Maintain an attitude of learning from colleagues, junior colleagues, and non-physician medical professionals.
Teach and learn from each other, colleagues, juniors, and non-physician medical professionals.
Study with colleagues, juniors, and non-physician medical professionals while training their successors.
Recognize the importance of domestic and international policies and the latest trends in medicine and medical care (including drug-resistant bacteria and genomic medicine).
Understand the latest trends in domestic and international policy, medicine, and medical care (including drug-resistant bacteria and genomic medicine).