- Research article
- Open Access
Assessment of medical professionalism using the Professionalism Mini Evaluation Exercise (P-MEX) in a multi-ethnic society: a Delphi study
BMC Medical Education volume 20, Article number: 225 (2020)
The importance of medical professionalism and its assessment has been well documented in the literature. However, there is currently no culturally-adapted tool to assess medical professionalism in Singapore. This study sets out to find consensus on relevance of the items from the Professionalism Mini Evaluation Exercise (P-MEX) for assessing medical professionalism in Singapore.
A two-round Delphi survey was completed by an expert panel consisting of program directors, associate designated institutional officials, and designated institutional official (n = 15) from residency programs in Singapore. Round 1 comprised of 23 items from the P-MEX rated on a 5-point scale (1 = Definitely include, 2 = Possibly include, 3 = Neutral, 4 = Possibly exclude, 5 = Definitely exclude). In round 2, the experts received feedback from the previous round, and were asked to re-rate the items which did not achieve consensus in the previous round. The threshold for consensus in the study was defined as 70% or greater agreement among experts.
Completed questionnaires for both rounds were received from all 15 experts. In round 1, 18 items (78%) achieved consensus to be included. In round 2, 1 (4%) item achieved consensus to be included. However, none of the remaining items achieved consensus to be removed, and they exhibited stability in responses. A list of 19 items covering four domains of medical professionalism (Doctor-patient relationship skills, Reflective skills, Time management and Inter-professional relationship skills) was obtained during the two-rounds of Delphi survey.
Nineteen items in the P-MEX had garnered consensus and is suitable for assessing medical professionalism in Singapore. The findings of this study can provide guidance for faculty and institutions who want to assess medical professionalism of their medical trainees.
Medical professionalism has been shown to affect doctors’ relationships with their patients, quality of care, and ultimately health and illness outcomes . In recent years, medical professionalism has been increasingly emphasized in medical undergraduate and post-graduate curricula [2,3,4,5]. Previously ‘good medical practice’ has been defined more broadly in terms of roles such as ‘professional’ and ‘healer’, with various medical councils, academic and professional bodies having produced clear documentations on these roles [6,7,8]. For residents in training, unprofessional behaviour during their training correlates with an increased risk of disciplinary action later on in their careers as physicians . Fortunately, medical professionalism can be nurtured , but in order for this to take place, it has to be adequately assessed . One of the dominant frameworks of medical professionalism is that professionalism consists of a set of behaviours and competencies that can be mastered by the physician; and these behaviours and competencies can be assessed . We had previously performed a systematic review of the quality and utility of observer-based assessment tools that could be used in residency programs and had identified the Professionalism Mini-Evaluation Exercise (P-MEX) as one of the assessment tools that could be useful in the assessment of medical professionalism in our residency programs . However, Chandratilake et al. and Jha et al. have demonstrated differences in understanding of professionalism in physicians from different regions [14, 15]. Moreover, according to consensus statement and recommendations from the Ottawa conference, the assessment of professionalism varies across different cultures and hence cross-cultural validation of the assessment tool is important . The P-MEX was originally developed in Canada by Cruess et al , and when the P-MEX was piloted in Japan  and Finland , additional culturally-relevant items were added.
Since the Ottawa report in 2011 , studies have explored the assessment of medical professionalism in various non-Anglo-Saxon/ Western contexts such as Korea, Japan and China [21, 22]. However, till date there has not been a study looking at the assessment of medical professionalism in a multi-ethnic Asian context such as Singapore. Singapore’s multi-ethnic landscape comprises of predominantly Chinese (74%), followed by Malay (13%), and Indian (9%) . The importance of cultural perspectives and its effect on medical professionalism has previously been highlighted by Jha et al . The intercultural development continuum  and cultural fit theory  both highlight that professional behaviour between healthcare professionals can potential be influenced by the shared values and societal culture, and this is because professional behaviours and values are socially constructed . Based on Hofstede’s cultural dimension theory, national culture consists of 6 dimensions: power distance, individualism versus collectivism, masculinity versus femininity, uncertainty avoidance, and long-term orientation versus short term normative orientation . Even among Asian countries, significant cultural difference across these dimensions may exist .
Our previous qualitative study with patients (healthcare recipients) and faculty (medical educators) had identified 23 items to assess medical professionalism, covering four domains of medical professionalism (Doctor-patient relationship skills, Reflective skills, Time management and Inter-professional relationship skills) . The aim of this study was to gain consensus among the stakeholders in residency programs on the items to be used to assess medical professionalism in a multi-ethnic and multi-cultural Asian context. These items can form the basis for an assessment tool for the assessment of medical professionalism within residency programs.
A modified Delphi technique with two iterative rounds was employed. To ensure strong retention of expert involvement, an upper limit of two rounds of investigation was set in this study . It is also acknowledged that having a planned number of rounds is an indicator of good quality in designing a Delphi study .
SingHealth is Singapore’s largest group of public healthcare institutions, consisting of four public hospitals, five national specialty centres and a network of community hospitals and polyclinics . There are over 1600 faculty and more than 900 residents in training.
Fifteen experts from the SingHealth residency programs were selected according to age, gender, ethnicity and disciplines. The experts selected had at least 3 years of experience in the residency program. They also had to be in involved in the disciplinary committees set up to investigate disciplinary issues related to residents, as well as be involved in the training and assessment of residents, especially in the area of medical professionalism. Each potential expert was sent an invitation email introducing the study objectives and the study procedures. Depending on the expert’s preference, either hardcopy or electronic questionnaire was provided upon agreement to participate in the study.
The first questionnaire contained 23 observer-based items assessing medical professionalism based on the results of the previous survey and qualitative study (Fig. 1) . Two new items which emerged from the qualitative study -(1) communicated effectively with patient and (2) demonstrated collegiality were included in this Delphi survey in addition to the original 21 items. The experts were asked to rate the level of agreement with each item as assessment of medical professionalism on a 5-point scale scored as follows: 1 = Definitely include, 2 = Possibly include, 3 = Neutral, 4 = Possibly exclude, 5 = Definitely exclude. The expert was also invited to provide any comment on each item.
The authors reviewed the consensus rating and feedback from Round 1. Respondents to round 1 were contacted and provided with the group and individual ratings from round 1, as well as any feedback obtained from the experts. The experts were asked to re-rate the items with less than 70% agreement rate in round 1. Similar to round 1, a 5-point scale (1 = Definitely include, 2 = Possibly include, 3 = Neutral, 4 = Possibly exclude, 5 = Definitely exclude) was used. The expert was also invited to provide any comment on each item.
Consensus was deemed to be achieved when 70% of the experts chose to include (definitely include and possibly include) or exclude (definitely exclude and possibly exclude) the item. There is no universally accepted threshold for defining consensus as part of the Delphi process, with thresholds for consensus ranging from 55 to 100% in the published literature . A predefined consensus level is an indicator of good quality Delphi research  and the consensus level is influenced by the study aims . A 70% threshold was considered appropriate for this study and is consistent with other research using a modified Delphi technique [34, 35]. It was decided a priori that items with no consensus in the two rounds of Delphi would be included in the list of items to be used for pilot assessment of medical professionalism in residency program. Stability of responses was determined using Wilcoxon signed-rank test. Data were analyzed with Stata SE15.0 (Stata-Corp, College Station, TX, USA).
The SingHealth Centralized Institutional Review Board approved this study (Reference Number: 2016/3009). We obtained informed consent, which conformed to the principle outlined in the 1964 Declaration of Helsinki, from all the experts before the commencement of study.
A total of 15 experts participated in this study (67% male, median age 45 years (37 to 66 years)), consisting of 11 program directors, three associate designated institutional officials (ADIO) and one designated institutional official (DIO). Around half of the faculty members came from medical disciplines, the rest spread across a wide spectrum of disciplines (surgical, diagnostic radiology, nuclear medicine and pathology, emergency medicine, pediatrics) (Table 1). All 15 experts participated in both rounds (response rate = 100%). Table 2 and Fig. 2 illustrate the summary of the results.
Completed questionnaires were received from all 15 experts. In round 1, 18 items (78%) achieved consensus for inclusion into the modified P-MEX tool. They included: listened actively to patient, showed interest in patient as a person, recognized and met patient needs, ensured continuity of patient care, maintained appropriate boundaries, communicated effectively with patient, demonstrated awareness of limitations, admitted errors or omissions, accepted feedback, maintained composure, was on time, completed tasks in a reliable fashion, was available to colleagues, maintained appropriate appearance, demonstrated respect for colleagues, avoided derogatory language, maintained patient confidentiality, demonstrated collegiality. The five remaining items (solicited feedback, advocated on behalf of a patient, extended his/herself to meet patient needs, used health resources appropriately, addressed own gaps in knowledge and skills) did not reach consensus for either inclusion or exclusion (Table 3).
Completed questionnaires were received from all 15 experts. In round 2, 1 (4%) item (addressed own gaps in knowledge and skills) achieved consensus to be included. The 4 remaining items (solicited feedback, advocated on behalf of a patient, extended his/herself to meet patient needs, used health resources appropriately) did not reach consensus for either inclusion or exclusion (Table 4).
A total of 19 items from the two rounds of the Delphi survey covering four domains of medical professionalism (Doctor-patient relationship skills, Reflective skills, Time management and Inter-professional relationship skills) were selected from the two-rounds of Delphi survey. Four items did not achieve consensus, and stability was observed for these items (Supplementary Table 1) . Further rounds of Delphi were terminated as decided a priori.
Professionalism refers to a “set of values, behaviours and relationships that underpins the trust that the public has in doctors”  and there are often differences in how professionalism is defined and observed in different countries and cultures . In Singapore, our residents comprising multi-ethnic groups often start their training of medical professionalism in medical school, where foundational platforms of medical professionalism included clinical competence, capacity to address ethical and legal issues and effective communication [37, 39], with virtues of medical professionalism such as honesty and integrity, responsibility and participation, respect and sensitivity, compassion and empathy taken as the foundational values of medical professionalism . Thus, it is expected that these values continue to be reflected in their clinical practice upon graduation. In our study, we were able to achieve consensus on 19 items covering four domains of medical professionalism (Doctor-patient relationship skills, Reflective skills, Time management and Inter-professional relationship skills). All the 19 items, inclusive of the two new items of ‘communicated effectively with patient’ and ‘demonstrated collegiality’, were aligned with the aforementioned aspirational values that identified medical professionalism in our multi-ethnic society. In particular, our findings underscore the importance of recognising acceptable behaviours according to social, religious and cultural norms in a multi-ethnic and multi-cultural country such as Singapore. For example, the item on maintaining appropriate boundaries received 100% agreement for inclusion into the assessment of medical professionalism. However, as the P-MEX was primarily designed to assess observable behaviours for use in the clinical setting, only 24 of the 142 behaviours were eventually chosen to be evaluated , and this has also resulted in the P-MEX having a higher utility score when used in the clinical setting to assess trainees . Thus, for practical reasons, it will not be able to assess all domains of professionalism completely e.g. altruism, excellence and humanism [39, 41]. In Korea, a previous study had identified eight categories of unethical and unprofessional behaviors- (a) substandard practice, (b) violation of work ethics, (c) misconduct related to conflict of interest, (d) dishonesty with patients, (e) violation of patient confidentiality, (f) lack of respect for patients, (g) lack of respect for colleagues, and (h) misconduct in research , of which six categories of unprofessional behaviour seem to mirror the 19 items found in our study.
In our study, 4 items “solicited feedback”, “advocated on behalf of a patient”, “extended his/herself to meet patient needs” and “used health resources appropriately” did not achieve the level of agreement to be included for both rounds, suggesting that these items may be less relevant in the local context as compared to other items in the list. This is largely in congruence with the findings of our previous qualitative study which showed that these items were less relevant to patients and/or faculty . Similarly, in a study to define professionalism in anaesthesiology, “resourcefulness” which is similar to “used health resources appropriately”, was deemed to be less important . This may be because of the difficulty to agree as to what is “appropriate use of health resources”, as raised by participants in this Delphi survey and in our previous qualitative study . Modifications to these items may be needed in the future after the pilot testing of the modified P-MEX in our local context.
In our study, the two new items derived from the previous qualitative study- ‘communicated effectively with patient’ and ‘demonstrated collegiality’ both achieved the consensus to be included. This reflects the importance of effective communication and collaborative practice in medical professionalism locally, which is corroborated by the development and incorporation of various communication and interprofessional courses in local undergraduate and postgraduate medical education [44,45,46,47]. Effective communication with patients and other healthcare personnel, as well as respect for patients and colleagues were recognized as increasingly important domains of medical professionalism that need additional attention and nurturing . The emphasis on collegiality may also reflect the collectivist nature of Asian culture based on the Hofstede’s cultural dimension theory [27, 42] and the increasing recognition that medical professionals practice in a community of practice . Other Asian countries exploring the use of the P-MEX should consider including “communicated effectively with patient” and “demonstrated collegiality” to better reflect the medical professionalism construct.
Key strengths of this study include involvement of senior faculty, including the DIO and ADIO, who are involved in the disciplinary committees handling issues of lapses in professionalism in the residency programs. In addition, faculty from a range of disciplines (both surgical and medical specialties) participated in the Delphi survey and of different ethnicities. There was also no expert drop-out in the two rounds of Delphi survey.
Limitations of this study include the arbitrariness of the cut-off point utilised. The cut-off point of 70% adopted in our study was widely used in other studies [34, 35]. However, at this threshold, our findings should be taken as the best achievable consensus given the lack of robust evidence in this field rather than as evidence of absolute unanimity. In addition, the study may have selection bias as the experts were selected by the researchers. However, considerable care was taken during the study to select the experts based on their experience in handling of disciplinary issues in the residency programs, and to include faculty of various years of experience, gender, ethnicity and across various surgical and medical disciplines. Moreover, the framework of medical professionalism in the P-MEX containing 4 domains - Doctor-patient relationship, Reflective, Time management, and Inter professional relationship - may not cover entire continuum of professionalism. Therefore, we may have missed certain attributes that define medical professionalism such as social accountability and altruism. Nevertheless, our qualitative  and survey studies (under review) with patients and faculty members showed that the four domains of P-MEX broadly covered medical professionalism in our setting with an addition of 2 new sub-domains- (1) communicated effectively with patient and (2) demonstrated collegiality.
A list of 19 items covering four domains of medical professionalism (Doctor-patient relationship skills, Reflective skills, Time management and Inter-professional relationship skills) of the P-MEX was obtained from the Delphi study. The 4 other items (solicited feedback, advocated on behalf of a patient, extended his/herself to meet patient needs, used health resources appropriately) did not reach consensus for either inclusion or exclusion. The findings of this study can provide guidance for faculty and institutions who want to introduce assessment of medical professionalism in the curriculum of medical trainees. In particular, other Asian countries exploring the use of the P-MEX may consider incorporating items such as “communicated effectively with patient” and “demonstrated collegiality” to have a more comprehensive and culturally pertinent assessment of the medical professionalism.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Associate designated institutional officials
Designated institutional official
Professionalism Mini Evaluation Exercise
Blasi ZD, Harkness E, Ernst E, Georgiou A, Kleijnen J. Influence of context effects on health outcomes: a systematic review. Lancet. 2001;357:757–62.
Medicine AFABoI. Medical professionalism in the new millennium: a physician charter. Ann Intern Med. 2002;136:243.
Swick HM, Szenas P, Danoff D, Whitcomb ME. Teaching professionalism in undergraduate medical education. JAMA. 1999;282:830–2.
Cruess RL, Cruess SR. Teaching professionalism: general principles. Med Teach. 2006;28:205–8.
Tweedie Jude JH. Dame Jane Dacre: advancing medical professionalism London Royal College of Physicians; 2018.
Blank L, Kimball H, McDonald W, Merino J. Medical professionalism in the new millennium: a physician charter 15 months later. Ann Intern Med. 2003;138:839–41.
Frank JR. The CanMEDS 2005 physician competency framework. Better standards. Better physicians. Better care. Ottawa: The Royal College of Physicians and Surgeons of Canada; 2005.
Tomorrow’s Doctors. General Medical Council 2009.
Papadakis MA, Hodgson CS, Teherani A, Kohatsu ND. Unprofessional behavior in medical school is associated with subsequent disciplinary action by a state medical board. Acad Med. 2004;79:244–9.
O'Sullivan H, van Mook W, Fewtrell R, Wass V. Integrating professionalism into the curriculum: AMEE Guide No. 61. Med Teach. 2012;34:e64–77.
Lynch DC, Surdyk PM, Eiser AR. Assessing professionalism: a review of the literature. Med Teach. 2004;26:366–73.
Irby DM, Hamstra SJ. Parting the clouds: three professionalism frameworks in medical education. Acad Med. 2016;91:1606–11.
Kwan YH, Png K, Phang JK, Leung YY, Goh H, Seah Y, Thumboo J, Ng APSC, Fong W, Lie D. A systematic review of the quality and utility of observer-based instruments for assessing medical professionalism. J Grad Med Educ. 2018;10:629–38.
Chandratilake M, McAleer S, Gibson J. Cultural similarities and differences in medical professionalism: a multi-region study. Med Educ. 2012;46:257–66.
Jha V, McLean M, Gibbs TJ, Sandars J. Medical professionalism across cultures: a challenge for medicine and medical education. Med Teach. 2015;37:74–80.
Hodges B, Paul R, Ginsburg S, The Ottawa Consensus Group M. Assessment of professionalism: from where have we come - to where are we going? An update from the Ottawa consensus group on the assessment of professionalism. Med Teach. 2019;41:249–55.
Cruess R, McIlroy JH, Cruess S, Ginsburg S, Steinert Y. The professionalism mini-evaluation exercise: a preliminary investigation. Acad Med. 2006;81:S74–8.
Tsugawa Y, Tokuda Y, Ohbu S, Okubo T, Cruess R, Cruess S, Ohde S, Okada S, Hayashida N, Fukui T. Professionalism mini-evaluation exercise for medical residents in Japan: a pilot study. Med Educ. 2009;43:968–78.
Karukivi M, Kortekangas-Savolainen O, Saxén U, Haapasalo-Pesu K-M. Professionalism mini-evaluation exercise in Finland: a preliminary investigation introducing the Finnish version of the P-MEX instrument. J Adv Med Educ Prof. 2015;3:154–8.
Hodges BD, Ginsburg S, Cruess R, Cruess S, Delport R, Hafferty F, Ho M-J, Holmboe E, Holtman M, Ohbu S. Assessment of professionalism: recommendations from the Ottawa 2010 conference. Med Teach. 2011;33:354–63.
Kwon H-J, Lee Y-M, Lee Y-H, Chang H-J. Development an instrument assessing residents’ attitude towards professionalism lapses in training. Korean J Med Educ. 2017;29:81.
Wang X, Shih J, Kuo F-J, Ho M-J. A scoping review of medical professionalism research published in the Chinese language. BMC Med Educ. 2016;16:300.
Noor NM, Leong C-H. Multiculturalism in Malaysia and Singapore: contesting models. Int J Intercult Relat. 2013;37:714–26.
Hammer MR. Additional cross-cultural validity testing of the intercultural development inventory. Int J Intercult Relat. 2011;35:474–87.
Lu L. " cultural fit": individual and societal discrepancies in values, beliefs, and subjective well-being. J Soc Psychol. 2006;146:203–21.
Goldie J. The formation of professional identity in medical students: considerations for educators. Med Teach. 2012;34:e641–8.
Hofstede G. National cultures in four dimensions: a research-based theory of cultural differences among nations. Int Stud Manage Organ. 1983;13:46–74.
Paik Y, Vance CM, Stage HD. The extent of divergence in human resource practice across three Chinese National Cultures: Hong Kong, Taiwan and Singapore. Hum Resour Manag J. 1996;6:20–31.
Fong W, Kwan YH, Yoon S, Phang JK, Thumboo J, Leung YY, Ng SC. Assessment of medical professionalism: preliminary results of a qualitative study. BMC Med Educ. 2020;20:27.
Ong B-H, Gonzalez-Rivas D. The inaugural SingHealth uniportal video-assisted thoracoscopic surgery masterclass in Singapore. J Vis Surg. 2018;4. http://jovs.amegroups.com/article/view/21361/20816.
Williams PL, Webb C. The Delphi technique: a methodological discussion. J Adv Nurs. 1994;19:180–6.
Humphrey-Murto S, Varpio L, Gonsalves C, Wood TJ. Using consensus group methods such as Delphi and nominal group in medical education research. Med Teach. 2017;39:14–9.
Vernon W. The Delphi technique: a review. Int J Ther Rehabil. 2009;16:69–76.
Robinson KR, Leighton P, Logan P, Gordon AL, Anthony K, Harwood RH, Gladman JR, Masud T. Developing the principles of chair based exercise for older people: a modified Delphi study. BMC Geriatr. 2014;14:65.
van Steenkiste BC, Jacobs JE, Verheijen NM, Levelink JH, Bottema BJAM. A Delphi technique as a method for selecting the content of an electronic patient record for asthma. Int J Med Inform. 2002;65:7–16.
Chalmers J, Armour M. The Delphi Technique. In: Liamputtong P, editor. Handbook of Research Methods in Health Social Sciences. Singapore: Springer Singapore; 2019. p. 715–35.
Physicians WPotRCo. Doctors in society. Medical professionalism in a changing world. Clin Med (London, England). 2005;5:S5.
Hafferty FW, McKinlay JB. The changing medical profession an international perspective; 1993.
Stern DT. Measuring medical professionalism. New York: Oxford University Press; 2006.
Macneill P, Joseph R, Lysaght T, Samarasekera DD, Hooi SC. A professionalism program in medical education and training–From broad values to specific applications: YLL School of Medicine, Singapore. Medical Teacher. 2020:1–1.
Ho M-J, Yu K-H, Pan H, Norris JL, Liang Y-S, Li J-N, Hirsh D. A tale of two cities: understanding the differences in medical professionalism between two Chinese cultural contexts. Acad Med. 2014;89:944–50.
Chang H-J, Lee Y-M, Lee Y-H, Kwon H-J. Investigation of unethical and unprofessional behavior in Korean residency training. Teach Learn Med. 2015;27:370–8.
Kearney RA. Defining professionalism in anaesthesiology. Med Educ. 2005;39:769–76.
Jacobs JL, Samarasekera DD, Chui WK, Chan SY, Wong LL, Liaw SY, Tan ML, Chan S. Building a successful platform for interprofessional education for health professions in an Asian university. Med Teach. 2013;35:343–7.
Chua AZ, Lo DY, Ho WH, Koh YQ, Lim DS, Tam JK, Liaw SY, Koh GC. The effectiveness of a shared conference experience in improving undergraduate medical and nursing students’ attitudes towards inter-professional education in an Asian country: a before and after study. BMC Med Educ. 2015;15:233.
Wong M, Koh D, Phua K, Lee H. Teaching community, occupational and family medicine at the National University of Singapore: past, present and future. Ann Acad Med Singap. 2005;34:102C–7C.
Samarasekera DD, Ooi S, Yeo SP, Hooi SC. Medical education in Singapore. Med Teach. 2015;37:707–13.
Ong YT, Kow CS, Teo YH, Tan LHE, Abdurrahman A, Quek NWS, Prakash K, Cheong CWS, Tan XH, Lim WQ, et al. Nurturing professionalism in medical schools. A systematic scoping review of training curricula between 1990–2019. Med Teach. 2020:1–14.
Wenger E. Communities of practice: learning, meaning, and identity: Cambridge University Press; 1999.
The authors thank all the participants for their time and contribution to the study.
This research was supported by SingHealth Duke-NUS Medicine Academic Clinical Programme Education Support Programme Grant (Ref No: 03/FY2017/P2/03-A47). Funder was not involved in the design, delivery or submission of the research.
Ethics approval and consent to participate
The SingHealth Centralized Institutional Review Board approved this study (Reference Number: 2016/3009). We obtained informed consent, which conformed to the principle outlined in the 1964 Declaration of Helsinki, from all participants before study commencement.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Fong, W., Kwan, Y.H., Yoon, S. et al. Assessment of medical professionalism using the Professionalism Mini Evaluation Exercise (P-MEX) in a multi-ethnic society: a Delphi study. BMC Med Educ 20, 225 (2020). https://doi.org/10.1186/s12909-020-02147-9