Pilot study to evaluate a novel measure of self-perceived competencies among dental students
BMC Medical Education volume 22, Article number: 582 (2022)
PRISM is a novel approach to support self-reflection and learning appraisal in dental students, based on a visual metaphor. The aim of this study was to evaluate whether PRISM measurments would be reproducible and sensitive to detect learning progress in undergraduate dental students in their clinical years.
Voluntarily participating dental students were included. To evaluate reproducibility, a mixed cohort of 10 students each in 3rd, 4th and 5th year (total n = 30) was recruited and received three identically structured PRISM interviews within one week without any other teaching events. To assess perceived learning progress, 29 volunteer 3rd year students participated in three interviews during their clinical simulation course (beginning, middle, end). Distances between Subject and Objects was measured in millimeter; objects were classified into close or branched clusters depending on their distance from each other on the PRISM board.
Values for perceived competencies within PRISM interviews during one week were comparable between the three time points in the mixed cohort (n = 30; p > 0.05). Comparing the three subgroups (3rd, 4th and 5th year, each n = 10), PRISM indicated that 3rd year students perceived their competencies are significantly lower than the 4th and 5th year students (p < 0.01). 3rd year students had less often a branched cluster of objects than the other two groups (p < 0.05).
PRISM showed that over time, 3rd year students perceived a gain in their competencies in conservative dentistry and its sub-disciplines (p ≤ 0.01). The PRISM data indicated that by the end of the simulation course, the students appeared to show higher discrimination of their self-perceptions between sub-topics in conservative dentistry than at the start of the course (p = 0.01).
PRISM yields a reproducible measure of individual students' learning progress. It is a promising novel approach for appraisal in dental education. Further work is needed to confirm the generalisability of the findings.
The ability for appropriate self-assessment and reflection appear crucial in medical professionals but this core competency is limited across physicians . It is therefore not surprising that self-reflection and reflective practice is an issue of high scientific relevance in medical education . As a theoretical basis, different models of reflection exist, which illustrate reflective practice within two potential dimensions, i.e. an iterative dimension, whereby reflection is triggered by experience and a vertical dimension, which includes different levels of reflection on experience . Against this theoretical background, many potential measures have been developed to assess the ability to self-reflect or self-assess. Williams et al. categorized them in a systematic review into three types, rubrics, self-reported scales and observed behavior . These instruments need to be applied individually, depending on different influential factors, while no instrument appears superior to others . Recently, further approaches to support and/or foster self-assessment and reflective practice have been reported, including virtual reality , briefing and debriefing sessions , or video-based approaches . Those are only several examples from the emerging field of self-assessment and self-reflection in medical education research.
Recently, a novel tool has been introduced to support self-reflection and to facilitate learning progress. PRISM is a visual tool based on a metaphor . The applications for PRISM are broad; while initially developed to visualize suffering, a growing variety of applications have been reported . The structure of the PRISM task as a visual metaphor allows it to generate information about personally salient appraisals . Thus, this method has also a potential in medical education. To the authors’ knowledge this is the first study to examine the use of PRISM in medical education.
The aims of this pilot observational study were to demonstrate the potential of PRISM as a tool to I) facilitate and II) quantify perceptions of learning progress in a small group of clinical dental students and to test the reliability of the measure. Based on these two aims, the study tested two hypotheses: I) repeated quantitiative measurements of PRISM are stable within one week without education events in a mixed cohort of undergraduate dental students. II) students’ PRISM responses will reflect expected improvements in competencies (including knowledge, skills and perceived training need) during a particular course.
This observational study has been reviewed and approved by the ethics committee of the Medical Faculty of University of Leipzig, Germany (No: 117/20-ek). All participants were informed verbally and in writing about the study and provided their written informed consent. The study consisted of two parts: (I) repeated application of PRISM in a cohort of 30 undergraduate dental students over one week without any education events and (II) the evaluation of PRISM findings during a clinical simulation course in conservative dentistry and periodontology.
Volunteers were recruited among undergraduate dental students in their clinical years of study. For part (I), 10 students each from 3rd, 4th and 5th year of study were included. The 3rd year students were recruited prior to their simulation course and 4th as well as 5th year students were recruited after their clinical course in conservative dentistry and periodontology. For part (II), 29 students in 3rd year were included and followed-up during eight weeks of their clinical simulation course in conservative dentistry. Students were included irrespective of their age or gender. There were no further inclusion or exclusion criteria.
PRISM was initially developed in the field of psychology/psychosomatic medicine to assess and visualize patient suffering . For the PRISM task, a white metal board (297 × 210 mm), represents a defined context (in this study, ‘your dental studies’). The board includes a fixed yellow circle (7 cm in diameter) at the bottom right hand corner representing the identified Subject (in this study, ‘myself as a X-year dental student’). Different coloured Object discs (5 cm diameter) can then be placed in relation to the Subject. The Object discs represent different study-related issues, e.g. “your theoretical skills in the field of conservative dentistry”. For quantitiative assessment, the distance between Subject and Object was measured in millimeters (Fig. 1). Beside of the total distance between Subject and Object, the relationship between Objects was also assessed according to the distance between them and whether different Objects were placed in a close or branched cluster (Fig. 2). This clustering was assessed to evaluate the perceived relationship between the different sub-topics of the same topic (disciplines of conservative dentistry). This allowed assessment of the ways in which students distinguish between the sub-topics and whether their perceptions change during the course. For example, placing sub-topics into a close cluster was taken to indicate a low distinction between the topics, what might be caused by limited experiences and knowledge in this respect. Placing topics in a branched cluster was taken to indicate a high distribution between topics, potentially related to a gain in experiences in the different sub-topics.
One dentist (GS), trained in the use of PRISM, performed all the interviews. This dentist was not involved in the general appraisal of the clinical courses of the students. The PRISM task was used in addition to and independently of students’ regular appraisals. Interview settings were standardized as far as was possible. Interviews started with an explanation of the task and one example question (“Imagine this blue disk is burger and this green disc is broccoli. Where would you put these disks to reflect how much do you like them at the moment?”). A short explanation was repeated at each interview appointment. Interviews consisted of 18 questions, regarding students; skills, knowledge and their perception of remaining training needs in different fields in dentistry (Table S1). The task aimed to evaluate students’ subjectively perceived competencies, i.e. their recent self-perceived knowledge, practical skills and related remaining training needs. In part (I), all participants underwent three interviews within one week (Monday – T1, Wednesday – T2, Friday – T3), including the same questions (each 10–12 min). For part (II), students were interviewed at the beginning, middle (after four weeks) and end of their simulation course in conservative dentistry. Each interview lasted 10–12 min.
The statistical analysis was performed with SPSS for Windows, version 24.0 (SPSS Inc., U.S.A.). Values are presented as mean values with standard deviation or percentage, respectively. For part (I), values between the three time points were compared to each other for the total cohort and also in the three sub-groups (3rd, 4th and 5th year). For part (II), values for the total cohort were compared between beginning, middle and end of the simulation course. Normal distribution was tested with Shapiro–Wilk test. Based on normality of distribution, the general linear model or Friedman test was applied, while sign test was used for ordinal data. More than two independent, non-normal distributed samples were compared using Kruskal–Wallis test. For the analysis of categorical data, a chi-square test was applied. The significance level was set at p < 0.05.
For Part (I), 30 students were included, which had a mean age of 23.9 ± 3.3 years and a balanced gender distribution (50% male and female, respectively). In this group, 10 students each were in 3rd, 4th and 5th study year. The sample for Part (II) consisted of 3rd year students (n = 29) undergoing a course in conservative dentistry. They had a mean age of 22.9 ± 3.0 years and had a majority of female participants (72% vs. 28%).
Evaluation of PRISM after 3 interviews
When PRISM was repeated at three times within one week (Monday, Wednesday, Friday), the variation within the whole group (N = 30) between values did not reach statistical significance (p > 0.05, Table 1). Among the three subgroups, 4th and 5th year dental students showed stable values across the week (p > 0.05, Table 2). Within the 3rd year students, significant differences were found between the three measurement points (p < 0.05) despite the students having had no educational events during the week, while only the clusters remained stable (Table 2).
Comparing the three subgroups, i.e. 3rd, 4th and 5th year students (each n = 10), 3rd year students had significantly higher values in theoretical knowledge, practical skills and training need, including the whole field of conservative dentistry as well as all sub-topics (p < 0.01, Table 3). This indicates that 3rd year students perceived their theoretical knowledge and practical skills as less well developed and their training needs as greater. Also, 3rd year students showed a lower amount of branched cluster of objects than the other two groups (p < 0.05, Table 3).
Evaluation of PRISM over 8 weeks of dental education
During the 3rd year stusents’ simulation course, the PRISM task showed significant reductions in the distances between the Subject and Objects reflecting conservative dentistry or its sub-topics (p ≤ 0.01, Table 4). This indicated that over the duration of the simulation course, students perceived improvements in their skills and knowledge as well as reduction of their training needs. There was also a substantial change in clusters during the course – there were more branched clusters at the end of the course than at the beginning across the sub-topics regarding their practical skills (p < 0.01) and training need (p = 0.01; Table 4). This indicates that students at the end of the course distinguished between the sub-topics, potentially as a result of increased experiences and knowledge in those topics.
In a mixed cohort of undergraduate dental students in their clinical years, PRISM values remained stable over one week in the absence of any education events. This supports the reproducibility of the PRISM task i.e. its reliability. Over the duration of a clinical simulation course, PRISM indicated that students’ perceived competencies improved, indicating that PRISM is sensitive to visualize self-perceived learning progress.
To the authors’ knowledge, this is the first quantitative assessment of PRISM in medical or dental education, or indeed any higher education. There is therefore no directly comparable published work. As noted in the introduction, PRISM is a visual metaphor initially applied to measure suffering  but its use has extended more recently to a growing range of different applications . In its original application, the PRISM task has been demonstrated to have good reliability, yielding consistent results when repeated at intervals of a few hours to three days [9,10,11]. Further studies confirmed the reliability of PRISM to assess the disease burden of patients, e.g. for patients with psoriasis , and even for measuring pain-related suffering across different countries . The present results of part (I) of the current study are consistent with this, indicating the PRISM yields reliable data in the setting of dental education.
In this context, it was conspicious that 3rd year students showed less stable results in part (I) than the 4th and 5th year students. This might be explained by the structure of dental studies in Germany. In the first two years of dental studies, students focus on basic subjects in medicine and dental technology. In their 3rd year, students encounter conservative dentistry and its sub-topics for the first time. Students might have difficulty understanding their competencies in a subject they have never heard about, or may even mix up the sub-topics across interviews. This would result in the limited reproducibility in this particular year group. On the other hand, one could argue that students should perceive themselves as incompetent in a subject in which the students had no experience or teaching. On this basis, PRISM values would be expected to be particularly stable in this year group. Another possibility is that, as expected, the PRISM task encourages self-reflection by the students and they attempt to guage to what extent their learning to date will be relevant to topics they have not yet encountered. Further work is needed to resolve this.
Quantitative analysis for PRISM has been repeatedly performed in its original application form; different clinical studies have measured the suffering or disease burden of patients, e.g. with ulcerative colitis, chronic inflammatory vulvar disease, liver cirrhosis or organ transplantation [14,15,16,17]. These studies choosed different ways of quantification, including measurement in centimentre, millimeter or distinction of values [14,15,16,17]. Based on the individuality of the PRISM task, a remarkable range is often observed, especially if values are measured in millimeter, however, this also brings a high sensitivity .
A reduction of the distance between Subject and Objects was observerd during the clinical simulation course, reflecting improvements in students’ perceived competencies. This supports PRISM as a sensitive tool to visualize and measure learning progress in undergraduate dental students. As expected, the largest difference was between 3rd year and the two other year groups (Table 3). This can be explained by the fact that 3rd year students undergo the simulation course, while 4th and 5th year students work with real patients. It is recognised that pre-clinical simulation in dental education is not entirely satisfactory, because simulated and real patients differ significantly . Considering the simulation course alone, 3rd year students perceive a substantial improvement in their competencies during the course (Table 4).
The clustering of Objects i.e. students’ perceived competencies yielded further information about the students’ perceptions. At the beginning of the 3rd year simulation course, the students placed all the sub-topics in a tight cluster, consistent with a lack of discrimination, expected because of their lack of prior experience of the sub-topics. As the course progressed, the sub-topics showed a branched structure, consistent with the students developing a more differentiated and nuanced appraisal of the sub-topics due to their teaching and their growing practical experience. As expected, the branched structure of sub-topics was also evident in the responses of the 4th and 5th year students. The clustering of PRISM Objects might therefore provide a visual summary of the student’s conceptualization of a domain of knowledge. How, using PRISM, a student clusters the sub-topics of one overall topic or domain e.g. the whole field of conservative dentistry, may provide insights into how the students perceives structural relationships and hierarchies among topics or sets of technical skills, and into the student’s capacity to distinguish among different components of knowledge or technical performance. This is also expected to substantially facilitate discussion between student and teacher not only because PRISM provides a visual summary as the basis for discussion but also because of the properties that PRISM shares with other metaphors . For these reasons, further work on clustering of Objects in the PRISM task in medical or dental education settings might be fruitful.
This pilot study has shown that PRISM can be applied reliably to assess quantitatively students’ perceptions of their learning needs and is sensitive to change. Further work is needed to assess how PRISM compares with existing numerical scales and other appraisal tools used in dental education. Numerical and similar scales are convenient for students and teachers because they are simple to administer and evaluate . PRISM, although brief to administer, takes longer to administer and evaluate that a umerical scale. However, PRISM has the advantage that it can be readily incorporated into the appraisal discussion between student and teacher, as has been shown in others of its applications .
Strengths and limitations: The main limitation of the present work is that it is a pilot study, involving a small number of selected students and a single teacher. Comparisons between the year groups were limited because of their small numbers. The students were volunteers and might thus have been particularly motivated to work with PRISM. The teacher was well trained and experienced in the use of PRISM and some training will be necessary for other teachers applying PRISM. Also, without knowing whether the PRISM interviews were likely to work, it would have been wrong to integrate them into the students’ course, but this meant that the PRISM interviews were separate from and additional to the course. However, the results indicate the feasibility of applying PRISM to dental education. Nevertheless, further work is necessary to confirm the generalizability of the findings. As noted above, there is no gold standard for student self-appraisal, but further work should at least compare PRISM with other self-appraisal tools.
The current study has several key strengths. It evaluated a novel tool for self-assessment of competencies and learning needs in undergraduate dental students. Although the PRISM measure has been applied in a variety of settings, to the authors’ knowledge this is the first study in which PRISM has been applied in higher education. It is also the first study in which interpretation of PRISM has been extended to include clustering of the objects under investigation. The results of the study have been promising – even with the small numbers of students recruited, the PRISM task has been shown to be reliable and to yield results consistent with expectations.
This pilot study has demonstrated that PRISM has potential as a reliable tool to quantify perceived competencies in undergraduate dental students in their clinical years of study. Furthermore, PRISM generates a visual summary of each student’s perceived learning progress. Even allowing for the limitations of the study, its results suggest that use of PRISM would be a promising approach to extend the instruments for appraisal in dental education. If the generalizability of PRISM is confirmed among dental students, it should also be applicable to medical students as well as to those following postgraduate studies.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request. The data are not publically available, because of the psedonymisation and data protection guidelines according to the ethics approval.
Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA. 2006;296(9):1094–102. https://doi.org/10.1001/jama.296.9.1094.
Mann K, Gordon J, MacLeod A. Reflection and reflective practice in health professions education: a systematic review. Adv Health Sci Educ Theory Pract. 2009;14(4):595–621. https://doi.org/10.1007/s10459-007-9090-2 Epub 2007 Nov 23.
Williams JC, Ireland T, Warman S, Cake MA, Dymock D, Fowler E, Baillie S. Instruments to measure the ability to self-reflect: A systematic review of evidence from workplace and educational settings including health care. Eur J Dent Educ. 2019;23(4):389–404. https://doi.org/10.1111/eje.12445 Epub 2019 Jun 6.
Wu JH, Du JK, Lee CY. Development and questionnaire-based evaluation of virtual dental clinic: a serious game for training dental students. Med Educ Online. 2021;26(1):1983927. https://doi.org/10.1080/10872981.2021.1983927.
Botelho M, Bhuyan SY. Reflection before and after clinical practice-Enhancing and broadening experience through self-, peer- and teacher-guided learning. Eur J Dent Educ. 2021;25(3):480–7. https://doi.org/10.1111/eje.12623.
Krause F, Ziebolz D, Rockenbauch K, Haak R, Schmalz G. A video- and feedback-based approach to teaching communication skills in undergraduate clinical dental education: The student perspective. Eur J Dent Educ. 2021. https://doi.org/10.1111/eje.12682. Epub ahead of print.
Büchi S, Sensky T, Sharpe L, Timberlake N. Graphic representation of illness: a novel method of measuring patients’ perceptions of the impact of illness. Psychother Psychosom. 1998;67(4–5):222–5. https://doi.org/10.1159/000012284.
Sensky T, Büchi S. PRISM, a Novel Visual Metaphor Measuring Personally Salient Appraisals, Attitudes and Decision-Making: Qualitative Evidence Synthesis. PLoS ONE. 2016;11(5):e0156284. https://doi.org/10.1371/journal.pone.0156284.
Büchi S, Buddeberg C, Klaghofer R, Russi EW, Brändli O, Schlösser C, Stoll T, Villiger PM, Sensky T. Preliminary validation of PRISM (Pictorial Representation of Illness and Self Measure) - a brief method to assess suffering. Psychother Psychosom. 2002;71(6):333–41. https://doi.org/10.1159/000065994.
Lima-Verde AC, Pozza DH, Rodrigues LL, Velly AM, Guimarães AS. Cross-cultural adaptation and validation for Portuguese (Brazilian) of the pictorial representation of illness and self measure instrument in orofacial pain patients. J Orofac Pain. 2013;27(3):271–5. https://doi.org/10.11607/jop.1070.
Kassardjian CD, Gardner-Nix J, Dupak K, Barbati J, Lam-McCullock J. Validating PRISM (Pictorial Representation of Illness and Self Measure) as a measure of suffering in chronic non-cancer pain patients. J Pain. 2008;9(12):1135–43. https://doi.org/10.1016/j.jpain.2008.06.016 Epub 2008 Aug 28.
Fotiou K, Hofmann M, Kaufmann R, Thaci D. Pictorial representation of illness and self measure (PRISM): an effective tool to assess the burden of psoriasis. J Eur Acad Dermatol Venereol. 2015;29(12):2356–62. https://doi.org/10.1111/jdv.13232 Epub 2015 Aug 31.
Brady B, Veljanova I, Andary T, Southwell T, Chipchase L. Recognising ethnocultural diversity in chronic pain assessment: validation of the Pictorial Representation of Illness and Self Measure (PRISM) for use with culturally diverse communities. Health Qual Life Outcomes. 2019;17(1):56. https://doi.org/10.1186/s12955-019-1126-9.
Ghosh S, Sensky T, Casellas F, Rioux LC, Ahmad T, Márquez JR, Vanasek T, Gubonina I, Sezgin O, Ardizzone S, Kligys K, Petersson J, Suzuki Y, Peyrin-Biroulet L. A Global, Prospective, Observational Study Measuring Disease Burden and Suffering in Patients with Ulcerative Colitis Using the Pictorial Representation of Illness and Self-Measure Tool. J Crohns Colitis. 2020;15(2):228–37. https://doi.org/10.1093/ecco-jcc/jjaa159 Epub ahead of print.
Corazza M, Virgili A, Toni G, Valpiani G, Morotti C, Borghi A. Pictorial Representation of Illness and Self-Measure to assess the perceived burden in patients with chronic inflammatory vulvar diseases: an observational study. J Eur Acad Dermatol Venereol. 2020;34(11):2645–51. https://doi.org/10.1111/jdv.16637 Epub 2020 Jun 29.
Kabar I, Hüsing-Kabar A, Maschmeier M, Völler C, Dümke M, Schmidt HH, Heinzow H. Pictorial Representation of Illness and Self Measure (PRISM): A Novel Visual Instrument to Quantify Suffering in Liver Cirrhosis Patients and Liver Transplant Recipients. Ann Transplant. 2018;23:674–80. https://doi.org/10.12659/AOT.910278.
Goetzmann L, Seiler A, Benden C, Boehler A, Büchi S, Jenewein J, Ruettner B, Mueller-Alcazar A, Weierstall R. Transplantation experience as a predictor for quality of life during the first 6 months after lung transplantation. Clin Transplant. 2018;32(10):e13393. https://doi.org/10.1111/ctr.13393 Epub 2018 Oct 8.
Tanzawa T, Futaki K, Tani C, Hasegawa T, Yamamoto M, Miyazaki T, Maki K. Introduction of a robot patient into dental education. Eur J Dent Educ. 2012;16(1):e195–9. https://doi.org/10.1111/j.1600-0579.2011.00697.x Epub 2011 Jul 14.
O’Donnell JA, Oakley M, Haney S, O’Neill PN, Taylor D. Rubrics 101: a primer for rubric development in dental education. J Dent Educ. 2011;75(9):1163–75 PMID: 21890846.
The authors acknowledge support from Leipzig University for Open Access Publishing.
Open Access funding enabled and organized by Projekt DEAL. This study did not receive funding.
Ethics approval and consent to participate
The study protocol was reviewed and approved by the ethics committee of the medical faculty of University of Leipzig, Germany (No: 117/20-ek). All participants were informed verbally and in writing and provided written informed consent. The authors confirm that all methods were performed in accordance with the relevant guidelines and regulations and were performed in line with the Declaration of Helsinki.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Schmalz, G., Kullmann, H., Sensky, T. et al. Pilot study to evaluate a novel measure of self-perceived competencies among dental students. BMC Med Educ 22, 582 (2022). https://doi.org/10.1186/s12909-022-03663-6
- Dental education
- Visual metaphor
- Undergraduate education