Research article | Open | Open Peer Review | Published:
Evaluating differently tutored groups in problem-based learning in a German dental curriculum: a mixed methods study
BMC Medical Educationvolume 16, Article number: 14 (2016)
It is still unclear to what extent the PBL tutor affects learning in PBL-sessions. This mixed-methods study (Part 1 and 2) evaluated the effects of facilitative (f) versus non-facilitative (nf) tutoring roles on knowledge-gain and group functioning in the field of endodontics.
Part 1 was a quantitative assessment of tutor effectiveness within a prospective, experimental, single-blind, stratified, randomized, two-group intervention study. Participants attended PBL in the context of a hybrid curriculum. A validated questionnaire was used and knowledge assessments were conducted before and after the intervention. External observers rated tutor performance. Part 2 was a qualitative assessment of tutor effectiveness and consisted of semi-structured expert interviews with tutors and focus group discussions with students.
Part 1: f tutors obtained significantly higher scores than nf tutors with respect to learning motivation and tutor effectiveness (p ≤ 0.05). nf tuition resulted in a slightly larger knowledge gain (p = 0.08). External observers documented a significantly higher activity among facilitative tutors compared to non-facilitative tutors.
Part 2: Tutors found the f role easier although this led to a less autonomous working climate. The students rated f tutoring as positive in all cases.
With respect to PBL-group performance, students felt that groups guided in a non-facilitative fashion exhibited a higher level of independence and autonomy, especially with increasing PBL experience. In addition, students reported that more preparation was necessary for sessions guided by a non-facilitative tutor. Tutors were able to modify their role and influence group processes in a controlled manner. Results are useful for future “Train-the-Teacher” sessions.
In their 2012 guidelines entitled “Recommendations on higher education qualifications for the healthcare system” for the advancement of dental medicine at German universities, the German Council of Science and Humanities endorses the problem-oriented and patient-centred learning as a great step forward .
Since the introduction of problem-based Learning (PBL) in the 1960s, many empirical studies employing various evaluation methods have evaluated the effectiveness of PBL [2–17]. Most authors chose a qualitative approach using interviews of various types [18–20]. Only approximately 10 % of these studies employed the mixed-methods approach, although such studies are particularly informative because they address the research question on various methodological levels [21–24]. Literature suggests that it can be used as an effective approach, but this is not always the case . Actually it is known, that PBL aims to improve the hypothetico-deductive reasoning (HDR) of students. However, HDR brings with it a high cognitive load, so the full use of PBL is not advocated for medical curricula . In the literature, there is general agreement on the definition of PBL: small groups of students work interactively on the solution of problems while being supported by a tutor. Thereby, distinction is made between sessions necessitating the student’s presence in the group and periods of individual learning. The tutor is required to accompany, rather than control, the learning process. The role of the tutor may be described as conducive or facilitative . Facilitation requires understanding of the learning process, primarily involves monitoring of student learning and promotion of effective group function . In order to effect student learning in the small group PBL session, the facilitator must be informed about and be acutely aware of his/her role . Such performance was aptly described by Hamdy as follows: “The key is when, how and how much guidance should be provided” . Nonetheless, clear description of the extent of guidance and precise delineation of tutor performance are rare in the literature as are reports of controlled, clinical studies of PBL. Dolmans et al.  alerted to this gap and interpreted it as a guiding principle for future research. However few studies aiming to elucidate open questions on the tutor’s role have been published since this paper.
The present prospective controlled study attempted to evaluate precisely defined tutoring styles. To account for the complexity of the topic, we chose a mixed-methods study design. Part 1 (quantitative part) addressed the following questions:
RQ1: How do the different tutoring styles (facilitative and non-facilitative) affect learning motivation, tutor effectiveness, group dynamics, and learning success in PBL sessions for dental students attending a hybrid curriculum?
RQ2: Can external observers distinguish between the different tutoring styles, based on the tutors‘activities performed?
Part 2 of the study (qualitative part) attempted to answer the following question:
RQ3: How do students and tutors rate the different tutoring styles in the PBL sessions?
Sample-size calculation was based on an intermediate effect size (d = 0.5). For a one-sided test (i.e., assuming that the effects of facilitative tutoring are not equal to those of non-facilitative tutoring) at a power of 0.8, the sample size per study group amounted to n = 50.
Study population and exclusion criteria
Overall, 4 PBL tutors and 106 students (41 men, 65 women; mean age = 25.2 years) participated in the study. The students were attending the first clinical semester (sixth subject-specific semester) of the dental school during summer term 2008, winter term 2008, or summer term 2009. Table 1 shows the distribution of the study population, stratified by age and gender. Inclusion criteria for the tutors were several years of experience with PBL tutorials and a complete dental training including the license to practice. All participating students had to be attending the sixth subject-specific semester and were not allowed to have any experience with PBL tutorials or to have attended any lectures on the subject of endodontology at study start. Due to previously defined exclusion criteria, a total of n = 5 were excluded from the final study design, so that the study population consisted of n = 101. Prior to study start, the study protocol was approved by the local Ethics Committee (164/08).
The Dental School at the University of Frankfurt am Main, Germany employs a hybrid curricular structure that combines conventional (i.e., lectures, courses, seminars) and modern (i.e., eLearning, PBL, blended learning) teaching methods. None of the participating students had previously worked with PBL, while all tutors had several years of PBL experience and were established experts (i.e., content expert tutors) in both theoretical and practical endodontics. This module focused on endodontics is curriculary integrated and part of the first clinical semester.
Concept of PBL
The PBL module consisted of 7 steps, in accordance with the procedure used at the University of Maastricht . Steps 1 to 5 (1. Clarification of terms and concepts; 2. Formulation of problem statement; 3. Brainstorm; 4. Categorizing and structuring of brainstorm; 5. Formulation of learning objectives) were dealt with during Session 1, while Step 6 (Self-study) consisted of the individual processing of learning objectives. Step 7 (Post-discussion and reflection on learning process) took place in Session 2 scheduled 1 week after Session 1. Each PBL session lasted 90 min, group work in connection with each PBL case lasted 180 min. A total number of eight cases were included into this study.
Instruction of students
Prior to the planned PBL module entitled “Endodontics”, the students received verbal instruction lasting approximately 60 min, focusing on the following issues: What does PBL stand for? Definition of steps, timeframe, tips for literature search, allocation of roles in the group (time keeper; summarist; moderator; writer), and the role of the tutor. In addition, a video film about PBL was shown, and a handout was provided (32).
Training of tutors
All tutors (n = 4; 3 women, 1 man) underwent a “Train-the-Teacher” seminar (60 min) to refresh their PBL experience and to familiarize themselves with the facilitative (f) and non-facilitative (nf) tutoring styles (Table 2). The tutoring styles were defined in accordance with publications by Walsh  and Tuckman  as well as observations made on occasion of a PBL-guest visit at Charité, University of Berlin. At the end of the “Train-the-Teacher” seminar it was checked, if the tutors can correctly act in the different styles using video-analysis. The PBL module and the “Train-the-Teacher” seminar were supervised by an established expert (Master of Medical Education) in this field.
Quantitative part (Part 1)
Part 1 was a prospective, experimental, single-blind, stratified, randomized, two-group intervention study comparing a number of variables before and after the PBL module using knowledge assessments. The students were stratified for age and gender using the software RandList 1.2 (DatInf GmbH, Tübingen, Germany). They were randomly allocated to one of the two groups. Randomization codes were generated locally and were treated confidentially before study start. The study participants were not informed of the planned interventions (single-blind study design).
In each semester students were allocated. Each group worked on two different PBL cases. Not all students worked on each case (Fig. 1). However, the contents developed by the group were presented to all students of the semester in a plenary lecture session at the end of each PBL case study, ensuring that all participants were informed. Thus, all participants were able to complete the post-PBL test.
PBL was scheduled on 2 days (Tuesday and Thursday) each week with different groups. One week intervened between the two sessions of each group (for example from Tuesday to Tuesday or from Thursday to Thursday). The weekly timetable of the hybrid dental curriculum included few PBL and more plenary lecture sessions. On the respective PBL days (Tuesday and Thursday), two groups were taught in the same location/class room (sufficiently far away from each other). To prevent any detection of the differences in the tutoring style we ensured that both groups were either accompanied in the same style in this joint session. Groups were tutored by just one style (facilitative or non-facilitative) that was determined before the start of the study. However, the tutors acted in both a facilitative or non-facilitative fashion, depending on the group. The group members were unaware to which one of the two tutoring styles they had been allocated. Each PBL case consisted of eight learning objectives that were known to the tutors only. Thus, a total of 64 learning objectives were elaborated for the eight PBL cases.
Each PBL group received an external observer who confirmed the allocated tutoring role and its proper application by the tutor. Before study start, the external observer was trained to assess the frequency of activities initiated by the tutor or the group.
Pre- and post-PBL tests
Both groups underwent a written test (multiple-choice questionnaire) before and after the PBL module to assess the participants’ level of knowledge. Neither test was announced. Both tests consisted of 40 multiple-choice questions that were different in the pre- and post-PBL tests. The tests were to be completed within 20 min in an anonymized fashion, ensuring however that the pre- and post-PBL tests could be matched for each participant. The contents of the tests remained unchanged during the study period (three semesters). Values of Cronbach‘s alpha amounted to 0.63 and 0.67 for the pre- and post-PBL tests, while the values for the difficulty of the pre- and post-PBL tests were 0.40 and 0.53, respectively.
Effectiveness of the tutoring styles was assessed on the basis of a questionnaire completed by the students. The contents of the questionnaire (n = 60 questions) were based on studies by Visschers-Pleijers et al.  and Dolmans & Ginns  and matched the PBL questionnaire used at Charité, Berlin . The questionnaire consisted of three sections (i.e., group interaction, tutor effectiveness, general comments on PBL), with a section for additional remarks provided (Table 3). Questions were answered using a Likert scale ranging from one to five (1 = completely disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = completely agree). In addition, an observation chart (“manipulation check“) was constructed on which the external observer recorded tutor performance and his/her activities as well as those of the group.
Qualitative part (Part 2)
The participating tutors were questioned about their experience with PBL by means of expert interviews that were half-standardized (semi-structured), i.e., the interviews were conducted using a guideline that only provided subject-related questions (Table 4).
One PBL group consisting of randomly allocated participants (n = 8 students) was exposed to both tutoring styles (facilitative or non-facilitative) in several sessions. These students participated in a focused group discussion to share their view about the two tutoring styles (Table 5). The setting of an open discussion allowed each participant to freely express his/her personal view and impression.
The guidelines for expert interviews and the focused group discussion were prepared in collaboration with the Institute of Psychology, Department for Educational Psychology of the Goethe University of Frankfurt am Main. All interviews and the focused group discussion were recorded in both audio and video forms, using a HDC-SD40EG-K Full HD Camcorder (Panasonic K. K., Kadoma, Japan).
The quantitative data (Part 1) were evaluated in line with the intention-to-treat (ITT) principle. The following analyses were conducted:
quality criteria of the multiple-choice tests
analysis of study questionnaire, determination of reliability values and factors
explorative data analysis
mixed-model analysis (analysis of variance), taking group and tutor effects into account (significance level p ≤ 0.05).
All data were statistically analyzed using the software SAS 9.1 (SAS Institute Cary, NC, USA).
In Part 2 of the study (qualitative data), the recorded expert interviews and focused group discussion were transcribed, and relevant information was grouped by topic. The codes generated in this way were processed with the program MAXQDA (Verbi GmbH, Marburg, Germany) and analyzed in terms of a “thematic analysis” . Two independent individuals who had been trained beforehand established the topic categories. These were based on interpretation of terms in the context in which they had been mentioned.
Quantitative part (Part 1)
The mean response rate for the total study population was 95.04 %, with a mean dropout rate amounting to 4.96 %.
Activities of tutors and groups as noted by the external observers
Table 6 lists the findings with respect to tutor and group activities as perceived by the external observers.
Determination of the proper values for questions 1 to 60 in the explorative analysis of the questionnaire proved a 3 or 8-factor solution to be appropriate. The 8-factor approach allows a more detailed analysis; thus it was used for subsequent analyses. Table 7 shows the reliability estimates (Cronbach’s alpha), mean data, and p-values for the analyzed factors with the corresponding items of the questionnaire. Support by the tutor, tutor effectiveness, and motivation were significantly better in groups receiving facilitative tutoring than those receiving non-facilitative tutoring.
The dimension “tutor effectiveness” consisting of 5 subdimensions achieved mean (± SD) values ranging from 3.77 ± 0.82 (context-relevant learning) to 4.05 ± 0.63 (intrapersonal behaviour). For facilitative tutoring, the subdimensions reached the following mean scores: constructive, active learning: 4.08 ± 0.76 (versus 3.77 ± 0.85 for non-facilitative tutoring); self-driven learning: 4.22 ± 0.78 (versus 3.83 ± 0.92); context-relevant learning: 3.92 ± 0.79 (versus 3.62 ± 0.82); collaborative learning: 4.15 ± 0.78 (versus 3.87 ± 0.80); and intrapersonal behavior: 4.15 ± 0.80 (versus 3.96 ± 0.84).
Statistical analysis of pre- and post-PBL tests
The overall results of the post-PBL test were significantly better than those of the pre-PBL test. The students reached a mean score of 16.05 ± 4.49 points in the pre-PBL test, and a mean of 21.38 ± 4.69 points computed for the post-PBL test (p ≤ 0.0001). After the pre-PBL test the students were stratified into the facilitative (with a mean of 16.30 points) and non-facilitative group (with a mean of 15.82 points) with no statistically significant difference (p = 0.590) between them.
In the post-PBL test both groups received significantly more correct answers than the respective pre-PBL group (p ≤ 0.05). Comparing the post-PBL results in the two tutoring styles (nf = 22.2 correct answers, f = 20.5 correct answers) no significant difference could be observed (p = 0.08).
Qualitative part (Part 2)
Table 8 shows the outcome (frequency of responses) of the focused group discussion with students and expert interviews with tutors. Additionally most tutors indicated that non-facilitative tutoring was difficult to apply (n = 7) and that previous training was necessary (n = 7). Irrespective of the tutoring style in their group, the students expressed uncertainty about the PBL procedure (n = 14). Moreover, the group composition was named to significantly influence the procedure (n = 8), and marked focusing on the tutor was observed (n = 2).
Similarly, the tutors considered PBL a challenging learning tool (n = 8) in which the group composition plays an important role (n = 3). They also concluded that PBL is easier to apply (n = 1) and requires less preparation time than classical teaching (n = 4). Suggestions for improving PBL included more practicing of the two different tutoring styles (n = 18), prolonged PBL meeting times (n = 3), and use of additional PBL cases (n = 4).
In the groups tutored non-facilitatively, tutors observed a high degree of student autonomy during the PBL sessions, particularly after the second session. They confirmed that “students found out very many things on their own in the second session“, and “needed less eye contact“. Furthermore, tutors concluded that the facilitative role is more suitable in a hybrid curriculum.
Focus group discussion
Table 8 shows the students‘assessment of tutoring styles and their effects on the PBL process. Additionally the students rated PBL as practice-oriented learning (n = 6) and favored the facilitative tutoring style (n = 20), with only one student rating the two tutoring styles as equally effective. Comments on the facilitative tutor were as follows: The facilitative tutor “tries to guide things into the right direction“; “is considered“, “does not use a poker face“, “was a supervisor and guide at the same time”; “provided feedback”; and “gave us an opportunity to ask questions”. In contrast, the non-facilitative tutor “was straight-faced“; tried to lead us astray“; and “solely acted as physician or patient”.
Suggestions for improving the module mostly concerned organizational issues (n = 10), followed in frequency by proposals of modifying the module contents (n = 6).
The students described the facilitative tutor role as a “guideline that leads you along“, and realized that “it is not that bad if you say the wrong thing“. In addition, they appreciated the fact that this tutoring style “lets you know whether you go off in the wrong direction”. The primary effect of non-facilitative tutoring on the group was uncertainty with respect to the PBL procedure. The students named the more practical mode of learning in PBL modules as the main advantage of PBL over conventional learning. Practice-oriented application of theoretical topics were considered “good training for work with the patient“; “attempts to solve problems in a practical way”, and a “stimulating exercise”. In the clinical sense, PBL was regarded as “more effective“, and “the awareness for practical problems is being triggered”. Students stated that for the first PBL cases, they would wish to have a facilitative tutor who subsequently “becomes increasingly non-facilitative”.
The tutor influences the success of PBL-groups. However the best way to perform the tutor’s role is a matter of debate. We compared two different tutor-styles. The facilitating tutor was more active, intervened and guided the group. The non-facilitating tutor only intervened in certain situations and kept still most of the time (see Tables 2 and 3).
There is general agreement in the available literature that PBL strengthens the learning motivation [2, 3, 6, 9]. In his flow theory, Csikszentmihalyi stated that “the structure of activity in the context of challenge, goal, feedback, concentration, and control has major influences on intrinsic motivation [38, 39]. “Flow” is defined as “the holistic experience that people feel when they act with total involvement”. In line with these statements, our study revealed a significant difference in the variable “motivation“between the two tutoring styles in favor of facilitative tutoring (4.01 ± 0.66 versus 3.85 ± 0.46).
The style of PBL tutoring affects the group work in a direct manner. For the present study, we used some of the dimensions published by Dolmans & Ginns . In line with the PBL form used at Charité, Berlin, our questionnaire consisted of 60 items. Some items (items 47–55) were phrased in a way that may have favored the facilitative style, which should be critically stated. The highest values in the rating of tutors occurred in the subdimension “intrapersonal behavior”. Facilitative tutors were more effective in “generating clear learning objectives“encouraging “the search for additional sources of information“, which was reflected in the group members’ assessment of tutors. To optimize group work, Wood  suggested allocating specific roles to the group members before sessions (e.g., reader, writer, time keeper, moderator, and summarist). We used these roles also in the present study.
PBL evaluation consists of assessing the students as well as the program [14, 41]. Such evaluations are difficult to conduct, and several differing approaches have been described in the literature. By employing multiple-choice tests, Budé et al.  demonstrated a significant difference (p = 0.072) between so-called directive and traditional PBL modules, with the directive programs achieving a better result. In our study, knowledge gain in the post-PBL test was significant in both groups, reflecting the newly acquired knowledge after completing the PBL module. There was a trend towards better results in the group tutored non-facilitatively, but the difference did not reach statistical significance.
In the expert interviews, tutors expressed clear difficulties with their non-facilitative role. Moreover, tutors documented more problems in implementing a non-facilitative tutoring style with PBL beginners, as apparent by the following comments: “My personal experience is that the non-facilitative role is not easy for a student who has never participated in a PBL module“; “If group members have already undergone a PBL session, I feel comfortable in either role”; or “The first session was really difficult when assuming the non-facilitative role”. The comment “The differences between the tutor roles and their practical implementation were not entirely clear in the role description“suggests that there was a need for more extensive tutor training. In line with this observation, the need for more extensive tutor training is emphasized in the literature [43–46].
With respect to internal study validity, the aspects to be considered include maturity, selection of study participants, study location, instrumentation, and the special study design used. During the 8-week PBL module „Endodontics“, students attended multiple other lectures and courses that were not directly connected with the topic of the PBL module but may have indirectly affected the monitoring of learning objectives. To minimize this effect, all programs on similar topics organized by the outpatient clinic for restorative dentistry were scheduled to take place after completion of the PBL module. Maturation’ may possibly also have affected the tutors involved in the study. It cannot be ruled out that the tutors became increasingly accustomed to their specific role during the study. This factor was controlled by the external observers who confirmed the assigned roles. To reduce potential problems associated with the study location, all PBL sessions were conducted in the same room (simulation laboratory) and took place on the same week day (Tuesday) at the same time (2 p.m. to 5 p.m.).
The issue of instrumentation concerned the two written tests that were conducted before and after completion of the PBL module. Since these had to remain the same throughout the three semesters, the questions and answers were not published or discussed at any time. For each semester, the test documentation contained identical questions, but they were presented in a different order than those of the previous semester. All tests were conducted without announcing them; neither the students nor the tutors knew the exact date of the tests.
The four tutors taking part in the study were responsible for the implementation of the different roles during the PBL sessions. All of them had many years of experience with PBL. However, it was feared that some of them might leave the university before the end of the study because of an expiring contract, which constituted a substantial complication in the planning of the study. The chosen study design (“nested“design) appeared to provide the best condition for this potential complication in that it allowed for a maximum size of factor variance. For this reason, the tutors had to switch roles each time, and the various PBL cases were exchanged during the module. In this way, replacing a tutor would have been less critical.
The chosen study design also addressed the potential problem of implementation, i.e., the specific selection of a role for each tutor and free allocation of students to a tutorial. Any interaction between groups (tutored by a facilitative or non-facilitative style) was minimized by the simultaneous scheduling of PBL-sessions with only the same tutoring style. Thus, potential variation in tutor performance was not evident in the parallel sessions. Any confounding by student interaction outside the PBL module was kept to a minimum by blinding the participants with respect to the study variables.
Our study design consisting of pre- and post-PBL assessments may be associated with certain limitations in connection with the testing condition . There is a danger that students anticipate the tests and will therefore not be unprepared when attending PBL sessions. Moreover, based on their experience with the nature of information and type of questions in the pre-PBL test, participants can prepare more effectively for the post-PBL test than for the pre-PBL test. Nonetheless, Fraenkel & Wallen  also point to the advantages of this study design; it provides information on the group composition with respect to the knowledge level and consequently on the comparability of the two groups. The authors point out that this study design is particularly useful if groups contain no more than 30 participants. With as many as 101 participants in our study, this limit was clearly exceeded. The large group size in our study resulted from the combination of three semesters. Consequently, pre-PBL testing was of critical importance to ensure comparable knowledge levels among the groups. In our study, the groups were, in fact, comparable with respect to their current knowledge on the subject.
An additional aspect of debate is the number of study participants. Within the study time frame, it was not possible to recruit more students (total number of students per semester = 48) and tutors (total number of experienced PBL tutors available at the entire dental school = 5) at our institute. Sample-size calculation prior to study start yielded a number of at least 50 participants per group (facilitative versus non-facilitative), but this number could not even have been met by adding students from an additional semester. Nonetheless, it would be desirable to confirm our findings in subsequent studies involving larger study populations.
With respect to external validity of the study, generalizability of our findings and the special setting of a replication study are of interest. To ensure that our results can be transferred to other persons, situations, conditions, and time frames, we allocated students to one of two groups by stratified randomization. Moreover, the low dropout rate increased the external validity of this study. Further strengthening of the external validity resulted from the nature of our evaluation that can be viewed as a replication study. Because the study ran over a period of three semesters, findings were transferable to other study subjects as different students were involved.
The “readiness” of tutors and students underwent a certain process. Initially, the facilitative style was favored by both the tutors and students. In the Results section we state that the facilitative tutoring style had resulted in less self-directed learning by the end of the PBL experiments. The tutors mentioned that the non-facilitative tutoring style greatly increased the students’ ability to learn on their own.
Within the limitations of the study that was based on a hybrid curriculum, we conclude that the tutor style should be modified during PBL training. Initially, the facilitative style may be more suitable, but the non-facilitative tutoring style should be introduced when students have gained sufficient knowledge in PBL. We believe that this approach would be of optimal long-term benefit to both students and tutors participating in PBL programs.
German Council of Science and Humanities. Recommendations on higher education qualifications for the healthcare system. Berlin 2012. Available from : http://www.wissenschaftsrat.de/download/archiv/2411-12_Executive-Summary.pdf
Albanese MA, Mitchell S. Problem-based learning: a review of literature on its outcomes and implementation issues. Acad Med. 1993;68:52–81.
Vernon DT, Blake RL. Does problem-based learning work? A meta-analysis of evaluative research. Acad Med. 1993;68:550–63.
Berkson L. Problem-based learning: have the expectations been met? Acad Med. 1993;68:S79–88.
Kasim R. What can studies of problem-based learning tell us? Synthesizing and modeling PBL effects on national board of medical examination performance: hierarchical linear modeling meta-analytic approach. Adv Health Sci Educ. 1999;4:209–21.
Colliver JA. Effectiveness of problem-based learning curricula: research and theory. Acad Med. 2000;75:259–66.
Smits PB, Verbeek JH, de Buisonje CD. Problem based learning in continuing medical education: a review of controlled evaluation studies. BMJ. 2002;324:153–6.
Dochy F, Segers M, Van den Bossche P, Gijbels D. Effects of problem-based learning: a meta-analysis. Learn Instr. 2003;13:533–68.
Newman M. A pilot systematic review and meta-analysis on the effectiveness of problem based learning: on behalf of the Campbell Collaboration Systematic Review Group on the effectiveness of Problem Based Learning: Catherine Cookson Centre for Medical Education and Health Informatics. Available from:http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.133.6561&rep=rep1&type=pdf
Gijbels D, Dochy F, Van den Bossche P, Segers M. Effects of problem-based learning: a meta-analysis from the angle of assessment. Rev Educ Res. 2005;75:27–61.
Polyzois I, Claffey N, Mattheos N. Problem-based learning in academic health education. A systematic literature review. Eur J Dent Educ. 2010;14:55–64.
Townsend G, Winning T. Research in PBL – where to from here for dentistry? Eur J Dent Educ. 2011;15:193–8.
Schmidt HG, Rotgans JI, Yew EH. The process of problem-based learning: what works and why. Med Educ. 2011;45:792–806.
Townsend GC, Kim M, Sankey D. Dynamic systems (Complexity) theory as a new conceptual model for researching PBL in dental education. Eur J Dent Educ. 2012;16:43–51.
Huang B, Zheng L, Li C, Li L, Yu H. Effectiveness of problem-based learning in Chinese dental education: a meta-analysis. J Dent Educ. 2013;77:377–83.
Bate E, Hommes J, Duvivier R, Taylor DC. Problem-based learning (PBL): getting the most out of your students - their roles and responsibilities: AMEE Guide No. 84. Med Teach. 2014;36:1–12.
Hayashi S, Tsunekawa K, Inoue C, Fukuzawa Y. Comparison of tutored group with tutorless group in problem-based mixed learning sessions: a randomized cross-matched study. BMC Med Educ. 2013;13:158.
Bearn DR, Chadwick SM. Problem-based learning in postgraduate dental education: aqualitative evaluation of students’ experience of an orthodontic problem-based postgraduate programme. Eur J Dent Educ. 2010;14:26–34.
Watmough SD, O’Sullivan H, Taylor DC. Graduates from a reformed undergraduate medical curriculum based on Tomorrow’s Doctors evaluate the effectiveness of their curriculum 6 years after graduation through interviews. BMC Med Educ. 2010;10:65.
Lee GH, Lin CS, Lin YH. How experienced tutors facilitate tutorial dynamics in PBL groups. Med Teach. 2013;35:935–42.
Grant A, Kinnersley P, Field M. Learning contexts at Two UK medical schools: a comparative study using mixed methods. BMC Res Notes. 2012;19:153.
Kamp RJ, Dolmans DH, Van Berkel HJ, Schmidt HG. The effect of midterm peer feedback on student functioning in problem-based tutorials. Adv Health Sci Educ Theory Pract. 2013;18:199–213.
Maudsley G, Williams EM, Taylor DC. Problem-based learning at the receiving end: a ‘mixed methods’ study of junior medical students’ perspectives. Adv Health Sci Educ Theory Pract. 2008;13:435–51.
Woltering V, Herrler A, Spitzer K, Spreckelsen C. Blended learning positively affects students’ satisfaction and the role of the tutor in the problem-based learning process: results of a mixed-method evaluation. Adv Health Sci Educ Theory Pract. 2009;14:725–38.
Qiao YQ, Shen J, Liang X, Ding S, Chen FY, Shao L, et al. Using cognitive theory to facilitate medical education. BMC Med Educ. 2014;14:79.
Azer SA. Problem-based learning: where are we now? Guide supplement 36.1 - viewpoint. Med Teach. 2011;33:121–2.
Groves M, Régo P, O’Rourke P. Tutoring in problem-based learning medical curricula: the influence of tutor background and style on effectiveness. BMC Med Educ. 2005;5:20.
McLean M. Clinical role models are important in the early years of a problem-based learning curriculum. Med Teach. 2006;28:64–9.
Hamdy H. The fuzzy world of problem based learning. Med Teach. 2008;30:739–41.
Dolmans DH, Gijselaers WH, Moust JH, de Grave WS, Wolfhagen IH, van der Vleuten CP. Trends in research on the tutor in problem-based learning: conclusions and implications for educational practice and research. Med Teach. 2002;24:173–80.
Maurer H, Neuhold C. Problems Everywhere? Strengths and Challenges of a Problem-Based Learning Approach in European Studies. Paper prepared for the Higher Education Academy Social Science Conference “Ways of Knowing, Ways of Learning” 28 and 29 May 2012. Liverpool. Available from:http://www.mcegmaastricht.eu/pdf/MCEG_part%20PBL_link2_%20PBL%20implementation%20challenges.pdf
Walsh A. The tutor in problem-based learning: a novice’s guide. Hamilton: McMaster University; 2005. Available from: http://fhs.mcmaster.ca/facdev/documents/tutorPBL.pdf.
Tuckman BW. Developmental sequence in small groups. Psychol Bull. 1965;63(6):384–99.
Visschers-Pleijers AJ, Dolmans DH, Wolfhagen IH, Van der Vleuten CP. Development and validation of a questionnaire to identify learning-oriented group interactions in PBL. Med Teach. 2005;27:375–81.
Dolmans DH, Ginns P. A short questionnaire to evaluate the effectiveness of tutors in PBL: validity and reliability. Med Teach. 2005;27:534–8.
Spiess C. Development of a PBL-mentoring program in preparation for the model course including a status analysis of the faculty role and implementation of the method PBL. 2008. Available from: http://www.reformstudiengang-medizin.de/attachment/1d598b30fe438f8e6a4d33a28e495d40/d173c66177a98611d1cd17dae7808a1e/Entwicklung%20eines%20POLMentorenprogramms%20in%20Vorbereitung%20auf%20den%20Modellstudiengang%20inkl.%20einer%20Statusanalyse%20der%20Dozentenroll.-e%20und%20Umsetzung%20der%20Methode%20POL in german language
Mayring P. Combination and Integration of Qualitative and Quantitative Analysis. Forum Qual Res 2(1):6. Available from: http://www.qualitative-research.net/index.php/fqs/article/view/967
Csikszentmihalyi M. Beyond boredom and anxiety. Experiencing flow in work and play. San Francisco: Jossey Bass Wiley; 2000.
Van Berkel H, Schmidt H. Motivation to commit oneself as a determinant of achievement in problem-based learning. High Educ. 2000;40:231–42.
Wood D. Problem based learning. In: Cantillon P, Hutchinson L, Wood D, eds. ABC of learning and teaching in medicine 2003: Vol 326. London: BMJ Electronic Production 8–12. Available from: http://edc.tbzmed.ac.ir/uploads/39/CMS/user/file/56/scholarship/ABC-LTM.pdf
Watmough S, Charry MG, O’Sullivan H. A comparison of self-perceived competencies of traditional and reformed curriculum graduates 6 years after graduation. Med Teach. 2012;34:562–8.
Budé L, Imbos T, Wiel M, Broers N, Berger M. The effect of directive tutor guidance in problem-based learning of statistics on students’ perceptions and achievement. High Educ. 2009;57:23–36.
Grand Maison P, des Marchais JE. Preparing faculty to teach in a problem-based learning curriculum: the Sherbrooke experience. Can Med Assoc J. 1991;144:557–62.
Maudsley G. Roles and responsibilities of the problem based learning tutor in the undergraduate medical curriculum. Br Med J. 1999;318:657–61.
Wetzel MS. Developing the role of the tutor/facilitator. Postgrad Med J. 1996;72:474–7.
McLean M. What can we learn from facilitator and student perceptions of facilitation skills and roles in the first year of a problem-based learning curriculum? BMC Med Educ. 2003;3:9.
Frankel JR, Wallen NE. How to design and evaluate research in education. New York: Mc Graw-Hill Humanities; 2008.
Thanks are due to Dr. Silvia M. Rogers (MediWrite GmbH, Basel, Switzerland) for editing and proofreading the manuscript. The present study was prized with the Dental Education Award (DEA) of the German Society of Dentistry and Oral Medicine in 2012.
The authors declare that they have no competing interests.
All authors read and approved the manuscript and have made substantive intellectual contributions to the published study. SG and FO proposed the PBL method, designed the experiment and evaluation method and played a leading role in writing the article. FK designed and carried out the questionnaire for the qualitative part of the study. AM collected and analyzed the data. AB helped draft and critically appraise the manuscript. MH helped design and carry out the theoretical frame for the qualitative part of the study. SG and SR revised the manuscript based on reviewers’ reports. All authors read and approved the final manuscript.
About this article
- Clinical tutor
- Faculty development
- Focus groups
- Mixed methods research
- Problem-based learning