- Research article
- Open Open Peer Review
Effectiveness of problem-based learning in Chinese pharmacy education: a meta-analysis
BMC Medical Educationvolume 16, Article number: 23 (2016)
This review provides a critical overview of problem-based learning (PBL) practices in Chinese pharmacy education. PBL has yet to be widely applied in pharmaceutical education in China. The results of those studies that have been conducted are published in Chinese and thus may not be easily accessible to international researchers. Therefore, this meta-analysis was carried out to review the effectiveness of PBL.
Databases were searched for studies in accordance with the inclusion criteria. Two reviewers independently performed the study identification and data extraction. A meta-analysis was conducted using Revman 5.3 software.
Sixteen randomized controlled trials were included. The meta-analysis revealed that PBL had a positive association with higher theoretical scores (SMD = 1.17, 95 % CI [0.77, 11.57], P < 0.00001). The questionnaire results show that PBL methods are superior to conventional teaching methods in improving students’ learning interest, independent analysis skills, scope of knowledge, self-study, team spirit, and oral expression.
This meta-analysis indicates that PBL pedagogy is superior to traditional lecture-based teaching in Chinese pharmacy education. PBL methods could be an optional, supplementary method of pharmaceutical teaching in China. However, Chinese pharmacy colleges and universities should revise PBL curricula according to their own needs, which would maximize the effectiveness of PBL.
Problem-based learning (PBL) is an educational innovation developed and first implemented in the 1960s in response to dissatisfaction with conventional medical education practices [1–5]. Many researchers have investigated the effectiveness of PBL in medical school curricula [6, 7] and findings indicate that PBL can contribute to knowledge retention, student satisfaction, motivation, and critical thinking [8, 9]. The first published application of PBL in pharmacy was a patient-oriented problem-solving instruction module for pharmacy students developed in 1983 by Love and Shumway . Since then, PBL has been applied in pharmaceutical education courses and several studies have reported results from the use of this educational method [4, 11, 12].
However, PBL is not often used in Chinese pharmacy education. There are several reasons for this. First, pharmacy tuition fees are less than 10,000 RMB (around 1600 USD). The Chinese government provides all the equipment and materials that pharmacy students require. The application of PBL is restricted by a shortage of educational equipment and materials. Second, both the standards and levels of pharmacy education vary significantly among different institutions in China. Third, almost all education is characterized by traditional educational methods that have been used for decades. Traditional education is teacher-oriented and places too much emphasis on disseminating factual knowledge and on passive student learning. Fourth, Chinese curricula are arranged differently from those of other countries. After completing all the pharmacy courses in 3 to 3.5 years, pharmacy students begin practical skills training. As teaching courses are limited, pharmacy colleges and universities tend to employ conventional teaching methods rather than innovative methods such as PBL. Furthermore, humility and courtesy are important qualities of Chinese communication. Research based on the Myers-Briggs Type Indicator (which measures differences in individuals’ psychological preferences for particular cognitive functions, such as perception and judgment ) has shown that Chinese pharmacy students with stronger introversion gain higher grades than those scoring strongly on extraversion . Because Chinese people are taught to be humble and to comply with their elders, they are more likely to express an introspective, quiet, and conservative personality . Studies using the Myers-Briggs learning styles have shown that pharmacy service professionals in Western countries have an obvious preference for the judging style [16, 17]. Introverted individuals participate less in open discussions because they are reluctant to express their opinions, especially if their opinions differ from those of others.
Because of the reasons discussed, China has been slower to incorporate PBL into pharmacy education than other countries, such as the United States. As there is great demand for highly qualified pharmacists in China, curriculum reform that includes the application of PBL teaching methods is urgently needed. In recent years, many Chinese pharmacy institutions have made tentative steps in PBL pedagogy. Studies on the effectiveness of PBL methods in pharmacy education in China have reported positive outcomes [18–33]. One recent systematic review and meta-analysis shows that the PBL method improved pharmacy students’ knowledge in the United States, the United Kingdom, and Canada . However, the benefits of PBL for undergraduate pharmacy students in China have not been clearly disseminated. Papers on PBL in Chinese pharmacy education have not been published in English and therefore cannot be accessed by non-Chinese-speaking researchers. The aim of this meta-analysis was to review the effectiveness of PBL in Chinese pharmacy education and to disseminate this research more widely to international education researchers. To evaluate the overall effectiveness of studies of PBL pedagogy, we used only studies conducted in China and focused on student-centered pharmacy education programs rather than traditional teaching methods.
We conducted a systematic review and meta-analysis following the guidelines of the Cochrane Handbook for Systematic Reviews of Interventions  and the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) 2015 statement recommendations (Additional file 1) .
We searched the electronic databases Chinese National Knowledge Infrastructure (CNKI), VIP Information (Chinese database), Wanfang Data (Chinese database), and Chinese Biomedical Literature (CBM). English-language computerized databases, such as PubMed, EMBASE, and the Cochrane Database, were also searched. Next, the reference list of selected articles was reviewed for additional related reports. The search was restricted to the period 1965 to December 2014, as PBL originated at the McMaster School of Medicine in Canada in 1965 . The mesh-terms or key words (“problem-based learning”) AND (“pharmac*” OR “pharmac* education” OR “pharmac* students”) were used for the search string. The literature search was conducted in April 2014; we updated the search on November 29, 2014. A total of 419 abstracts were retrieved.
Two reviewers independently selected the studies; any discrepancies were resolved by discussion. The studies were first selected according to the title and abstract. We included articles if they (1) investigated students of pharmacy institutions in China; (2) described randomized controlled trials; (3) included pharmacy students from nonpharmacy institutions; (4) used PBL as an educational approach in the intervention group; (5) used traditional lectures in the control group and exposed neither group to supplementary teaching methods that could have an impact on the results; (6) evaluated theoretical scores and questionnaires as outcomes; and (7) reported the sample size and the mean difference in theoretical scores for the intervention group and control group. Articles were excluded if they (1) were non-randomized controlled trials; (2) included subjects other than pharmacy students; (3) utilized interventions other than PBL; and (4) had incomplete data, such as not reporting the mean difference in theoretical scores. We finally selected 13 randomized controlled trials.
Data were extracted by two independent reviewers. Disagreements about eligibility were resolved by consensus. For each study, the following information was extracted in the current analysis: the first author, publication year, the pharmacy disciplines involved, sample size (intervention group and control group), characteristics of participants, intervention method, teaching method for the control group, outcomes (the mean difference in theoretical examination scores for the intervention group and control group), time of outcome measure, outcomes measured, and length of intervention.
The quality of the included studies was assessed as adequate, uncertain, or inadequate by two reviewers and was based on the six general sources of bias described in the Cochrane Handbook for Systematic Reviews of Interventions . The following quality items were checked: adequacy of the generation of the allocation sequence, concealment of allocation, blinding procedures, incomplete outcome data, selective outcome reporting, and other sources of bias. Information addressed by these items was obtained from the published reports and authors were contacted if additional information was required . The Cochrane Collaboration’s tool for assessing risk of bias is available on line at http://handbook.cochrane.org/ .
Review Manager 5.3 software  was used to test the data for heterogeneity and to carry out the meta-analysis. We analyzed the theoretical scores of the PBL groups and the control groups. As continuous data from different scales were extracted, the standardized mean difference (SMD) was calculated for effect size based on sample size  and 95 % confidence intervals for each study, and for the pooled studies using variance analysis. Weighted mean differences and 95 % confidence intervals were calculated for continuous data from the same scale. A two-sided P value of less than 0.05 was regarded as significant for all analyses. We used two meta-analysis models. A fixed-effects model was used to pool data if there was no heterogeneity, otherwise we used a random-effects model. Heterogeneity was considered significant for a P value of Cochran’s Q statistic <0.10 and I2 > 50 % [39, 40]. I2 is the percentage of variation attributed to heterogeneity and is easily interpreted. An I2 statistic of 25–50 % was considered low, 50–75 % was considered moderate, and ≥75 % was considered high.
If heterogeneity was revealed, we conducted a sensitivity analysis to assess if this significantly altered the results of the meta-analysis. We performed the sensitivity analysis by excluding studies associated with heterogeneity and then recalculating the pooled estimates for the remaining studies. However, this did not significantly alter the results.
Publication bias was assessed with a funnel plot and Egger’s test of asymmetry  using STATA 13.0. The funnel plot shapes did not reveal any obvious evidence of asymmetry, and all P values for Egger’s tests were more than 0.05, providing statistical evidence of the funnel plot symmetry.
The search results
The literature search identified 419 abstracts, out of which 166 duplicates were removed. A further 48 articles were excluded after reading the titles and abstracts, among which 40 were review articles and 96 were not relevant to this investigation. In accordance with the inclusion and exclusion criteria, relevant full-text articles (n = 56) were assessed for eligibility. Eighteen studies did not measure theoretical scores and only used questionnaires, six did not include a control group, three used PBL and other teaching methods for the intervention group, four were non-randomized controlled trials, and nine did not report detailed outcome data. We e-mailed the corresponding authors to ask for the available outcome data but received no reply. Thus, 16 articles were included in the meta-analysis. The data abstraction process is shown in Fig. 1.
Characteristics of included studies
Sixteen articles [18–33] representing 17 studies were included in the meta-analysis. They were all published between 2006 and 2014 and were in Chinese. The sample sizes ranged from 28 to 112 participants and the pooled sample size was 1826 (PBL group = 929, control group = 897). The studies examined teaching methods in different areas of pharmacy, as follows: three studies of pharmaceutical analysis [20, 24, 28]; three of natural medicine chemistry [18, 30, 33]; two of biopharmaceutics and pharmacokinetics [22, 31]; one of pharmaceutical formulation design ; one of pharmaceutical affairs, law, and regulation ; one of Chinese materia medica pharmacology ; one of basic pharmacy ; one of pharmaceutical botany ; one of pharmaceutical literature retrieval ; one of organic chemistry ; and one of pharmaceutical molecular biology . All study outcomes were measured using theoretical scores and questionnaire surveys after PBL teaching. The length of intervention varied from one semester to two semesters. Table 1 shows the characteristics of the 16 included studies.
Table 2 shows the risk of bias assessment of the 16 included studies. Most studies had a low risk of bias across the six domains. The allocation sequence in one study  was adequately generated by random numbers. In another study , the allocation sequence was generated by odd and even numbers based on the college entrance examination scores. The allocation sequence of four studies [21, 23, 28, 31] was based on the preference of the researchers, who decided which students to assign to the experimental or control groups. The other 10 studies did not report the allocation method used, so this information was not available. So it was not possible to assess the blinding of the students and tutors to the interventions. All studies reported complete outcome data. We assessed whether each study was free from selective outcome reporting by checking whether all outcomes mentioned were adequately reported in the results section. All studies adequately reported the results. All studies seemed free from “other sources of bias” as defined in the Cochrane Collaboration’s domain-based evaluation.
Effects of interventions on theoretical examination scores
The effects of the PBL methods were evaluated using both quantitative data and description. Sixteen studies involving 1826 participants (PBL group = 929, control group = 897) reported theoretical scores. Yang and Li  reported two types of outcome data; thus, there were 17 reports of outcome data in the meta-analysis. This meta-analysis reviews the effects of PBL on theoretical scores, which are more objective than other indices. All studies showed statistically significant differences between the PBL and the control groups in pharmacy students’ theoretical scores. There was high heterogeneity (I2 = 93 %, P < 0.00001); thus, the random-effects model was used. The pooled effect size showed a significant difference in theoretical scores (SMD = 1.17, 95 % CI [0.77, 1.57], P < 0.00001) in favor of PBL, compared with traditional lectures (Fig. 2). The results indicate that PBL improves students’ academic achievement. The fixed-effects model was also applied to pool the data. The pooled effects favored the PBL group (SMD = 1.06, 95 % CI [0.95, 1.16], P < 0.00001).
Because we observed heterogeneity between studies reporting theoretical scores, a sensitivity analysis was carried out to verify the reliability of the results. This was performed using sequential omission of individual studies. After excluding five studies [21–23, 28, 31] with inadequate generation of a randomized sequence from the analyses of theoretical scores, the pooled effect size favored the PBL group (SMD = 1.06, 95 % CI [0.77, 1.35], P < 0.00001) and did not change the effects observed in the primary analysis.
The funnel plot for the 17 reports on the theoretical scores analysis is shown in Fig. 3. The shape of the funnel plot is symmetrical indicating no significant publication bias.
Effects of interventions assessed by questionnaires
According to the questionnaire results, most students at the PBL pilot institutions showed more enthusiasm for PBL methods than for conventional teaching methods. The questionnaire results also show that the PBL methods are superior to the conventional teaching methods in improving students’ learning interest, independent analysis skills, scope of knowledge, self-study, team spirit, and oral expression.
The pharmacy students in the PBL groups had better theoretical examination scores than those in the traditional learning method groups. This result is in accordance with the findings of a previous review of the efficiency of PBL learning methods in pharmacy education in United States, United Kingdom, and Canada, but not in China . To our knowledge, this is the first systematic review and meta-analysis of the effects of PBL in Chinese pharmaceutical education.
There are several possible reasons why pharmacy students who received PBL teaching had higher theoretical examination scores that those who received traditional teaching. First, the main difference between PBL and a traditional curriculum is that teachers and other faculty personnel in a PBL program engage more with students [42–44]. Second, the PBL teaching model can inspire students to engage in proactive learning and thinking initiatives, facilitating a stronger grasp of experimental processes and logic. Third, PBL encourages students to think about and resolve practical problems. It may also help pharmacy students to apply their knowledge to work situations, improve their professional reasoning, and encourage self-directed learning during their professional careers [45, 46].
The questionnaire results indicated that students showed more enthusiasm for PBL than for traditional learning. The questionnaire results also show that PBL methods are superior to conventional teaching methods in improving students’ learning interest, independent analysis skills, scope of knowledge, self-study, team spirit, and oral expression. In PBL teaching methods, pharmacy students were asked to laying out the problem, search information, group discussion, summary, and effectiveness evaluation, which are the basic progress for PBL teaching methods but not for conventional teaching methods. After the related training process of PBL teaching methods, the pharmacy students showed more enthusiasm and raised abilities of learning interest, self-analysis, self-study, team collaboration, and spoken expression.
Learning is most effective when students are actively involved in PBL . The present results for pharmacy education are consistent with recent research in nursing education , medical education , and Chinese dental education . This suggests that PBL is the best education method for healthcare courses . The main obstacles to implementing PBL include training of teachers and other staff and necessary decreases in class size, which could raise pharmacy education fees [45, 51]. Because most of the 16 studies reviewed here used random allocation or blinding methods, the methodological quality of the included articles was relatively high. Differences of age, gender, and scores on college entrance exams between the PBL and control groups were mentioned in most studies, so the sampling error was minimal. Those studies that did not describe random allocation or blinding methods were not included in the analysis.
There was an obvious heterogeneity among the 16 pooled studies for the theoretical scores, which may have resulted from the following factors. First, there were differences in the educational levels of the students and the pharmacy schools in the included studies. In the Chinese pharmacy education system, pharmacy schools are established by various colleges or universities, which provide different levels of education. The criteria for admission of pharmacy students are not based on a standard examination, such as the Pharmacy College Admission Test used in the United States. For example, of the 16 pooled studies, 10 studies [18, 19, 22, 24, 25, 28, 29, 31–33] were conducted in colleges and the other six studies [20, 21, 23, 26, 27, 30] were conducted in universities (higher level institutions). This diversity is a fundamental cause of the heterogeneity among those studies. Second, the pooled studies focus on different pharmacy disciplines. In this review, we synthesized these studies to assess the total effectiveness; however, these differences in discipline may have resulted in heterogeneity. The third reason concerns the examination method. Sixteen studies employed different examinations. There are no unified criteria for the evaluation of the effectiveness of PBL pedagogy on theoretical knowledge. Fourth, the pharmacy institutions did not use a standard PBL pedagogy; these different teaching methods and objectives may have resulted in different effects.
In China, the implementation of the PBL method is still at an early stage. Some pharmacy colleges and universities have made tentative steps to utilize PBL methods in pharmacy education. Because PBL is an effective way to prepare pharmacy students for careers in the pharmaceutical industry, it should be used more widely in China, rather than only being applied in an exploratory way. However, it is difficult for pharmacy colleges and universities in China to model PBL teaching methods that have been successful in other countries, because Chinese pharmacy education is characterized by special teaching conditions. With the exception of curricula for particular specialties, PBL should be aimed at educating general pharmacists to provide recommendations for the most optimized treatment schedules for patients.
Limitations and future studies
The methodological quality of the included articles was low. The included studies involved research in the field of pharmacy education, so it was impossible for the researchers to implement allocation concealment and blinding. Although the studies we included were randomized controlled trials, and two investigators examined the studies and extracted the data independently, selection bias and performance bias were unavoidable. Moreover, there are no standard criteria for assessing the effectiveness of PBL pedagogy. Therefore, this meta-analysis is likely to contain measurement bias. However, we found no evidence of attrition bias. Lastly, the methodological qualities of studies in the field of pharmacy education are not comparable to those of pharmaceutical science research for multiple reasons. However, this review does provide information about implementing PBL pedagogy and suggests the utilization of a more standardized method to assess PBL.
The 16 included articles did not adopt uniform outcome measures. There was no standard examination to test the theoretical outcomes, which are designed to reflect precisely the effect of the PBL pedagogy. Before a standard evaluation system of PBL pedagogy is established, existing standard examinations, such as the North American Pharmacist Licensure Examination (NAPLEX) and the Multistate Pharmacy Jurisprudence Examination (MPJE) in the United States, or corresponding examinations in other countries, could be used as evaluation tests. In 1993, Vernon and Blake  conducted a meta-analysis of PBL in medical education using the National Board Medical Examination (NBME) as a theoretical test. The scaled scores and pass rate on the North American Pharmacist Licensure Examination for the University of Mississippi were both above the national scores in all reporting periods from 2001 to 2005 except one . In China, a standard examination such as the Chinese Pharmacy License Examination could be adopted as the standard examination for PBL pedagogy evaluation. High-quality randomized trials are needed to eliminate bias. Most of the 16 included studies did not describe random allocation methods in detail. PBL curriculum design should include precise educational objectives of PBL pedagogy. The impact of PBL on postdoctoral plans and extracurricular activities, especially research, should also be assessed in future studies .
Another limitation of this review was the absence of studies that assessed effects on the professional achievements of pharmacists who experienced the PBL method compared with those who received traditional methods of instruction. The present findings indicate that PBL students demonstrated higher theoretical examination scores; however, we cannot infer that these students will become better professionals.
Future research should prioritize experimental designs that evaluate effects that are more directly related to professional effectiveness and workplace performance. Greater recording and publication of evidence about PBL implementation would facilitate the adoption of this method without substantially raising fees.
The PBL curriculum seems to improve the academic performance of pharmacy students when compared with the traditional method of instruction. However, the heterogeneity of the selected studies needs to be considered. Most of the questionnaire surveys indicated the method’s positive effects on students’ learning interest, independent analysis skills, scope of knowledge, self-study, team spirit, and oral expression. Directors and teachers of pharmacy courses should consider gradually implementing PBL methods into their programs, because PBL pedagogy may be superior to traditional lecture-based teaching. PBL methods could be an optional, supplementary method for pharmacy teaching models in China. However, Chinese pharmacy schools should devise PBL curricula according to their own needs, which would optimize the effectiveness of PBL. Reporting the results from such initiatives is likely to improve the quality of the existing evidence in support of PBL methods.
Standardized mean difference
Dochy F, Segers M, Van den Bossche P, Gijbels D. Effects of problem-based learning: a meta-analysis. Learn Instruct. 2003;13(5):533–68.
Cisneros RM, Jawaid SP, Kendall DA, McPherson 3rd CE, Mu K, Weston GS, et al. International practice experiences in pharmacy education. Am J Pharm Educ. 2013;77(9):188.
Galvao TF, Silva MT, Neiva CS, Ribeiro LM, Pereira MG. Problem-based learning in pharmaceutical education: a systematic review and meta-analysis. Sci World J. 2014;2014:578382.
Cisneros RM, Salisbury-Glennon JD, Anderson-Harper HM. Status of problem-based learning research in pharmacy education: a call for future research. Am J Pharm Educ. 2002;66(1):19–26.
Donner RS, Bickley H. Problem-based learning in American medical education: an overview. Bull Med Libr Assoc. 1993;81(3):294–8.
Kwan CY. Learning of medical pharmacology via innovation: a personal experience at McMaster and in Asia. Acta Pharmacol Sin. 2004;25(9):1186–94.
Abraham RR, Ramnarayan K, George BM, Adiga I, Kumari GR, Suvarna N, et al. Effects of problem-based learning along with other active learning strategies on short-term learning outcomes of students in an Indian medical school. Int J Health Allied Sci. 2012;1(2):98.
Tayyeb R. Effectiveness of problem based learning as an instructional tool for acquisition of content knowledge and promotion of critical thinking among medical students. J Coll Physicians Surg Pak. 2013;23(1):42–6.
Yadav A, Subedi D, Lundeberg MA, Bunting CF. Problem-based learning: influence on students’ learning in an electrical engineering course. J Eng Educ. 2011;100(2):253–80.
Love DW, Shumway JM. Patient-oriented problem-solving instruction in pharmacotherapeutics. Am J Pharm Educ. 1983;47(3):228–31.
Romero RM, Eriksen SP, Haworth IS. A decade of teaching pharmaceutics using case studies and problem-based learning. Am J Pharm Educ. 2004;68(2):31.
Romero RM, Eriksen SP, Haworth IS. Quantitative assessment of assisted problem-based learning in a pharmaceutics course. Am J Pharm Educ. 2010;74(4):66.
Myers IB, McCaulley MH, Quenk NL, Hammer AL. MBTI manual: a guide to development and use of the Myers-Briggs Type Indicator, vol. 3. Palo Alto: Consulting Psychologists Press; 2003.
Shi R, Shan S, Tian J. Psychological type and undergraduate student achievement in pharmacy course in military medical university. US-China Edu Rev. 2007;4(9):20–4.
Y-y H, Ip G, C-y C, Morris MW, Menon T. Construction of the self: collective duties and individual rights in Chinese and American cultures. Soc Cognition. 2001;19(3):251–68.
Garvey M. An assessment of learning styles among pharmacy students. Am J Pharm Educ. 1984;48(2):134–40.
Shuck AA, Phillips CR. Assessing pharmacy students’ learning styles and personality types: a ten-year analysis. Am J Pharm Educ. 1999;63(1):27–33.
Che Y. Application of problem-based learning in the experiment course of natural products chemistry. Sci Technol Innov Her. 2014;14:125.
Du S, Zhu H, Dong Y, Cheng X. Comparison of PBL and LBL in the teaching of the design of pharmaceutical formulation. J Pharm Pract. 2012;30(1):75–7.
Fang H, Sun C, Ren H. Teaching method of problem based learning used in theory course of pharmaceutical analysis. Heilongjiang Med Pharm. 2006;29(1):46–7.
Ge J, Wu F, Zhao B, Lv X, Li J. PBL teaching method in “pharmaceutical affairs law and regulation” teaching. Northwest Med Educ. 2008;16(2):299–302.
Huang H, Lv P, Jia Y, Han D. Application of problem-based learning in the experimental teaching of biopharmaceutics and pharmacokinetics. Chin Educ Innov Her. 2014;7:63.
Li L. PBL pedagogy in pharmacy professional “pharmacology of Chinese materia medica” experimental teaching in agricultural colleges and universities. Chin Med Pharm. 2011;1(24):137–8.
Pu Q. Teaching evaluation of PBL in pharmaceutical analysis. Neijiang Technol. 2011;8:87.
Shen J. Comparison of the effects of PBL and LBL teaching methods in teaching of “basic pharmacy”. Forefront Med Pharm. 2013;4:325.
Wang L, Tang Z, Zhang L. Application of PBL teaching method in practical teaching of “pharmaceutical literature retrieval”. Chin J Med Lib Inf Sci. 2012;21(12):56–9.
Wang Z, Tong Q, Zhou R, Liu X. Application of problem-based learning in the experiment course of pharmaceutical botany. J TCM Univ Hunan. 2009;29(9):86–8.
Yang X, Sun Y, Gao Y, Zhu Y. Evaluation of PBL teaching in pharmaceutical analysis. Chin J Anim Husbandry Vet Med. 2010;4:131–2.
Yang Y, Li P. PBL teaching method in organic chemistry teaching. Health Vocational Educ. 2007;25(23):71–2.
Yu H, He B, Wang K, Hu W, Hu J, Cao S, et al. Application of PBL in natural medicine chemistry experimental teaching. J Kunming Med Univ. 2013;34(5):163–5.
Zhang M. Discussion of PBL teaching method in “biopharmaceutics and pharmacokinetics”. J Shayang Teach Coll. 2011;12(4):26–8.
Zhang Y, Shi S, Su J. A comparative study of PBL with LBL teaching mode in pharmaceutical molecular biology teaching. Guide Chin Med. 2010;8(9):160–1.
Zhuo S, Wu G. Application of problem-based learning in the teaching of natural products chemistry. Course Educ Res. 2014;3:20.
Higgins JP, Green S. Cochrane handbook for systematic reviews of interventions version 5.1.0 (updated March 2011). The Cochrane Collaboration. 2011: Available from www.cochrane-handbook.org. Accessed 15 April 2014.
Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1.
Berkson L. Problem-based learning: have the expectations been met? Acad Med. 1993;68(10 Suppl):S79–88.
Collaboration RTC. Review Manager (RevMan). 5.3. Copenhagen: The Nordic Cochrane Centre: The Cochrane Collaboration; 2014.
Cohen J. Statistical power analysis for the behavioral sciences. Hillsdale: L. Erlbaum Associates; 1988.
Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21(11):1539–58.
Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557–60.
Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315(7109):629–34.
Hamdy H, Agamy E. Is running a problem-based learning curriculum more expensive than a traditional Subject-Based Curriculum? Med Teach. 2011;33(9):e509–14.
Mennin SP, Martinez-Burrola N. The cost of problem-based vs traditional medical education. Med Educ. 1986;20(3):187–94.
Donner RS, Bickley H. Problem-based learning: an assessment of its feasibility and cost. Hum Pathol. 1990;21(9):881–5.
Jones RW. Problem-based learning: description, advantages, disadvantages, scenarios and facilitation. Anaesth Intens Care. 2006;34(4):485–8.
Barrows HS. A taxonomy of problem-based learning methods. Med Educ. 1986;20(6):481–6.
Carrero E, Gomar C, Penzo W, Rull M. Comparison between lecture-based approach and case/problem-based learning discussion for teaching pre-anaesthetic assessment. Eur J Anaesthesiol. 2007;24(12):1008–15.
Kong LN, Qin B, Zhou YQ, Mou SY, Gao HM. The effectiveness of problem-based learning on development of nursing students’ critical thinking: a systematic review and meta-analysis. Int J Nurs Stud. 2014;51(3):458–69.
Huang B, Zheng L, Li C, Li L, Yu H. Effectiveness of problem-based learning in Chinese dental education: a meta-analysis. J Dent Educ. 2013;77(3):377–83.
Williams B. Self direction in a problem based learning program. Nurse Educ Today. 2004;24(4):277–85.
Finucane PM, Johnson SM, Prideaux DJ. Problem-based learning: its rationale and efficacy. Med J Aust. 1998;168(9):445–8.
Vernon DT, Blake RL. Does problem-based learning work? A meta-analysis of evaluative research. Acad Med. 1993;68(7):550–63.
Hogan S, Lundquist LM. The impact of problem-based learning on students’ perceptions of preparedness for advanced pharmacy practice experiences. Am J Pharm Educ. 2006;70(4):82.
Susarla SM, Medina-Martinez N, Howell TH, Karimbux NY. Problem-based learning: effects on standard outcomes. J Dent Educ. 2003;67(9):1003–10.
This study was supported by grants from Education Project of Chinese Military Colleges and the National Natural Science Foundation of China (grant no. 81471040, 81100597).
The authors declare that they have no conflicts of interest. The authors alone are responsible for the content and writing of the paper.
JZ, SZ, and CH conceived the paper, and collected and analyzed the data. RX, ZZ, SZ, and GQ participated in the critical review and preparation of the manuscript. GQ made the final revision to the manuscript. All authors read and approved the final manuscript.
Jiyin Zhou and Shiwen Zhou contributed equally to this work.
PRISMA 2009 Checklist. (DOC 65 kb)