Skip to main content

A model six-month workshop for developing systematic review protocols at teaching hospitals: action research and scholarly productivity

Abstract

Background

Research engagement contributes to the improvement of patient care. A systematic review is a suitable first scholarly activity because it entails summarization of publicly available data and usually requires neither rigorous ethical review nor research funding.

Methods

This study aimed to develop a model workshop for healthcare staff to acquire skills in creating systematic review protocols based on their own clinical questions at teaching hospitals. We used an action research method to create a model workshop at four hospitals in Japan from April 2015 to March 2017. To improve the program, we solicited reflections using participant questionnaires for each lecture and examined the quality of homework submitted by participants after each lecture. We administered a revised final version of the workshop at five hospitals from April 2016 to March 2017. We evaluated the participants’ scholarly productivity related to these workshops. The observation period was a minimum of 2 years following the workshops.

Results

Most participants had never developed a formal clinical research protocol and voluntarily participated in the workshop. The action research was developed and implemented at nine teaching hospitals in Japan, including one university hospital. The study developed a model nine-step workshop curriculum: 1) Research question development, 2) Search strategy development, 3) Search strategy brush-up, 4) Exclusion and inclusion criteria development, 5) Risk of bias assessment planning, 6) Meta-analysis planning, 7) Subgroup and sensitivity analysis planning, 8) Planning the presentation of results, and 9) Presentation protocols. A total of 233 participants, including medical doctors and other health professionals, produced 414 research questions. Seventy-nine participants (34%) completed the workshop, and 47 review teams accomplished systematic review protocols. The participants published 13 peer-reviewed articles as a result of the workshop.

Conclusions

We developed a structured scholarly productive model workshop for healthcare staff working at hospitals. We found healthcare staff with clinical subspecialties were able to develop an unexpectedly high number of research questions through this workshop. Medical teachers at hospitals with prior systematic review experience could teach how to develop systematic review protocols using this model. Further research is needed to increase the academic productivity of such workshops.

Trial registration

UMIN (https://www.umin.ac.jp/ctr/), UMIN000017107 (4/15/2015), UMIN000025580 (1/10/2017).

Peer Review reports

Background

Research engagement is generally believed to contribute to the improvement of patient care [1, 2]. Residency programs in several countries include support for residents in their scholarly activities [3]. However, these programs may not involve adequate supervision and appropriate training in research methods [4, 5]. Systematic reviews (SRs) in medicine are a type of research that attempts to collate all empirical evidence to answer a specific research question [6]. The research method is a summarization of publicly available electronic patient data and usually requires neither rigorous ethical review nor research funding. This is an established and widely used method [7], allowing even research novices to perform SRs for their first clinical research projects.

Quality SRs require guidance and support of experienced researchers; however, these opportunities are rarely available for healthcare staff in their first SR. The first step in conducting an SR is to structure clinical questions for the research. Vague clinical questions often set too wide of a scope, requiring support from a senior with prior research experience to ensure the appropriateness of clinical questions. Unlike narrative literature reviews, SRs a priori specify eligibility criteria to be reviewed [7]. Authors of SRs usually consider eligibility criteria based on both clinical questions and specification of the types of studies that have addressed these questions [7]. Another feature of SRs is a comprehensive literature review to achieve unbiased knowledge syntheses [8]. If a modification of search strategies is required, the review authors are forced to repeat the entire time-consuming literature review process. Therefore, such a repetition can be avoided with a review of SR search strategies before running them. Protocol preparation for SRs requires an analysis of pre-specification of outcomes. Changes in SR outcomes observed in published SRs may lead to a biased interpretation of SR results [9, 10]. Therefore, step-by-step peer review of the protocols is essential for producing quality SRs. Although several universities and organizations have developed massive open online courses (MOOCs) for SRs [11,12,13], MOOCs involve large groups of people and render providing step-by-step feedback from teachers and advice tailored to the participants’ level of understanding a major challenge [14]. Furthermore, MOOCs are generally known for their high dropout rates [14,15,16].

In this study, we developed a model continuous workshop (WS) utilizing an action research method to develop SR protocols with step-by-step peer-review. We subsequently evaluated its scholarly productivity.

Methods

We used an action research method to create a model WS and implemented it at nine teaching hospitals in Japan. The institutional ethics committees of each hospital approved our research. Action research is a research method that aims at both taking and creating knowledge or theory relating to that action [17,18,19]. The action research process involves a cyclical process of (1) planning, (2) implementing the plan (action), (3) observation, and (4) critical and self-critical reflection on the results of (1–3) and making decisions for the next cycle of (1–4) to improve the action. As such, the model WS was developed through a cyclical progress in four phases: planning, action, observation, and reflection [17]. The reflection phase after each lecture focused on attempting to improve both the next lecture for the same participants and a lecture on the same topic in the next hospital setting. The lectures were given by three teachers who were physicians with prior SR experience and one teacher who had been one of the participants of the third WS and completed an SR through the WS.

Phase 1: planning

The goal of the WS was for healthcare staff to learn how to answer their own clinical questions through SRs. We recruited participants from each of the nine hospitals. Two teachers first constructed an overall framework for the WS based on prior experience with writing SRs, the graduate school syllabus at Kyoto University Graduate School of Medicine/School of Public Health, and an introductory textbook for SRs [7].

Phase 2: action

We held the first WS from April to August 2015 at Hyogo Prefectural Amagasaki General Medical Center (HPAGMC) with 21 healthcare staff participants and held the second WS from October 2015 to February 2016 at Tenri Hospital with 14 healthcare staff participants. We held the third WS from April 2016 to September 2016 at Shiga University of Medical Science Hospital with 30 healthcare staff participants, including medical students and held the fourth WS from October 2016 to March 2017 at Kameda General Medical Center with 45 healthcare staff participants.

Phase 3: observation

After each lecture, participants developed their SR protocols as their homework based on the lecture they took part in. We then assessed the quality of their protocols in terms of whether the study could actually be conducted in a reproduceable way and with reference to the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) [20]. We also administered an open-response evaluation questionnaire and a Likert scale questionnaire (see Supplementary Table 1, Additional file 1) to participants after each lecture and developed a list of common questions and mistakes.

Phase 4: reflection

Immediately after each lecture, we held a formal debriefing to evaluate the lecture and the homework. We read the questionnaire responses and discussed curriculum improvement, and thenmodified the subsequent lecture. All modifications were shared among the core members (HT, YK, and YT) and the external institutional members (ETT, YS2, and RS1) by email and approved through an interactive process until consensus was reached.

Finally, we discussed the framework of the model WS sessions in August 2016 for external validation. We conducted a WS using the revised final version of the curriculum at three hospitals (number of beds) in Japan: Sakai City Medical Center (487), National Hospital Organization Kinki-Chuo Chest Medical Center (385), Amagasaki Iryoseikyo Hospital (199). In addition, teachers who had not previously lectured in previous WSs independently conducted the WS at Kurashiki Central Hospital (1166) and Seirei Mikatahara General Hospital (934). Two years after the last WS, we analyzed the scholarly productivity related to the WS. The following terms were used to describe the participants’ outcomes with the following definitions:

  • Research questions: a structured approach for framing questions that uses five components—the patient population or the disease being addressed, the interventions or exposure, the comparator group, the outcome or endpoint, and the study design chosen [7, 21]

  • Protocol: a document that presents an explicit plan for a systematic review [20]

  • Systematic review: an attempt to collate all relevant evidence that fits pre-specified eligibility criteria to answer a specific research question 

The key characteristics of a systematic review are (1) a clear objective with an reproducible methodology, (2) a comprehensive search, (3) an assessment of the validity of the findings, and (4) systematic presentation and synthesis [20].

The observation period was a minimum of 2 years following completion of the workshop because a previous study showed that the median time from protocol registration to publication was over 1 year [22].

Results

The first WS included seven interactive lecture-series from April 2015 to July 2015 at HPAGMC. Participants were mainly medical doctors, but other healthcare professionals were also present (Table 1) and voluntarily participated in the workshop. This section discusses our results under three main headings: structure of the final model WS (Table 2), quantitative data of each WS, and themes identified from the observations and reflections. The differences between the initial and final models are outlined in Table 3.

Table 1 Participant demographics
Table 2 The final WS model
Table 3 Differences between initial and final models

Structure of the final model WS

We set specific objectives for each lecture, with the teaching method including short lectures and homework-based discussions (Table 2). Participants were expected to develop a part of their SR protocols based on the previous lecture as homework. They submitted the homework through an online platform (Google Forms, https://www.google.com/intl/ja_jp/forms/about/). We gave feedback using the online platform within 48 h of the submission and accepted resubmissions to ensure that participants would not lose the research rhythm [14]. All participants could see other participants’ homework and feedback and could use it as a reference for their own homework. Learning and practicing the items in Table 2 was essential for creating a research protocol that complies with the items in PRISMA-P. The process of action research and evaluation of the participant-created protocols improved the initial model.

Themes identified from observations and reflections

During the observation and reflection phases, we carried out the following evaluations and then made modifications to develop the final model WS described above.

Participants’ readiness

Some participants had difficulty coming up with research questions due to a lack of clinical experience and knowledge. In such cases, the research scope tended to be overly broad and difficult to handle as an SR. On the other hand, participants with clinical subspecialties, such as specialized nurses and specialists in physical therapists, occupational therapists, or speech-language-hearing therapists, had many viable research questions. Therefore, we particularly encouraged clinical staff with a subspecialty to join the WS.

Most participants had never developed a formal clinical research protocol before, so some participants found it difficult to grasp the overall picture of the SR protocol and the connection between each lecture. Ultimately, some of these participants quit the WS if they found it challenging to develop a single section of the protocol, particularly the search formula development. To remedy this, we provided a model SR protocol in the first lecture and explained how each section of the protocol is related to each lecture.

Some participants also dropped out after finding existing SRs that covered their research questions. Although there is no way to find existing SRs without conducting an SR, previous SRs can be found quickly by using clinical practice guidelines and the Cochrane library (https://www.cochranelibrary.com/). To address this, we added a short lecture on finding existing SRs in the first meeting of the WS starting with the second hospital.

Time management for research activity

Most participants were full-time healthcare staff and had no time allocated for research within their work schedule. Unfortunately, lack of time was a consistent reason for participants dropping out of the WS. To address this, we provided an estimated time commitment for each homework assignment in advance and incorporated a time-management skill lecture in the first meeting of the WS [14]. In this lecture, we emphasized the importance of setting aside scheduled research time at least once a week. In addition, we extended the intervals between WSs from 2 to 3 weeks to allow more time for homework assignments.

In addition, as most participants were healthcare professionals, their priority was their patients, sometimes making it difficult for them to attend a scheduled lecture. To remedy this, we implemented flipped classrooms [23], where participants watched lectures online (https://youtu.be/OT5C9vbNYAc), collaborated in online discussions, and then carried out research activities as homework. Implementing flipped classrooms allowed participants to skip some WS meetings if necessary. To facilitate participation in the meetings without having questions to ask, some participants requested that the lecture be both provided on video and given at the WS meeting. However, it was not possible to do both because of limited time available for the meetings in the evening after work.

Research team

We identified one successful case where busy junior residents were able to create an SR protocol. We believe the reason for this success could be the support of their team members. The members of this research team met every day in the hospital ward. As one participant in the WS at HPAGMC observed, “Every time they met, they reminded each other that they had a research project to tackle together.”

On the other hand, teams with too many members had trouble finding time when all members could meet. These issues were not considered in the initial planning; therefore, at the first meeting, we facilitated participants in setting up their review team with two or up to three members who could meet frequently [24]. We encouraged choosing participants who could meet every day to facilitate quick peer support for their task.

Since attending meetings was not required, review teams were not always able to meet. To reduce communication obstacles and to foster camaraderie among the WS participants, we changed the feedback platform from email-based to a shared web cloud-based message board that every participant could read and participate in.

Scholarly productivity

A total of 233 participants produced 414 research questions. Seventy-nine participants (34%) finished the WS. They wrote 47 protocols adhering to current standards [20]. If the participants were interested, we offered support in implementing the protocols and with publication. As a result, they published 13 peer-reviewed articles. We searched PubMed for SRs associated with affiliations of the hospitals that conducted the workshops in the 3 years before and after the workshops and found a threefold increase in the number of publications from before to after the WSs (Supplementary Tables 2, 3, Additional file 1). This is a larger number compared to about a 1.6 times increase for SRs associated with Japanese affiliations, suggesting an indirect effect of the WSs’ implementation. One of the early participants ultimately became a teacher of this WS. Although unanticipated at the time of planning, due to the success of this WS, some teachers were invited to several clinical practice guideline working groups, where experts teach SR research methods. In addition, the WS helped teachers develop a meta-epidemiological perspective on the process of SR creation, enabling them to publish 47 papers inspired by the WS as of May 2020 (Supplementary Table 4, Additional file 1; Supplementary data, Additional file 2).

Discussion

During this project, we used action research methods to build and refine a model WS on developing SR protocols. We conducted the WS at 9 hospitals for over 200 healthcare staff, improved the quality of the lectures based on feedback and self-reflection, and derived a workable model. We found that healthcare staff were able to develop an unexpectedly high number of clinical questions through this WS.

The WS model has several strengths. Since the lecture had already been video recorded for the flipped classrooms, the WS was easily transferable between institutions in Japan. We also made the lecture materials available to everyone via the Internet (https://drive.google.com/drive/folders/1IfO9rNbGf2F_Tc7iJammVwl69vXxJ8P_?usp=sharing). The workshop eventually had a form of recruitment that could be easily transformed into a location-independent format, and we were able to start a crowdfunded online salon (https://community.camp-fire.jp/projects/view/187310). In the first year, this led to 252 participants, 16 protocols, and 1 paper accepted. Although we did not anticipate this at the beginning, the salon has become a form of learning that aligns well with the “stay-at-home” situation of the COVID-19 pandemic [25]. In addition, the model requires only one or two tutors, a small room with a projector, internet access, desks, chairs, and no standardized participants; therefore, it can be easily used in locations where educational resources are limited. Furthermore, the model allows participants easy access to peer reviewers through a peer discussion of their protocols. The participants may have played a role similar to that of teaching assistants in the previously reported university course, leading to the success of this workshop [26]. A notable success story is when one of the participants (MB), who was new to SR during the third WS, became a competent tutor in the eighth and ninth WSs, contributing greatly to the WS’s sustainability. Our WS also had a higher completion rate (34%) than most MOOCs, which generally have a completion rate of less than 13% [15]. We believe these results demonstrate that, rather than providing uniform information to all learners, teachers must strive to support individual learners [16, 27]. Given the participants’ readiness in this WS, this appears to be particularly important for learners at hospitals. Such individually catered education cannot be provided by MOOCs. In addition, we believe our online-based quick feedback system helped participants manage their time and maintain a rhythm of engagement in the study.

There were three main limitations to this WS model. First, the number of participants was limited to about 20–30 learners per teacher at a time, whereas MOOCs usually accept an unlimited number of applicants. Second, the proposed model WS was only validated in a limited number of institutions in one country in one language. We encourage further validation in different languages and settings. Finally, due to scheduling constraints, we could not perform a formal focus group session for the groups that dropped out or those who successfully published a paper. We believe that future research comparing the results of such focus groups would improve the academic productivity of the WS.

Conclusion

We developed a structured model for interactive lecture-series WSs at hospitals. We showed that medical teachers in hospitals with prior SR experience could use this model to teach health professionals how to develop SR protocols. Further research is needed to increase the academic productivity of our model.

Availability of data and materials

The data used during the current study are available from the corresponding authors on reasonable request.

Abbreviations

WS:

Workshop

SR:

Systematic review

MOOCs:

Massive open online courses

HPAGMC:

Hyogo Prefectural Amagasaki General Medical Center

References

  1. 1.

    Harding K, Lynch L, Porter J, Taylor NF. Organisational benefits of a strong research culture in a health service: a systematic review. Aust Health Rev. 2017;41:45–53.

    Article  Google Scholar 

  2. 2.

    Boaz A, Hanney S, Jones T, Soper B. Does the engagement of clinicians and organisations in research improve healthcare performance: a three-stage review. BMJ Open. 2015;5:e009415.

    Article  Google Scholar 

  3. 3.

    Noble C, Billett SR, Phang DTY, Sharma S, Hashem F, Rogers GD. Supporting resident research learning in the workplace: a rapid realist review. Acad Med. 2018;93:1732–40.

    Article  Google Scholar 

  4. 4.

    Stehlik P, Noble C, Brandenburg C, Fawzy P, Narouz I, Henry D, et al. How do trainee doctors learn about research? Content analysis of Australian specialist colleges’ intended research curricula. BMJ Open. 2020;10:e034962.

    Article  Google Scholar 

  5. 5.

    Kataoka Y, Ikegaki S, Kato D, Takada T, Tsujimoto Y, Sasaki S, et al. Scholarly activity support systems in internal medicine residency programs: a national representative survey in Japan. Intern Med. 2019;58:1859–64.

    Article  Google Scholar 

  6. 6.

    Green S, Higgins J, Alderson P, Clarke M, Mulrow C. What is a systematic review? In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.handbook.cochrane.org.

  7. 7.

    Higgins JPT, Green S. Chapter 5: Defining the review question and developing criteria for including studies internet source (Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from https://www.handbook.cochrane.org.

  8. 8.

    McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016;75:40–6.

    Article  Google Scholar 

  9. 9.

    Kirkham JJ, Altman DG, Williamson PR. Bias due to changes in specified outcomes during the systematic review process. PLoS One. 2010;5:3–7.

    Article  Google Scholar 

  10. 10.

    Tricco AC, Cogo E, Page MJ, Polisena J, Booth A, Dwan K, et al. A third of systematic reviews changed or did not specify the primary outcome: A PROSPERO register study. J Clin Epidemiol. 2016:1–9.

  11. 11.

    Polanin JR, Pigott TD. The Campbell Collaboration’s systematic review and meta-analysis online training videos. Res Soc Work Pract. 2013;23:229–32.

    Article  Google Scholar 

  12. 12.

    Li T, Dickersin K. Introduction to systematic review and meta-analysis. Johns Hopkins University. https://www.coursera.org/learn/systematic-review. Accessed 1 June 2020.

  13. 13.

    Cochrane Sweden, The medical degree Programme and the medical faculty library & ICT unit at Lund University. Interactive Learinig. Cochrane Interactive Learning. https://training.cochrane.org/interactivelearning. Accessed 1 June 2020.

  14. 14.

    Nawrot I, Doucet A. Building engagement for MOOC students: Introducing support for time management on online learning platforms. In: WWW 2014 Companion - Proceedings of the 23rd International Conference on World Wide Web. Seoul: Association for Computing Machinery, Inc; 2014. p. 1077–82.

  15. 15.

    Onah DFO, Sinclair J, Boyatt R. Dropout rates of massive open online courses: behavioural pattern. In: 6th International Conference on Education and New Learning Technologies. Barcelona: IATED; 2014. p. 5825-5834. EDULEARN14 Proceedings.

  16. 16.

    Chen BY, Kern DE, Kearns RM, Thomas PA, Hughes MT, Tackett S. From modules to MOOCs: application of the six-step approach to online curriculum development for medical education. Acad Med. 2019;94:678–85.

    Article  Google Scholar 

  17. 17.

    Coghlan D, Casey M. Action research from the inside: issues and challenges in doing action research in your own hospital. J Adv Nurs. 2001;35:674–82.

    Article  Google Scholar 

  18. 18.

    Nishigori H, Masuda K, Kikukawa M, Kawashima A, Yudkowsky R, Bordage G, et al. A model teaching session for the hypothesis-driven physical examination. Med Teach. 2011;33:410–7.

    Article  Google Scholar 

  19. 19.

    Cohen L, Manion L, Keith M. Research methods in education. 6th ed. New York: Taylor & Francis; 2007.

    Book  Google Scholar 

  20. 20.

    Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1.

    Article  Google Scholar 

  21. 21.

    Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009;62:e1–34.

    Article  Google Scholar 

  22. 22.

    Tsujimoto H, Tsujimoto Y, Kataoka Y. Unpublished systematic reviews and financial support: a meta-epidemiological study. BMC Res Notes. 2017;10:703.

    Article  Google Scholar 

  23. 23.

    Moffett J. Twelve tips for flipping the classroom. Med Teach. 2015;37:331–6.

    Article  Google Scholar 

  24. 24.

    Chase J, Topp R, Smith CE, Cohen MZ, Fahrenwald N, Zerwic JJ, et al. Time Management Strategies for Research Productivity. West J Nurs Res. 2013;35:155–76.

    Article  Google Scholar 

  25. 25.

    Gordon M, Patricio M, Horne L, Muston A, Alston SR, Pammi M, et al. Developments in medical education in response to the COVID-19 pandemic: A rapid BEME systematic review: BEME Guide No. 63. Med Teach. 2020;42:1–14.

    Article  Google Scholar 

  26. 26.

    Li T, Saldanha IJ, Vedula SS, Yu T, Rosman L, Twose C, et al. Learning by doing-teaching systematic review methods in 8 weeks. Res Synth Methods. 2014;5:254–63.

    Article  Google Scholar 

  27. 27.

    Setia S, Tay JC, Chia YC, Subramaniam K. Massive open online courses (MOOCs) for continuing medical education – why and how? Adv Med Educ Pract. 2019;10:805–12.

    Article  Google Scholar 

  28. 28.

    Tsujimoto H, Kataoka Y, Tsujino E, Hozumi T, Sada R, Sato Y. A Model Workshop for Developing Skills of Systematic Reviews at Hospitals. Jeju: 2015 JEJU Cochrane Workshop; 2015. p. p376.

    Google Scholar 

  29. 29.

    Tsujimoto H, Kataoka Y, Tsujino E, Nagano H, Hozumi T, Sada R, et al. A model workshop for developing skills of systematic review protocols at hospitals: midterm report. Singapore: Asia Pacific Medical Education Conference; 2016. p. 410.

    Google Scholar 

  30. 30.

    Tsujimoto H, Kataoka Y, Tsujimoto Y, Tsujino E, Nagano H, Sumi Y, et al. A Model Workshop for Systematic Review Protocols at Teaching Hospitals: The Second Midterm Report. Barcelona: AMEE 2016; 2016. p. 796.

    Google Scholar 

  31. 31.

    Kataoka Y, Tsujimoto H, Tsujino E, Sada R. Developing A Model Workshop for Systematic Review Protocols at Teaching Hospitals: Midterm Report of Action Research. Seoul: The 24th Cochrane Colloquium; 2016. p. 84.

    Google Scholar 

  32. 32.

    Kataoka Y, Tsujimoto H, Banno M, Tsujimoto Y. A Model Workshop for Writing Systematic Review Protocols at Teaching Hospitals: The Final Report. Vienna: AMEE 2019; 2019. p. 7GG.

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank Prof. Toshi A. Furukawa, Kyoto University, for providing the schedule of his lectures at Kyoto University Graduate School of Medicine/School of Public Health as well as all of the participants and supporters of SRWS. The authors give special thanks to Dr. Hidemichi Yuasa, National Hospital Organization, Toyohashi Medical Center, Aichi, Japan, and Dr. Eishu Nango, JCHO Tokyo Joto Hospital, Tokyo, Japan, who attended the presentation day of the first WS at HPAGMC and provided peer comments for participants’ protocols. Special thanks are also given to Akira Onishi, Kobe University, Hyogo, Japan, who gave a lecture about meta-analysis in the first WS at HPAGMC. The authors also extend a thank you to Dr. Shin-Ichi Kitamoto, Awachiiki Iryo Center Chiba, Japan; Dr. Akira Sato, Kameda General Medical Center, Chiba, Japan; Dr. Satoshi Jujo, Kameda General Medical Center, Chiba, Japan; Ms. Yukiyo Tada, Kameda General Medical Center, Chiba, Japan; Dr. Ryohei Yamamoto, Kameda General Medical Center, Chiba, Japan; Dr. Hideto Yasuda, Kameda General Medical Center, Chiba, Japan; Dr. Yuda Adachi, Kameda General Medical Center, Chiba, Japan; Ms. Ritsuko Ohse, Kameda General Medical Center, Chiba, Japan; and Dr. Yukiko Okami, Shiga University of Medical Science for their support in running the workshop. The authors would also like to thank Cochrane Japan (https://japan.cochrane.org/ja) for their support in promoting this WS. The authors further thank Dr. Hiromu Kutsumi and Dr. Katsutaro Morino, Shiga University of Medical Science, for their support in running and promoting the second iteration of this WS. Finally, the authors thank Editage (www.editage.jp) for their English language editing services. Aspects of this article were presented as the following poster presentations: “A Model Workshop for Developing Skills of Systematic Reviews at Hospitals” at the 2015 JEJU Cochrane Workshop, June 12, 2015, Seogwipo, Jeju, Korea [28]; “A Model Workshop for Developing Skills of Systematic Review Protocols at Hospitals: Midterm Report” at the Asia Pacific Medical Education Conference, Jan. 16, 2016, Singapore [29]; “A Model Workshop for Systematic Review Protocols at Teaching Hospitals: The Second Midterm Report” at the 2016 conference of The Association for Medical Education in Europe (AMEE), Aug. 30, 2016, Barcelona, Spain [30]; “Developing A Model Workshop for Systematic Review Protocols at Teaching Hospitals: Midterm Report of Action Research” at the 24th Cochrane Colloquium, Oct. 25, 2016, Seoul, Korea [31]; “A Model Workshop for Writing Systematic Review Protocols at Teaching Hospitals: The Final Report” at the 2019 conference of the AMEE, Aug. 27, 2019, Vienna, Austria [32].

Funding

Transportation expenses to some hospitals were paid for by the Japan Primary Care Association (http://www.primary-care.or.jp/jpca_eng/).

Author information

Affiliations

Authors

Contributions

HT and YK: concept design, analysis and interpretation of data, drafting of manuscript. Implementing and improving the workshop: HT, YK, YS1, MB, ETT, YS2, RS1, TF, YO, JK, HI, YM, RS2, TK, TY, YU, HN, MA, TH, and YT. All authors read and approved the final manuscript.

Corresponding authors

Correspondence to Hiraku Tsujimoto or Yuki Kataoka.

Ethics declarations

Ethics approval and consent to participate

This study was approved by institutional ethics committees (institution, date, reference number): Hyogo Prefectural Amagasaki General Medical Center, Feb. 2015, July 2015 and Jan. 2017, 26–42, 27–6, and 28–129; Tenri Hospital, June 2015, 683; Shiga University of Medical Science, Apr. 2016, 27–238; Kameda General Medical Center, Aug. 2016, 16–062; Sakai City Medical Center, Feb. 2017, 30; National Hospital Organization Kinki-Chuo Chest Medical Center, Feb. 2017, 585; Kurashiki Central Hospital, Feb. 2017, 2441; Amagasaki Iryoseikyo Hospital, July 2017, 29–3; Seirei Mikatahara General Hospital, Aug. 2017, 17–23. Protocol registration: UMIN (https://www.umin.ac.jp/ctr/), UMIN000017107 (4/15/2015), UMIN000025580 (1/10/2017). Written consent was obtained from the participants.

Consent for publication

Not applicable.

Competing interests

Y. Kataoka, Y. Tsujimoto, M. Banno, and R. So are the founding members of SRWS-PSG, which is a non-profit, crowdfunded, and managed group. By using a portion of the framework of this WS, they help their supporters in producing papers on clinical research, including systematic reviews. Y. Tsujimoto is one of the directors of Cochrane JAPAN (https://japan.cochrane.org/ja, since Jun. 2019). The authors declare that they have no competing interests. The authors alone are responsible for the content and writing of the paper.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Supplementary Table 1.

Questionnaire for participants. A questionnaire given to workshop participants. Supplementary Table 2. Number of SRs in PubMed affiliated with the hospitals for three years before and after the WS. PubMed search results for SRs three years before and after the WS. Supplementary Table 3. PubMed search details related to Supplementary Table 2. Search details related to Table 2. Supplementary Table 4. Issues covered by participants' SRs. Research topics for participants’ SRs

Additional file 2: Supplementary data.

Publications related to the systematic review workshop since 2015.4

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tsujimoto, H., Kataoka, Y., Sato, Y. et al. A model six-month workshop for developing systematic review protocols at teaching hospitals: action research and scholarly productivity. BMC Med Educ 21, 98 (2021). https://doi.org/10.1186/s12909-021-02538-6

Download citation

Keywords

  • Systematic reviews
  • Action research
  • Scholarly activity
  • Continuing professional development