Over the past 25 years the philosophy and practice of EBP has evolved rapidly. This is reflected by the number of instruments developed to evaluate the efficacy of the teaching of EBP . This rapid growth has resulted in heterogeneity of studies and inconsistencies in reporting of data. To address this problem, Simera, Moher, Hirst, Joey, Schulz and Altman (2010)  recommend that academic and other research institutions “promote and support accurate and transparent reporting of health research studies and the use of reporting guidelines…” Despite the recent rise in the number of reporting guidelines for health research, there are no reporting guidelines for educational interventions. The aim of this program of research is to develop a reporting guideline (GREET statement) and an explanation and elaboration (E&E) paper to enable the consistent and transparent reporting of EBP educational interventions and teaching.
Following a recent systematic review of reporting guidelines for health research, Moher, Weeks, Ocampo, Seely, Sampson, Altman, Schulz, Miller, Simera, Grimshaw and Hoey (2011) produced a guidance statement for developers of reporting guidelines. This guidance statement forms the backbone for this program of research. To develop a reporting guideline for health research, Moher, Schulz, Simera and Altman (2010) recommend undertaking ‘initial steps’, which include identifying the need for a guideline and reviewing the literature, followed by ‘pre-meeting activities’ which refer to the preparation required for the next stage, the ‘face to face consensus meeting’ where consensus is achieved. This ‘pre-meeting stage’ will often include a Delphi survey to generate a list of items to be further discussed at the ‘face to face consensus meeting’. This is then followed by the ‘post meeting activities’ which includes the development of the reporting guideline, E&E document and pilot testing and publication strategy. The ‘post-publication activities’ form the final stage in the development process and includes dealing with the feedback and criticism and updating of the guidelines.
The inclusion of the Expert Panel representing the fields of EBP education, research guideline development, epidemiology and journal editing will guide the development and testing processes used in this study. Engaging an expert panel is not a requisite step in the development process for reporting guidelines, however it provides a safeguard to ensure the entire development process is transparent and comprehensive .
In this research protocol, the systematic review and Delphi consensus survey amalgamate the recommended ‘pre-meeting’ and ‘face-to face consensus meeting’ stages . While a ‘face to face consensus meeting’ allows for a captive audience and the potential for clearer communication there are several possible limitations. The process can be time consuming and expensive, with previous reporting guidelines requiring 2–3 days for the initial face to face meeting and a recommended funding requirement of $75K for this meeting alone [15, 28]. The travel required to attend a face to face meeting may preclude international participants thus limiting attendance and potentially introducing personal bias . For these reasons, this protocol plans to use a Skype conference meeting in lieu of the recommended ‘face to face consensus meeting’ to discuss the results of the Delphi consensus survey. The Delphi survey is intended to be used as the primary means of achieving consensus with the Skype conference to discuss and finalise the content and proposed layout of the GREET statement and E&E paper.
With respect to the Delphi consensus stage, this research protocol is based upon the recommendations from a recent systematic review as the consensus method . There are currently no internationally accepted criterion standards to determine the number and composition of participants for a Delphi survey . Previous Delphi surveys have used as few as 10 to as many as 1685 participants . Delphi surveys have previously been used in the development of nine reporting guidelines for health research [22, 30] with the number of participants (reported by only seven studies) ranging from 11 (CONSORT) to more than 50 (SQUIRE) . The potential number of participants planned for this program of research (likely maximum of 110 participants) is larger than has been used in previous guideline development because all corresponding authors of articles included in the systematic review (encompassing researchers and educators in the field of EBP education, statisticians and epidemiologists) will be invited to participate in the Delphi survey .
Traditionally the Delphi survey uses a minimum of three and a maximum of four rounds or until consensus is achieved . The number of rounds used in Delphi surveys to date varies from as few as two [12, 28, 30] to as many as six . Four rounds will be used in the proposed Delphi survey to maximise open ended survey responses in the first two rounds, and allow for additional items identified in the systematic review to be introduced in round three. Commencing the Delphi survey with an open ended question rather than using a pre-determined list of items will ensure that the researchers’ views are not imposed upon the participants, thus enabling the participants to suggest all possible outcomes for consideration .
Previous studies have used a systematic review to generate a pre-determined list of items to open a Delphi survey . In the current program of research, a systematic review will be used to inform the Delphi survey with a list of items identified in the review incorporated into the Delphi at the completion of the second round. This should provide sufficient transparency and time over two further Delphi rounds to determine the opinion of the panel regarding the importance of the items and whether or not they should be included as reporting requirements.
It is recommended that the level of consensus for the Delphi survey is both clearly defined and declared ‘a priori’ . However there are no current internationally accepted criterion standards to determine whether a consensus has been reached . Previous Delphi studies used in the development of reporting guidelines for health research do not report adequate detail to provide a guide for this study. Only two of the Delphi surveys used in the development of reporting guidelines for health research reported the level of agreement required to achieve consensus [30, 33] and consensus was not declared ‘a priori’ for any of the Delphi surveys [22, 30]. Consensus can be defined as a level of agreement and calculated using a mathematical formula or equation (e.g. percentage agreement, kappa analysis). However a consensus opinion is not necessarily the ‘correct’ opinion nor is it necessarily an accurate answer to a question . For the purpose of this study consensus will be defined ‘a priori’ and deemed to be achieved for each item using descriptive statistics of >80 per cent agreement .
The importance of reporting guidelines are well recognised and the benefits of reporting guidelines that have been developed using robust and widely accepted methodologies include improving the standard and transparency of reporting, assisting peer reviewers and editors to strengthen manuscripts and potentially streamlining research funding applications [14, 22]. The advantages of E&E documents, previously described as an ‘essential element’ in the development of reporting guidelines, are less widely recognised, and as a result, these documents have been largely overlooked by guideline developers. The E&E document provides the background and justification for the reporting guideline and can assist users by providing examples of how to report information . There are very few E&E documents available in comparison to the large number of reporting guidelines available, with fewer than 15 per cent of reporting guidelines providing an accompanying E&E document . The development of an E&E paper forms an essential part of the final stage of this program of research.
To determine how useable documents are, requires testing in the real world and Moher, Schulz, Simera and Altman (2010) recommend considering a pilot testing process in the development of the reporting guideline. However most reporting guidelines do not include a pilot testing process, with the minority of reporting guidelines (n = 11, 13.6%) reporting a pilot testing process . Pilot testing is planned for this program of research to determine whether the GREET statement and E&E paper are written at an appropriate level and whether they provide sufficient instruction for potential users. This will be achieved by gaining two perspectives from participants; an editorial perspective and a user perspective. The Delphi participants will be invited to review the documents and provide feedback with respect to the layout, wording and structure of the documents. Researchers will be invited to use the GREET statement and E&E paper to report their educational interventions and to provide feedback on the usability of the GREET statement and E&E paper.
There are knowledge frameworks other than EBP, but this decision making paradigm is rapidly becoming part of accreditation requirements for entry-level and post graduate health programs in educational institutions and various health care registration bodies. There is a rapidly growing body of research concerning educational practices used to teach and facilitate knowledge and skills in EBP, hence the consensus papers concerning what should be taught in EBP (Sicily 1) and the development of specific outcome instruments for use in assessing EBP foundation knowledge, skills, behaviours and attitudes (Sicily 2). There are philosophical differences in terms of which frameworks are appropriate for exploring educational interventions and definitions of what evidence should be considered. Regardless of which framework is proposed or accepted, there is value in having a guideline for use when reporting the educational intervention. This guideline would allow consistency in describing the educational intervention irrespective of the study design / methodological approach used by researchers.
The development of the GREET statement and E&E paper will be the culmination of a development process that is both robust and applies widely accepted methodologies. This will be the first reporting guideline for EBP educational interventions and as such is an important milestone. As the reporting of EBP educational interventions is inconsistent at best, improving the standard of how educational research is conducted, reported and published is imperative. Developing the GREET statement will provide a reporting guideline for EBP educational interventions that can be used in conjunction with existing guidelines for research design to potentially benefit all stakeholders in EBP education: researchers, educators, editors, reviewers and students.