We’re sorry, something doesn't seem to be working properly.
Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.
How do junior medical officers use online information resources? A survey
BMC Medical Educationvolume 16, Article number: 120 (2016)
Online information resources function dually as important learning tools and sources of the latest evidence-based recommendations for junior medical officers (JMOs). However, little is currently known about how JMOs utilise this information when providing care for their patients. This study aimed to examine the usage and experience of online information resources amongst JMOs in South Australia to ascertain (i) the type of resources accessed, (ii) the frequency, (iii) the intended purpose, and (iv) the perceived reliability.
A survey instrument using multiple choices, five-point Likert scales and free-text comments was developed and distributed through SurveyMonkey to South Australian JMOs between 1 May 2014 and 30 June 2014.
Of the 142 surveyed, 100 JMOs (70.4 %) used online information resources as their first approach over all other resources available. JMOs overwhelmingly (94.4 %, n = 134) used online information resources at least once per day, with the most frequent purpose for use being information regarding prescription medication (82.4 %, n = 117, reported ‘very frequent’ use). JMOs stated online resources were necessary to perform their work and, of the different types of information accessed, they rated peer-reviewed resources as the most reliable.
JMOs strongly rely upon online clinical information in their everyday practice. Importantly, provision of these resources assists JMOs in their education and clinical performance.
The Internet has caused a significant shift in the way doctors source clinical information, with an increasingly greater amount of medical knowledge now available and being accessed online . The importance placed on evidence-based practice ensures that the Internet has become an essential tool as it contains the most up-to-date information in vast quantities . With the field of medicine being a continually renewing and expanding body of knowledge and clinical practice, it is challenging for all doctors to keep abreast of the latest evidence and therapeutic recommendations. This challenge is particularly so for junior medical officers (JMOs), who do not have the experience and knowledge-base of their senior colleagues .
This increased role for information technology appears to be particularly embraced by junior doctors , who are typically younger and more intuitive with technology. Given their limited clinical experience, it is reasonable to expect that online clinical information resources would serve to equip JMOs with additional knowledge to assist them in diagnosing and managing patients’ conditions. Such sources would offer an important mechanism for ongoing education, as well as assisting in the clinical and diagnostic process. From clarifying a medication dosage to looking up clinical approaches for complex medical presentations, evidence-based platforms such as electronic Therapeutic Guidelines  or UpToDate  are designed to help doctors to make the best decisions for their patients.
Many hospitals provide open access to various clinical resources online through hospital servers for their doctors to use in their work. This represents a significant financial commitment to assist hospital doctors to achieve best practice. However, little is currently known about the way junior medical officers rely on such clinical information resources or how they use these resources when providing health care. This is particularly important as much misinformation exists online from non-peer reviewed sites, and this could have adverse patient outcomes if inappropriately relied upon [7, 8].
The aim of this study was to evaluate how South Australian hospital JMOs used the clinical resources online and their experience of them. It aimed to ascertain (i) the type of online information resources accessed, (ii) the frequency of access, (iii) the intended purpose for consulting the resource, and (iv) the perceived reliability of the information retrieved. This information may have implications in guiding how financial resources are allocated for JMOs, improvement of training experience for JMOs and, potentially, maximising patient outcomes.
A 16-question online survey instrument was developed by the authors to address the stated aims of the investigation. This included a combination of multiple choice questions, five-point Likert scales and free-text comment boxes. The survey questions were arranged into five distinct sections: demographics, types of resources used, resource reliability, frequency/ease of access, and general comments. These sections contributed to addressing the four key aims of the study, which were (i) the type of resources accessed, (ii) the frequency of use, (iii) the intended purpose, and (iv) the perceived reliability. The questions were uploaded onto the SurveyMonkey website (SurveyMonkey, Palo Alto, CA, USA) for distribution and data collection. The full survey used in this study is included in the Additional file 1.
Junior doctors were recruited for voluntary participation in this study, which solely consisted of completion of the survey as described above. The inclusion criterion for each participant was a requirement to hold current employment as an Intern or prevocational Resident Medical Officer (RMO) in a South Australian public hospital or affiliated training location. In 2014, the Department of Health in South Australia employed 278 Interns and 271 RMOs, totalling 549 Junior Medical Officers across the different Local Health Networks .
Participants were recruited via hospital email contact lists, the South Australian Junior Medical Officers Forum newsletter and verbally at face-to-face hospital training sessions. Potential participants were provided with a URL which would direct them to information about the study with a link for commencement of the survey. The survey was open for the period between 1 May 2014 and 30 June 2014 and took approximately seven minutes to complete.
Participation in this study was strictly voluntary. Detailed explanation was provided prior to the commencement of the survey regarding the purpose of the study and the way the collected information would be used. Informed consent was implied by the potential participant clicking the option to proceed to answering the questions after reading the information slide. Participants had the option to withdraw consent at any time prior to submitting responses.
All information was anonymous, although participants were given the option to submit their contact details to enter a prize draw for a tablet device. If a participant provided this information, the data were de-identified prior to analysis.
All data were computed using Microsoft Excel 2010 (Microsoft Corporation, Redmond, WA, USA). Results are presented either as percentage or as mean ± standard deviation.
In total, 142 junior doctors (74 Interns and 68 RMOs) were surveyed from multiple different sites across South Australian health services. Given that the total number of prevocational junior doctors in South Australia was 549, the response rate was 25.9 %.
Types of resources accessed by junior doctors
When junior doctors were asked what their first approach would be to answer a clinical query, online information resources were the most frequent option selected. Indeed, 70.4 % (n = 100) of junior doctors replied that this would be their first approach a “few times each day” (Table 1). The next most preferred options were asking a senior colleague or a peer colleague, which were only pursued a “few times a day” by 50.7 % (n = 72) and 29.5 % (n = 42) of junior doctors respectively. Print textbook and original articles are the least likely first approach, with almost half of junior doctors using them less than once per week.
Junior doctors were given a list of the online information resources available either through the South Australian Health online portal or were anecdotally reported to be used frequently by junior doctors. They were asked which in the list they had used in the month prior to the survey and almost all respondents had used Australian Medicines Handbook  (95.1 %, n = 135), electronic Therapeutic Guidelines  (94.4 %, n = 134) and UpToDate  (88.0 %, n = 125) (Table 2). Respondents were additionally given a free-text box to list other information resources they have accessed which were not present in the list shown in Table 3. The resources mentioned were South Australian Perinatal Practice Guidelines  (n = 3) and DermNET  (n = 2).
Frequency of use
In addition to their preferred approach for addressing a clinical query, the respondents were also asked about how frequently they used such resources. Regarding online information resources, 117 of the 142 junior doctors surveyed (82.4 %) used these resources a few times daily, 17 (12 %) use them daily and eight (5.6 %) use them once every few days (Table 3). No response was received for “once a week”, “less than once a week” and “less than once every few weeks”.
Of the common online information resources specifically referenced in the survey, the frequency of usage was examined. Australian Medicines Handbook  was “used daily” by 59.9 % (n = 85) of all junior doctors (Table 4). Additionally, 40.1 % (n = 57) of junior doctors use electronic Therapeutic Guidelines  daily. Although local hospital clinical guidelines were used by 76.8 % (n = 109) of junior doctors in a month prior (Table 2), they were “often used” by 49.3 % (n = 70) and “used daily” by 28.9 % (n = 41) of junior doctors (Table 4). In contrast, while 88.0 % of junior doctors had used UpToDate  (n = 125) in the prior month, this resource was only used “daily” by 22.5 % (n = 32) of junior doctors. Research databases were only used “rarely” and “sometimes” by most junior doctors (26.8 %, n = 38, and 45.1 %, n = 64, respectively).
The purpose junior doctors had for accessing online resources was assessed via the respondents being presented with a list of common reasons for use and they were then asked how frequently they accessed the Internet for each reason. This employed a five-point Likert scale that ranged from “very frequently” to “never”. The survey showed that 82.4 % (n = 117) of junior doctors reported that, on average, they access online information resources for medication information “very frequently”. Therapy information was the next most common, where 54.2 % (n = 77) of junior doctors reported “frequently” seeking related information. Less common purposes were for choosing investigations, researching complex clinical presentations and clarifying diagnoses, where 56.3 % (n = 80), 52.1 % (n = 74) and 43.7 % (n = 62) of junior doctors respectively reported that they only “sometimes” seek this information for these purposes (Table 5).
Participants were also invited to provide free-text responses at the conclusion of the survey. The majority of responses in this section stated that online information resources were considered necessary, and sometimes critical, to the job by the surveyed cohort of junior doctors.
Using a scale from 1 to 5 where 5 is “most reliable” and 1 is “not reliable”, junior doctors rate electronic Therapeutic Guidelines  at 4.83 ± 0.03 and Up-to-Date at 4.65 ± 0.04. 118 of 142 junior doctors (83.1 %) rated electronic Therapeutic Guidelines as very reliable, while 94 (66.2 %) gave UpToDate  the same rating. Other online information resources assessed for reliability included print textbooks and journals (3.85 ± 0.07), Wikipedia  (2.47 ± 0.08) and random Google  search results (2.48 ± 0.08). All of these options were rated as being markedly less reliable.
Online information has become a significant source of knowledge to complement junior doctors’ ongoing education and learning. Previous literature has highlighted that online clinical resources are important for junior doctors as learning tools . We found that the reliance on online clinical resources is overwhelming, as more than 80 % of junior doctors surveyed in the present study accessed them multiple times per day. When in a moment of doubt, the convenience of online information resources enables clinicians to readily seek and clarify the information they need. We expect the reliance on information resources to be more pronounced in junior doctors who are still in the steeper phase of learning at this early stage of their career.
Results from this study allude to the importance of reliable and valid online clinical resources for junior doctors to perform on the job. More than half of the respondents are first year doctors who, as part of their medical clerkship, gain the majority of their learning and experience through direct clinical contact with patients. In addition to promoting work-based learning, this has significant direct flow-on benefits to standards of healthcare. A previous investigation by Westbrook et al.  demonstrated both objectively and subjectively that the primary intended purpose of clinicians’ use of online information is for patient care, rather than self-directed study or research, and this use is particularly intense when a new patient is initially admitted.
Selected online information resources provide current information in contrast to textbooks. Indeed this study observed that print textbooks are the least preferred first option for junior doctors when seeking information. Previous studies have noted that most interns and residents do not frequently use research databases and original articles compared to online information resources and print textbooks [14, 16]. This trend was observed in our study as most junior doctors do not prioritise research databases and original articles as a first approach when seeking information. Our study shows that junior doctors show a significant inclination to access electronic resources, which is in contrast to an earlier study that found that many senior doctors (specialist consultants and general practitioners) reported that they did not have the required computing skills to efficiently search online and therefore used it less commonly than printed texts . This earlier study also found that senior doctors had a distinct preference for sourcing information by asking a colleague or using a printed text . The findings of the present survey indicate that junior doctors are leading the trend towards using technology in answering clinical questions.
Our results show that Australian Medicines Handbook , Therapeutic Guidelines , UpToDate  and local hospital clinical guidelines are the most commonly used and most reliable sources available to junior doctors. This suggests that this group of clinicians generally access online information resources in order to answer a specific clinical question. Particularly in this survey, medication and therapy information are the most common information junior doctors seek, which is consistent with previous studies [17, 18].
The use of high-quality evidence from online literature has also been shown to be beneficial for solving clinical quandaries. Westbrook et al.  conducted an experiment where experienced clinicians were asked to provided answers to questions relating to various clinical scenarios before and after the use of online information. The use of the online resources was found to significantly improve the accuracy of the answers provided, indicating the potential benefit to patients if their doctors are able to consult such sources when looking after them. Indeed, when surveyed, clinicians reported that they had personally experienced online information resources improving patient care . The benefits could be expected to be even more pronounced for junior doctors who, unlike the senior clinicians in this study, do not yet have the same wealth of experience of medical practice to assist them. Given the uptake of use amongst junior doctors as shown in our study, hospitals should provide the infrastructure for junior doctors to access online information resources.
There are some limitations to our study. The actual usage records of junior doctors were not directly examined. Instead the data analysed were junior doctors’ subjective self-reports and this would be limited both by their recall, but also by a potential bias to the needs of the current clinical rotation they were undertaking. The survey instrument designed was specifically brief and intended to maximise responses in a cohort that is notoriously time-poor. This was successful, as a quarter of potential respondents agreed to provide the information anonymously. Care should also be taken to generalise the results to other health jurisdictions outside South Australia, as the online clinical resources available could be different in terms of hospital subscriptions and ease of access.
Green et al.  has previously shown that a junior doctor develops two clinical questions for every three patients encountered including diagnosis and therapy . Alarmingly, only 29 % of the questions were ever pursued for an answer. That study was conducted in a time where print medical textbooks were the main media for information and learning. The results shown in the present survey will be heartening for clinical teachers as junior doctors today are actively seeking answers for questions they have relating to patient care. Online clinical resources can effectively integrate education with everyday clinical practice. Additionally, a better understanding of the purpose junior doctors have for using the resources and the specific nature of the resources accessed could be beneficial to clinical teachers aiming to address potential education gaps.
This study indicates for the first time that online information resources are strongly relied upon by junior medical officers in their everyday clinical practice. This is part of a generational shift from print media to Internet-based information. Online clinical resources are important learning tools and sources of the latest evidence-based recommendations for JMOs. These are highly valued by the doctors who use them to make informed decisions for their patients. We recommend the provision of these resources as essential for JMOs’ clinical performance and patient safety.
Ethics approval and consent to participate
Ethical Approval for this study was obtained from the SA Health Human Research Ethics Committee (ref: HREC/15/SAH/24). All participants read a ‘participant information’ before taking part and completion of the survey was taken as implied consent.
Availability of data and materials
Survey questionnaire is listed in the Additional file 1. The complete dataset can be requested from the first author.
junior medical officer
Younger P. Internet-based information-seeking behaviour amongst doctors and nurses: a short review of the literature. Health Info Libr J. 2010;27(1):2–10.
Hartzband P, Groopman J. Untangling the Web--patients, doctors, and the Internet. N Engl J Med. 2010;362(12):1063–6.
Starmer K et al. Blind prescribing: a study of junior doctors’ prescribing preparedness in an Australian emergency department. Emerg Med Australas. 2013;25(2):147–53.
Payne KB, Wharrad H, Watts K. Smartphone and medical related App use among medical students and junior doctors in the United Kingdom (UK): a regional survey. BMC Med Inform Decis Mak. 2012;12:121.
electronic Therapeutic Guidelines. Therapeutic Guidelines Limited, Melbourne. Available from: http://www.tg.org.au/index.php?sectionid=71. (Accessed 20 September 2015).
UpToDate, UpToDate, Massachusetts. Available from: http://www.uptodate.com/contents/search. (Accessed 20 September 2015).
Scullard P, Peacock C, Davies P. Googling children’s health: reliability of medical advice on the internet. Arch Dis Child. 2010;95(8):580–2.
Tang H, Ng JH. Googling for a diagnosis--use of Google as a diagnostic aid: internet based study. BMJ. 2006;333(7579):1143–5.
A Report on the Junior Doctor Allocations in South Australia for positions commencing January 2014. Medical Workforce, South Australian Health. April 2014, Adelaide.
Australian Medicines Handbook. Australian Medicines Handbook Pty Ltd, Adelaide. Available from: https://amhonline.amh.net.au/. (Accessed 20 September 2015).
DermNet NZ. DermNet New Zealand, Hamilton. Available from: http://www.dermnetnz.org/. (Accessed 20 September 2015).
Wikipedia: The Free Encyclopedia. Wikimedia Foundation. Available from: https://en.wikipedia.org/wiki/Main_Page. (Accessed 20 September 2015).
Google. Google Inc.; Available from: https://www.google.com.au/. (Accessed 20 September 2015).
Phua J, Lim TK. Use of traditional versus electronic medical-information resources by residents and interns. Med Teach. 2007;29(4):400–2.
Westbrook JI, Gosling AS, Coiera E. Do Clinicians Use Online Evidence to Support Patient Care? A Study of 55,000 Clinicians. J Am Med Inform Assoc. 2004;11(2):113–20.
Ely JW et al. Obstacles to answering doctors’ questions about patient care with evidence: qualitative study. BMJ. 2002;324(7339):710.
Green ML, Ciampi MA, Ellis PJ. Residents’ medical information needs in clinic: are they being met? Am J Med. 2000;109(3):218–23.
Schilling LM et al. Residents’ patient-specific clinical questions: opportunities for evidence-based learning. Acad Med. 2005;80(1):51–6.
Westbrook JI, Coiera EW, Gosling AS. Do Online Information Retrieval Systems Help Experienced Clinicians Answer Clinical Questions? J Am Med Inform Assoc. 2005;12(3):315–21.
Wheeless, C.R., Wheeless’ Textbook of Orthopaedics; Available from: http://www.wheelessonline.com/. (Accessed 20 September 2015).
Australian Injectable Drugs Handbook. Society of Hospital Pharmacists of Australia, Melbourne; Available from: http://www.shpa.org.au/Publications/Australian-Injectable-Drugs-Handbook. (Accessed 20 September 2015).
TOXINZ. New Zealand National Poisons Centre, Dunedin; Available from: http://www.toxinz.com/Search. (Accessed 20 September 2015).
Best Practice. BMJ Best Practice; Available from: http://bestpractice.bmj.com/. (Accessed 20 September 2015).
Clinical Practice Guideline. Royal Children’s Hospital, Melbourne; Available from: http://www.rch.org.au/. (Accessed 20 September 2015).
PubMed. The National Center for Biotechnology Information, Maryland; Available from: http://www.ncbi.nlm.nih.gov/pubmed/. (Accessed 20 September 2015).
Google Scholar. Google Inc.; Available from: http://scholar.google.com.au/. (Accessed 20 September 2015).
Medscape. WebMD Health Professional Network, New York; Available from: http://www.medscape.com/. (Accessed 20 September 2015).
MIMS Online. MIMS Australia, Sydney; Available from: https://www.mimsonline.com.au/Search/Search.aspx. (Accessed 20 September 2015).
Pharmaceutical Benefits Scheme. Department of Health Pharmaceutical Benefits Scheme, Sydney; Available from: http://www.pbs.gov.au/. (Accessed 20 September 2015).
The authors would like to thank Dr Minh Nguyen, Chair of the South Australian JMO Forum 2014 and Ms Mary Peterson, SA Health Library Service Knowledge Manager, for their insight and assistance in developing the conceptualisation for the research question of this project. The iPad Mini as a participation prize was kindly provided by Avant Mutual Group Limited.
The authors receive no funding for this research. An iPad Mini as a participation prize for the survey was sponsored by Avant Mutual Group Limited, which did not have any role in the design, collection, analysis, and interpretation of data, in the writing of the manuscript or decision to submit the manuscript for publication.
The authors declare that they have no competing interests.
HC contributed to the conception and design of the study. He was involved in the acquisition, analysis, interpretation of data and drafting the manuscript. MW contributed to the conception and design of the study. He was involved in the analysis and interpretation of data. He also co-wrote the first draft of the manuscript. PS contributed to the analysis and interpretation of data, and participated in the revision of the manuscript. AJ contributed to survey design and content, and she was involved in the critical revision of the manuscript. All authors read and approved the final manuscript. All authors agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Dr Heng T Chong is a junior doctor currently in his prevocational training.
Dr Michael Weightman is currently a vocational psychiatry trainee in South Australia.
Dr Peranada Sirichai is a prevocational junior doctor at the Westmead Hospital in Sydney, Australia.
A/Prof Alison Jones is the manager of the South Australian Medical Education Training Unit. She is also an academic staff at the Faculty of Medicine, Nursing and Health Sciences at Flinders University, Australia.
JMOs and Clinical Information Resource Survey. (PDF 235 kb)