Effectiveness of smart phone application use as continuing medical education method in pediatric oral health care: a randomized trial
BMC Medical Education volume 19, Article number: 431 (2019)
Continuing education aims at assisting physicians to maintain competency and expose them to emerging issues in their field. Over the last decade, approaches to the delivery of educational content have changed dramatically as medical education at all levels is now benefitting from the use of web-based content and applications for mobile devices. The aim of the present study is to investigate through a randomized trial the effectiveness of a smart phone application to increase public health service physicians’ (PHS physicians) knowledge regarding pediatric oral health care.
Five of all seven DHCs (District Health Center) in Tehran, which were under the supervision of Tehran University of Medical Sciences and Iran University of Medical Sciences, were selected for our study. Physicians of one DHC had participated in a pilot study. All PHS physicians in the other four centers were invited to the current study on a voluntary basis (n = 107). They completed a self-administered questionnaire regarding their knowledge, attitudes, practice in pediatric dentistry, and background. PHS physicians were assigned randomly to intervention and control groups; those in the intervention group, received a newly designed evidence-based smartphone application, and those in the control group received a booklet, a CME seminar, and a pamphlet. A post-intervention survey was administered 4 months later and t-test and repeated measures ANCOVA (Analysis of Covariance) were performed to measure the difference in the PHS physicians’ knowledge, attitude and practice.
In both groups, the mean knowledge scores were significantly higher (p-Value < 0.001) in post-intervention data compared to those at baseline. Similar results existed in attitude and practice scores. Although the scores in knowledge in the intervention group indicating potentially greater improvement when compared to those of the control group, the differences between the two groups were not statistically significant (dif: 0.84, 95% CI − 0.35 to 2.02).
In the light of the limitations of the present study, smart phone applications could improve knowledge, attitude and practice in physicians although this method was not superior to the conventional method of CME.
Our clinical trial had been registered in Iranian Registry of Clinical Trials (registration code: IRCT2016091029765N1).
Continuing education aims at assisting physicians to maintain competency and to learn about emerging topics in their field. It is an important part of medical practice. The traditional in-person lecture has been considered the best method for continuing education ; however, it suffers from the limitations of being instructor-centered and the need for the presence of instructor and learner at the same time and place [2, 3]. On the other hand, educational booklets with a combination of images and text have been used for continuing medical education (CME) as a learner-centered method . In order to benefit from the strengths of these two methods, some programs have combined the use of traditional lecture sessions and booklets or pamphlets [5,6,7]. For instance, a study in Iran reported the effectiveness of delivering an educational booklet followed by a lecture session in improving nurses’ knowledge and attitudes regarding oral health .
Several methods such as films, television programs, and audio programs have been used for CME along with the development of distance education facilities. Distance learning may reduce inequalities in health education  and has found its place among other training methods as it has been used in a number of previous studies with promising results [9,10,11,12,13]. Online CME websites can provide easy access, and their interaction potential promises more effectiveness compared to traditional methods . However, insufficient access to evidence-based information, lack of sufficient searching skills, time shortage and financial cost are major barriers to access information via this approach .
In the last decade, approaches to the delivery of educational content have changed dramatically as medical education at all levels is now benefitting from the use of web-based content and mobile device applications, including smart phone applications [16,17,18,19,20,21,22,23]. Mobile phones and tablets offer communication, access to the scientific literature in real time, are portable, and provide easy access to information at the point of care . Also, smartphone applications could provide interactive learning and constant connection through question and answer sections. This seems particularly useful, since studies have concluded that, widely used CME delivery methods such as conferences and lecture sessions without practice-reinforcing strategies have little direct impact on improving professional practice . Moreover, compared to traditional lecture-based CMEs, interactive CMEs are more effective in promoting knowledge and changing physicians’ practice [9, 25]. Thus, interactive methods have been proposed as a tool to be used in CME [18, 19, 22]. Also, online CME methods may offer greater flexibility in training times, improve access by geographically dispersed learners, reduce travel expenses and time, and adapt to individual learner styles . Despite the emergence of smartphone applications as a potential approach to deliver CME, almost no study exist that investigated its effectiveness.
The American Academy of Pediatric Dentistry (AAPD) recommends a child’s first dental visit to occur within 6 months of the eruption of the first tooth and no later than 12 months of age . However, most children do not visit a dentist before the age of 3 in several countries [28, 29]. Very often, a child’s first visit with a family physician or pediatrician occurs earlier than the child’s first visit to a dentist. According to guidelines [30, 31], primary health care providers have to counsel families on teething and dental care [32, 33]. However, studies indicate family physicians and other primary care providers lack sufficient knowledge and have received little training in medical school regarding preventive dental care [34, 35]. Also, these studies reported physicians’ lack of knowledge and training as barriers for providing preventive oral health care to their patients specifically for children .
The aim of this study was to investigate the effectiveness of smartphone applications as a continuing education (CE) method to improve self-reported knowledge, attitudes and practice of public health service (PHS) physicians regarding pediatric oral health care.
Study design and subjects
The study population was a sample of general practitioners (n = 107) working in the District Health Centers (DHC) of Tehran. Each DHC supervises 15 to 20 public health centers with one to three PHS physicians in each center.
There are seven DHCs in Tehran and its satellite towns. We selected five of them which were under supervision of Tehran University of Medical Sciences and Iran University of Medical Sciences. Physicians of one DHC (South West) participated in our pilot study. All PHS physicians in the other four centers were invited to participate in this larger study on a voluntary basis (n = 107). The inclusion criteria was being a general practitioner and working in DHC. The randomization was done at DHC level. Two DHCs were selected through a simple randomization (by flipping coin) process for intervention so that all PHS physicians in these two DHCs received intervention. The other two DHCs served as controls (Fig. 1).
Assuming an equal standard deviation of two intervention groups at 80% power, the minimum difference between the two groups was calculated to be 1.704 in the knowledge and 1.818 in the attitude and 1.242 in the practice scores.
Questionnaire and variables
A questionnaire developed in a previous study and evaluated for content validity and reliability [37, 38] was selected as the data collection tool (Additional file 1). No personal identifiable information was collected. The questionnaire requested information on participant’s demographic characteristics (age, gender, work experience, whether or not working in private sector, and whether or not having a dentist in first-degree family), as well as items in the following domains:
Knowledge of pediatric oral health
The knowledge domain included four multiple-choice questions and ten questions with five-point Likert scale responses ranging from strongly agree to strongly disagree and including an option for “don’t know”. The responses were assigned a score of one for correct answers, and zero for incorrect and don’t know answers. For true statements, “strongly agree” and “agree” answers were given score one, and the other answers score zero. For false statements, “strongly disagree” and “disagree” answers were given score one and the other answers score zero. Questions tested the participant’s knowledge regarding the timing of primary and permanent tooth eruption, the time/age when tooth cleaning and brushing for children should begin, usage of fluoride (toothpaste and varnish), transmission of the bacteria that cause dental decay, the effects of pacifier sucking and mouth breathing, the advantages of sealant therapy, and dental trauma. By summing the scores, final scores, with a range of zero to 14 were calculated and sub-grouped into quartiles.
Attitudes toward pediatric oral health
The attitudes section comprised eight questions with five-point Likert scale response alternatives which ranged from strongly disagree to strongly agree and was scored from one to five. The range of final scores was from eight to 40. The questions asked PHS physicians’ opinions about oral health care, and the preventability of dental caries and periodontitis. They also were asked about the responsibility of PHS physicians to examine children’s oral cavity, the effectiveness of routine dental visits in preventing dental disease, importance of PHS physicians’ role in preventing oral diseases, association of oral health problems and general health problems, and tendency to implement preventive oral health activities.
Practice in pediatric oral health
The practice section contained two multiple choice questions, eight five-item Likert type questions with options very likely, likely, medium, unlikely, very unlikely (scored from 1 to 5, respectively), 11five-item Likert type questions with options strongly agree to strongly disagree (scored from 0 to 4, respectively), and 12 four-item Likert type questions with options never, rarely, occasionally, very frequently (with the first two options scored 0, and the second two scored 1). By summing the scores, final scores were calculated from 31 to 107 and sub-grouped into quartiles as described above.
Intervention and control groups
The intervention group received training through an evidence-based smart phone application (hereby referred as ‘application’ in this paper) designed for the purpose of the study. Although participants were instructed on how to use the application, there was a help section in the menu of the application that explained how to use it. PHS physicians could also submit their questions online and receive answers within 2 days. A reminder message was sent to the intervention group through the application itself 1 month following the first session.
PHS physicians in the control group received the same educational content as a booklet offered in the traditional method of CME. In addition, there was a Q&A session for this group 2 weeks after the first session. Also the “education and health promotion unit” staff of the health network sent a reminder in the form of a pamphlet to the booklet group.
The seminar and booklet covered the same topics as the application: information on pediatric oral and dental disease; caries and its etiology, signs and care; dietary habits; fluoride therapies and fissure sealant; and dental trauma.
Baseline data collection
One of the researchers (MB) visited all the PHCs and administered the baseline questionnaire in-person to the participants. One week after each visit, the same researcher collected the completed questionnaire. Baseline data collection was performed from November to December 2016.
Four months after baseline data collection, in one of the monthly meetings of the DHCs, the study questionnaire was distributed among the participants, and collected after 1 h. To measure changes at the participant level, we requested each participant to enter a person-specific code when completing the pre- and post-intervention questionnaires.
Figure 1 shows the flow diagram of the present study.
All numerical data were entered and analyzed using the IBM Statistical Package for Social Sciences (SPSS version 21.0). Descriptive statistics were obtained for gender, age, working experience and working sector. T-test and repeated measure analysis of covariance (ANCOVA) served to assess the statistical significance of differences between knowledge, attitude and practice scores of intervention and control groups.
Participation in the study was voluntary, and the responses were anonymous. All respondents provided their written informed consent. The Ethics Committee of Tehran University of Medical Sciences approved the study (IR.TUMS.REC.1395.2252). In addition, the study was registered in Iranian Registry of Clinical Trials (IRCT2016091029765N1).
Of the 107 physicians invited for the baseline data collection (50 in intervention and 57 in control group), 86 physicians (43 in intervention and 43 in control group) completed the questionnaire (total response rate = 80.3%). In both intervention and control group, all physicians completed baseline questionnaire participated also in post-intervention data collection. A quarter of the PHS physicians completing both baseline and post-intervention questionnaire were men, and the majority (n = 68, 79%) of them worked solely in the public health sector. The mean age was 39.2 years among the smartphone intervention group and 44.3 years among the control group (Table 1).
No significant differences existed between intervention and control group regarding demographic information.
The rate of using application in the last week leading to post intervention evaluation was 68.4% (varied from once a week to everyday).
The mean knowledge score among participants at baseline was 8.17 ± 2.03 (Table 2). At baseline, only 9.5% of the PHS physicians in the control group knew the correct answer to the question “Pacifier sucking in under-4-year-old children is a risk factor for dento-alveolar malformation” while the percentage of correct answers in the intervention group was 11.6%. In the control group, the biggest change (13.5%) in PHS physicians’ responses before and after the intervention were related to the question “Physicians should examine the oral cavity and teeth throughout their routine patient’s visits”. In the intervention group, the biggest change (20%) in the PHS physicians’ responses before and after the intervention was related to the question “Oral health care delivered by physicians is not efficient for patients”.
In both groups, the mean scores for knowledge, attitudes and practice were significantly higher at post-intervention data collection compared to that at baseline (Table 2).
Table 3 displays the knowledge, attitudes and practice differences of study participants in the intervention and control groups. Although the scores of intervention group in knowledge, attitudes and practice showed larger differences in pre and post scores compared to that of the control group, the differences between the two groups remained insignificant (Table 3).
Subgroup analysis by ANCOVA showed that the improvement of knowledge, attitudes, and practice scores in each study group remained independent from background factors (P > 0.05).
The present study investigated the effectiveness of CME on oral health delivered through a smartphone application and a booklet among PHS physicians. Both methods could improve knowledge, attitude and practice in physicians. However, the difference between the two groups was insignificant, showing no superiority of the smartphone app over the conventional method.
Many surveys and studies exist about benefits, barriers and risks of online CME. Although studies on health-oriented patient centered applications are available, research on using particular smart phone as a medium for CME among physicians is scarce. Certain studies [39,40,41,42] have found results similar to ours. Short et al., in 2005, conducted an online interactive CME in the field of intimate partner violence for physicians. The control group in their study received no training. Similar to our study, they concluded that online interactive CME made persistent changes in knowledge, attitudes and self-reported practice . Ryan et al., in 2009, compared the effectiveness of face-to-face and online CME among 62 general physicians. The course was about accreditation as pharmacotherapies prescribers for opioid dependence. Similar to the findings of our study, they reported significant improvement of knowledge among participants in both groups. Comparison of post-test scores of knowledge among the two groups also showed no significant difference. The same pattern also occurred in the attitude scores. They concluded that online CME was as effective as the face-to face method for increasing the knowledge of treatment and management of opioid dependence .
Similar to our study results in the knowledge section, Kim et al. in a study on educating nursing students to provide care for infant airway obstruction reported no significant difference in the knowledge score between the smart phone-based group and the lecture group . A Canadian study in 2011 evaluated the outcomes of an online CME course in the field of asthma without a control group and reported significantly increased level of knowledge in clinical area among health professionals . Their finding is in agreement with the results of the present study.
Other studies have reported significant differences in their findings. A study conducted by Pelayo in 2011 in Spain, which compared online training on palliative care to traditional self-training method, found a 14 to 20% increase in knowledge through the online method among primary care physicians. Moreover, this method led to significant improvement in attitudes and perception of confidence in symptom management and communication . Also, Kim et al. compared the effects of one-time lecture and smartphone application on skill of nursing students regarding infant airway obstruction. They reported that the skill score of students in the smart phone application group was significantly higher than that in the lecture group .
One of the advantages of CME through smartphone over conventional methods is the accessibility that smartphones provide. It is worth mentioning that smartphones, when used as a tool for CME, can provide access to educational content at the point of care at any time without adding any new asset to pocket . Moreover, the high rate of adoption of smartphone by physicians (84.5 to 94%) in 2012, indicates its potential to be used for CME .
The high response rate in baseline data collection (80.4%), and the fact that all participants who completed the baseline questionnaire also participated in post-intervention data collection can be considered as strengths of our study.
Limitation of study
The main reason that some of the physicians refused to participate was that they were too busy, which seems not to be unusual in studies on professional groups. To alleviate this limitation, physicians were given the privilege of continuing education credits for free. Also, gifts including toothbrushes and tooth pastes were given to the respondents. As a result of our sample size and according to wide range of CI in differences in main variables in Table 3, a possibility exists that a significant difference does exist between intervention and control, but the sample size was underpowered to detect it. Downloading the application in app group and the coverage of network in health centers was another limitation which was eliminated by using mobile modem in training sessions of application group. On the other hand, having a self-administered questionnaire may cause social desirability bias and lead to overestimation rather than underestimation of the reported attitudes and practice. Moreover, a risk of under-estimation exists in questionnaire surveys answered by lay people . Our study investigated the short-term outcome of CME through smartphone, and the long term effectiveness of this method needs to be further studied.
In the light of the limitations of the present study, smart phone applications could improve knowledge, attitude and practice in physicians although this method was not superior to the conventional method of CME. Other aspects of the use of this method such as cost and time savings, its widespread use and higher ease of accessibility need to be further investigated.
Availability of data and materials
The datasets generated and analyzed during the current study are available from the corresponding author on reasonable request.
American Academy of Pediatric Dentistry
Analysis of Covariance
Continuing Medical Education
District Health Center
Public Health Service
Question and Answer
Statistical Package for Social Sciences
Stephens MB, McKenna M, Carrington K. Adult learning models for large-group continuing medical education activities. Fam Med-Kansas City. 2011;43(5):334.
Anahita Enzevaei AM, Mojtahedzade R. Medical education, curriculum development, assessment of the principles of education and learning, derived from the education program at Dundee University of England. Tehran: Tehran Universiy of Medical Sciences; 2007.
Zohre Khakbazan SMR, Zeinalou A, Soheili S, Qarib M, Meshkati ZS, Mohamadi M. The process of teaching in medical science. Tehran: Tehran University of Medical Sciences; 2005.
Badiei M, Gharib M, Zolfaghari M, Mojtahedzadeh R. Comparing nurses’ knowledge retention following electronic continuous education and educational booklet: a controlled trial study. Med J Islam Repub Iran. 2016;30:364.
Abdollahy A, Cheraghi M, Yazdanic A, Rabiei M. Comparing the educational effects of lecturing method and using educational pamphlets and a combination of these on high-school students'awareness of aids [Summary in English]. Iranian Journal of Medical Education. 2004;Suppl10:126.
Yaghobian M, Yaghobi T, Salmeh F, Golmohammadi F, Safari H, Savasari R, et al. Comparing the effect of teaching using educational booklets and lecture along with educational booklets on nurses' knowledge about professional laws and regulations. Iran J Med Educ. 2010;9(4):372–80.
Hundley SA. A comparative study of traditional lecture methods and interactive lecture methods in introductory geology courses for non-science majors at the college level: The Ohio State University; 2007. Available at: https://etd.ohiolink.edu/!etd.send_file?accession=osu1196191640&disposition=inline. Accessed Mar 2019.
Tavukcu T, Arapa I, Özcan D. General overview on distance education concept. Procedia Soc Behav Sci. 2011;15:3999–4004.
Forsetlund L, Bjørndal A, Rashidian A, Jamtvedt G, O'Brien MA, Wolf FM, et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Libr. 2009;15(2):CD003030.
Klein KP, Miller KT, Brown MW, Proffit WR. In-office distance learning for practitioners. Am J Orthod Dentofac Orthop. 2011;140(1):126–32.
Ruf D, Berner MM, Kriston L, Maier I, Härter M. General practitioners online: the conditions are good, but use of the internet for continuing medical education found to be poor. Z Evid Fortbild Qual Gesundhwes. 2008;102(5):291–7.
Brown M, Bullock A. Evaluating PLATO: postgraduate teaching and learning online. Clin Teach. 2014;11(1):10–4.
Pouliquen B, Le Duff F, Delamarre D, Cuggia M, Mougin F, Le Beux P. Managing educational resource in medicine: system design and integration. Int J Med Inform. 2005;74(2):201–7.
Sherman A. Continuing medical education methodology: current trends and applications in wound care. J Diabetes Sci Technol. 2010;4(4):853–856.
Virtanen JI, Nieminen P. Information and communication technology among undergraduate dental students in Finland. Eur J Dent Educ. 2002;6(4):147–52.
Wallace S, Clark M, White J. It’s on my iPhone’: attitudes to the use of mobile computing devices in medical education, a mixed-methods study. BMJ Open. 2012;2(4).
Aungst TD. Medical applications for pharmacists using mobile devices. Ann Pharmacother. 2013;47(7–8):1088–95.
Kiser K. 25 ways to use your smartphone. Physicians share their favorite uses and apps. Minn Med. 2011;94(4):22–9.
Ozdalga E, Ozdalga A, Ahuja N. The smartphone in medicine: a review of current and potential use among physicians and students. J Med Internet Res. 2012;14(5):e128.
Yoo JH. The meaning of information technology (IT) Mobile devices to me, the infectious disease physician. Infect Chemother. 2013;45(2):244–51.
O’Neill KM, Holmer H, Greenberg S, Meara JG. Applying surgical apps: smartphone and tablet apps prove useful in clinical practice. Bull Am Coll Surg. 2013;98(11):10–8.
Mosa ASM, Yoo I, Sheets L. A systematic review of healthcare applications for smartphones. BMC Med Inform Decis Making. 2012;12(1):67.
Divall P, Camosso-Stefinovic J, Baker R. The use of personal digital assistants in clinical decision making by health care professionals: a systematic review. Health Inform J. 2013;19(1):16–28.
Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA. 1995;274(9):700–5.
Garattini L, Gritti S, De Compadri P, Casadei G. Continuing medical education in six European countries: a comparative analysis. Health Policy. 2010;94(3):246–54.
Horton WK. Designing web-based training: how to teach anyone anything anywhere anytime. New York: Wiley; 2000.
American Academy of Pediatric Dentistry. Oral health policies. American Academy of Pediatric Dentistry. Pediatr Dent. 2003;25(7 Suppl):11–49.
Douglass JM, Douglass AB, Silk HJ. A practical guide to infant oral health. Am Fam Physician. 2004;70(11):2113–20.
Wendt L-K, Carlsson E, Hallonsten A-L, Birkhed D. Early dental caries risk assessment and prevention in pre-school children: evaluation of a new strategy for dental care in a field study. Acta Odontol Scand. 2001;59(5):261–6.
Richerson JE, Abularrage JJ, Almendarez YM, Boudreau ADA, Cantrell PE, Hackell JM, et al. 2019 Recommendations for Preventive Pediatric Health Care. Pediatrics. 2019;143(3).
Futures B. Pediatrics AAo. Recommendations for preventive pediatric health care. Elk Grove: American Academy of Pediatrics; 2008.
Pediatrics AAo. Recommendations for preventive pediatric health care. Pediatrics. 1995;96(2):373–4.
Panagiotou L, Rourke LL, Rourke J, Wakefield JG, Winfield D. Evidence-based well-baby care. Part 1: Overview of the next generation of the Rourke Baby Record. Can Fam Physician. 1998;44:558.
Khami M, Bonabi M, Mohebbi S. Knowledge and attitude of health-network physicians toward pediatric oral health in Tehran in 2016. J Contemp Med Sci. 2018;4(1):12–16.
Rabiei S, Mohebbi SZ, Yazdani R, Virtanen JI. Primary care nurses’ awareness of and willingness to perform children’s oral health care. BMC Oral Health. 2014;14(1):26.
Lewis CW, Boulter S, Keels MA, Krol DM, Mouradian WE, O'Connor KG, et al. Oral health and pediatricians: results of a national survey. Acad Pediatr. 2009;9(6):457–61.
Rabiei S, Mohebbi SZ, Patja K, Virtanen JI. Physicians’ knowledge of and adherence to improving oral health. BMC Public Health. 2012;12(1):855.
Di Giuseppe G, Nobile CG, Marinelli A, Angelillo IF. Knowledge, attitude and practices of pediatricians regarding the prevention of oral diseases in Italy. BMC Public Health. 2006;6(1):176.
Short LM, Surprenant ZJ, Harris JM Jr. A community-based trial of an online intimate partner violence CME program. Am J Prev Med. 2006;30(2):181–5.
Ryan G, Lyon P, Kumar K, Bell J, Barnet S, Shaw T. Online CME: an effective alternative to face-to-face delivery. Med Teach. 2007;29(8):e251–e7.
Kim S-J, Shin H, Lee J, Kang S, Bartlett R. A smartphone application to educate undergraduate nursing students about providing care for infant airway obstruction. Nurse Educ Today. 2017;48:145–52.
Fleet LJ, Fox G, Kirby F, Whitton C, McIvor A. Evaluation outcomes resulting from an internet-based continuing professional development (CPD) asthma program: its impact on participants' knowledge and satisfaction. J Asthma. 2011;48(4):400–4.
Pelayo M, Cebrián D, Areosa A, Agra Y, Izquierdo JV, Buendía F. Effects of online palliative care training on knowledge, attitude and satisfaction of primary care physicians. BMC Fam Pract. 2011;12(37):1–11.
Helöe LA. Comparison of dental health data obtained from questionnaires, interviews and clinical examination. Eur J Oral Sci. 1972;80(6):495–9.
This study has been supported by Research Center for Caries Prevention, Dentistry Research Institute (95–01–194-31526), Tehran University of Medical Sciences, Tehran, Iran. The funding was used as personnel costs and costs of materials and trips for collecting data.
Ethics approval and consent to participate
All respondents provided their written informed consent. The Ethics Committee of Tehran University of Medical Sciences approved the study (IR.TUMS.REC.1395.2252). In addition, the study was registered in Iranian Registry of Clinical Trials (IRCT2016091029765N1).
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Bonabi, M., Mohebbi, S.Z., Martinez-Mier, E.A. et al. Effectiveness of smart phone application use as continuing medical education method in pediatric oral health care: a randomized trial. BMC Med Educ 19, 431 (2019). https://doi.org/10.1186/s12909-019-1852-z