Pilot implementation of a telehealth course for health professions students

Abstract

Objectives

This study examined the level of technology proficiency amongst healthcare professions students. Additionally, the study provides an evaluation of the pilot implementation, as well as the effect of a 7-module telehealth course on the level of adoption and future use of telehealth amongst future Australian healthcare workforce.

Methods

Students from four health-sciences departments at the University of Melbourne, Australia, participated in this pilot study by completing the course and an online questionnaire, which included both structured and open-ended questions. The questionnaire included: 12-items on socio-demographic and Internet utilization; 34-items about acceptance and use of telehealth adapted from the Unified Theory of Acceptance and Use of Technology (UTAUT2) questionnaire; and 22-items about confidence in using the Internet and ICT, adapted from Technology Proficiency Self-Assessment Questionnaire for 21st Century Learning (TPSA-C-21).

Results

The evaluation included 26 students who expressed confidence in their Internet/ICT skills They showed enthusiasm for telehealth and recognized its potential benefits, but also emphasized the value of face-to-face interactions. They requested information on legal and aspects and additional learning. Post-test assessments indicated improvements in overall acceptance and use attitudes towards telehealth and on six dimensions of the UTAUT2 instrument. Participation in the course indicated improvements in students’ overall acceptance and use attitudes and on six of the ten dimensions of the UTAUT2 instrument (p < 0.05).

Conclusion

This preliminary evaluation indicated that the telehealth course was a positive and enjoyable learning experience for students with appropriate structure and information. The course was successful in improving students’ acceptance and use of health technology. The study identified areas in which further development might be required. As such, the course represents a helpful approach for telehealth training among health professions students. Further evaluation with larger samples is required.

Introduction

Telehealth (inclusive of phrases; telemedicine, virtual care, etc.) is a technology that has matured significantly, with accelerated adoption in the last few years. It is characterized by rapid evolution and ongoing advancements. This progress has been driven by various factors, including (but not limited to): advancements in Information and Communication Technologies (ICT), rising patient expectations, preferences for innovative healthcare approaches, and the demand for flexible models of care [1, 2]. Telehealth serves as an effective tool to enhance access and opportunities for care, address specific health challenges, and broaden the scope for continuous training and professional development for healthcare professionals [1].

More recently, in a short period of time and largely due to the COVID-19 pandemic, there has been an expansion in the use and demand of telehealth services and ICT in all its modalities of health practices. This has been mostly to ensure the uninterrupted provision of health services to the population [3]. This situation has highlighted the urgent need and obligation to properly train the future health workforce on these new modalities of practice, which undoubtedly will involve some form of telehealth to operate effectively [4, 5].

Health profession students, as the future health workforce of the digital age, must be exposed to these technologies in a comprehensive and systematic manner so they can develop proficiency and competencies in this area, understanding the opportunities and limitations of working under this “new normal” [6]. Hence, there is a need to incorporate telehealth training in health professions curriculum. This would enable the acquisition of the necessary knowledge and skills to effectively incorporate telehealth technologies into their practice [7, 8].

A few months after the WHO declared the COVID-19 pandemic, The Faculty of Medicine, Dentistry, and Health Sciences (MDHS) at the University of Melbourne called for a task group to discuss initiatives to further expand, embrace, and develop the use of telehealth as a faculty, as opposed to each school/department working separately. The work group agreed that the implementation of telehealth in Victoria should incorporate two components: education, and implementation of telehealth models of care; both would work collaboratively, but in separate groups. The telehealth education arm oversaw the design and assessment of a telehealth learning program.

This telehealth learning program aimed to furnish entry-to-practice health professions students with essential skills and knowledge required for success as virtual healthcare practitioners. The learning materials were formulated by a panel of experts from the MDHS, as well as in-line with contemporary digital health capabilities [9]. They were crafted with the goal of providing health professionals with capabilities to meet the evolving demands of healthcare through the utilization of telehealth and ICT, equip them with fundamental knowledge that extends beyond teleconsultation and remote triage, enable them to identify healthcare services and procedures suitable for telehealth delivery, and prepare them for real-world practice by emphasizing telehealth integration and healthcare regulations.

It was expected that course modules would grant access to the theoretical foundations of state-of-the-art telehealth, fostering reflective spaces for health professions students. Modules were designed in alignment with contemporary best-practices surrounding digital health workforce education capability [9, 10]. The focus was not solely on technology but on developing students’ abilities to communicate with diverse stakeholders (such as patients, fellow clinicians, and care teams), provide patient-centered education and care, adeptly adopt new technologies, identify barriers and facilitators to care within this model, and collaborate effectively in multidisciplinary healthcare teams.

This telehealth course was developed as a blended learning experience. The course included seven modules:

• Module (1) Introduction to telehealth.

• Module (2) Telehealth foundations.

• Module (3) Telehealth platforms, service modalities, and their architectures.

• Module (4) Ethical aspects in telehealth and telehealth care services. Cybersecurity, privacy, and confidentiality.

• Module (5) Normative and regulatory, financial/reimbursement service models.

• Module (6) Interactive face-to-face workshop (i.e., mini practicum activities, interaction with telehealth software, and standard remote assessments, records management systems, virtual home visits, tele-referrals, communications: interprofessional and with patients).

• Module (7) Career specific scenarios/case-studies and future directions. This module in the first iteration of the course was intended for dentistry and physiotherapy students. Therefore, the module contained specific scenarios for those professions.

Each module was the equivalent of 60 min learning time - which included engaging with online media, reading, video-based content, audio-visual materials, and activities. Module 6 was longer, lasting around 3 h.

This study provides an evaluation of the effect of a seven-module telehealth course on the level of adoption and future use of telehealth amongst a group of the future Australian healthcare workforce. More specifically, the objectives were to report on the level of self-assessed technology proficiency, acceptance, and future adoption of these technologies by members of the future healthcare workforces. This assessment was undertaken with students across four interprofessional health sciences Departments/Schools at the University of Melbourne, Australia: Dentistry, Physiotherapy, Social Work, and Audiology and Speech Pathology.

Several universities have introduced these contents in the curriculum. However, evaluation is not commonly undertaken. Still, Schools and Faculties should make evidence-based decisions about contents in their curriculum. There is limited evaluation of effectiveness in dentistry [11], medicine [5], nursing [12], and physiotherapy [13]. Thus, this evaluation was considered an important first step alongside an understanding of implementation challenges and associated issues to aid in integrating telehealth into the educational environment.

Methodology

With the approval of the Human Research Ethics Committee from The University of Melbourne (Study ID: 20529), students from five health science award courses (across four departments) at the University of Melbourne were recruited to participate in this project. In its first iteration the course was piloted amongst Dentistry, Oral Health Therapy, Physiotherapy, Social Work, and Clinical Audiology students.

Data for this pilot test was collected between August and September 2021. During this period students were invited to participate in the study by first completing an online anonymous, 68-item questionnaire; completing six self-paced online telehealth modules and one interactive live workshop module (i.e. Module 6); and an online post-course anonymous questionnaire. To ensure that students remained anonymous, a code was created by the participants themselves. The post-course questionnaire contained 38 items about perceptions surrounding using telehealth.

In addition to the structured questions, the post-course form included a free-text section that allowed participants to provide additional comments or feedback free of coercion. Their feedback fell into the categories of “comments about the course” or “constructive feedback.” This categorization process was designed to facilitate the analysis of participants feedback and gain a deeper understanding of the participants’ perspectives and gather more comprehensive data for analysis.

The questionnaire included items on socio-demographic characteristics (i.e., age, sex, and course of study) and course level data, as well as Internet utilization information (i.e., frequency and devices used). Additionally, the instrument included 22 items asking about Internet and information communication technology (ICT) use, adapted from Technology Proficiency Self-Assessment Questionnaire for 21st Century Learning (TPSA C-21) [14]. The instrument also contained a subsequent 34 items about perceptions surrounding using telehealth [15]. Students’ perceptions, acceptance around, and future use of telehealth were captured using questions according to the Unified Theory of Acceptance and Use of Technology (UTAUT2).

The UTAUT2 was developed by Venkatesh, Thong, and Xu, with the aim of further addressing and explaining technology adoption and use [16]. The UTAUT2 model is a validated and reliable framework that has been widely adopted in research in several fields, including digital health, to understand the factors that affect the adoption and use of technologies in various organizational settings [16, 17]. To ensure the validity and reliability of the instruments, the models were adopted with minimal modifications. The post-course questionnaire contained questions regarding the UTAUT2 only.

Internet utilization information included: Frequency of online access; participants classified themselves according to frequency of visit to Internet sites, as ‘At least hourly, ‘At least daily, ‘At least weekly’, ‘At least monthly’, and ‘Less than once a month’. The device use list included 9 alternatives: Mobile smartphone; tablet (i.e., iPad); desktop computer; laptop; smart TV; gaming console (i.e., Xbox, PlayStation); smartwatch (i.e., Apple watch, Fitbit); eBook reader (i.e., Kindle, Kobo, etc.); and smart home assistant (i.e., Google Home, Amazon Alexa).

The TPSA C-21 was assessed on a 5point ordinal Likert scale, according to the response that best described their confidence in using the Internet and ICT as ‘Strongly disagree’ to ‘Strongly agree’. These responses were weighted as ‘Somewhat agree’= 0.5, ‘Strongly agree’= 1, all other responses = 0. The weighted values were added to yield a total confidence (proficiency) score.

The UTAUT2 model consists of 10 constructs that describe perceptions about using telehealth technology. These are performance expectancy; behavioural intention to use the system; effort expectancy; social influence; facilitating conditions; hedonistic motivation; price value; habit; self-perception; and usage behaviour. Students were asked before and after the course about their level of perceptions about using telehealth technology. Items in the UTAUT2 were assessed on a 5point ordinal Likert from ‘1’: ‘Strongly disagree’ to 5: ‘Strongly agree’. The responses were added to yield a pre- and a post-usability score.

Sample size calculation were based on the minimum requirements to detect a change from pre- to post-intervention (i.e., participation in the telehealth course), it was estimated that a total sample size of 26 participants would be necessary to detect a mean difference of five-tenths (0.50) of the standard deviation in major outcomes between paired observations of participants (pre-test vs. post-test), at the uni-dimensional significance criterion of 0.05, and a power of 0.80 [18]. The study used convenience samples of male and female students, 18 years or older, enrolled in the aforementioned courses.

Quantitative data were analysed using IBM SPSS to statistically compare results between different socio-demographic and Internet use variables. The analyses included descriptive statistics (means, standard deviation, and frequencies). A one group pre-test–post-test quasi-experimental design was chosen to evaluate the telehealth course. The analysis tested the hypothesis that those who participated in the course would exhibit significant improvements after adjusting for pre-test scores. Categorical and ordinal variables were analysed utilizing chi-square analysis. For continuous variables (technology proficiency results), due to the small sample size, data were analysed using the Wilcoxon signed-rank test to compare pre-test and post-test scores. Statistical analysis was conducted using statistical analysis software (SPSS v.22.0 for Windows, SPSS Inc., Chicago, Illinois). An a-priori level of significance was set at p < 0.05.

In addition to the quantitative analysis, the inclusion of thematic content analysis of free text responses provided better understanding of students’ satisfaction with the telehealth course. The analysis comprised identifying themes and key words. Themes were data-driven and categorised based on questions asked to participants.

Results

In total, 26 students enrolled in the training course and completed the pre-assessment instrument. The majority (82.6%) were female and aged 29 years-old or younger (76.9%). Nine participants (34.6%) were from the Dental School (i.e., dental, and oral health therapy), 23.1% were from Physiotherapy, another 23.1% were Social Work students, and the remaining 19.2% were Audiology students.

All participants had access to a smartphone and a laptop. Frequency of use of the Internet was high, 43.3% of the students accessed the Internet at least every 10 min. Another 15.4% and 23.1% accessed every 30 min or every hour, respectively. No statistically significant differences in frequency of use were found by profession, sex, or age group.

Level of confidence with technology

When students were asked about their level of confidence with technology (their self-assessed proficiency), results indicated (Table 1) that they were confident in their ability to perform most of the tasks. Confidence scores ranged from 13.0 to 22.0, with an overall mean of 19.8 (s.d. 2.5). Half of the participants scored 18.0 points or more on the confidence scale. There were no significant differences by gender, age group, frequency of use, or profession.

Table 1 Students’ responses to technology proficiency questionnaire (%)¹

Except for one item (item 11), participants scored highly in all the items (> 55.0%). In particular, they all (100%) strongly agreed that they were able to: find webpages related to my subject matter interests (Item 1); use the computer to create a slideshow presentation (Item 7); and download and view streaming movies/video clips (Item 20). In another seven items, all students were either Strongly or Somewhat confident that they would (Items 3,6,9,18,19,21,22); and that they confidently would download podcasts and audio books; or send phots via a smartphone; or and safe and retrieve files from the cloud.

On the other hand, although the majority (> 50%) were strongly/somewhat confident in their proficiency, they were less confident in areas requiring deeper skills such as: creating a database of information (26.9%) (Item 8); integrating mobile technology in their work or creating a blog (26.9%) (Item 14). Students were also less confident in describing software programs or apps they would use in their role as healthcare professionals (19.2%) (Item 10), and to a lesser extent using social media tools as part of their role as health professional (7.6%) (Item 13).

Healthcare profession students felt less confident in two items: writing a plan with a budget to buy technology that would support me in my role as health professional (Item 11); and on how to create their own webpage (Item 2). When asked about how satisfied they were writing a plan with a budget to buy technology that would support me in my role as health professional, although the majority was either somewhat confident or slightly confident (57.7%), 15.4% was neutral and, more importantly, another 26.9% somewhat or strongly disagree with the statement.

The majority were also confident (strongly: 30.8%; or somewhat: 30.8%) that they could create their own webpage. However, 15.4% were neutral, and another 19.2% were somewhat confident and 3.8% were not confident.

Course evaluation

Six participants provided free text comments. Qualitative analysis of these data in indicated that while preferring face-to-face interaction with clients/patients, they also showed a strong interest in incorporating telehealth into their future career noting the potential benefits and opportunities it offers.

Not only for convenience as a substitute but it can also be implemented as an additional service in their care plan for better on going patient support and care (ID:12)]

Nonetheless, participants expressed a need for more specific information on the legal and insurance aspects of telehealth to ensure compliance in their practice.

“It would be good to have more specific information about the legal and insurance aspects” (ID:2).

Accessibility was also an important consideration, with the informants highlighting the importance of accommodating individuals with disabilities or different learning preferences {”Please ensure that access requirements are taken into consideration in advance” (ID2)], such as using captioning for videos or providing alternate ways to access information.

Regarding the course structure, overall, participants provided positive feedback regarding the structure and informativeness of the course, mentioning a satisfactory learning experience. The participants stated that the course was well-structured and delivered effectively, and expressed interest in gaining further practical experience, such as simulations or shadowing of real client calls, to enhance their telehealth skills and knowledge.

“overall very helpful intro to telehealth. Would have liked to see more examples of telehealth consultations being done” (ID:7).

Perceptions about using telehealth technology

Regarding students’ perceptions about using telehealth technology, 18 students complete the before and after assessments (retention rate: 69.2%). Of them, six were dental students; six were from the Physiotherapy course; and another six from Social Work or Audiology courses. After participating in the telehealth course, students significantly improved their overall acceptance and use attitudes compared to baseline (13.56 vs. 17.60; p = 0.007; See Table 2). No statistically significant differences in overall UTAUT2 scale were found, sex, age group, or frequency of use of internet. However, there were significant differences in the final score by profession. Dentistry was lower at pre-test, however, after participating in this telehealth course, dental students showed statistically significant improved post-test scores (8.2 vs. 14.5; p < 0.05).

Table 2 Students’ responses to acceptance of ICT and digital health questionnaire (UTAUT2)

Overall, at the post-test students significantly improved in the expected direction, in six of the ten dimensions of the UTAUT2 instrument, at least at p < 0.05 level of significance. The exceptions were “Behavioural intention to use the system”, “Hedonistic motivation”, which increased after the interventions without reaching significant levels, and “Habit”, and “Self-perception” (See Table 2).

Discussion

A telehealth course was designed to provide health professions students with a set of foundational knowledge and skills needed to succeed as a virtual health care practitioner. The preliminary data analysis conducted in this pilot study revealed positive feedback regarding the structure and informativeness of the course, and high level of satisfaction, increased confidence, and a positive overall experience among the students who participated in the course. This suggests that the course was able to effectively educate the participants on the topic. Feedback provided also suggested that including real-life case studies or demonstrations of telehealth consultations could enhance the learning experience and provide a better understanding of how telehealth is implemented in practice.

This study also provided an initial evaluation of the course effectiveness in improving health professions students’ acceptance of ICT and digital health on future daily practice.

Additionally, this study investigated the level of technology proficiency (reported as self-assessed ‘confidence’) amongst the future healthcare workforces. The information provided by this cohort of students would also indicate that students in healthcare professional courses at the University of Melbourne are highly proficient, and able to use a wide range of technologies regularly in their daily lives.

This was a cohort of students who have been exposed to ICT in education since, at least, high school. Students were confident in their ability to perform all the tasks included in the TPSA C-21. This was important to verify, as it has been found among healthcare students that not all are frequent users of ICT [19]. Furthermore, studies have also purported that Internet use by students was mostly for non-professional related purposes [9, 19]. Students in this study felt less confident in some administrative uses of ICT technology and in its use as a tool for their future role as health professionals. Thus, although students seem to have adequate proficiency and confidence, the study also identified some areas in which support and further development may be required, suggestion important issues to be considered in the design and delivery of technology-enhanced curricula in the future.

Concerning the acceptance and aaal health (i.e., UTAUT2 model), scores had significant increases after the course in six subscales, particularly, the ”Facilitating conditions” and “Price values” scales. These subscales are about the belief in the organizational and technical infrastructure to support the use of technology, and perceptions that the cost of using technology has worth compared to the benefit, respectively [16]. On the other hand, results indicate that some dimensions did not significantly change after participating in the course. For “Behavioural intention to use the system” and “Self-perception, they were relatively high before the course. For “Hedonistic motivation” and “Habit”, they reflect, the pleasure and enjoyment derived in the use of the technology, and a routine, less conscious use of technology. In any case, they may not be as relevant when related to their use as health professionals, but these dimensions play a moderating role in technology use [15, 16].

While this study offers valuable insights into the course acceptability, as well as technology proficiency, acceptance, adoption, and confidence in using these technologies among health professions students, it is not without its limitations. The most apparent one is the cross-sectional design, which precludes drawing definitive conclusions about longer-term technology adoption and proficiency among future healthcare professionals. Additionally, the assessment relied on a self-selected sample, introducing potential variations between participants and non-participants concerning technical self-competency, technology experience, and other factors. Moreover, the study depended on self-reported data, raising concerns about the accuracy of the relationship between self-perceived and actual technical competency. Geographically, the study was confined to a small sample from one country and one university. Despite the mentioned limitations and the pilot study being conducted in 2021, the current approached was deemed appropriate given the exploratory nature of the study.

The ability to integrate 21st century technology for learning is an expectation for educators [20]. If we wish to ensure that the future healthcare workforces are digitally trained, then self-efficacy, acceptance, and technological abilities are important constructs [21]. In fact, one of the reasons identified for failure of implementation of technology is the users’ lack of understanding of how health professionals accept digital health [22]. However, the acceptance of and attitude to use technologies does not guarantee that use will follow. It is difficult to assume that health professionals will incorporate these new technologies if they do not have the skills and support to integrate these technologies into their practice. Present findings highlight the different areas of competence that healthcare students must acquire for the use of ICT and digital health as a competence [9, 23].

Learning telehealth care is not only about learning a new technology; several barriers, both internal and external, will influence its adoption and employing that technology into practice [14]. External barriers include tools, training, and support, while internal barriers include attitudes, confidence, and beliefs in the need to incorporate technology. This telehealth course was designed to address some of those external and internal barriers. Students need also to be aware of the legal responsibilities and regulatory, and funding issues [24]. These aspects are covered in the telehealth course. Consequently, it would is an advantage for health students, as the future workforce, to be exposed to these technologies in a comprehensive manner during their professional training and preparedness to practice and become familiar and understand the impact of them on service provision [7].

The advancements in ICT, including artificial intelligence, robotics, self-learning machines, and the necessity to analyze substantial data sets, will necessitate the cultivation of novel competencies among healthcare professionals [25, 26]. Many educational programs in health are adopting digital health. Furthermore, healthcare service institutions have also implemented digital health in the context of workplace learning [27], to decrease disparities in access to services and specialists. Under these scenarios, it can be expected that health professionals will incorporate these new technologies if they have the skills a support to integrate them. Nevertheless, as previously noted, there is a limited comprehensive evaluation for telehealth course initiatives. Some evidence suggests that the outcomes of the existing evaluation served as the foundation for subsequent implementations of the course [28].

Conclusions

Students in healthcare professional degrees at the University of Melbourne indicated they were able to use a wide range of technologies regularly in their daily lives. Furthermore, their responses indicate a balanced perspective on telehealth, recognizing its potential while also acknowledging its limitations and the need for further education and experience. The analysis of the qualitative date also showed a positive feedback on the structure and informativeness of the course, and the enjoyable learning experience. Feedback also provided suggestions for future implementation.

They seem to have adequate proficiency and confidence in their ability to perform all the tasks explored in this evaluation. Nonetheless, the study identified some areas in which support and further development may be required. Large-scale testing and validation of the course should follow. This study also provides further understanding of the health professional acceptance of ICT and digital health and investigates. This initial evaluation provides valuable information, which could be used during the redesign of health sciences curricula to enable it to meet the needs of students, the healthcare professions, and the community.