Tutorless board game as an alternative to tabletop exercise for disaster response training: perception of interaction engagement and behavioral intention

Abstract

Background

Although tabletop exercise is a commonly used method for disaster response training, it is labor-intensive, requires a tutor for facilitation and may not be ideal in a pandemic situation. Board game is a low-cost and portable alternative that can be utilized for this purpose. The purpose of this study was to compare the perception of interaction engagement and behavioral intention to use a newly developed board game with tabletop exercise for disaster training.

Methods

Using the Mechanics-Dynamics-Aesthetics’ (MDA) framework, a new, tutorless educational board game known as the Simulated Disaster Management And Response Triage training (“SMARTriage”) was first developed for disaster response training. Subsequently, the perceptions of 113 final year medical students on the “SMARTriage” board game was compared with that of tabletop exercise using a crossover design.

Results

Using Wilcoxon signed rank test, it was that found that tabletop exercise was generally rated significantly higher (with p < 0.05) in terms of perceived usefulness, perceived ease of use and behavioral intention compared to tutorless “SMARTriage” board game. However, in terms of attitude and interaction engagement, there was no significant difference between these two learning methods for most of the items.

Conclusion

Although a clear preference for tutorless board game was not demonstrated, this study suggests that board game was not inferior to tabletop exercise in fostering interaction engagement suggesting that “SMARTriage” board game could potentially be used as an adjunct for teaching and learning activities.

Introduction

Disaster response management is an important subject that should be incorporated in medical and nursing education [1, 2]. This subject should be taught using a multi-modal approach by leveraging on a variety of educational tools [1]. One of these frequently used tools is tabletop exercise [1, 3,4,5]. Tabletop exercise is a cost-efficient, classroom-type disaster response simulation activity where students are presented with an unfolding scenario; and through the iterative interactions with one another, the students learn to make decisions in a disaster scenario [5]. Tabletop exercise, however, is labor-intensive and it requires a tutor to facilitate or coordinate the session. Only a limited number of case scenario combinations could be discussed for the entire class in a tabletop exercise. As different students are assigned different roles and tasks, the learning experience of each student cannot be standardized [5]. Furthermore, tabletop exercises would require the gathering of the entire group of students which may not be ideal if the size of the group is becoming too large particularly in a pandemic situation.

One option that could be used as an alternative or a supplement to tabletop exercise in disaster training is through games and gamification strategies. Game is defined as a set of problem-solving activities approached with a playful attitude [6] whereas gamification is defined as the application of game design elements in a non-game context [7]. Generally, a major advantage of using games and gamification strategies for teaching and learning is the fact that it affords the opportunity for students to explore and to learn from mistakes in a safe environment [8]. A type of low-cost game that is portable enough and does not require internet connection or any electronic device to play is the board game.

To the best of our knowledge, although a number of games for disaster response training had been described in literature [9], the use of board game in disaster response training specifically designed for healthcare students is rarely reported. One recently published board game for medical students’ disaster training was the “AFTERSHOCK” game designed by a group of German medical students. The aim of this board game is to train medical students to respond to the aftermath of an earthquake [10]. The authors found that “AFTERSHOCK” game was a suitable and acceptable method for disaster training. In that study, however, the level of interaction engagement was not measured and neither was a comparison between the board game with another conventional teaching and learning method made.

To fill in this gap, we conducted a study on a newly developed disaster training board game with the primary objective of comparing the perception of interaction engagement and behavioral intention to use this board game as compared to the conventional tabletop exercise.

Methodology

Setting

In our institution, i.e., Universiti Malaysia Sarawak (UNIMAS), disaster response training is part of our final year medical students’ emergency medicine rotation. In this training, a 1 to 2-h tutorial session was first given to all students on the basic concepts of disaster, the different phases of a disaster, the principles of multi-agency field and hospital management and the contingency planning to minimize the impact of any future recurrence. This was followed by a practical session on disaster triaging and response skills, conventionally conducted using tabletop exercise.

Study design

We divided our study into two main phases. In Phase 1, we first described the processes of designing a new educational board game known as the Simulated Disaster Management And Response Triage training (“SMARTriage”) (Phase 1 of our project). Subsequently, in Phase 2, we compared the medical students’ perception on this tutorless “SMARTriage” board game with the conventional tabletop exercise.

The board game was designed in the year 2020 and the study was conducted throughout in the entire year 2021 (2 semesters). The board game sessions were conducted in our clinical skills training room. The room was spacious enough to accommodate about 12—14 students performing cardiopulmonary resuscitation on manikins at any one time. The tabletop exercise sessions were conducted in smaller tutorial rooms that allow students to break out into 3 smaller groups. Ethical approval was obtained from the institutional medical research ethics board of UNIMAS (reference no UNIMAS/NC-21.02/03–02 Jld.4 (22)).

Participants

For Phase 1, all authors participated in group discussions with the aim of designing and developing the board game. For Phase 2, the entire cohort of 113 final year medical students were recruited sequentially in groups of 43, 36 and 34 students. The medical students had to be recruited sequentially as they came in groups of 30 – 40 students per group for their emergency medicine rotation.

Materials

For Phase 1, the Mechanics-Dynamics-Aesthetics’ (MDA) framework by Hunicke et al. [11] for game design was applied. “Mechanics” refers to a set of rules that define what can or cannot be done in the game. “Dynamics” refers to the players’ experience and interaction with the rules of the game. “Aesthetics” refer to the intended emotional experience evoked when a player participates in the game. Hunicke et al. [11] further described a taxonomy of at least 9 types of aesthetics, i.e., sensation (game as sense-pleasure), fantasy (game as make-believe), narrative (game as drama), challenge (game as obstacle course), fellowship (game as social framework), discovery (game as uncharted territory), expression (game as self-discovery) and submission (game as pastime). In “SMARTriage” board game, the intended aesthetics are (1) fellowship and (2) challenge (i.e., the experience of different players collaborating, rather than competing, to beat the time limit in order to correctly triage and transfer all patients to the definitive wards). We further considered the dynamics of our board game from five different elements of user experience described by von Wangenheim et al. [12]. These five elements are (1) the immersion element; (2) the challenge element; (3) the competence element; (4) the fun element and (5) the social interactions element.

Phase 2 of the study was conducted using the psychometric instrument that we adapted from a previously validated instrument [13]. However, we differed from the previous instrument in that we included “behavioral intention” as our dependent variable. Furthermore, in the previous instrument, students’ engagement was further divided into two separate constructs: (1) “skill engagement” (referring to learning skills such as taking good notes and listening well in class, etc.) and (2) “interaction engagement” [13]. In our study, “skill engagement” was excluded in our instrument as these items (e.g., measuring “taking good notes in classroom”, “listening carefully in classroom”, etc.) were deemed irrelevant in our practical disaster response training. All items in “interaction engagement” construct listed in the previous instrument were included in our instrument development except for the item “asking questions when I did not understand”. This was because unlike the tabletop exercise, the “SMARTriage” board game session was purported to be a tutorless session (except at the beginning of the session for briefing on how to play and the end of the session for debriefing). Hence, it was felt that it would not be a fair comparison to include this item “asking questions when I did not understand” as the amount of contact time with the tutor was significantly low in the “SMARTriage” compared to in tabletop exercise.

Procedures

For Phase 1, group discussions were held during the first 2 months of this study to develop the mechanics, dynamics and aesthetics components of “SMARTriage” board game using the framework by Hunicke et al. [11]. In the first two sessions, all authors KSC, SSLW, IST, JWK, NAAR and SASWA attended and contributed to this development and design. Author KSC (who is an emergency physician with special interest in medical education) then invited three emergency residents (Dr. ET, Dr. YYT and Dr. CYT) from the emergency and trauma department of Sarawak General Hospital (who were not part of this project) to contribute some case scenarios for this game. The subsequent iterative process of vetting and fine tuning of these case scenarios were conducted through emails and the short messaging application, WhatsApp. The details of the conduct of both tabletop exercise and board game are detailed in Table 1.

Table 1 Comparison of the learning objectives, combination of case scenarios and conduct of tabletop exercise and “SMARTriage” board game

For Phase 2, we first performed confirmatory factor analysis using SmartPLS software version 3.3.3 [14] to ensure that the psychometric instrument that was adapted from Ab. Rahman et al. [13] had good validity. Specifically, convergent validity (using factor loadings and average variance extracted or AVE), discriminant validity and internal consistency reliability analyses (using Cronbach’s alpha and composite reliability) of the items were determined [15]. This validated psychometric instrument has five constructs, i.e., (1) “Perceived Usefulness”, (2) “Perceived Ease of Use”, (3) “Attitude”, (4) “Interaction Engagement” and (5) “Behavioral intention” (see Table 2). The detailed process and results of validating the instrument is described in the Supplementary material.

Table 2 List of items and constructs in the psychometric instrument

Subsequently, using this newly validated psychometric instrument, we compared the medical students’ perception on the tutorless “SMARTriage” board game with the conventional tabletop exercise specifically on “interaction engagement” and “behavioral intention” using a crossover design. Crossover design is a study design where participants are assigned randomly to a sequence of interventions [16]. For example, in the crossover study of AB/BA design, participants would be randomized to receive either intervention “A” followed by “B” or “B” followed by “A”. The advantage of crossover design is that each participant can serve as his or her own control for comparison.

Results

Phase 1: Development of the board game (SMARTriage)

Based on the findings from the group discussions, “SMARTriage” board game was designed and developed using the MDA framework by Hunicke et al. (2004). The details of these MDA components are described in Table 3. Examples of the components used in “SMARTriage” board game, i.e., the cards and the game board are illustrated in Fig. 1.

Table 3 The Mechanics, Dynamics, Aesthetics and Components of “SMARTriage” Board Game

Fig. 1

Components of the “SMARTriage” Board Game

Phase 2: Comparison of “SMARTriage” with Conventional Tabletop Exercise

Using Wilcoxon signed rank test, it was that found that overall, tabletop exercise was still rated significantly higher in terms of perceived usefulness, perceived ease of use and behavioral intention compared to tutorless “SMARTriage” board game (refer Table 4). However, in terms of attitude and interaction engagement, there was no significant difference between these two learning methods for most of the items. In particularly, there were no significant difference in 2 out of the 3 items in attitude construct and 5 out of 6 items in interaction engagement construct suggesting that board game was non-inferior in terms of generating interaction engagement among students.

Table 4 Comparison of Perception of Tabletop Exercise with Tutorless “SMARTriage” Board Game

Discussion

Overall, our findings suggest that students prefer tabletop exercise compared the tutorless “SMARTriage” board game in disaster response training. Specifically, the students rated “perceived usefulness”, “perceived ease of use” and “behavioral intention” significantly higher in tabletop exercise compared to that in the board game. However, a caveat in deciphering the meaning of our findings is the fact that “SMARTriage” board game was intended to be a tutorless, self-contained learning method. While it is true that the proliferation of gamification in teaching and learning has further pivoted the shift from a teacher-centric to a student-centric environment, this study suggests that the presence of tutor is still important in disaster response training. This is consistent with a review by Molin [17] where the author explained how the roles of tutors had often been sidelined or ignored in gamified learning when in fact, tutors play a crucial role to facilitate learning. In this context, the tutor can promote deep and rich collaborative learning rather than simply being an agency of passive knowledge transmission.

Finn and Zimmer [18] highlight on the positive impact of interaction engagement on learning outcomes, suggesting that it leads to improved learning, increased motivation, and better academic performance. Similarly, according to a systematic review by Vlachopoulos and Makri [19], game-based learning approaches confer significant benefits beyond mere cognitive knowledge acquisition. Their findings show that game-based learning not only increases interaction engagement and student motivation, but also promotes the development of critical soft skills such as teamwork, collaboration, organizational skills, adaptability, leadership, as well as a greater ability to resolve conflicts. Consistent with these findings, the rating for most items in the “interaction engagement” construct in “SMARTriage” board game arm were similar with that in the tabletop exercise arm. This suggests that the tutorless “SMARTriage” board game can be as good as tabletop exercise in fostering deep engagement. Previous studies had also shown that gamified learning can be a fun and feasible alternative to foster deep discussion [20, 21] to complement conventional teachings [22, 23].

Few important limitations of this study deserve mentioning. Inherent to the objective of “SMARTriage” to be a tutorless learning activity, the tutor was not present throughout the entire session (unlike in Tabletop exercise). Tutor availability might have confounded the results of the study. To ensure a fairer comparison between these two interventions, future studies could be repeated by controlling for the presence of tutor in both study arms. Secondly, in order to ensure that a game is crisp enough to be completed within a stipulated time bounded by the rules and regulations of the game (“mechanics”), the game has to be necessarily reductionistic. This, however, is unlikely to reflect the complexity of managing disaster in real world. For example, in a real disaster, the physiological status of the patients is dynamic. The patient could deteriorate due to a variety of insults including airway compromises, respiratory fatigue or hemorrhagic shock. This kind of physiological dynamism, however, could not be simulated in a board game; hence, reduces its authenticity. Thirdly, in ideal crossover study design, adequate washout period is recommended until the effect from the first intervention subsides before embarking on the second intervention. In our teaching session, however, due to time restriction, washout period was not possible to be imposed.

Finally, although it was initially mentioned that tabletop exercises may not be suitable in a pandemic as this would require the gathering of the entire group of 40 over students, board game is still not an ideal solution as it would require the gathering of smaller groups of 4 students (although small groups of 4 students would be easier to contain and control). In this regard, a future option would include the creation of a digital or online version of “SMARTriage” board game. This could enable remote gamified learning and exclude any form of physical contact in the event of a pandemic.

Conclusion

Although a clear preference for tutorless board game was not demonstrated in this study, this study suggests that board game was not inferior to tabletop exercise in fostering interaction engagement. Hence, board game such as the “SMARTriage” could potentially be used as an adjunct for teaching and learning activities.