- Research article
- Open Access
- Open Peer Review
Brain computerized tomography reading in suspected acute ischemic stroke patients: what are essentials for medical students?
BMC Medical Education volume 19, Article number: 359 (2019)
Few systematic methods prioritize the image education in medical students (MS). We hope to develop a checklist of brain computerized tomography (CT) reading in patients with suspected acute ischemic stroke (AIS) for MS and primary care (PC) physicians.
Our pilot group generated the items indicating specific structures or signs for the checklist of brain CT reading in suspected AIS patients for MS and PC physicians. These items were used in a modified web-based Delphi process using the online software “SurveyMonkey”. In total 15 panelists including neurologists, neurosurgeons, neuroradiologists, and emergency department physicians participated in the modified Delphi process. Each panelist was encouraged to express feedback, agreement or disagreement on the inclusion of each item using a 9-point Likert scale. Items with median scores of 7–9 were included in our final checklist.
Fifty-two items were initially provided for the first round of the Delphi process. Of these, 35 achieved general agreement of being an essential item for the MS and PC physicians. The other 17 of the 52 items in this round and another two added items suggested by the panelists were further rated in the next round. Finally, 38 items were included in the essential checklist items of brain CT reading in suspected AIS patients for MS and PC physicians.
We established a reference regarding the essential items of brain CT reading in suspected AIS patients. We hope this helps to minimize malpractice and a delayed diagnosis, and to improve competency-based medical education for MS and PC physicians.
Stroke is one of the leading causes of death and disability worldwide. The most influential factor predicting the severity of disability and mortality in patients with acute ischemic stroke (AIS) is the time required to restore cerebral blood flow. Currently, AIS treatments including intravenous thrombolytic therapy (IVT) and intra-arterial mechanical thrombectomy (IA-MT), which have been shown to effectively improve stroke-related morbidity, disability and mortality . The phrase “time is brain” is a reminder that AIS treatment should be completed promptly on arrival at the emergency department (ED) . The assessment, evaluation and interpretation of the brain images of suspected AIS patients should therefore be completed in a timely, precise and comprehensive manner. In contrast to the IVTs in ED patients, the IVTs in in-hospital AIS patients are usually delayed . Inadequate awareness of the urgency of AIS patients, insufficient knowledge of ordering and interpreting stroke examinations, and inadequate initiation and interpretation of brain computed tomography (CT) scan would result in a delay and quality gap of acute in-hospital stroke care [2, 3]. Therefore, dealing with AIS patients should also be an important entrustable professional activity (EPA) for primary care (PC) physicians in training.
Diagnostic studies are sub-competencies of patient care for ED physicians, and brain image reading is an important milestone in this sub-competency. Brain CT reading is as essential as physical examinations in daily healthcare practice. Although radiologists provide the official medical reports, all PC physicians still need to swiftly interpret CT findings in emergency cases in order to propose adequate treatment plans . It would thus be beneficial for neurology residents, medical students (MS), and PC physicians to master these CT reading skills, including comprehensive reading, important positive and negative signs of a main diagnosis and major differential diagnoses. In patients presenting with suspected AIS, the goals of brain CT reading are not only to select the patients who are candidates for IVT and IA-MT, but also to identify the patients who have symptoms mimicking AIS . Missing the subtle signs of life-threatening diseases other than AIS, such as subarachnoid hemorrhage, epidural or subdural hemorrhage, and skull fractures may also result in a wrong diagnosis, morbidity, mortality, and severe complications if receiving IVT. Moreover, doctors of different specialties may place emphasis on different factors and read in a different sequence when interpreting the same brain images . The lack of standardization and of a standard reading sequence may make instructional designs for teaching and learning brain CT reading difficult . Therefore, the goal of this study was to obtain a consensus on essential items and to establish a checklist for brain CT readings of patients with suspected AIS for MS and PC physicians. We applied a web-based modified Delphi method over several rounds to build consensus among a selected panel of experts.
The generation of a list of essential items for brain CT images of suspected AIS patients
Three attending physicians (TYC, SCC, and CJS) from the Neurology, Radiology, and ED at Chang-Gung Memorial Hospital, Linkou were enrolled in the pilot group to draft a list of essential items of brain CT images from patients with suspected AIS. All three participants had actively contributed to the treatment of AIS for more than 5 years. Their draft list was further discussed during the Delphi process. The Ethics Institutional Review Board of Chang Gung Memorial Hospital approved this study (IRB NO. 201601984B0).
The modified web-based Delphi process
It is important to consider diversity and representativeness during the selection of panelists in the modified Delphi process [8, 9]. We selected 15 expert panelists for the modified Delphi process, including three general neurologists, three stroke neurologists, three neurosurgeons, three neuroradiologists, and three ED physicians from different medical schools and medical centres. All these panelists were experienced in brain CT reading and have actively contributed to clinical care and teaching of neurological diseases for more than 5 years. Given the busy schedule of the clinicians, getting them together at the same time would have been difficult. Therefore, we designed a web-based, anonymous, and modified Delphi process, with a dedicated URL to the free software SurveyMonkey (www.surveymonkey.com), to calculate the rating results of each item on the checklist inspired by existing literature [10, 11]. Of note, all of the chosen panelists should be familiar with a mobile device. We used personal e-mails as the main connection between the principal investigator and the panelists. First, we sent a background survey regarding their instructing experience and expertise to all of the panelists. We then started the first round of the modified Delphi process. A personalized follow-up reminder e-mail message was sent to absent panel members with a uniform resource locator link to the survey and to the webpage. The list of responders from each round was copied into new a recipient list for the subsequent rounds. All ratings, suggestions, and discussions were recorded, transcribed and anonymized.
In the first round of questionnaires, the 15 panelists were asked to rate for whether each item should be included or excluded. The survey was conducted by mailing a questionnaire with the draft checklist to all panelists. Each panelist indicated their agreement or disagreement by using a 9-point Likert scale (1 = strongly disagree; 9 = strongly agree). The response rate indicated the proportion of the panelists who rated on each item. We discarded the items with a less than 10% response rate from the panelists. The items with response rates between 10 and 90% were discussed in the next round, while the items with a response rate over 90% were discussed in this round. Among these, the items with a median rating score of 9 were defined as “strong agreement”, those with median rating scores of 7 and 8 were defined as “agreement”, and those scored between 4 and 6 were discussed in the next round. Finally, the items with a median rating score between 1 and 3 were discarded (Fig. 1). We also asked the 15 panelists to provide feedback and suggest important positive or negative signs that were not initially included.
During the second round, we rated the items without a conclusive consensus, feedback and the items suggested by the panelists in the first round. Different to the first round, the items with a response rate of less than 80% from the panelists were discarded. Of those with a response rate over 80%, the items with a median rating score between 1 and 6 were left out regardless of their response rates; those with a median rating score between 7 and 9 were saved only if the response rate was more than 90% (Fig. 1). The items with a median rating score of 9 were defined as “strong agreement”, and those with median rating scores of 7 and 8 were defined as “agreement”.
All statistical analyses were performed using the Statistical Package for the Social Sciences (IBM SPSS Statistics version 22.0). The Likert scale rating for each item from all of the panelists was expressed as median (quartile 1, quartile 3). Items with median scores between 7 and 9 in the first or second round were defined as being recommended brain CT reading items. In addition, we compared differences in perspective between the stroke specialists and non-stroke specialists using the Mann-Whitney U test (nonparametric data). Statistical significance was set at p < 0.05.
The proposed essential brain CT items for the MS and PC physicians from the pilot group
After reviewing text books and published important image signs , a total of 52 essential items of brain CT reading were initially proposed from the pilot group [13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30]. All of these items were categorized into six parts, including checking the patients identification (ID), checking CT scan quality, reading bone windows, extra-axial lesions, intra-axial lesions, and identifying abnormal densities on brain CT.
Background characteristics of the panelists
Among the 15 panelists, eight (53.3%) had more than 11 years of experience in instructing brain CT reading to students, and seven (46.7%) had less than 10 years of experience. The panelists included three (20%) stroke neurologists, three (20%) non-stroke neurologists, three neurosurgeons (20%), three neuroradiologists (20%), and three ED physicians (20%). The most common expertise of the panelists was cerebrovascular disease (53.3%), followed by vascular intervention (26.7%), spinal disease (26.7%), neuro-critical care (26.7%), medical education (26.7%), neuroimaging (20.0%), brain tumor (20.0%), and emergency medicine (20.0%). The panelists in this study therefore covered most fields of subspecialties of clinical neuroscience (Table 1).
Results of the modified Delphi process
All of the 15 panelists completed the first and second rounds of ratings (Response rates of the both rounds: 100%; Fig. 1). All 52 of the essential items of brain CT reading were used for the first round of the modified Delphi process, of which 35 (67.3%) had agreement or strong agreement. Of these 35 items, three were strongly recommended as essential brain CT reading items for MS and PC physicians with a rating of 9, including checking the patient’s ID, identifying a mass effect, mid-line shift, or herniation, and identifying low density lesions (edematous lesions and recent infarcts). The other 17 (32.7%) of the 52 items were entered into the second round for further confirmation. The panelists also provided feedback in the first round (Additional file 1: Table S1) and suggested adding another two items to the essential list of brain CT reading in the first round (Fig. 1), which were identifying the image symmetry of the bilateral cerebral hemispheres, and reviewing the symmetry, hyperdensity, and hypodensity of the bilateral cerebellar hemispheres.
Among the 19 items discussed in the second round of the modified Delphi process, only three (15.8%) were regarded to be essential brain CT reading items for MS and PC physicians, while the other 16 items were discarded after thorough consideration. Finally, 38 items were included in the recommended essential list of brain CT reading items for MS and PC physicians (Tables 2 and 3; Additional file 1: Tables S2 and S3).
Differences in viewpoints between the stroke and non-stroke specialists
Comparing the eight stroke specialists with the seven non-stroke specialists, the stroke specialists placed less emphasis on epidural and subdural spaces [stroke specialists vs. non-stroke specialists, 6.5 (6, 8) vs. 8 (7, 9), p = 0.03], lateral ventricles [stroke specialists vs. non-stroke specialists, 6.5 (5.3, 8) vs. 9 (8, 9), p < 0.03] including anterior and posterior horns and temporal horns [stroke specialists vs. non-stroke specialists, 6 (5.3, 7.8) vs. 8 (8, 9), p < 0.01], third ventricle [stroke specialists vs. non-stroke specialists, 5.5 (5, 6) vs. 7 (7, 8), p = 0.01] and fourth ventricle [stroke specialists vs. non-stroke specialists, 6 (6, 6) vs. 7 (7, 8), p = 0.02]. The stroke specialists also gave a lower rating for reading the medulla [stroke specialists vs. non-stroke specialists, 6.5 (5, 7) vs. 8 (8, 8), p = 0.03] (Table 4).
Brain CT reading is a key process in the diagnosis and treatment of AIS. Our modified Delphi study provided a useful checklist for essential items of brain CT reading in suspected AIS patients. A standardized and sequential reading from each anatomical location may reduce errors in interpreting brain images in these patients. Although there were some studies discussing the reading patterns of neurologists, agreements between different specialties, and common errors in brain CT interpretations [28, 31, 32]. The structured checklist of brain CT reading for MS and PC physicians remain scarce. Furthermore, the common sites of residents’ misinterpretation in these studies were thoroughly discussed during our Delphi process . A mobile stroke team or pre-hospital telemedicine may allow for faster image reading and treatment of AIS in the current era [33,34,35], however insufficient manpower with regards to stroke specialists could be a potential problem worldwide. A delay or misinterpretation of brain CT may result in a delay of door-to-needle time and decrease the potential for brain tissue salvage . A previous study further demonstrated that the image-to-needle time could be a more common contributor to a delay in timely thrombolytic therapy . Therefore, improving the competency of stroke image interpretation of MS and PC physicians should also play an important role in the quality of AIS care .
AIS is a life threatening disease, and IA-MT or earlier decompressive craniectomy has demonstrated clinical benefits in selected patients [39, 40]. Being unaware of the possible signs of large vessel occlusion (LVO) may delay the initiation of the IA-MT process . The consensus of our panel suggested the importance of dense middle cerebral artery and dense basilar artery signs when reading brain CT. These signs are clues to LVO, and identifying these signs early may help prompt transferal to a stroke center or the initiation of IA-MT. Our study also suggested that the most important items when interpreting the brain CT of suspected AIS patients include low density lesions suggesting recent infarcts, a mass effect, mid-line shift, and herniation. We suggest that identifying and properly managing these life-threatening conditions in a timely manner by neurosurgeons should be a crucial EPA for all clinical physicians in training . Recognizing emergency imaging findings and describing abnormalities on brain CT should also be an important milestone in the sub-competency of patient care for MS and PC physicians. Mastering the important signs of brain CT reading should be integrated into the EPA of AIS patient care. We hope our work may help to guide brain CT reading for MS and PC physicians.
Being too focused on one diagnosis can be a major source of diagnostic errors . We noted that there was a discordance between the stroke and non-stroke specialists when rating the items in our study, and the stroke specialists placed less emphases on reading the medulla, ventricles, epidural and subdural spaces (Table 4). The incidence of stroke may increase after traumatic brain injury, and the risk of traumatic brain injury and associated mortality could also be increased in stroke patients [43, 44]. Of note, obstructive hydrocephalus and brain stem compression may cause mortality in space-occupying cerebellar infarcts . In addition, managing hydrocephalus may be an emergency in patients with raised intracranial pressure . It is essential to read the ventricles, epidural and subdural spaces carefully regardless of whether or not there are prominent signs of AIS. A structured reporting template can help to reduce errors in interpreting these findings . We hope our standardized reading sequence may remind all physicians not to miss these important findings other than AIS.
There were several strengths to our modified Delphi study. First, we overcame the barriers of distance, time, and expense and achieved a high adhesion rate through this web-based method . Second, we expanded the heterogeneity of our panelists. We enrolled four specialty groups (neurologists, neurosurgeons, radiologists, and ED physicians), and we expected that this could enrich the viewpoints of the panelists . However, there are also several limitations to this study. First, on-line panel discussion may be associated with lower levels of interaction and engagement , and this could prejudice the consensus gathering process. Instead, we anonymously and individually provided the median ratings of each item and the overall comments of the panelists to reduce this confounding. Second, not all of the panelists in this study were course directors, and we only enrolled one international panelist. This may have influenced the generalizability of our results. However, we tried to include panelists from different medical centers or tertiary hospitals, and we hoped that the diversity of our panel may broaden the viewpoints in this study.
We established a reference regarding the essential items of brain CT reading in suspected AIS patients for MS and PC physicians. This could be the first step towards a standardized educational materials of brain CT for MS and PC physicians by identifying the consensus among specialists. We hope this may help to minimize malpractice and a delayed diagnosis in these patients. Future studies are needed to evaluate the reliability and validity of this tool.
Availability of data and materials
Anterior cerebral artery
Acute ischemic stroke
Cerebral venous thrombosis
Entrustable professional activity
Intra-arterial mechanical thrombectomy
Intravenous thrombolytic therapy
Large vessel occlusion
Middle cerebral artery
Posterior cerebral artery
Powers WJ, Rabinstein AA, Ackerson T, Adeoye OM, Bambakidis NC, Becker K, et al. 2018 Guidelines for the early Management of Patients with Acute Ischemic Stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. 2018;49(3):e46.
Kurz MW, Kurz KD, Farbu E. Acute ischemic stroke--from symptom recognition to thrombolysis. Acta Neurol Scand Suppl. 2013;127:57–64.
Cumbler E. In-Hospital Ischemic Stroke. Neurohospitalist. 2015;5:173–81.
Erly WK, Ashdown BC, Lucio RW, Carmody RF, Seeger JF, Alcala JN. Evaluation of emergency CT scans of the head: is there a community standard? AJR Am J Roentgenol. 2003;180:1727–30.
McClelland G, Rodgers H, Flynn D, Price CI. The frequency, characteristics and aetiology of stroke mimic presentations: a narrative review. Eur J Emerg Med. 2018;26(1):2.
Evans LR, Fitzgerald MC, Mitra B, Varma D. Emergency department interpretation of CT of the brain: a systematic review. Postgrad Med J. 2017;93:454–9.
Evans LR, Fitzgerald MC, Varma D, Mitra B. A novel approach to improving the interpretation of CT brain in trauma. Injury. 2018;49:56–61.
Goldacre MJ, Taylor K, Lambert TW. Views of junior doctors about whether their medical school prepared them well for work: questionnaire surveys. BMC Med Educ. 2010;10:78.
Goldacre MJ, Lambert TW, Svirko E. Foundation doctors’ views on whether their medical school prepared them well for work: UK graduates of 2008 and 2009. Postgrad Med J. 2014;90:63–8.
Thomson R, Lewalle P, Sherman H, Hibbert P, Runciman W, Castro G. Towards an international classification for patient safety: a Delphi survey. Int J Qual Health Care. 2009;21:9.
Bisson J, Tavakoli B, Witteveen AB, Ajdukovic D, Jehel L, Johansen J, et al. TENTS guidelines: development of post-disaster psychosocial care guidelines through a Delphi process. Br J Psychiatry. 2009;196:69–74.
Osborn AG, Salzman KL, Jhaveri MD, Barkovich AJ. Diagnostic Imaging: Brain. 3rd ed. Salt Lake City: Amirsys; 2016.
Broder JS. Head computed tomography interpretation in trauma: a primer. Psychiatr Clin North Am. 2010;33:821–54.
Ernst M, Romero JM, Buhk JH, Cheng B, Herrmann J, Fiehler J, et al. Sensitivity of Hyperdense basilar artery sign on non-enhanced computed tomography. PLoS One. 2015;10:e0141096.
Jensen-Kondering U, Riedel C, Jansen O. Hyperdense artery sign on computed tomography in acute ischemic stroke. World J Radiol. 2010;2:354–7.
Bahrami S, Yim CM. Quality initiatives: blind spots at brain imaging. Radiographics. 2009;29:1877–96.
Poon CS, Sze G, Johnson MH. Orbital lesions: differentiating vascular and nonvascular etiologic factors. AJR Am J Roentgenol. 2008;190:956–65.
Li Q, Zhang G, Huang YJ, Dong MX, Lv FJ, Wei X, et al. Blend sign on computed tomography: novel and reliable predictor for early hematoma growth in patients with intracerebral hemorrhage. Stroke. 2015;46:2119–23.
Selariu E, Zia E, Brizzi M, Abul-Kasim K. Swirl sign in intracerebral haemorrhage: definition, prevalence, reliability and prognostic value. BMC Neurol. 2012;12:109.
Li Q, Zhang G, Xiong X, Wang XC, Yang WS, Li KW, et al. Black hole sign: novel imaging marker that predicts hematoma growth in patients with intracerebral hemorrhage. Stroke. 2016;47:1777–81.
Schneider H, Huynh TJ, Demchuk AM, Dowlatshahi D, Rodriguez-Luna D, Silva Y, et al. Combining spot sign and intracerebral hemorrhage score to estimate functional outcome: analysis from the PREDICT cohort. Stroke. 2018;49(6):1511.
Bonneville F. Imaging of cerebral venous thrombosis. Diagn Interv Imaging. 2014;95:1145–50.
Barber PA, Demchuk AM, Zhang J, Buchan AM. Validity and reliability of a quantitative computed tomography score in predicting outcome of hyperacute stroke before thrombolytic therapy. ASPECTS study group. Lancet. 2000;355:1670–4.
Treadwell SD, Thanvi B. Malignant middle cerebral artery (MCA) infarction: pathophysiology, diagnosis and management. Postgrad Med J. 2010;86:235–42.
Bruno A, Zahran A, Paletta N, Maali L, Nichols FT, Figueroa R. A standardized method to measure brain shifts with decompressive hemicraniectomy. J Neurosci Methods. 2017;280:11–5.
Mangla R, Kolar B, Almast J, Ekholm SE. Border zone infarcts: pathophysiologic and imaging characteristics. Radiographics. 2011;31:1201–14.
Takhtani D, Dundamadappa S, Almast J. Role of noncontrast head CT in the assessment of vascular abnormalities in the emergency room. Emerg Radiol. 2013;20:529–41.
Yaniv G, Mozes O, Greenberg G, Bakon M, Hoffmann C. Common sites and etiologies of residents’ misinterpretation of head CT scans in the emergency department of a level I trauma center. Isr Med Assoc J. 2013;15:221–5.
Zee CS, Go JL. CT of head trauma. Neuroimaging Clin N Am. 1998;8:525–39.
Zaheer BM. 101 clinical cases in emergency room: Jaypee brothers; 2014.
Matsumoto H, Terao Y, Yugeta A, Fukuda H, Emoto M, Furubayashi T, et al. Where do neurologists look when viewing brain CT images? An eye-tracking study involving stroke cases. PLoS One. 2011;6:e28928.
Demaerschalk BM, Bobrow BJ, Raman R, Ernstrom K, Hoxworth JM, Patel AC, et al. CT interpretation in a telestroke network: agreement among a spoke radiologist, hub vascular neurologist, and hub neuroradiologist. Stroke. 2012;43:3095–7.
Wei D, Oxley TJ, Nistal DA, Mascitelli JR, Wilson N, Stein L, et al. Mobile interventional stroke teams Lead to faster treatment times for Thrombectomy in large vessel occlusion. Stroke. 2017;48:3295–300.
Grunwald IQ, Ragoschke-Schumm A, Kettner M, Schwindling L, Roumia S, Helwig S, et al. First automated stroke imaging evaluation via electronic Alberta stroke program early CT score in a Mobile stroke unit. Cerebrovasc Dis. 2016;42:332–8.
Taqui A, Cerejo R, Itrat A, Briggs FB, Reimer AP, Winners S, et al. Reduction in time to treatment in prehospital telemedicine evaluation and thrombolysis. Neurology. 2017;88:1305–12.
Kamal N, Sheng S, Xian Y, Matsouaka R, Hill MD, Bhatt DL, et al. Delays in door-to-needle times and their impact on treatment time and outcomes in get with the guidelines-stroke. Stroke. 2017;48:946–54.
Sauser K, Levine DA, Nickles AV, Reeves MJ. Hospital variation in thrombolysis times among patients with acute ischemic stroke: the contributions of door-to-imaging time and imaging-to-needle time. JAMA Neurol. 2014;71:1155–61.
Heikkila I, Kuusisto H, Stolberg A, Palomaki A. Stroke thrombolysis given by emergency physicians cuts in-hospital delays significantly immediately after implementing a new treatment protocol. Scand J Trauma Resusc Emerg Med. 2016;24:46.
Mian AZ, Edasery D, Sakai O, Mustafa Qureshi M, Holsapple J, Nguyen T. Radiological imaging features of the basal ganglia that may predict progression to hemicraniectomy in large territory middle cerebral artery infarct. Neuroradiology. 2017;59:477–84.
Elsawaf A, Galhom A. Decompressive craniotomy for malignant middle cerebral artery infarction: optimal timing and literature review. World Neurosurg. 2018;116:e71.
Smith EE, Saver JL, Cox M, Liang L, Matsouaka R, Xian Y, et al. Increase in endovascular therapy in get with the guidelines-stroke after the publication of pivotal trials. Circulation. 2017;136:2303–10.
Singh H, Giardina TD, Meyer AN, Forjuoh SN, Reis MD, Thomas EJ. Types and origins of diagnostic errors in primary care settings. JAMA Intern Med. 2013;173:418–25.
Burke JF, Stulc JL, Skolarus LE, Sears ED, Zahuranec DB, Morgenstern LB. Traumatic brain injury may be an independent risk factor for stroke. Neurology. 2013;81:33–9.
Chou YC, Yeh CC, Hu CJ, Meng NH, Chiu WT, Chou WH, et al. Risk and mortality of traumatic brain injury in stroke patients: two nationwide cohort studies. J Head Trauma Rehabil. 2014;29:514–21.
Jeon SB, Koh Y, Choi HA, Lee K. Critical care for patients with massive ischemic stroke. J Stroke. 2014;16:146–60.
Leinonen V, Vanninen R, Rauramaa T. Raised intracranial pressure and brain edema. Handb Clin Neurol. 2017;145:25–37.
Chami K, Gavazzi G, de Wazieres B, Lejeune B, Carrat F, Piette F, et al. Guidelines for infection control in nursing homes: a Delphi consensus web-based survey. J Hosp Infect. 2011;79:75–89.
Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One. 2011;6:e20476.
Khodyakov D, Hempel S, Rubenstein L, Shekelle P, Foy R, Salem-Schatz S, et al. Conducting online expert panels: a feasibility and experimental replicability study. BMC Med Res Methodol. 2011;11:174.
We particularly thank Professor Lynn Monrouxe from Chang Gung Medical Education Research Centre, Chang Gung Memorial Hospital for her suggestions on study design. We thank all of the panelists who participated in the modified Delphi process, and all of the participants of the pilot group in this study for their help. We also thank Miss Elaine Shinwei Huang for her administrative work. We also thank (1) the Division of Medical Education, Graduate Institute of Clinical Medical Sciences, College of Medicine, Chang Gung University, (2) Chang Gung Medical Education Research Centre, Chang Gung Memorial Hospital, (3) Department of Neurology, Chang Gung Memorial Hospital, Linkou, and (4) Ministry of Science and Technology for their assistance in study design or financial support.
Funding was provided by Chang Gung Memorial Hospital (grant number: CMRPG3G0261) and Ministry of Science and Technology (grant numbers: 106–2511-S-182A-002 -MY2 and NMRPG3G6411–2). The both funding supported the financial and administrative affairs regarding study design, data collection, and statistical analyses.
Ethics approval and consent to participate
The Ethics Institutional Review Board of Chang Gung Memorial Hospital approved this study (IRB NO. 201601984B0). All of the participants have signed the informed consent and they agreed with the publication of the results of this Delphi study.
Consent for publication
All the panelists participated in this study agreed with this publication.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.