Construction and preliminary evaluation of the inpatient glycemic control questionnaire (IGCQ): a survey tool assessing perceptions and knowledge of resident physicians

Background Uncontrolled hyperglycemia in hospitalized patients, with or without diabetes mellitus, is associated with many adverse outcomes. Resident physicians are the primary managers of inpatient glycemic control (IGC) in many academic and community medical centers; however, no validated survey tools related to their perceptions and knowledge of IGC are currently available. As identification of common barriers to successful IGC amongst resident physicians may help foster better educational interventions (ultimately leading to improvements in IGC and patient care), we sought to construct and preliminarily evaluate such a survey tool. Methods We developed the IGC questionnaire (IGCQ) by using previously published but unvalidated survey tools related to physician perspectives on inpatient glycemic control as a framework. We administered the IGCQ to a cohort of resident physicians from the University of Mississippi Medical Center, University of Louisville, Emory University, and the University of Virginia. We then used classical test theory and Rasch Partial Credit Model analyses to preliminarily evaluate and revise the IGCQ. The final survey tool contains 16 total items and three answer-choice categories for most items. Results Two hundred forty-six of 438 (56.2%) eligible resident physicians completed the IGCQ during various phases of development. Conclusions We constructed and preliminarily evaluated the IGCQ, a survey tool that may be useful for future research into resident physician perceptions and knowledge of IGC. Future studies could seek to externally validate the IGCQ and then utilize the survey tool in pre- and post-intervention assessments. Electronic supplementary material The online version of this article (10.1186/s12909-019-1657-0) contains supplementary material, which is available to authorized users.


Background
Hyperglycemia is common in the inpatient setting and affects up to one-third of patients admitted to general medical and surgical wards [1][2][3]. Uncontrolled hyperglycemia in hospitalized patients, with or without diabetes mellitus (DM), is associated with adverse outcomes including increased rates of infection and mortality and longer hospital stay [4][5][6][7]. Various studies in both critically and noncritically ill hyperglycemic inpatients demonstrate that improved inpatient glycemic control (IGC) can reduce rates of hospital complications, infections, and cost [8][9][10][11]. As more than 90% of patients with DM are admitted for reasons unrelated to the disease and may be cared for by staff without specific DM expertise, IGC is often poor [12]. The recent consortium for Planning Research in Inpatient Diabetes (PRIDE) was formed to promote clinical research leading to advancement and improvement of IGC. The consortium outlined eight aspects of IGC which needed to be addressed; the first suggested development and evaluation of provider education tools to improve knowledge of and address barriers to achieving appropriate IGC [13]. Since resident physicians are the primary managers of IGC in many academic and community medical centers, it is important to understand their baseline knowledge and perceptions. Currently, scant data are available [14][15][16][17] and no validated survey tools for this topic exist in the medical literature.
Numerous strategies have recently been employed in an attempt to improve IGC, including standardized insulin order sets [18][19][20][21][22][23][24][25][26], mentoring [27], diabetes care team intervention [28][29][30], computerized systems [31,32], physician and nurse education [19,33,34], and resident education [35,36]. While guidelines and interventions designed to improve IGC are gaining attention, knowledge of and barriers to successful implementation of these guidelines along with how they are being translated into clinical practice by resident physicians remains unclear [14]. Identification of common barriers to successful IGC may help foster better educational interventions, ultimately leading to improvements in IGC and patient care. We therefore aimed to construct and evaluate an easy-to-use survey tool for perceptions and knowledge of IGC among resident physicians.

Research locations
We performed this study at four locations: University of Mississippi Medical Center (UMMC), Jackson, MS; University of Virginia Health System (UVA), Charlottesville, Virginia; University of Louisville Health Sciences Center (UL), Louisville, KY; and Emory University Healthcare (Emory), Atlanta, GA.

Questionnaire design and administration
To identify relevant prior work on resident physician perspectives of IGC, we searched PUBMED and Google Scholar using search terms "resident physician AND inpatient glycemic control," "resident AND inpatient glycemic control," and "resident physician AND inpatient hyperglycemia" (Fig. 1). We then expanded the search by examining citations referenced in the manuscripts initially retrieved. A total of 18 manuscripts were retrieved, four [14][15][16][17] of which were deemed highly relevant for inclusion in our study.
The inpatient glycemic control questionnaire (IGCQ) was constructed by using the previously published but unvalidated surveys [14][15][16][17] as a framework. We then created a novel survey tool by consolidating and adapting these previously published questionnaires, specifically by decreasing the amount of demographic data collected, expanding the scope and focus of question material, utilizing Likert-scale answer choices, and inserting questions designed to assess knowledge of suggested inpatient glycemic targets from consensus guidelines [4]. Institutional Review Board (IRB) approval at each institution and verbal consent from each participant were obtained prior to questionnaire administration. The IGCQ was then administered to internal medicine (IM) and medicine-pediatric resident and chief resident physicians to determine their comfort with managing IGC, knowledge of inpatient glycemic target values, and perceived barriers to successful IGC. We distributed questionnaires in person during resident physician educational lectures at UMMC, used Google Forms (Google; San Francisco, California) for data collection at Emory, utilized IRB-approved software (QuestionPro Inc.; San Francisco, California) at UVA, and used SurveyMonkey Pro (SurveyMonkey; San Mateo, California) at UL. Anonymous results were collected during February-May 2015 (UMMC), March-June 2016 (Emory), November-December 2016 (UVA), and March-May 2017 (UL). Survey results were tabulated in an Excel (Microsoft; Redmond, Washington) spreadsheet for data analyses.

Rasch partial credit model
RPCM is a unidimensional model that enables "specifically objective" comparisons of persons and items when analyzing responses recorded in two or more ordered categories [37,38]. RPCM was conducted using Winsteps software (version 3.70.0.5) to examine data for item fitting, dimensionality, and category response functioning (thresholds).

Item fitting, dependency, and dimensionality
Fit statistics examine data in comparison with expectations of RPCM. Item fitting is calculated using chi-square statistics and may be reported as mean square (MNSQ), an unstandardized average value of squared differences between the RPCM's expected and actual values for an item. This value for each item should ideally fall between 0.50 and 1.70 for clinical tools [39]. Item dependencies represent correlation between item difficulties, identifying items potentially measuring the same concept, which could form a sub-dimension and thereby affect overall unidimensionality of the test. Principal component analysis (PCA) of the differences between observed and expected scores or residuals can reveal contrasting items, which can potentially breach the unidimensionality of outcome measure [39].

Category response functioning
RPCM compares the probability of a category response to other category responses of the same item as well as category responses from other items [39].

Classical test theory
CTT is a traditional quantitative approach to testing the reliability and validity of a scale, based on that scale's individual items. CTT assumes each subject has a true score, T, that would be obtained if there were no errors in measurement [40]. True scores quantify values on an attribute of interest, defined as the underlying concept, construct, trait, or ability of interest. As values of the true score increase, responses to items representing the same concept should also increase, assuming that item responses are coded so that higher responses reflect more of the concept [40]. We used a Kruskal-Wallis test to evaluate differences in total scores by several variables, including postgraduate year (PGY), program, and gender.

Preliminary evaluation Pretest (phase 1)
Three IM attending physicians, two endocrinology attending physicians, and two IM resident physicians performed pretest review of the survey tool. Our goals were to ensure that: (1) the IGCQ adequately covered key aspects of both IGC and resident education and (2) question construction was neither too leading nor confusing. Feedback received in this stage raised concerns about two Likert scale questions being "vague and open for interpretation." Based on these recommendations, we revised the IGCQ accordingly. The IGCQ as presented in Additional file 1: Appendix 1 reflects the survey tool after pretest revision but before distribution to resident physicians.

Further study (phase 3)
For prospective evaluation of the revised IGCQ, the survey was administered to 64 resident physicians at UL.
RPCM was then applied to UL cohort data and revealed no disordered thresholds (Fig. 4), confirming three category responses as the better answer choice scale for non-knowledge based questions. Fit statistics showed improvement in MNSQ for Questions 2 and 3 ( Table 2). Questions 4 and 5 trended toward misfit (but not  enough to degrade quality of scale) while Question 17 showed severe misfit ( Table 2). We then applied RPCM to collective data from all four centers. Analyses demonstrated no disordered thresholds (Fig. 5), though Questions 2 and 3 again showed little discrimination with 3choice answer scale. Fit statistics demonstrated moderate misfit for Question 17 (Table 3). We also used PCA to assess dimensionality. For Questions 1-9, the first principal component was essentially the average of the items and accounted for 25% of the total variance. A second principal component roughly separated Questions 1-4 from Questions 5-9, indicating that construction of separate scales could be considered. Overall, PCA indicated that it was reasonable to tabulate the responses from Questions 1-9 into a "comfort with managing IGC" scale (with higher scores indicating greater comfort) and Questions 14-19 into a "barriers to IGC" scale (with higher scores indicating lower perception of barriers). For Questions 14-19, the first principal component accounted for 31% of the total variation, which was again essentially the average (without Question 17). The second principal component accounted for 18% of the variation and was mostly due to Question 17, suggesting that the scale would improve if Question 17 were removed. CTT analyses using Kruskal-Wallis test indicated no difference in performance by gender, program, or PGY for the "barriers to IGC" scale. However, the same analyses performed on the "comfort with managing IGC" scale indicated differences in performance by gender and PGY (Table 4). Specifically, comfort with  management scores increased as PGY increased (i.e., resident physicians grow more comfortable managing inpatient glycemic control as they progress through training). Figure 6 demonstrates total score frequencies for both scales.

Final revision (phase 4)
Cumulative RPCM data from all phases of study were considered when making final modifications to the IGCQ. We removed Question 17 as fit statistics demonstrated misfit throughout phases 2-3. We also removed Questions 2 and 3 since RPCM analyses demonstrated threshold disorder in phase 2 testing and little discrimination, even with 3-choice answer scale, in both phases 2 and 3. Finally, Question 4 was removed because its data trended toward mild misfit and because it addressed time spent on education in the outpatient setting, which was ultimately viewed as extraneous in light of the IGCQ's focus on IGC. These final refinements completed preliminary evaluation and led to the 16-item IGCQ (Additional file 2: Appendix 2). Questions 1-6 represent the "comfort with managing IGC" scale, Questions 7-10 represent the "knowledge of IGC" scale, and Questions 11-16 represent the "barriers to managing IGC" scale.

Questionnaire participation
Previous work demonstrated that an estimated minimum sample size range of 108-243 subjects is needed for Rasch analysis for item calibration with ± 0.5 logits at 99% confidence, even if the scale is poorly targeted [41]. However, a minimum sample size of 150 subjects is considered to be adequate in most cases at this confidence level [41]. In our study, 246 of 438 (56.2%) eligible resident physicians completed the IGCQ during various phases of development, including 182 in the pilot study.

Discussion
To our knowledge, the IGCQ is the first preliminarily evaluated survey tool specifically constructed for assessment of perceptions and knowledge of IGC among resident physicians. Positive attributes of the IGCQ include its basis on previous work in the field, evaluation through RPCM analysis at multiple centers, ease of use, and availability for future research. The IGCQ is primarily appropriate for use in assessing resident physician perspectives on proper glycemic control of the hospitalized patient. The survey tool has many potential applications and could be used to evaluate the effect of educational interventions on resident physician knowledge of IGC. In the future, it could also be used to compare how resident physician knowledge correlates with real-world clinical care received by hospitalized patients. The IGCQ might also be useful in various other assessments of resident physician perceptions and knowledge. For example, the IGCQ could be easily modified to focus on outpatient glycemic control by reframing Likert scale questions for that specific topic and having knowledge-based questions focus on appropriate outpatient glycemic targets from recent consensus guidelines [42].
Our study has several limitations that should be noted. First, while our results indicate that the IGCQ fits well the Rasch model standards, this analysis is only for one cohort of resident physicians and, thus, further analyses are indicated for future cohorts. Second, each participating institution had different IRB-approved survey software, so we were unable to standardize survey administration across all centers. Third, no differential item functioning (DIF) analyses were performed. DIF occurs in situations where members of different groups show differing probabilities of endorsing an item despite possessing the same level of the ability that the item is intended to measure [43]. Fourth, neither Rasch analysis of knowledgebased questions nor external validation of the survey tool were performed. Future studies could evaluate external validity of the IGCQ and psychometric properties of knowledge-based questions. Further evaluation of any performance differences by gender, ideally with psychometric statistics such as DIF analyses, would also be valuable.

Conclusion
Herein we presented the construction and preliminary evaluation of the IGCQ. Development of the IGCQ was informed by a review of the current literature on resident physician perspectives of IGC. Examination of the IGCQ utilizing RPCM yielded satisfactory results; however, a few potential issues were identified. We analyzed these issues accordingly and restructured the IGCQ based on study data. The preliminarily -evaluated IGCQ could be valuable for studies seeking to examine the effect of educational interventions on resident physician knowledge of IGC. Future studies could evaluate external validity of the IGCQ and psychometric properties of knowledge-based questions. Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Authors contributions WBH designed the survey tool, obtained IRB approval at UVA and UMMC, collected and analyzed data from all four study centers, and authored the manuscript. SL obtained IRB approval and collected data at Emory and assisted with writing and revising the manuscript. MD collected data at UL and assisted with writing and revising the manuscript. MRC performed all statistical analyses and assisted with writing the manuscript. NTK obtained IRB approval and oversaw data collection at UL and assisted with writing and revising the manuscript. JLK oversaw data collection at UVA and assisted with writing and revising the manuscript. SCT designed the survey tool, oversaw data collection at UMMC, and assisted with writing and revising the manuscript. All authors read and approved the final manuscript.

Funding
The authors have no sources of funding to disclose for this study. No funding body played any role in the design of the study; collection, analysis, and interpretation of data; or writing of the manuscript.

Availability of data and materials
Please contact the first author for data requests.
Ethics approval and consent to participate Institutional Review Board (IRB) approval for study design and verbal consent to participate was granted separately at each institution (Emory University, University of Virginia, University of Louisville, and University of Mississippi) prior to questionnaire administration. Verbal consent was obtained from all participants in this study.

Consent for publication
Not applicable.