Skip to main content

Extending the use of the conceptions of learning and teaching (COLT) instrument to the postgraduate setting

Abstract

Background

Several studies have shown that conceptions of teachers on teaching and learning can influence the teaching practices and behavior in higher education. This association is also found in undergraduate medical education but not yet established in postgraduate medical setting. An instrument, Conceptions of Learning and Teaching (COLT) was developed to measure conception of teachers in undergraduate medical education. COLT is a 3-factor 18-item questionnaire. The objective of this study is to evaluate if COLT is valid for postgraduate medical education.

Methods

We invited postgraduate clinical faculty from 3 hospitals in the Netherlands to fill out the COLT. Confirmatory and exploratory factor analysis were performed to evaluate the fit of the postgraduate clinical faculty data to the COLT. Analysis of variance was done to evaluate if there was difference among the 3 hospitals in terms of the response by the clinical faculty.

Results

Confirmatory factor analysis showed that the postgraduate faculty data had a 2 factor structure after removal of five items. These factors were Teacher Centeredness (TC) and combined Appreciation of Active Learning and Orientation to Professional Practice (A-P) and were considered as comparable to the factors in the original COLT, expressing the post-graduate learning and teaching setting. As several items were removed, the fit was suboptimal, yet did suggest validity for use of the COLT for postgraduate medical education.

Conclusion

The modified COLT can be used to measure conceptions of teaching and learning in postgraduate medical education. We recommend further study to improve the factor structure of the modified COLT.

Peer Review reports

Background

Several studies have shown that conceptions can influence the teaching practices and behavior in the classroom [1,2,3,4]. Many educational researchers have adapted the definition by Pratt [5] that “conceptions are specific meanings attached to phenomena which then mediate our response to situations involving those phenomena … in effect we view the world through the lenses of our conceptions, interpreting and acting in accordance with our understanding of the world … ..our conceptions significantly influence our perception and interpretation of events, people and phenomena surrounding us” [p. 204]. Pratt [5] posits that this relation between conceptions and behavior impacts on how educators think about teaching. Additionally, innovations in teaching and learning are unlikely to happen without concomitant changes in the conception of teaching of the faculty and staff [6, 7]. Kember [6] in his review of teaching conceptions of university academics stated that teaching conceptions are related to measures of quality of learning, and also impact teaching approaches which in turn affect the learning outcomes and how student learn. Kember and Kwan [7] in a study of 17 lecturers in a university about conceptions of teaching, noted that teachers who conceive of teaching as transferring knowledge were more likely to utilize content-centered approaches to teaching, whereas those whose conception of teaching is being facilitators are inclined to utilize learning-centered approaches. These above mentioned studies clearly illustrate a relationship between conceptions of teaching and approaches to teaching and learning. Devlin, however, challenged the associations between conceptions and methods of teaching and the purported relationship between conceptions and improvement on teaching [8]. In their study, Peeraer et al. [9] were not able to confirm a relationship between conceptions of learning and approaches to teaching. These statements, which are contrary to the prevailing notion on the role of conception in teaching, underscore the need for further research regarding conceptions of teaching and its influence on faculty and staff and learning outcomes.

In undergraduate medical education, it was recognized that conceptions of teaching and learning of faculty can promote reflections of teachers and stimulate changes in the teaching behavior [10]. However, clinical teachers rarely receive formal instruction in teaching. Moreover, in medical education, expertise in a particular field is considered equivalent to being a qualified clinical teacher [11, 12]. There is a scarcity of research regarding conceptions of teaching of clinical faculty, in spite of the potential relation between teaching beliefs and behavior of clinical teachers [12, 13]. Since in most institutions the faculty involved in postgraduate education are also involved in undergraduate medical education, it is tempting to assume that the conceptions of undergraduate medical faculty who are also involved in post-graduate education are similar. However, the contexts of undergraduate and postgraduate learning and teaching are different. Many studies on conceptions of learning and teaching are done in secondary and higher education and recently in undergraduate medical education, yet little is done in the postgraduate medical setting. Studying conceptions of teaching in postgraduate medical education is more complicated since a significant part of medical education takes place in clinical settings, presenting with unique challenges to teaching and learning and the circumstances are such that educators cannot control the cases one will encounter and trainees have different levels of competence [12]. Jacobs et al. [10] put forward an instrument, the COLT, that measures conceptions of learning and teaching among undergraduate faculty in two academic institutions for undergraduate medical education in the Netherlands. COLT is a 3 factor, 18 item instrument, the 3 factors are Teacher Centeredness, Appreciation of Active Learning and Orientation to Professional Practice. Using this instrument, they were able to show differences in the conceptions of the faculty between an institution which has a student-centered orientation and an institution which has been using a teacher-centered curriculum [10]. COLT, by design, has neither been used nor shown to be valid in postgraduate education. The objective of this study is to investigate whether the COLT, designed for undergraduate medical education is valid in postgraduate medical education.

Methods

Setting

We sampled in 3 medical centers in the Netherlands: 1 academic medical center or hospital (denominated as center 1) and two generic teaching medical centers (denominated as centers 2 and 3). The academic medical center has an established tradition of student-centered education and problem-based learning approach.

Data collection

Instrument

Since COLT was originally designed for undergraduate medical institutions [10] with student-centered curricula, we invited four physicians involved both in undergraduate medical education and postgraduate training to evaluate the original COLT to make it more appropriate in the context of a postgraduate environment. Minor revisions were done and it was agreed that the content of COLT was applicable to the setting of postgraduate residency training. These revisions included replacing the word “students” in the original COLT to “residents” and modifying item number 1 which originally was phrased as “Students should first master basic science knowledge before they can formulate their own goals” to “Residents should first master general medical principles before they can formulate their own learning goals”. COLT is an 18-item 3-factor questionnaire with a five-point Likert scale (1=strongly disagree, and 5=strongly agree). See supplement Table 1.

Participants

The COLT questionnaire was distributed to the physician staff of the 3 hospitals. The e-mail invitation was sent via the overarching secretaries of staff at each of the 3 hospitals, the e-mail contained information on the research, informed consent and a link to the questionnaire. One reminder was sent after 4 weeks.

Data analysis

We performed a confirmatory factor analysis (CFA) and exploratory factor analysis (EFA) to determine if the postgraduate data fitted the 3-factor structure of the original 18-item COLT [10]. We used the following criteria and associated pre-determined cut-off values to gauge goodness of fit: Tucker-Lewis index (TLI > 0.9), comparative fit index (CFI > 0.9), root mean square error of approximation (RMSEA < 0.08) and standardized root mean square residual (SRMR < 0.08). In the present study, a sufficient fit was deemed to have been achieved when 4 criteria produced significant results. Analysis of variance (Anova) and multivariate analysis of variance (Manova) were done to see if there was a significant difference in the scores on the 2 factors among the participants from the three hospitals.

Results

There were 417 participants in total, 245 male (58.8%) and 170 female (40.8%) while 2 participants did not indicate their gender. Ninety eight percent belonged to age group 30–60 years, as to number of years involved in clinical training 25.7% have been involved for < than 5 years, 28.1% for 5–10 years, and 46% for more than 10 years. There were 173 participants from the academic medical center, 142 from center 2 and 102 from center 3.

A CFA showed that the four fit measures (SRMR = 0.053, RMSEA = 0.047, CFI = 0.886 and TLI = 0.864) were just outside the range for adequate fit criteria, as indicated by Jacobs et al. [10]. The CMIN/DF was 2.02, which is within the range (< 3), but p < 0.05, suggesting the fit was not optimal. As part of the statistical validation of the questionnaire, the overview of possible disturbances, ordered by the importance provided in the CFA analysis was used. Based on this list, a decision was made which items caused the most disturbance. Then the model was recalculated to determine the fit without those items. Exploratory factor analysis (EFA) was done, and a repeat CFA afterwards showed a good fit if 5 items were removed from the original COLT which led to a 2-factor model. It was shown that removal of several items indeed improved the model and led to an acceptable fit.

Items removed were no. 8 in the original COLT, under the factor Teacher-centeredness (TC), item no. 9, under the factor Appreciation of Active Learning and under the factor Orientation to Professional Practice items 15 and 18. The removal of these items led to the proposed 2 factor model TC and combined Appreciation of Active Learning and Orientation to Professional Practice (A-P). The fit of the 2-factor model was better than the 3-factor model. Removing 1 additional item, item 17, under Orientation to professional practice resulted in good fit: SRMR = 0.049, RMSEA = 0.042, CFI = 0.919 and TLI = 0.902, these are well in the range for adequate fit as indicated by Jacobs et al. [2012]. The CMIN/DF is 1.72, which is within the range (< 3), but still p< 0.05, suggesting that the fit is not optimal but usable.

There was a preference for the combined factors A-P across all age groups, regardless of gender, duration of teaching and the hospital surveyed. Tables 1 and 2 summarize these data.

Table 1 Mean scores and standard deviation (S.D.) for TC and A-P based on gender
Table 2 Mean scores and standard deviation (S.D.) according to duration of teaching

There was no significant difference in the responses of the participants among the three hospitals surveyed in this study using Anova and Manova. The Cronbach’s alpha overall was 0.477, for TC 0.703 and for A-P 0.504. Supplement Table 2 shows the postgraduate COLT questionnaire, and the overall responses.

Discussion

Although the COLT [10] was designed for student-centered medical education in an undergraduate setting, this study showed the questionnaire is applicable in postgraduate medical education, although with some modifications. The postgraduate COLT met the criteria for goodness of fit although the fit is not optimal. Our study also revealed that among the 417 physicians who participated there was a preponderant preference for the items originally included in the factors “appreciation of active learning” and “orientation to professional practice” which were combined into a single factor after the CFA. In the original COLT [10] both these factors exemplified concepts consistent with student-centered education in a higher education setting. Most of the instruments available in the literature to measure conceptions of teaching and learning are designed for higher education and undergraduate medical education.

Postgraduate medical training, in contrast with undergraduate medical education, is faced with distinct challenges in the context of daily clinical care. Both clinical teachers and residents have to combine the responsibilities of daily clinical care with learning and learning opportunities. The trainees have limited options to choose and select and therefore must deal with the available cases as learning opportunities. Likewise, the faculty has to adjust to the cases to teach or discuss. Postgraduate trainees in the setting of work-based learning have dual roles as learners and as ‘employees’ wherein they are also expected to provide service to their patients [14]. In addition, in postgraduate training, the educational alliance between trainee and clinical teacher takes a next level, as the trainee is also a junior-colleague and responsible member of the team [15, 16]. These contextual factors affect the learning environment [12, 17]. The trainees are expected to cope with the limitations presented by the above mentioned conditions, and try to minimize the distractions these can have on their training.

Our study showed some important differences in the factor structure of the COLT in post-graduate settings in contrast with the original COLT, which was intended for undergraduate medical education. In the postgraduate COLT, using statistical analysis 5 items were removed to improve the fit. These differences were anticipated, given that these could be related to the specific context of workplace-based teaching and learning in postgraduate training as delineated in the previous paragraph. For the separate items, some additional remarks can be made:

For the item: “When residents collaborate they teach each other the wrong things”, this item was part of the teacher-centeredness factor in the undergraduate COLT but has to be removed in the current study to improve the fit. We speculate that physicians who participated in the study most probably felt that at the level of postgraduate setting, residents cannot afford to make mistakes or teach each other ‘wrong things’ since they are involved in the management of patients already, which would make the item irrelevant.

For the item: “Residents learn a great deal by explaining the subject matter to each other.” This item may have given the impression that in a postgraduate setting theoretical learning is still dominant which is opposite the actual condition, where decisions made can impact a person’s life or quality of life. Sociocultural theories are increasingly used in graduate medical education to explain how learning occurs, this includes the theory of situated learning. Situated learning emphasizes that learning is intertwined with the context, the social relations and practices [18,19,20]. Lave and Wenger [18] expanded this to legitimate peripheral participation as a process by which trainees start as unrecognized members to being respected members of communities of practice.

For the item: “Being introduced to the day-to-day practice of their future profession motivates residents to learn”. Although this item fits the student-centered education of center number 1 in undergraduate medical education, such may not be the case in postgraduate educational setting and this may be less evident in the workplace learning where clinical duties prevail. In addition, the residents are already working in their ‘future’ profession, so this may have given a variety of responses that did not lead to a fit.

For the item: “I think that interactions between me and the residents are an important aspect of my teaching”. The removal of this item was not expected, in a study done among residents involving internal medicine and pediatric residents, a strong motivator to learning for the residents was the engagement between staff and residents [21]. A similar finding although involving undergraduate medical education but done in the same cultural context as our current study is reported by Strand et al. [22]. They described learning as partnership, as one of the main three main themes, where the interaction with the supervisor plays a significant role, and in addition the supervisory relationship gave way to reciprocal learning [22]. The fact that the responses of the physicians surveyed did not support this item could reflect an ongoing dichotomy between the perceptions of staff and perception of the trainees. In our view this might require further investigation, in the light of the above studies which strongly support that interaction between trainees and supervisors play a crucial role and promote learning among undergraduate and postgraduate medical trainees.

For the item: “Discussing topics with each other helps residents learn how to deal with different points of view, so as to gain a deeper understanding”, Teunissen et al. [23] proposed a framework on how residents learn which postulates that the starting point of learning is getting involved in work-related activities. For residents the ‘experience‘initiates the construction of meaning which generates personal knowledge [23]. This is similar to experiential learning theory which highlights that trainees construct knowledge and meaning based on their real-life experiences [24]. Thus, for the residents the emphasis at this point is on actual activities that induce learning and not on theoretical discussion.

The principal advantages and benefits of measuring the conceptions of the faculty were found to include evaluation and improvement of teaching, faculty development and peer reviews of teaching [25]. This is echoed by the study of Strand et al. [22] who investigated the physicians’ conception focusing on learning in the clinical workplace and how clinical supervisors contribute to learning. One of their conclusions was that mapping the workplace supervisor conceptions of learning can be a valuable starting point for medical schools and educational developers working with changes in clinical educational and faculty development practices.

In the postgraduate setting, Calkins et al. [26] believe that helping the faculty to develop a complex understanding of teaching is a fundamental component in a faculty member’s professional development. Perspectives on teaching are philosophical orientations to knowledge, learning and the responsibility of an educator [27] and healthy discussion on a teacher’s perspectives is a good way to understand philosophies their colleagues hold onto [28].

In this study, an existing instrument was used as the starting point, which has certain advantages which includes time and the availability of a preexisting construct. We agree that further study and validation are needed to come to an instrument to measure conception among postgraduate clinical teachers. We recommend future studies that can improve the reliability of the current postgraduate COLT, possibly with additional items tailored specifically to workplace learning in the postgraduate setting. We consider it a limitation of our study that although we found a new factor structure, the factors are not optimally represented by the items. Given the deletion of items, a smaller number of items remained, also explaining the suboptimal Cronbach’s alpha overall and of the two factors.

Conclusion

There are established benefits in being able to measure the conceptions of teaching and learning, however in postgraduate medical education there is a lack of instruments that can be used to measure faculty conceptions. The postgraduate COLT, although the psychometric properties are not optimal, may be used for this purpose with the limitations presented. For clinical teachers, becoming aware of their conceptions can be a stepping stone to improve their teaching approaches, benefitting not only the medical trainees but the patients as well.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

COLT:

Conceptions of learning and teaching

TC:

Teacher centeredness

A-P:

Appreciation of active learning-Orientation to professional practice

CFA:

Confirmatory factor analysis

EFA:

Exploratory factor analysis

TLI:

Tucker-Lewis index

CFI:

Comparative fit index

RMSEA:

Root mean square error of approximation

SRMR:

Standardized root mean square residual

Anova:

Analysis of variance

Manova:

Multivariate analysis of variance

SD:

Standard deviation

References

  1. 1.

    Pajares MF. Teachers’ beliefs and educational research: cleaning up a messy construct. Rev Educ Res. 1992;62(3):307–32.

    Article  Google Scholar 

  2. 2.

    Trigwell K, Prosser M. Changing approaches to teaching: a relational perspective. Stud High Educ. 1996;21(3):275–84.

    Article  Google Scholar 

  3. 3.

    Trigwell K, Prosser M, Waterhouse F. Relations between teachers’ approaches to teaching and students’ approaches to learning. High Educ. 1999;1:57.

    Article  Google Scholar 

  4. 4.

    Kember D, Kwan KP, Ledesma J. Conceptions of good teaching and how they influence the way adults and school leavers are taught. Int J Lifelong Educ. 2001;20(5):393–404.

    Article  Google Scholar 

  5. 5.

    Pratt DD. Conceptions of teaching. Adult Educ Q. 1992;42(4):203–20.

    Article  Google Scholar 

  6. 6.

    Kember D. A reconceptualization of the research into university academics’ conceptions of teaching. Learn Instr. 1997;7(3):255–75.

    Article  Google Scholar 

  7. 7.

    Kember D, Kwan KP. Lecturers’ approaches to teaching and their relationship to conceptions of good teaching. Instr Sci. 2000;28:469–90.

    Article  Google Scholar 

  8. 8.

    Devlin M. Challenging accepted wisdom about the place of conceptions of teaching in university teaching improvement. Int J Teach Learn Higher Educ. 2006;18(2):112–9.

    Google Scholar 

  9. 9.

    Peeraer G, Donche V, De Winter BY, Muijtjens AMM, Remmen R, Van Petegem P, Scherpbier AJJA. Teaching conceptions and approaches to teaching of medical school faculty: the difference between how medical school teachers think about teaching and how they say that they do teach. Med Teach. 2011;33(7):e382–7.

    Article  Google Scholar 

  10. 10.

    Jacobs JCG, Van Luijk SJ, Van Berkel H, Van Der Vleuten CPM, Croiset G, Scheele F. Development of an instrument (the COLT) to measure conceptions on learning and teaching of teachers, in student-centered medical education. Med Teach. 2012;34(7):e483–91.

    Article  Google Scholar 

  11. 11.

    McLeod PJ, Meagher T, Steinert Y, Schuwirth L, McLeod AH. Clinical teachers’ tacit knowledge of basic pedagogic principles. Med Teach. 2004;26(1):23–7.

    Article  Google Scholar 

  12. 12.

    Taylor EW, Tisdell EJ, Gusic ME. Teaching beliefs of medical educators: perspectives on clinical teaching in pediatrics. Med Teach. 2007;29:371–6.

    Article  Google Scholar 

  13. 13.

    Williams RG, Klamen DL. See one, do one, teach one - exploring the core teaching beliefs of medical school faculty. Med Teach. 2006;28(5):418–24.

    Article  Google Scholar 

  14. 14.

    Swanwick T. Informal learning in postgraduate medical education: From cognitivism to “culturism”. Med Educ. 2005;39:859–65.

    Article  Google Scholar 

  15. 15.

    Telio, S., Ajjawi, R., & Regehr, G. (2015). The “educational Alliance” as a framework for Reconceptualizing feedback in medical education. Acad Med 2015;90:609–614.

  16. 16.

    Telio, S., Regehr, G., & Ajjawi, R. (2016). Feedback and the educational alliance: examining credibility judgements and their consequences. Med Educ 2016;50:933–942.

  17. 17.

    Stenfors-Hayes T, Hult H, Dahlgren LO. What does it mean to be a good teacher and clinical supervisor in medical education? Adv Health Sci Educ. 2011;16(2):197–210.

    Article  Google Scholar 

  18. 18.

    Lave J, Wenger E. Situated learning: legitimate peripheral participation. United Kingdom: Cambridge University Press; 1991.

    Google Scholar 

  19. 19.

    Mann KV. Theoretical perspectives in medical education: past experience and future possibilities. Med Educ. 2011;45(1):60–8.

    Article  Google Scholar 

  20. 20.

    Hodges BD, Kuper A. Theory and practice in the design and conduct of graduate medical education. Acad Med. 2012;87(1):25–33.

    Article  Google Scholar 

  21. 21.

    Pacifico JL, Villanueva JAS, Heeneman S, van der Vleuten C. How perceptions of residents toward assessment influence learning: a qualitative study. Asia Pac Scholar. 2020;5(1):46–53.

    Article  Google Scholar 

  22. 22.

    Strand P, Edgren G, Borna P, Lindgren S, Wichmann-Hansen G, Stalmeijer RE. Conceptions of how a learning or teaching curriculum, workplace culture and agency of individuals shape medical student learning and supervisory practices in the clinical workplace. Adv Health Sci Educ. 2015;20(2):531–57.

    Article  Google Scholar 

  23. 23.

    Teunissen PW, Scheele F, Scherpbier AJJA, Van Der Vleuten CPM, Boor K, Van Luijk SJ, Van Diemen-Steenvoorde JAAM. How residents learn: qualitative evidence for the pivotal role of clinical activities. Med Educ. 2007;41(8):763–70.

    Article  Google Scholar 

  24. 24.

    Yardley S, Teunissen PW, Dornan T. Experiential learning: transforming theory into practice. Med Teach. 2012;34(2):161–4.

    Article  Google Scholar 

  25. 25.

    Collins, J. B., & Pratt, D. D. (2011). The teaching perspectives inventory at 10 years and 100,000 respondents. Adult Educ Q 2011;61(4):358–375.

  26. 26.

    Calkins S, Johnson N, Light G. Changing conceptions of teaching in medical faculty. Med Teach. 2012;34:902–6.

    Article  Google Scholar 

  27. 27.

    Pratt DD, Arseneau R, Collins JB. Reconsidering “good teaching” across the continuum of medical education. J Contin Educ Heal Prof. 2001;21:70–81.

    Article  Google Scholar 

  28. 28.

    Pratt, D. D., Smulders, D., & associates. Five perspectives on teaching: mapping a plurality of the good. Malabar, Florida: Krieger Publishing. 2016.

Download references

Acknowledgements

The authors are grateful to all the physician staff of the three hospitals who took part in the study.

Funding

None.

Author information

Affiliations

Authors

Contributions

All authors contributed significantly to the conception and execution of the study. JLP conceptualized and initiated the study, wrote the first draft up to the final draft. WvM facilitated the data collection. JD facilitated the statistical analysis and commented up to the final draft. CvdV critically commented on the initial draft up to the final copy. JCGJ critically commented up to the final draft. SH facilitated the data collection, critically commented and contributed to the final draft. All authors read and approved the final draft.

Corresponding author

Correspondence to Jaime L. Pacifico.

Ethics declarations

Ethics approval and consent to participate

The study was approved by the ethical review board of the Dutch Society for Medical Education (approval no. NVMO-ERB-882). Participation was voluntary.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Table S1.

Original COLT Questionnaire. Table S2. Postgraduate COLT Questionnaire and overall scores.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Pacifico, J.L., van Mook, W., Donkers, J. et al. Extending the use of the conceptions of learning and teaching (COLT) instrument to the postgraduate setting. BMC Med Educ 21, 32 (2021). https://doi.org/10.1186/s12909-020-02461-2

Download citation