In three previous studies, it was observed that knowledge structure, as evaluated by CMA, was a correlate of diagnostic performance [7–9]. It was also observed that using a diagnostic scheme during learning increased the odds of having expert-type short-term knowledge structure, as did the use of a diagnostic scheme by preceptors during the learning experience [7, 9]. In the present study, the focus was on long-term knowledge structure and, in particular, the association between this dependent variable and two explanatory variables; short-term knowledge structure and the use of a diagnostic scheme during learning.
Changes in the medical knowledge structure of students over time
Perhaps not surprisingly, the first finding in this study was that knowledge structure appears to change with time. In the group of students taken as a whole, there was a non-significant trend towards a greater number of students changing from expert-type to novice-type structure than vice versa. However, somewhat surprising was the fact that the change in knowledge structure was unpredictable, suggesting that variables other than decay over time may influence changes in knowledge structure.
Variables associated with expert-type knowledge structure one year later
Two explanatory variables were considered as potential correlates of long-term knowledge structure; short-term knowledge structure and use of a diagnostic scheme during learning. On its own short-term knowledge structure was not observed to be a correlate of long-term knowledge structure, whereas use of a diagnostic scheme was. There was an interaction between these variables, suggesting that the association between short and long-term knowledge structure is conditional upon whether or not a diagnostic scheme was used during learning. In the group of students who did not use a diagnostic scheme there was a greater number of students who changed from expert-type to novice-type structure, whereas no such change was observed in the group using a diagnostic scheme. This observation suggests that students who developed expert-type knowledge structure during the period of instruction, and who use a diagnostic scheme in so doing, were more likely to keep expert-type knowledge structures than those who did not use a diagnostic scheme.
The apparent 'instability' of knowledge structure over time raises concerns regarding validity of studying knowledge structure. While these results suggest that short-term knowledge structure is not predictive of long-term knowledge structure, previous studies have shown that it is a correlate of [short-term] diagnostic performance. We would propose that evaluation of knowledge structure be used to try and explain the present [poor diagnostic performance] rather than predict the future [7–9]. In a recent study on diagnostic performance, Woods et al observed somewhat parallel results to these, i.e., that diagnostic performance immediately after instruction did not predict performance at a later time, whereas the method of learning was a correlate of later performance.
By virtue of the observational design of this study it was not possible to test the hypothesis that use of a diagnostic scheme during learning is responsible for the maintenance of expert-type knowledge structure. To test this hypothesis, a trial in which subjects are randomized to receive a diagnostic scheme or otherwise would be required. If such a benefit to the use of diagnostic schemes were confirmed, we would speculate that the mechanism by which diagnostic schemes might help maintain expert-type knowledge structure is by providing an organized and meaningful relationship between concepts. The way in which experts store knowledge of a clinical presentation [at least in the nephrology presentations that we have studied] is consistent with how knowledge is usually retrieved during the process of diagnostic reasoning (scheme-inductive reasoning) [16]. Thus, diagnostic schemes may provide internal encoding specificity that enhances the formation of, and retention of, expert-type knowledge structure [17].
Cognitive strategies to facilitate learning and knowledge retention
In cognitive psychology the explicit use of cognitive strategies during educational encounters to facilitate learning and knowledge retention is well described and is referred to a 'reciprocal teaching' [10]. Conceptual frameworks, such as diagnostic schemes, are one of many strategies available. They have been studied widely in university students in a variety of domains and have also been studied to a lesser extent in medical students and health care professionals. While most studies show a short-term improvement in knowledge retention and problem solving, few studies have examined the effects of these strategies on longer-term knowledge retention. In the studies that evaluated knowledge after the initial learning period, the impact of conceptual frameworks appeared to be variable. This study adds to the existing body of literature by suggesting that the use of a conceptual framework to facilitate the development of expert-like knowledge structure may attenuate the loss of knowledge structure that was seen in students who did not use a conceptual framework.
Study limitations
The present study has some important limitations. Results from an observational cohort study such as this are hypothesis generating and cannot be interpreted as showing that it is the use of a conceptual framework that facilitates (or 'causes') expert-type knowledge structure. As discussed above, to test this hypothesis a study in which subjects were randomized to receive the intervention (conceptual framework) or not, would be required. The small number of students who were studied at the two time points and the fact that only a single presentation was studied also limits generalizability of these results to other presentations. The stability of knowledge structure over time, including day-to-day, is not known and the results from this study would have been strengthened by several repeated measures of knowledge structure, rather than simply two measures separated by one year. In the present study we did not re-evaluate diagnostic performance in the clinical presentation of metabolic alkalosis and we cannot necessarily assume that the correlation between knowledge structure and diagnostic performance, evident in the short-term, persists. While the results of the clerkship examinations might have offered some overall assessment of diagnostic ability, given the importance of content specificity it was felt that this would not serve as an accurate assessment of diagnostic ability in the specific clinical presentation of metabolic alkalosis. Finally, the long follow-up period of one year, while advantageous in minimizing the influence of pre-examination cramming on knowledge structure, introduces the possibility of performance bias, although this would be likely to affect the groups equally.