Factors affecting the intention of providers to deliver more effective continuing medical education to general practitioners: a pilot study

Abstract

Background

Despite the importance of continuing medical education (CME) for GPs, there has been little research into how providers decide what types of CME to deliver to GPs. This study aimed to identify factors affecting the intention of providers to provide more effective types of CME; and to design a survey instrument which can be used to test the applicability of Triandis' model of social behaviour to the provision of CME to general practitioners.

Methods

This was a cross-sectional study on a convenience sample of 11 Australian providers of CME for interviews and a random sample of 25 providers for the pilot test. Open-ended interviews structured on Triandis' theory were performed with key informants who provide CME to GPs. These were used to develop a pilot survey instrument to measure the factors affecting intention, resulting in a revised instrument for use in further research.

Results

There was a broad range of factors affecting providers' intention to deliver more effective forms of CME identified, and these were classifiable in a manner which was consistent with Triandis' model. Key factors affecting providers' intention were the attitude toward CME within organisations and the time and extra work involved.

Conclusions

We identified a range of potential factors influencing the intention of providers to provide more effective forms of CME, in all categories of Triandis model. Those interested in increasing the choice of more effective CME activities available to GPs may need to broaden the methods used in working with providers to influence them to use more effective CME techniques. The interview material and questionnaire analysis of the pilot survey support the use of Triandis model. Further research is needed to validate Triandis'model for the intention to deliver more effective forms of CME. Such research will inform future strategies aimed at increasing the amount and choice of effective CME activities available for GPs.

Background

In Australia, to maintain vocational registration as a general practitioner, GPs participate in ongoing continuing medical education (CME). The Royal Australian College of General Practitioners (RACGP) administers a Quality Assurance and Continuing Professional Development (QA&CPD) program. Those delivering CME activities to GPs (providers) apply to the RACGP to have their activities allocated points under the QA&CPD program. GPs need to gain 130 points over a three year period (triennium). Research into CME has identified factors in educational activities that are associated with better educational results [1–3]. The RACGP QA&CPD Program has incorporated this research into the point allocation process for CME activities. Differential point allocation means that activities allocated 3 points per hour (3 pph)ph and clinical audit (CA) points in this system have attributes consistent with more educationally effective CME activities. Despite point allocations favouring more effective categories of CME activity the percentage of educational activities developed which fall into these categories is low. In the 1996–98 triennium clinical audit activities made up 10% of approved applications and 3 point per hour activities made up about 2% of applications. Comparative figures for the 1999–2001 triennium are 5% clinical audits and 4% 3 pph. There is a significant gap between the amount of more effective activities being produced by providers and the amount that would be desirable. This in turns limits GPs choices of topics covered by more effective forms of CME.

The delivery of CME is an important issue. Considerable money is spent delivering CME to GPs. A total of 3.78 million dollars is spent by divisions of general practice on CME for GPs in Victoria alone [4]. As is reflected in the GP Strategy Review [5] government and consumers expect to see outcomes from the resources spent on CME.

One possible theoretical framework for investigating providers' decisions about the types of activities they choose to deliver is Triandis' theory [6] of social behaviour. This seeks to explain the reasons for behaviour using a comprehensive multifactorial model. The equation for this model is: Probabilty of Act = (Habit + Intention)(Motivation × Facilitating Conditions). The intention component of the model is defined as the self-instruction to perform an act and is represented by the equation:

Intention = Social Factors + Affect toward Behaviour + Value of Perceived Consequences of Behaviour.

In this equation, intention is defined as above. Social factors are subjective factors based on what the individual sees as the correct thing to do from the point of view of the individual's moral code and previous interaction with others. These include personal normative beliefs, normative beliefs, role beliefs, self-concept and interpersonal agreements to act. Affect describes the direct emotional response the person feels at the thought of doing the behaviour. Perceived consequences are the individual's perception of the likely outcome of a particular behaviour, and the value the individual puts on this outcome.

Triandis' model has been assessed in the context of predicting intention for health related behaviours [7–9] and it has been used as a framework for providing information to guide developing education in the context of health promotion programs [10–13]. Triandis model has also been used to examine the attendance of interns at resident rounds training sessions in a large Australian metropolitan teaching hospital [14].

A review of the literature shows that there has been relatively little research into providers of CME. Studies of the barriers facing providers in the provision of CME activity have explored such barriers in a limited way [4, 15–18], mainly by assessing the effect of infrastructure and concrete support such as training. In Triandis' terms, they have examined facilitating conditions, not determinants of intention. Provision of CME is a social behaviour. Hence, one might expect that the probability of a provider performing the act of offering a more effective CME activity would be influenced by factors that affect other forms of social behaviour. Triandis' model has been shown to be effective in other social behaviours and in the absence of other models, Triandis' theory was chosen as a framework to investigate the factors affecting provider's intention to develop more effective CME.

The aim of this study was to identify factors affecting the intention of providers to provide more effective types of CME; and to design a survey instrument which can be used to test the applicability of Triandis' model of social behaviour to the provision of CME to general practitioners.

Methods

Ethics approval for this study was obtained from the RACGP Ethics Committee. Structured open-ended interviews aimed at exploring factors influencing intention were performed with key informants. The number of interviews performed was guided by the number of new themes introduced at interview. By interview 11, there was a high level of redundancy of themes and interviewing ceased. There were 9 Tasmanian and 2 Victorian providers interviewed face to face by the investigator, recorded with the permission of interviewees and transcribed by the same investigator. Frequency tables were drawn up, detailing the number of times an item was mentioned, and in what interview it was mentioned.

Views were also sought from each educational and development officer in the QA&CPD program by an open e-mail query asking: "What do you feel the barriers to putting on 3 pph/CA are for providers and what might be the advantages and disadvantages for providers?" The interview results, input from other education and development officers and the investigator's own experience in dealing with providers were used to design the pilot survey instrument. The instrument included eight demographic items.

Themes from the interviews were not used if:

• The item came from a very specific scenario. For example, the RACGP training program is concerned almost exclusively with GP registrar training so one item dealing with this aspect from an interview with a training program staff member was not used.

• The item was too universal eg almost all interviewees suggested that more time would facilitate CME delivery.

• The item was suggested by < 3 interviewees, and its use was not supported by the feedback of other education and development officers or the investigator's experience with providers.

• The item was close in content to another that was used.

In stage two, the pilot instrument and a debriefing questionnaire (see Box 1 [Additional File 1]) were distributed to 25 providers selected randomly from the RACGP QA&CPD Program database. This database classifies providers according to the type of organisation they represent, into 13 categories (Diagnostic Services, Divisions of General Practice, Educational Institution, Government Agency, GP Association, GP Group, Health Organisation, Hospitals, Medical Colleges, Pharmaceutical Companies, RACGP, Universities, Other CME Providers). Diagnostic services refer mainly to radiology and pathology providers. Divisions of general practice are regional organisations for GPs, largely funded by the government, with the aim of supporting general practice and through this, improving health outcomes in their local area. GP groups refer to small groups of GPs who meet for educational purposes, commonly being educational meetings held within practices. The sample frame was of providers who had had an activity approved between 1/7/99 and 31/12/99.

Statistics

Analysis was performed in Stata version 7.0. Items that had consistent negative correlations with other items in the same subscale were re-coded, and the following analyses performed:

• Frequency distributions of responses – items were considered for discarding if they were highly skewed;

• Inter-item correlation – any item with a Pearson's product moment correlation of <0.2 was considered for discarding;

• Item-total correlation – items whose correlation with the total of their subscale was <0.2 were considered for discarding;

• Cronbach's Alpha – this was calculated for each scale of affect, perceived consequences, social factors and intention. Each item was eliminated from its scale in turn. If alpha increased significantly when the item was omitted, the item was considered for discarding.

The criteria by which items were eliminated were:

• poor performance on all 4 analyses;

• very highly skewed responses;.

• poor performance on 2 or more analyses, if the elimination improved Cronbach's alpha.

The results of this analysis and the questionnaire feedback were used to design the final survey instrument.

Results

Five male and six female key informants were interviewed. They were representative of the following provider types: RACGP (1), hospitals (2), health organisations (2), divisions of general practice (2), GP association (Rural Doctor's Association) (1), government department (1), pharmaceutical companies (2). Three were practicing GPs. Key findings from the interviews are given in Box Two [see Additional File 1]. The frequency tables in full are given in Additional File 2.

The response rate for the pilot was 13/25 (52%). All respondents provided written feedback. Responders and non-responders were similar in their gender breakdown (38% male responders, 33% male non-responders). Responders were representative of all provider types, except diagnostic services, educational institutions and the RACGP, and all states were represented except the ACT, NT and overseas providers.

The pilot instrument took between 5 and 25 min to complete (mean of 10.5 minutes, standard deviation 5.5 minutes). Three respondents said the questionnaire was right, 3 too long. All respondents accurately described what the questionnaire was about. Most respondents thought that instructions were clear, though two noted that the repetition of scales was too much. All respondents were satisfied with the information given with the cover letter to the questionnaire. No questions caused any irritation or uncomfortable feelings in respondents. As a result of item analysis, 8 items were eliminated from the pilot instrument (Box 3 [see Additional File 1]).

Of the respondents, 46% (n = 11) had been involved in providing one or more 3-pph activity (median number of activities = 3). 31% had been involved in designing or producing clinical audit activity (median = 2). A high proportion had a background in education (77%).

Twenty-three percent of respondents were GPs. Fifty-four percent of respondents were in the age range 31–45 years, 38% in the range 46–60, and 8% in the 18 to 30 range.

The final instrument [see Additional File 3] of 49 items consisted of:

• 8 items measuring background information (gender, age, whether they are a GP, whether they have a background in education, number of 3 pph and clinical audit activities in which they have been involved.)

• 6 items measuring intention

• 17 items measuring social factors

• 13 items measuring perceived consequences

• 5 items measuring affect

Examples of items from each scale are given in box four [see Additional File 1].

Discussion

The Qualitative Data

The qualitative data from the interviews demonstrate clear themes. Overall, the intention of the interviewed providers to produce 3 pph or CA activities in the future was low. This was despite the fact that provision of CME was a high priority for the organisations to which the interviewees belonged. Interestingly, there was much weaker agreement with considering CME as part of the occupational role of the individuals, despite the priority CME has within their organisations. Individuals with an educational background felt that this was an important influence on their decision-making in CME provision. Most providers had made no commitments to other people to put on 3 pph or CA activities.

Most providers (8/11) nominated staff or management in their organisation as people who would affect decisions about types of education activities to produce. This underlines the importance of the culture of CME within organisations as a factor. GPs themselves were also frequently listed, giving yet another potential way of influencing intention by influencing the demand for more effective activities from GPs. Divisions of general practice were by far the most influential outside organisations that would influence other providers. At a state level, Divisions are responsible for around 50% of CME for GPs in Australia, so this emphasizes that they are key organisations to influence when trying to improve CME quality. By comparison, the RACGP, despite its QA&CPD Program, was listed by only 3 providers as affecting their decisions. The RACGP may need to address either changing their profile with providers, or working with other organisations to improve CME effectiveness. Throughout interviews, a recurring theme was that of the time pressure individuals were working under and the effect this had on their ability to produce 3 pph and CA activities. The extra work involved was a major concern and concern and anxiety were common affects reported in relation to producing 3 pph and CA activities. Time and extra work were very clear disadvantages raised when exploring perceived consequences. This is also reflected in the fact that being organised was a key feature of the type of person individuals expected to be involved in 3 pph and CA provision.

Most providers felt that a big advantage of 3 pph and CA would be in getting GPs to attend CME. Commercial providers such as pharmaceutical companies then felt that this was worthwhile in terms of marketing advantage and accessing GPs: "we can sell more product". Only 4/11 interviewees raised improved educational outcomes as an advantage, which is of concern given that this perceived consequence has in the past been a key feature of how the QA&CPD Program promoted 3 pph and CA activities to GPs and providers.

The investigator performing the interviews worked as an educational and development officer for the RACGP QA&CPD Program in Tasmania. When selecting key informants, selections were made from outside Tasmania to ensure that the sample included informants who had not been previously exposed to the investigator's opinions. It also meant that potential differences in providers from another state could be identified. Victorian interviews were done last. The degree of redundancy and the lack of new items resulting from these last two interviews, indicates that there was actually little difference between providers in at least Victoria and Tasmania, and made further interviews in other states unlikely to yield significant new information.

Questionnaire Analysis

Respondents to the pilot sample a mix of provider types, state of origin and experience level in providing CA and 3 pph activities, which was helpful when using the pilot data to produce a final survey instrument.

Most items eliminated (19, 24, 34, 36, 40, 41, 42, 43) were measures of social factors. Item 19 was designed to measure the effect of competition from other organisations on normative belief, but it performed poorly. This may be because it deals with aspects of organisational rather than individual performance. The original item came from an interview with a person in a commercial organisation and so may lack meaning in the broader context of organisations providing CME which include high numbers of not-for-profit organisations. Thirty-eight percent (5/13) of respondents chose the neutral response to item 24 which dealt with colleagues perception of providers of 3 pph/CA as workaholics. The item has content validity and there is no obvious explanation for its poor performance. Item 34 dealing with importance of quality of education was highly skewed towards a positive result and had little discriminatory power. The most likely explanation for this is that most providers were unlikely to feel that the quality of their activities was unimportant (and be unlikely to admit this even if they felt so).

Items 36, 40, 41,42 and 43 were all from the subscale of self-concept. A negative response to items 41,42, 43 and to a lesser extent 36 would have required an admission of lack of ability (conscientiousness, competence, flexibility and organisational ability); this could explain why there were no negative responses for these items and the skewed response distribution reduced the usefulness these items. Item 40 was less skewed, but performed poorly in other analyses. This item, designed to measure tolerance to bureaucracy, may not have been sufficiently explicit, or such tolerance may not be a component of self-concept.

The concerns about questionnaire length and formatting were addressed by reducing the number of items, changing the format and reducing the redundancy in the instructions, to maintain clarity but at the same time improve readability.

Table 1 gives the Cronbach's alpha for each subscale of the pilot instrument after the elimination of poorly performing items, showing that the inter-item consistency of the survey instrument was high. Reliability was assessed by examining internal consistency i.e. by Cronbach's alpha, rather than by test-re-test reliability as the logisitics of getting a high enough response rate within a reasonable time with a lengthy questionnaire precluded the latter approach.

Table 1 Cronbach's Alpha for Each Scale After Elimination of Poorly Performing Items.

The changes discussed above were used to produce the final survey instrument for use in further research [see Additional File 3].

In the past the QA&CPD Program has mainly addressed the issue of increasing the availability of more effective forms of continuing education activity such as 3 pph and clinical audit by promoting their educational effectiveness and by training providers in educational principles and how to produce these types of activity. The material obtained from the interviews with providers, and further investigated with the pilot survey, indicate that there are potentially many factors involved in the decisions that providers make regarding the type of continuing medical educational activity they choose to produce. Box 4 [see Additional File 1] gives some examples of items exploring potential factors.

The intention to produce more effective forms of CME may be influenced by a diverse range of issues, in all categories of Triandis' model for intention. To enable or influence providers to provide these more effective forms of CME, organisations concerned with CME provision to GPs may have to widen the methods used in working with providers towards this end. While the qualitatitive data are supportive of Triandis model being a potential model to examine the intention of providers to provide more effective forms of CME, the numbers in the survey are low and further research is needed both to validate the final survey instrument in a wider and larger sample of providers; and to test the hypothesis that Triandis'model for intention applies to the intention to deliver more effective forms of CME. This study has focussed on intention alone. Further research is needed to identify and measure the other determinants of behaviour according to Triandis model, namely habit, motivation and facilitating conditions.

This study was performed on Australian providers working within the Australian system of CME provision. The behavioural intention in the Australian context is the intention to provide 3 pph and clinical audit activities. Nonetheless, it is likely that many of the issues raised regarding how providers choose methods of CME delivery will apply to providers working in other settings. An example of this is the concern with time availability, raised by Australian providers but also by providers in the UK and Canada [17, 18].

Conclusions

We identified a range of potential factors influencing the intention of providers to provide more effective forms of CME. Key issues were the attitude toward CME within organisations and the time and extra work involved producing more effective forms of CME. The qualitative data provide organisations with an interest in increasing the range of more effective CME activities available to GPs with a broad range of factors that they may consider addressing with providers to achieve this end. The types of factors identified were classifiable into groups consistent with Triandis' model, providing additional support for the use of this model in further research. A pilot survey instrument designed to determine how well these factors fit Triandis' model of social behaviour was developed and tested. Further research is needed both to validate the final survey instrument in a wider and larger sample of providers, and to test the hypothesis that Triandis'model for intention applies to the intention to deliver more effective forms of CME. This will provide more information to inform future strategies to increase the amount of and choice of effective CME activities available for GPs in Australia and internationally.