Table 1 Linkage between CHIPPS training evaluation data and Kaufman’s Model of Learning Evaluation

Level 1a Input

Did we deliver topics which were useful?

Pre-training preparation

‘Preparing a Case Study that I brought to that day was good just to sort of reframe how you think about things and I suppose feeling confident to make the actions really’ Interview PIP17

‘I felt the input from geriatricians of significant benefit. In particular discussing case studies and down titrations of various medications in real life practice.’ PIP15

‘I found the sessions considering treatment of Dementia in Parkinson's and also the psychiatrist’s session on reviewing and reducing antipsychotics very useful. I didn't have this knowledge’ PIP 22

Areas to improve

‘In Scotland some of the Mental Health legislation differs from how you do things in England so I recall there was a lot of talking about incapacity and certifications and documentation was slightly different …just maybe more inclusive in terms of the documentation and things’ PIP15

‘Maybe a bit of negotiation training’ PIP22

Level 1 b Process

Did we deliver training in effective ways?

Usefulness of mentor

‘It was actually the mentor chatting around situations she’d encountered, and what we could do, and how we should approach it, that was beneficial, because I wouldn’t have considered how I would go about going in and introducing myself, the mentor giving us these tips on how to build these relationships, has definitely helped’ PIP14

Time to prepare for assessment of competency

‘There was a lot of self-directed work to do and I think this was too much- some more of this could have been covered in the training’ PIP13

Areas to improve

‘Information about the study procedures was helpful but there was an element of duplication’ PIP8

Level 2 Acquisition

Did the PIPs acquire new knowledge?

Refreshing knowledge

‘It certainly refreshed my knowledge, and I’m probably a bit more geared up to do the nursing home reviews’ PIP21

‘It was good for me to do it because it has made me think about things, but I knew I needed to refresh some things’ PIP2

Increasing competency

‘There were some clinical things that covered areas I wasn’t very competent for example dementia, and the only time I called my mentor was when I had a question about ‘is it okay for me to wean a patient off this particular drug?’ so I obviously learnt from that and got more confident’ PIP8

Areas to improve

‘I wouldn’t say it was a futile exercise [completing professional competency portfolio] it was good to do it, so I won’t really get rid of that but I would definitely make it a lot more concise, a bit more smarter’ PIP16

Level 3 Application

Did the PIPs apply new knowledge?

Making medication changes

‘Quite a lot of medicines were PRN anti-psychotics which is one of the things we done on the training and once the Carers were aware of what they should be looking out for and me making sure that the MAR chart said ‘please follow the PRN protocol more closely’ quite a lot of patients managed to come off them and their sleeping tablets’ PIP19

‘Covert meds I had been doing already, but it made me reflect, we shouldn’t be leaving them for a year, they need reviewed sooner and do they need the medicine at all, so I think it made me be more thorough as a prescriber and a pharmacist’ PIP22

Increasing confidence

‘There were a few people we’d got off a medication, antipsychotics particularly, because that’s something I probably wouldn’t have touched, but after having the training session, and the group discussions, and more of an awareness, I felt more comfortable ‘ PIP14

‘I’d certainly challenge the consultant more, I was a bit more confident, having had the training I’m thinking, hang on a minute, maybe we don’t want to do this, we don’t want to do that’ PIP21

Level 4 Organisational results

Were there impacts on the intervention and stakeholder?

GP reactions

‘PIP knew about nursing home prescribing very well. Offered different options for covert medication which was very helpful. Wrote in the notes clear plans from discussion with staff, patients and family’ GP16

‘PIP stopped medication that didn’t need to be given; a couple of patients with low blood pressure, picked up that they didn’t need to be on so much medication, that theoretically reduces the risk of falling’. GP8

Improved review

‘I try and look in advance of 6 monthly review, when did resident last have a blood test, or is there a psychiatric letter, if I see some drugs that I’m thinking, I’m not sure about that one, I’m probably faster at it, I’m thinking right, I’ve got that knowledge in my head, it’s a bit fresher’ PIP21

Benefit to resident

‘There were people that we took off [antipsychotics] and they were actually brighter and happier, and more smiley and that is definitely good’ PIP14

Level 5 Societal/customer consequences

Were there impacts on wider organisations?

Knowledge carried forward

‘I am in a new job now and based on the experience [in CHIPPS] I’ve delivered my own model of working in my new job as a Care Home Pharmacist’ PIP16

PIP sharing expert advice with GPs changed practice

‘I did start to change my practice when I prescribed a cream for a course of treatment I actually put on the directions ‘discard after a week’ or ‘discard after a fortnight’. GP19

‘There was a lot of links that they gave us at training days which were very useful, I shared a lot of this at our clinical meeting, areas that are very good to have a reflect on, we’ve had a few different meetings on how we go forward with all the frailty’ PIP22