Table 2 Example quotes for Context, Mechanisms, and Outcomes

Context

“I think a lot of the problems were that nobody knew what software platform we were going to use. And then a month beforehand it switched. So, I think there was a lot of problems with the roll-out” (PD4_2019)

“The whole ePortfolio thing has not helped. We're getting ready for one system, now you've got another system, and at the same time we're introducing other clinical systems.” (F1_2019)

“I do think that it should have probably been planned a little bit more carefully, because we're rolling out a system that doesn't have all the components in place. There's [sic] new versions coming out all the time” (PA6_2020)

Mechanisms

Participation

“The fact that the university is using just the University of Manitoba email addresses. Well, I'd say 95% of our attendings don't routinely use their University of Manitoba email addresses and I'd say 50% or 60% don't even know how to access them and that is with several dedicated sessions on how to access your University of Manitoba email” (PD6_2020)

“No one is logging into this thing. The app doesn't work, and the website is not conducive to data collection, so you're not going to get a bunch of 60-year-old surgeons pulling out their phone.” (PD11_2021)

Communication

“I need to know how Entrada works because that's just the main communication tool of getting the EPAs completed, and that's essentially where I've been having my most difficulty is understanding how Entrada works because it's not the most straightforward program I'm learning” (PA1_2020)

“[Faculty development] was mainly about making sure that they could actually get in Entrada, and have their U of M account working. The hardest part was going to them and saying, “This is how you claim your ID. This is how you do this.” (PA2_2020)

Adaptation

“Most of us have actually created a separate spreadsheet or an Excel document of some sort where you have to say, okay I need this, and then you need to get it from, oh, I need this one from a nurse and an anesthetist and then someone else.” (R1_2020)

“I would say the most difficult part or one challenge I guess is just making sure the [faculty] appointments are all set up on [Entrada] because if they're not then […] the resident has to print it, the EPA, and then they have to get it filled out and then I have to input it.” (PA3_2020)

Outcomes

Change Fatigue

“Entrada is probably the biggest difficulty. It just was a lot of change at once. And I'm fine with change, but not everyone loves change. So, when we are doing different forms, different ways to teach, now we're using a different system [...] There's a lot of changes at once, because everything was July 1st for us. Like with Entrada, competency, all of it, was July 1st for us.” (PA6_2020)

The Family Medicine curriculum “sounds super similar and it's on Entrada. Now [faculty] think everything on Entrada has got to be CBD. It's not.” (PA3_2020)

They used the UM email, which most staff frankly don't use,...is another barrier. It just adds another layer and more workarounds and more education... but if nobody's using it, it's just been more work. (PD3_2021)

Unmet Expectations

“all of the effort that went into showing [faculty members] what it was, then three months go by and they haven't done a single evaluation, then they probably forgot what they were supposed to do as well. So then when it did start happening, there was a lot of, “Well, I don't know how to do this. I don't have an account.” My residents are scrambling trying to get it done before their deadline, but faculty didn't know how to do it.” (PD4_2019)

“I think we've had a lot of issues with staff not having accounts set up or […] different emails or these little things that really aren't a big deal, but have been a big barrier to getting evaluations completed.” (PD7_2020)

“Some don't know their log-in info, for instance, and can't do [the assessment] on the spot and then it doesn't get done or doesn't get done until weeks later, which then isn't the point of all of this in the first place, in which case it's really not adding anything to anyone's learning. So that's the issue with Entrada.” (R1_2020)

Earlier Problem Identification

“I would say that we have a better sense of where our residents are at any given time and can quickly move towards meeting with our residents if there's some difficulty, identify and address it quickly.” (PD2_2020)

“It's easier as a program director to see where a resident stands amongst their peers, and if they are struggling or not meeting certain components, it's easier to identify those and to try to act on them early on.” (PD7_2020)