Table 3 Themes, sub-themes, and selected quotes from in-depth key informant interviews describing experiences of racism in medicine, anti-racism teaching in medicine, and strategies to improve and sustain anti-racism curricula

1) Experiences of racism in medicine are ubiquitous.

A) Everyone has either witnessed and/or experienced racism.

“I have so many examples, we would honestly need to interview for weeks or months on end.”

B) Outright racism was not uncommon.

“When I was called the 'N word' by a patient, they changed them to another provider, nobody followed up with me.”

C) Racism exists on a spectrum.

“I’ve seen [hospital] managers treat [staff] differently based on race, like having a different standard for certain … tasks.”

D) Racism exists in the medical system in multiple forms.

“The structural racism that is inherent in medicine I think is really important to acknowledge as well. Things like different values for GFR for patients with chronic kidney disease [based on race]. That’s an example of structural racism, where the system is … saying that a lower GFR is less serious in a [black] patient than in a white patient.”

E) Racist policies exist and contribute to the problem.

“We officially are not allowed to say the word ‘diversity’ anywhere in our documents.... Nothing [with] state funding can have the D word in it.”

F) Important to consider intersectionality.

“[A patient} asked me if I was Jewish and [said] they wouldn’t want a Jewish doctor because of whatever various anti-Semitic ideas they had, and I did have [other] encounters with patients who were either neo-Nazi or [part of] the KKK, but there was no specific support structure around that, and I didn’t report it to anybody, I just carried on with my duties as a resident.”

G) Tokenism contributes to the problem.

“...the oppressed are usually being put in the position and tokenized to fix the problem on behalf of the oppressing system.”

2) Experiences of teaching anti-racism in medicine are variable.

A) Lack of anti-racism teaching is common.

“There wasn’t any education provided or any training provided in our curriculum that pertain[ed] to racism.”

B) The tone for anti-racism teaching is set from the top.

One participant noted that when patients made racist comments during rounds, “[as a] more a junior person on the team, you always look to your attending or your senior resident ... to see what their reaction is and often [that] sets the tone for the rest of the encounter.”

C) Most respondents reported at least some engage in anti-racism efforts in their program.

“There’s been a huge effort both at the residency and faculty level in recruiting trainees and faculty who are historically underrepresented in medicine and supporting [them] once they’re there.”

D) Participants reported a range of activities across their programs & institutes.

“Our residency lives within a larger Medical Center that has a robust equity inclusion and diversity [program that] puts on events for the entire Medical Center quite regularly.”

E) Some emphasized community collaboration and representation.

“They oftentimes will have a community or patient representative so that they have a voice [in program planning and design].”

3) Sustainability considerations for anti-racism curricula should focus on recruitment & retention, resources, and systemic changes.

A) Integrate outcome measures into credentialing & accreditation

“There [are] some outcomes ... like testing scores or step scores [where anti-racism knowledge could be tested].”

B) Ensure all lecture topics covered include a slide on equity.

“...we provide a PowerPoint slide showing an example of a framework that they can use and that has been really nice and ... so even if they’re talking about OB... or GI, then [disparities or racism] components can be included.”

C) Ensure dedicated staff with protected and compensated time.

“...having a champion, someone in the program who will continue to help lead it and to make sure that it’s integrated year after year into the curriculum.”

D) Ensure permanent resources and support are present.

“[Hire] someone to specifically look at diversity and inclusion.”

E) Ensure URM are recruited, supported, and promoted.

“... this training pathway is very complicated and what you have to do, to get into medical school, to get into residency, to be successful, [is difficult]. The training is not easy, and having somebody who can coach you through [all of this] is essential... [providing this] kind of mentorship connections at much earlier stages [is necessary].”

F) Promote STEM in local communities.

“[Encourage staff to support elementary and high school] students from historically under-represented minorities in STEM [programs].”

G) Consider additional training tracks for residents and/or faculty.

“[An equity fellowship supporting an annual] cohort of 30 to 40 fellows and residents from all across different departments [who participate longitudinally to learn about equity].”

H) Change mindset from “We are doing this to help URM,” to “We are doing this to make us all better.”

“We [are] moving away from [saying] ‘let’s bring in trainees [from] under representative minorities for diversity’s sake,' to ‘it makes us a better program, a better healthcare system, because there is just so much added benefit to having our workforce represent the communities that we’re serving and to have diverse viewpoints...’”

I) Continuous quality improvement efforts are needed.

“[Another important] piece of this of course was the quality improvement piece where we tried to actualize an intervention to try and reduce disparities.”

J) Create equitable ‘scoring rubrics’ for resident selection.

“When we were looking at our process, we realized that there were a lot of issues … [for example] when we asked our current residents [what sorts of questions they were asked in interviews, we found] a lot of applicants and residents were picked or ranked highly [based on] having more similar hobbies as opposed to clinical skills or life experiences that were important … Now we [use elements of] our program mission and values [in] deciding what to score ... for example [skills related to] clinical medicine, social medicine, and community medicine.”

4) Tools for curriculum building are abundant if you know where to find them.

A) Have residents present on books and podcasts.

“[Have residents review] podcasts, print media, videos, [etc., and then assign them to give] their own presentation [to faculty and co-residents on a topic they choose in anti-racism or disparities].”

B) Use problem and case-based learning.

“[Have residents watch and evaluate [case scenarios].”

C) Use online simulations and other tools.

“A poverty simulation ... on[line] that you can walk through, and you you have X amount dollars, and you have to make decisions [about feeding your kids vs taking medications etc].”

D) Invite and compensate speakers or consultants who are URM.

“There’s a bunch of [URM] people that they own their own [consultant] companies [that can support anti-racism work].”

E) Invite speakers from the community.

“We .. had.. patients talk to us about their experience with racism in the hospital setting... they [shared] their perspective, what they go through when they walk in the door of the hospital [and it was eye opening].”

F) Shadow other staff in clinic/hospital.

“Integrate residents into working with our chaplaincy service [and have them work] with our transitional care team to try to understand the barriers that our patients face.”

G) Have colleagues share their own experiences.

“I really believe in the power of testimony. So listening to either minority patients or minority physicians talking about what their experience has been ... that perspective is really useful.”

5) Outcome measures should be multilayered and include resident/faculty, patient, community, and system metrics.

A) Resident pre-post surveys or tests

“Test or quiz … to reflect whatever knowledge was gained during that portion of the curriculum.”

B) Demographics tracking (%URM considered for interviews, matched, faculty, fellowship)

“ . . . whether or not the number of URM residents and faculty had increased over the years to have ... racial concordance to match our patient population.”

C) Patient surveys

“Ask the patients to... to fill out a survey that says, hey, how did you feel about this?... Did you feel like your needs were met? [Then track responses over time]”

D) Track patient access & health outcomes.

“Are minority patients having shorter visits than white patients?... Getting more diagnosis codes that are associated with things like malingering or drug seeking behavior?... [Look at] data points like the way care is delivered … whether it’s effective care.. [do] they [get] preventive medications? How many calls it takes to reach a physician? … All these sorts of access to care, metrics... You could see how those metrics change over time, particularly with the URMs.”

E) Track system changes such as resources allocation and fundings.

“You’re not serious if it’s not embedded into the priorities of your strategic plan, if it’s not embedded with a person who’s actually hired, actually empowered by protected time and money, and if there aren’t evaluation systems to [ensure] this is happening... [You’re not serious if you’re] not giving them the same things that we give traditional researchers and predominantly white people in academic spaces.”

F) Hire equity/disparity outcome metrics consultants.

“There are black ... or people of color who have spent years training and building entire business[es] around antiracism work… budgets must be made to pay those people.. the same way as hospitals ... pay millions of dollars for other things, they need to invest in this.”

6) Curricular strategies should be multipronged, integrated, and longitudinal.

A) Integration into all parts of the program

“Something that starts in orientation and continues longitudinally throughout the program is really helpful.”

B) Provide evidence and data to support strategies.

“Physicians respond to data and numbers... [so you need to include this in] presentations.”

C) Incorporate personal stories.

“It’s not real until they actually see a person and they’re like, hey, you know, this is what happened to me.... Someone who already has some social capital so that they felt like they could speak in this space... another physician [to] talk about their experience as a patient or their experience … [of] racism so that they humanize it.”

D) Create an equity journal club.

“A good ... anti-racist library and journal club, [can be] really effective.”

E) Include a broad range of topics.

“...other components that factor into healthcare would include housing discrimination, the criminal justice system, and even components of reproductive justice, which has been a very important talk about.... social political climate, and also... what’s happening in their own specific [city/town/state].”

F) Open safe space and allow for anonymous feedback.

“Develop that kind of space where people could discuss things as they come up is pretty important, and … [utilizing] a trauma informed [approach] where people can contribute from all levels.”

G) A broad range of “what not to do” recommendations were made.

“...these one-time events [like posting a] mission statement... on a website, and that’s it.”

H) Come up with unbiased ways to deal with “concerns” raised about residents of color and recognize that URM residents might not always feel “safe” speaking out when they have questions.

“[as a white male] I always feel confident and comfortable enough to say I don’t understand that or I can’t figure it out [but URM trainees might not feel they have the privilege or they will be penalized more harshly or perceived as incompetent].”

I) Consider creating safe “meeting” spaces where trainees of color can discuss their experiences with one another.

“... a group that was formed for minority students and they got a separate room, separate tutors, a space where they

could talk about their experiences”

7) Self-reflection, discomfort, and engagement are all necessary.

A) Advocate for change on a societal level.

“There’s so much work to do and we’re barely scratching the surface with what we are doing..."

B) Address uncomfortable issues such as white fragility.

“[as a white person sometimes] you have to be uncomfortable, have to get to a place where you feel uncomfortable to grow.”

C) Each person should critically examine their own biases.

“... you know, I don’t believe I’m a racist and probably most people in medicine don’t, and yet we have these differing outcomes [about] which we have to be honest with ourselves … what can we do to bridge the gap?”

D) Strategies that work for one type of program might not work others.

“... academic centers are often urban and almost always large well run systems ... when you try to take that same idea and move it to a [smaller more rural system] the concepts are very different, and the people are very different. If you don’t consider [this, then your efforts may] fall short.”