Table 1 Key themes identified (first phrase in bracket refers to the group the participant belonged, second phrase refers to the anonymized identity of the participant)

Learnability

Learnability is believed to be facilitated should the checklist is introduced early.

Illustrative remarks:

“Yeah, this tool should be taught earlier... as early as possible, maybe while in medical school because once they already out working, the junior doctors would have already developed their own ways of approaching patients, and it might be difficult to introduce new tools for them” (Doctor 10)

“…this checklist would be useful but it should be introduced much earlier at the beginning of our clinical rotations in Year 3; then we would be more familiar with it.” (Student 4, Group 1)

Learnability can be hampered by lack of emphasis on critical thinking in medical school

Illustrative remarks:

“Yes, it takes some time for us to learn this checklist, I mean, this is something new to us, and it is because I think many of our lecturers have seldom been emphasizing on questioning the rationale behind why we choose this working diagnosis.” (Student 1, Group 1)

Its simplicity makes the checklist easy to learn

Illustrative remarks:s

“The checklist is not that complicated to learn with only four items to it” (Student 1, Group 2)

Effectiveness of the checklist

Effectiveness in resolving diagnostic dilemma

It is an effective tool in helping to resolve decision dilemmas, particularly, whether to discharge apparently stable patients with non-specific complaints.

Illustrative remarks:

“…this tool is very helpful in evaluating patients in the Green zone. This is because of the volume of patients that come into the Green zone, especially for patients that come in with very nonspecific complaints. So, by just applying this checklist, it can help us to rule out the life threatening causes for these nonspecific complaints…” (Doctor 9)

Item W = “What else?” should especially be activated if there are data that does not fit into the overall clinical picture of the patient.

Illustrative anecdote:

“I asked myself, why was the patient on wheelchair? He does not need to be on wheelchair if he just had URTI. How long has he been on wheelchair? Is it because he’s not able to walk by himself? And if he’s not able to walk, how long has he been in this state? Then I asked myself, “What else could this be then?” And then I asked the family members, “Why is he on a wheelchair? Has he always been on wheelchair, not able to walk on his own?” It was only then that the family member said, “Oh, I am sorry. I forgot to inform you just now. He suddenly could not walk anymore. Just about 2 h ago.” (Doctor 2)

Effectiveness in promoting metacognition

It is also an efficient tool in promoting metacognition.

Illustrative remarks:

“This tool will at least help me not to simply discharge the patient, reducing my risk of misdiagnosing and mismanaging the patient.” (Doctor 10)

Parental anxiety should prompt the doctor to be extra careful in considering the question of “What else?”

Illustrative anecdote:

“So, initially I thought of discharging this patient who looked so well but ehh… at that time, I applied this tool and asked myself what else I could have missed given the unusual presentation of left knee pain. Furthermore, the parents were quite worried. So, I decided to do an x-ray of for him and lo and behold, I’ve found out that this pain is due osteochondroma.” (Doctor 10)

Item E is especially important in bringing more objectivity to the diagnostic process and to counter authority gradient in diagnoses consideration.

Illustrative anecdote:

“I remember a patient I encountered. Every clinician said that was the case of Guillain-Barre syndrome but it turns out not to be so. Err. it was likely because initially the visiting neurologist told every clinician at that time that the case was Guillain-Barre syndrome. But actually when I elicited the reflexes, I found it to be rather brisk. So I went back to the checklist and ask myself “what else could it be?” And I began to look hard for other evidence. Eventually after some other investigations done by the doctors in charge, it turned out that the patient actually had mononeuritis multiplex.” (Student 1, Group 1)

Memorability

Relevance of checklist determines its memorability

Illustrative remarks:

“…generally the contents of this tool are exactly what we do in our day-to-day clinical work. Basically when we see patients, no matter which zones we are working in, the first thing is to rule out life threatening causes; only then we start ruling out all other possible diagnoses, and only then we make decisions on whether to admit or to discharge the patients. So, basically, thought processes embedded in the tool are very simple and generic…” (Doctor 5)

Items “T” (Ruling out life or limb threatening conditions) and “W” (What if I am wrong? what else?”) are the most important/useful components of the checklist to both medical students and the doctors and is the only item that some medical students can remember.

Illustrative remarks:

“..first two items (i.e., the items “T” and “W”), they are easier for us to remember because these two items are relevant to us in our clinical encounters as medical students, because we apply them everyday. The other components or items are more difficult to remember.” (Student 2, Group 1)

“Parts of this checklist are user friendly; but other parts are not. Like for example, item no. 1, the “T”. and item no. 2, “W”, “Is there any life threat?” “What if I am wrong”, the words themselves are self-explanatory. I know what it is about. But item no. 4 “dispositional factors” is complicated. I wouldn’t understand what it means and I would have to read further on the fine prints to understand it. And in an emergency situation, I wouldn’t be able to do that.” (Student 5, Group 1)

items “W” and “E” are inter-related as the consideration of one of these 2 items may trigger the consideration of the other item.

Illustrative remarks:

“In my opinion, I think after we consider ‘E = evidence’, we should go back to ‘W’, i.e., whether we are wrong or not? Or whether the evidence support my diagnoses or not? And what else it could be? (Student 5, Group 1)”

“…I think that there are some overlaps between item no. 2 (“W”) with item no. 3 (“E”). Because, by the fact that I can say I might be wrong means that I have evidence to show it to be so…” (Student 1, Group 1)

Familiarity of the checklist determines its memorability

The items in the checklist where the participants can remember are the items that they are familiar with. Ironically however, familiar items are precisely the items that they have been practicing in their daily clinical work, hence, many believe they do not need a checklist for these processes.

Illustrative remarks:

For me, I think the components “T”, “W”, “E” are something which we are already practicing on a daily basis even without referring to the checklist but the “D = Disposition” component of the tool is something we need to pay particular attention too. (Doctor 9)

Most of the medical students perceive “item D” (the dispositional influence of emotional and environmental factors on the clinical decisions) as not applicable, not important and not relevant.

Illustrative remarks:

“I don’t know… I just remember the item “T”, to rule out the emergency conditions. (Student 2 nodded in agreement). The other items “E = Evidence” and “D = the Dispositional factors” do not occur to me as relevant most of the time.” (Student 1, Group 1)

On the contrary as illustrated in the remark in the column on the left, most of the doctors can relate on the importance of “item D” as an essential but often neglected group of factors influencing the quality of their clinical decisions.

Illustrative remarks:

“…I think the components “T”, “W”, “E” are something which we are already practicing on the daily basis but the “D = Disposition” component of the tool is something we need to pay particular attention too. The “environment” we are working in can influence our judgment.” (Doctor 9)

Errors/Pitfalls/limitations

The checklist may slow down the entire working process

Another limitation of the checklist is that it can be time-consuming and slow down the entire work process, especially in a busy clinical setting.

Illustrative remarks:

“To me the limitation of this tool is that when we keep thinking too much on the patient, we may then be worrying too much about the case, spending too much time thinking of what could the errors be, etc. I mean, being a bit skeptical, applying critical thinking is good, but sometimes, this can become too time-consuming especially when we are too skeptical, which in turn, delays our management, leading to stress and frustration and prolonging the waiting time for the patients. This, I think is particularly true for cases that are stable or relatively stable; in Green zone, for example, although, I mean, this tool would be helpful for patients with unstable vital signs, but for patients who are stable in Green zone, I think, applying this tool is too time consuming. In other words, where the diagnosis is clear cut, I would probably not likely to apply the tool, but where the diagnosis is not clear cut, but I know something is not right the patient, I would probably apply it. The challenge for us then is to know when and for which case do we need to apply the checklist, and which ones we do not.” (Doctor 8)

The checklist requires adequate prior medical knowledge. Hence, the effectiveness of the checklist is hampered by the lack of prior medical knowledge. In fact, as one student put it, this checklist is not pleasant to use because it reminds him of his own inadequacies:

“I believe the challenge in using this checklist is contributed by our own lack of knowledge. It is mostly because of our own inadequacies; for example, we can usually only think of 3–4 differential diagnoses.” (Student 1, Group 2)

Only one student agreed that item “D” is relevant, but as he has rightly said (see remarks below), even if students are aware of their own fallacy, they may feel helpless as their knowledge base is inadequate for them to generate another differential diagnosis:

“I think the item “D” is still relevant to us as medical students. Since we are humans, our judgment can also be influenced by the emotional state that we are in. But the real problem is, I think, even if we know the emotional and environment dispositions that may influence our judgment, often we are still not able to generate alternative diagnoses due to the lack of knowledge.” (Student 3, Group 2)

On the other hand, some perceived the checklist as an unpleasant tool to use as it mirrors their own inadequacies:

“To me, this tool is not pleasant to use. It is supposed to be a checklist, but to me this is too complicated. Maybe because I am not familiar with it. I think there are just many items in it. I cannot remember all. I think it is supposed to be simpler than this.” (Student 2, Group 2)

Satisfaction or pleasantness in using the checklist

Its mnemonic and simplicity makes it pleasant to use

Illustrative remarks:

it is pleasant to use this checklist with its mnemonic structure and also because, since it only has four items, it is simple enough (Student 2, Group 2)

The fact that it reminds the participant of his or her own inadequacies and shortcomings (of not able to generate more differential diagnoses) makes it unpleasant to use

Illustrative remarks:

“Actually I think this checklist is not very pleasant to use in the sense that it reminds me of my own inadequacies and shortcomings but it is still a good checklist to use to help us remember to check for things we might have missed.” (Student 3, Group 2)