This article has Open Peer Review reports available.
Educational potential of a virtual patient system for caring for traumatized patients in primary care
© Ekblad et al.; licensee BioMed Central Ltd. 2013
Received: 3 February 2013
Accepted: 13 August 2013
Published: 19 August 2013
Virtual Patients (VPs) have been used in undergraduate healthcare education for many years. This project is focused on using VPs for training professionals to care for highly vulnerable patient populations. The aim of the study was to evaluate if Refugee Trauma VPs was perceived as an effective and engaging learning tool by primary care professionals (PCPs) in a Primary Health Care Centre (PHC).
A VP system was designed to create realistic and engaging VP cases for Refugee Trauma for training refugee patient interview, use of established trauma and mental health instruments as well as to give feedback to the learners. The patient interview section was based on video clips with a Bosnian actor with a trauma story and mental health problems. The video clips were recorded in Bosnian language to further increase the realism, but also subtitled in English. The system was evaluated by 11 volunteering primary health clinicians at the Lynn Community Health Centre, Lynn, Massachusetts, USA. The participants were invited to provide insights/feedback about the system’s usefulness and educational value. A mixed methodological approach was used, generating both quantitative and qualitative data.
Self-reported dimensions of clinical care, pre and post questionnaire questions on the PCPs clinical worldview, motivation to use the VP, and IT Proficiency. Construct items used in these questionnaires had previously demonstrated high face and construct validity. The participants ranked the mental status examination more positively after the simulation exercise compared to before the simulation. Follow up interviews supported the results.
Even though virtual clinical encounters are quite a new paradigm in PHC, the participants in the present study considered our VP case to be a relevant and promising educational tool. Next phase of our project will be a RCT study including comparison with specially prepared paper-cases and determinative input on improving clinical diagnosis and treatment of the traumatized refugee patient.
At the end of 2011, forcibly displaced people worldwide exceeded 42 million. This figure has been about the same during the last fifth consecutive years (2007–2011). This figure included 26.4 million internally displaced persons (IDPs), 15.2 million refugees, 10.4 million under UNHCR’s mandate, 4.8 million Palestinian refugees and 895,000 asylum seekers . In the refugee group, about half are women, nearly half are children under the age of 18 and they live mainly in low-income countries, close to wars and mass violence, and they are not yet resettled. In 2010, it was estimated that there were 40 million foreign-born people or 12.9 percent of the total population living in the United States . Since 1975, the U.S. has resettled over 3 million refugees from all over the world . Most immigrants and all refugees come to the US to escape poverty, mass violence and/or political oppression. Immigrants and refugees in the US have been demonstrated to have major issues in seeking primary health care leading to significant problems in health disparities [4–17].
However, primary care practitioners (PCPs) have limited training in identification and treatment of mental health problems such as depression and Posttraumatic Stress Disorders (PTSD) , which calls for improved training on the management of traumatized patients in both undergraduate and postgraduate education. The Harvard Program in Refugee Trauma (HPRT) at Massachusetts General Hospital and Harvard Medical School has been a pioneer over the past 30 years in the identification and treatment of highly traumatized refugee patients from culturally diverse backgrounds .
Traumatic life events and health
Evidence shows that trauma is a risk factor for both physical and mental health. Exposure to traumatic life events has been demonstrated to be highly correlated with smoking mortality, an increase in alcohol abuse, drug use and direct physical health problems as well as other long term physical illnesses . It has been well established that cumulative trauma is associated with the psychiatric diagnosis of PTSD and depression in a dose-effect relationship, i.e. increasing levels of trauma lead to higher rates and severity of PTSD and depression [21, 22]. Over the past 25 years major community studies  have demonstrated the high rates of PTSD, depression and physical disability in highly traumatized refugee populations [23–26]. Mental health and physical illness are directly related to major lifestyle factors such as diet, smoking, obesity, lack of exercise and alcohol/substance abuse that can be directly managed and improved in the primary health care (PHC) setting and through community-based interventions. Therefore, PCPs need to receive training how to accurately identify trauma as a major medical and mental health risk factor.
Virtual patients as a training model
Virtual Patients (VPs) are interactive computer simulations of patient encounters used in health care education. The VP is a virtual representation of a patient encounter for learning and assessment, typically including interactive features for illness history taking, physical examination, laboratory tests as well as features for suggesting diagnosis and treatment plan [27, 28]. According to situated learning theory, students can acquire knowledge by being engaged in tasks that in an authentic way parallel real world activities . Communication research has shown that highly engaging and challenging interactive media interfaces can promote deeper motivation and concentration . Those factors have a major significant influence on the way we interact with others, eg. authentic patient encounters and how we interpret verbal and non-verbal communication cues. Research in education has reported that emotional mediated experiences have a positive impact on cognitive learning outcomes [31, 32]. Thus, they lead to heightened involvement and decreased cognitive overload. Authenticity is critical to whether a virtual patient can be considered to be part of a situated learning endeavor, indicating that VPs may provide reliable, valid, and applicable representations of live patients . VPs can emulate a problem-based learning environment to assist medical practitioners in active, independent (and group) learning and problem-solving and are also giving the learner automatic feedback on the patient management process.
VPs have been shown to have a great educational value especially for training clinical reasoning . Virtual patients are also shown to have a potential to emphasize socio-cultural aspects and cultural differences as they pertain to healthcare education [28, 35]. The use of VPs has been proven to be effective in the training of surgeons , medical students [27, 37, 38] as well as psychology students [39, 40]. Research into the use of VPs in psychotherapy training is quite new . Little is known about the efficacy of the VP in training PCPs for traumatized refugee populations and for patients with psychiatric symptoms, although an evaluation by medical students of a VP with PTSD in an adolescent female has been published .
VPs may offer a holistic approach to medical education using a model that emphasizes both the patient and the doctor’s perspective. Therefore VPs might have a potential to dramatically increase skills and knowledge as well as foster trust, respect and empathy in the doctor-patient relationship.
Our team has performed a pilot study in Sweden with VPs for training the management of treating culturally diverse and traumatized refugees . That study indicated that the Refugee trauma VP was well received by the participants and that they appraised it as depicting in a realistic way a real life doctor–patient encounter and having a good potential for training patient management of mentally traumatized refugees.
In this study, we have set up an international American-Swedish collaboration, aiming to further develop the initial Refugee trauma VP case system and investigate the potential of that system to train PCPs in the management of culturally diverse, highly traumatized refugee patients with co-morbid health and mental health problems.
Refugee trauma system
The initial prototype is designed in such a way that it makes it possible for user interaction in the following areas of medical care: (1) medical interview, including a comprehensive illness history dialogue with the virtual patient for investigating the chief complaint, history of the present illness, and social history; (2) physical examination (including mental status examination); (3) screening instruments, including the Harvard Trauma Questionnaire (HTQ) and the Hopkins Symptom Checklist (HSCL-25); (4) laboratory tests and imaging studies; (5) additional data (i.e. information about the country of origin, laws about migration in the host country and links to relevant sources of information in the world wide web); and (6) preliminary assessment (i.e., treatment plan). The preliminary system included open-ended questions which make it possible for the participant to present a structured summary of the patient’s history, a probable diagnosis and a summary of a plan for treatment. An automated and individualized feedback upon actions taken, their relevance and the quality of case management appears after this module. The content of this feedback include a list of activities performed during the medical examination (e.g. questions asked, ordered laboratory tests and examination of the virtual patient) followed by a comment on its relevance. Further, a summary of the case management as suggested by a domain expert is presented to the user, followed by feedback from a “virtual advisor” (VA).
The Refugee trauma pilot study  was redesigned to increase the realism and engagement in the patient interview, incorporate established trauma and mental health instruments as well as to facilitate and improve the feedback to the learners. Proposals by the participants of the Swedish pilot study were taken thoroughly in consideration in order to achieve this.
11 PCPs at the Lynn Community Health Centre (LCHC), Lynn, Massachusetts, were invited to participate in the study and all of them volunteered to be included in the study. The study was performed in three steps. The first step started with giving informed consent to the participants. After the invited participants had returned the signed consent form, they got access to an on-line version of the actual VP-system. In session 1, the PCPs received an introduction to the VP system and they were administered a pre-test questionnaire regarding their IT Proficiency, overview of clinical worldview and current motivation to use VPs for training.
In Session 2, all PCPs run the system on their own and were asked to encounter and manage the virtual refugee as if it had been a real patient. Upon completion of this session all PCPs received a post-test questionnaire.
One month later, after the PCPs had some time to reflect on their experience, they were invited for Session 3, a telephone interview. During the interview they were able to reflect on the quality and usefulness of the Refugee trauma VP and made recommendations for further improvements of the system.
Written informed consent for participation in the study was obtained from the participants. This study was approved by the IRB at Massachusetts General Hospital (IRB protocol number 2011P001736). Attention was paid to all ethical considerations during the recruitment procedure of the participants and at each step during preparation, performance and data analysis, including consent form and information sheet.
The Institute of Medicine defines primary health care as “integrated and accessible care by clinicians who are responsible for addressing a majority of personal health needs through a sustained partnership with patients and practicing in a family and community context” . PHC is therefore considered an ideal health care environment of addressing the health and mental health needs of traumatized persons from culturally diverse communities. Primary health care, for example, serves as the initial point of contact for patients with health related trauma problems, depression and PTSD . Yet, the usual care by PCPs may be less than optimal with studies indicating the recognition of trauma-related distress as less than 40% , diagnosis of PTSD as low as 2%  and depression less than 50% . In primary health care veteran clinics where PTSD and depression should be routinely diagnosed, less than 50% of diagnosable patients were identified . In a Swedish study at a psychiatric outpatient clinic in a suburb south of Stockholm in a multicultural immigrant/refugee population only 40% of the diagnosable patients were recognized with diagnosis of PTSD . Under-diagnosis and under-treatment for historically disadvantaged ethnic groups may be especially high [50–52]. Despite the inherent and current limitations of primary care, such as its fast-pace and time constraints, it remains the ideal place for diagnosis and treatment of health and mental health problems and primarily health care can be less stigmatizing than special mental health clinics and despite the many barriers, can meet the immigrants’ entire spectrum of mental health and medical needs.
The PCPs were randomly selected from the entire list of 55 PCPs at LCHCs, however professionals that reported that they were not motivated in learning to use the VPs or stated that they had “poor” internet skills were excluded from being invited. That sample size allowed for intensive discussion and feedback with the PCP study group. All PCPs had previously been encountering traumatized refugees, but none was considered as a domain expert.
The 11 selected PCPs participated in the three 60 to 75 minute sessions led by the HPRT team leaders and received a US$ 225.00 dollar honorarium for each session. The participants were middle career practitioners (5 men and 6 women).
All out of the 11 initial PCPs completed the study and returned completed questionnaires. One of the initial PCPs was unable to take part to the follow up interview due to illness.
The pre and post questionnaires included survey questions on the PCP’s IT proficiency, clinical worldview, and current motivation to use VPs in the learning exercise. A similar questionnaire set was also used in the Swedish pilot study  with good results, making us believe that the questions had acceptable validity. The pre-test version included two parts. The first (“Overview of Clinical Worldview”) with the aim to examine the participants’ self-reported important issues during a “real life” medical examination, on a scale from 1 (no emphasis) to 5 (full emphasis). It consists of 10 items, divided into two parts; regarding the level of emphasis the clinician usually places on (a) data (chief complaint, history of present illness, physical exam, mental status exam, laboratory tests, and traditional healing exam) collected during the medical examination and (b) root causes of the disease (biological, psychological, social/economic and spiritual).
A follow-up interview
A follow-up interview by telephone or face-to-face was performed by fifth author (JL) after one month to receive more insights into each participant’s learning experience and attitudes toward the VP system. These were opened ended questions regarding the participant’s perception of educational potential and usefulness of the VP case, engagement perception, empowerment, virtual interpreter, virtual advisor and feedback.
We quantified all findings from the answers in the questionnaires in the pilot study using SPSS 19.0. Analysis of the quantitative data were primarily descriptive since the small N = 11, does not allow for significance testing and will not allow for generalizability of gender and age. The analysis from the pre and post data included item-by item measures and median values for measuring the average rating of the Likert scale questions included in the revised KI-VP-LEQ. Focus was on an evaluation of the realistic nature and usefulness of the VP-system when analyzing the participants’ answers. Evaluation of the face validity, defined as “the extent to which the examination resembles real life situations” [53, 54] was analysed in the issue of acceptance of the degree of the realistic nature of the patient simulation in relation to the actual task as it has been used before including the Swedish pilot study .
The qualitative data included the answers from the participants during the 10 telephone interviews lasting between 15 and 30 minutes. The data was analyzed according to inductive content analysis based on Graneheim and Lundman’s model . Anonymous citations will be used to exemplify the quantitative data. Sometimes the grammar has been changed to increase readability, but in a way that the content has not been changed.
All 11 participants reported long experience of using computers in their daily clinical work and were able to run the RTSim system without any major problems. None of them mentioned that they had used virtual patients before.
Overview of clinical worldview
Self-reported dimensions of clinical care (pre-test questionnaire and post-test questionnaire) ranked by level of emphasis (1 = no emphasis; 5 = full emphasis)
All (N = 11)
All (N = 11)
History of present illness
Mental status examination
Traditional healing examination
Motivation to use VPs for training
Self-reports of current motivation (pre-test questionnaire and post-test questionnaire) ranked by level of emphasis (1 = highly disagree; 4 = highly agree)
All (N = 11)
All (N = 11)
1. I am motivated to use VP as it leads to better care
2. I am motivated to use VP as I will feel more competent
3. I am motivated to use VP as I will have a better relationship with the patient
4. I am motivated to use VP as it will provide better treatment outcomes
5. I am motivated to use VP as I can meet more patients per hour
6. I am motivated to use VP as I can have more time with the patient
7. I am motivated to use VP as I can use VP to educate my staff
8. I am motivated to use VP as it helps to improve interdisciplinary communication
9. I am motivated to use VP as it helps me to understand the mental health problems of my patient
10. I am motivated to use VP as it helps me to understand medical problems of my patient
11. I am motivated to use VP as it helps me to understand social problems of my patient
12. I am motivated to use VP as it helps me to understand spiritual problems of my patient
13. I believe the VP will help me provide better care to all of my patients
14. I believe the VP will help me provide better care of my traumatized patients from any cultural background
15. I believe the VP will help me provide better care to my traumatized patients who are from culturally diverse backgrounds and are low-English speakers
16. This system is only good for helping me to manage traumatized refugees from Bosnia
General opinions and perceptions in the questionnaires after the learning experience
It [VP] appeared real. When I started the program and was trying to figure out what to press, Katharina (VP) started to cough as a sign that she was waiting for me to start. I thought that was interesting.
I liked it. It’s a great learning tool, many things can be done with it. Listening/watching a real person add a lot to the experience.
- I can’t think of any reasons why we should be using (this). There are better reasons to use it to improve our patient’s quality of regarding critique. Thu, its will improve our knowledge.
Emotions, tone of voice, body language enhanced the experience [of VP].
When first working with people who are victims/survivors of trauma a clinician can be overwhelmed to the point of that they themselves being traumatized. Through this learning method, the clinician can feel those moments within themselves, learn to deal with them, put the aside to a more appropriate place/time to talk about them etc. But initially those “Oh my God” moments maybe hard to deal with appropriately.
…her frustration with waiting for the next question, her body language, her tone of voice all were key aspects to the assessment–certainly it added to the learning experience.
Most people I see are further out from their refugee/trauma experiences than this patient, and less likely to have semi-acute medical issues like gonorrhea.
Others felt that after being trained with the VP, they became more open to ask people about their stories and to have full assessment of the patient’s status–physical, emotional, social and spiritual, which reinforces the complexity of patients.
It was common that the participants wanted to have the possibility to formulate their own questions in free text and experienced that scrolling among the pre-defined illness history questions in order to find the most suitable question was in some way odd. Some felt that the VP tool was somewhat artificial due to the limited options to ask questions in the order they wanted. They also reported that the feedback of the VP and VA was not totally clear and needed to be technically improved.
Several issues: (1) Traditional (folks) folder was empty; (2) Did not like read to scroll the questions; (3) unable to access background info–it did not open anywhere. Also the ending was confusing. I did put my assessment and submitted it, but did not get feedback. Then pressed wrong button and case was over. May be a warning “It would end your session, are you sure? Or would you like to hear feedback?
Feedback during the follow up interview regarding VP as authentic, acceptable, and ease to use
The participants recalled VP as a realistic and relevant virtual interface (“realistic responses to the questions”) with an interesting story that was very well done. Some remembered that the questions were very good and especially liked the four patient management domains. One participant hoped that this version could be rolled out for other languages and cultures, as it had great potential; the resource section needs a lot of work.
One of the participants remembered the VP as a useful tool especially for medical students but not for practicing doctors, as it was experienced as too artificial. It reminded some that most of their patients have some trauma in their lives and thought in a real patient would have had trouble dealing with all of that material.
- I think it was a very sad story, I feel compelled by all her losses. I hear similar stories from my patients and this story confirms my experience.
- I was moved by her. She reminded me of many of my patients.
A common answer from the participants (8 out of 10) was that VP could be used as an examination tool on many levels of competence, since it eliminates a lot of biases as everything which is used with this VP is the same.
- I think that my feeling is… it could eliminate a lot of biases because that is used with this VP is the same so the learner gets organized to think through the case.
- It gives you some distance to consider best questions and what directions should I go. The physical examination part one can’t do too much.
I am more aware of listening more attentively to patients.
The majority of the participants answered that they would recommend the VP to a colleague as a training tool in assessing a refugee trauma patient.
Suggestions for improvement of the system were given by the participants also during the follow up interviews. They wanted it to be more user friendly, for instance giving the possibility to ask their own follow-up questions. The VA should be more easily accessible and more interactive and give more feedback about the participants’ interviewing skills. Further suggestions were that physical examination module needs to be expanded or should be skipped and some of the information was a little disjointed. It was recommended that the flow of questions should be increased and made more natural and with less structure. The participants would also have liked to be informed that it was not obligatory to ask all of the questions provided by the system.
- The patient and the answers were very real life. It was clear that a lot of work was put into it and it was very believable. However, the questions and the system were not easy to use with the click and submit methods.
The main aim of this study was to present results from developing and evaluating a virtual patient simulation for training the management of traumatized refugees and how the VP system was perceived by PCPs. In summary, this study demonstrated that the participants’ responses were positive and this VP system was seen as a promising educational tool for increasing their knowledge in the area of caring for patients with trauma, mental illness and refugee background. Other positive comments paid attention to being more relevant than a paper case in the training of medical students. The participants ranked the mental status examination more positive after the simulation exercise compared to before the simulation. Only minor changes was found in datapoints between the pre- and post-tests of self-reported motivation to use VP as an educational tool. Although the general motivation as ranked during the pre-test was high, the participants exhibited high expectations and a positive attitude towards the VP system. Some items in post-test scored, however lower in the post-tests (including items 9, 11 and 13) which can be interpreted as that the participants acquired a more realistic view on the current version of our VP having it’s limitations and areas that need improvement. The results were supported by the follow up interviews. Comparing answers to motivation in the questionnaire versus telephone/face-to-face follow up, most are moving in the same direction. In terms of validity, this pilot study also favors the system’s face validity as the participants had a positive attitude to the VP system which was also found in the study by Pantziaras et al. .
In complex patients with multiple problems and poor communication skills such as refugee patients, traditional PCP approach is often inadequate. Siebens  has responded to these limitations by creating a systematic approach to organizing the PHC use of clinical information called the Domain Management Model (DMM). The DMM provides a standard approach and language to the entire clinical care process consistent with the principles of evidence and culture-based medicine. The DMM is a practical application of the biopsychosocial approach first described by Engel  and primarily used by behavioral health clinicians. Due to the results in this study, that the participants had increased awareness of the continuing effect of past trauma, we recommend an additional domain, a trauma domain in a forthcoming RCT as the PCPs maybe will not pay attention to severe traumatic life events in the assessment and follow up of the patient with trauma.
Even though virtual clinical encounters are quite a new paradigm in PHC, the participants in the present study considered our VP prototype to be a relevant and promising educational tool. The participants also experienced VP as a real clinical assessment tool, which let them use their clinical competence.
Most of the respondents suggested that the VP case may also be useful for exams. This is positive and something that is used as a routine nowadays at both medical schools and as board exams cf. [37, 38]. This also indicated that the PCPs in our study believe the RTSim system is capable of visualizing cases that resemble real patient cases.
However, the most common application of VPs is for learning and training clinical reasoning. Our VP educational model based PHC imply a cognitive constructivist pedagogy and a situated learning approach [29, 58] which may promote medical students’ patient-centered skills in interviewing for assessment, diagnosing and follow up. Good patient-centered interviewing skills have been connected with improved health outcomes and these strategies can according to Lein and Wills  “enhance effectiveness of patient care processes and outcomes while retaining efficiency of patient management” (page 215). Previous research has pointed out that VPs cases need to be realistic and preferably also based on real cases [60–62]. The more authentic a virtual patient case is, it is more challenging , which may explain some of the reservations expressed by one of the participants who remembered the VP as a useful tool especially for medical students but not for practicing doctors, as it was experienced as too artificial.
The reduction in post motivation scores in items 9, 11 & 13 seems to show that the VP training was negative in some aspects, mainly because of the high expectations before the training procedure. This is called expertise reversal effect, i.e. the relative challenges in expert learners’ performance may happen “when there are overlaps between their well-learned and proceduralized knowledge structures and provided instructional guidance” , p. 333. This evidence has significant implications for research in educational theory for adult, experienced learners as compared with novices, like medical students.
Practical implementation of VPs for clinical training
This study investigated how a small number of PCPs at a local clinic could make use of VPs for training their skills and even though none of them had used virtual patients before, they seemed to be able to run the RTSim system without any major problems. This indicates that VPs also might be used in clinical settings, where normally no teachers or facilitators are available. Therefore, we interpret our results that VPs might be used in for example CME in also other settings without too much need of support. This is in line with previous research , where CME has been indicated as one potential use of VPs.
The practical implementation of VPs for training has also been discussed in other studies, and indicated that it might be advantageous to use VPs in PBL settings or in other small groups as pairs of learners . However, in clinical settings and/or for CME, individual use is probably the method of choice due to limitation in time and space for clinicians with heavy clinical burdens.
One of the limitations in this study was the rather small N, which did not allow for generalizability of gender and age, which may have some impact on the PCPs’ perceptions of the utilization of the VP in PHC as suggested by the initial study .
The results were based on the subjective experiences of the participants without any external video observations, physiological recordings or other objective data  which could more deeply pay attention to the participants’ thoughts and attitudes to the VP.
There were some limitations due to technical issues which may have influenced the motivation of using the VP. Positive findings from the questionnaires included that the PCPs put more focus on mental status examination after the VP-based training. This is very positive since traumatized refugees often have mental impairment that needs to be identified, understood and taken into consideration by the PCP. Other non-significant, but still interesting positive findings included that the self-reported dimensions of clinical care showed positive trends in the post-test indicating that the PCPs tended to put more emphasis on root causes (social and spiritual) after the VP-training.
On the other hand, negative fluctuating datapoints in results came from the post-test in terms of that the participants disagreed more to use VP to help them to understand the mental health problems in their patients and to use VPs to help them to understand spiritual problems of their patients. These negative fluctuating datapoints might be connected with someone’s wish to apply open ended free text questions in order to build trust, which was not possible in the current version of the RTSim system. Such a comment was also given by one of the interviewed participants, who said that he lacked a feature to ask open ended questions to the Virtual Refugee. However, such VPs systems with free text dialogues are very expensive to develop, even with modern techniques.
The need for improvement of training of PCPs in the management of traumatized refugee patients with physical and psychiatric impairment is substantial. Even though virtual clinical encounters are quite a new paradigm in PHC, the participants in the present pilot study considered RTSim to be a relevant and promising educational tool for PHC. The participants also experienced VP as a realistic clinical assessment tool, which let them use their clinical competence. The next stage of our project, Phase 2, will be to test the impact of the VP prototype compared to a paper case on improving clinical diagnosis and treatment of the traumatized refugee patient. Then we plan a RCT study to test the impact of data outcome.
Professor Richard F Mollica, MD is foreign adjunct professor at Cultural Medicine Unit, Department of Learning Informatics, Management and Ethics, Karolinska Institutet, Stockholm, Sweden (2010-05-10—2016-05-09).
This work was supported by the NIH PAR-08-269 Grant Exploratory and Developmental Grant to Improve health care Quality through Health Information Technology.
Special thanks to Olivier Courteille, PhD at the time of the study, medical informatics researcher at the Centre for Medical Education, Department of Learning, Informatics, Management and Ethics, Karolinska Institutet, Stockholm, Sweden.
- The UN Refugee Agency: http://www.unhcr.org.
- U.S. Census Bureau: Census of population, 1850 to 2000, and the American community survey. 2010, http://www.census.gov/newsroom/pdf/cspan_fb_slides.pdf.Google Scholar
- U.S. Department of State: Diplomacy in action. Refugee admissions. http://www.state.gov/j/prm/ra/index.htm.
- AHRQ: U.S. Department of Health and Human Services, Agency for Healthcare Research and Quality. 2007 National Healthcare Disparities Report. 2008, Rockville, MD: AHRQ Pub, 08-0041.Google Scholar
- Ngo-Metzger Q, Legedza ATR, Phillips RS: Asian Americans’ reports of their health care experiences: results of a national survey. J Gen Int Med. 2004, 19: 111-119. 10.1111/j.1525-1497.2004.30143.x.View ArticleGoogle Scholar
- Weiss BD, Schwartzberg J, Davis T, Parker R, Williams M, Wang C: Health literacy, A manual for clinicians. 2003, Chicago: American Medical Association Foundation and American Medical AssociationGoogle Scholar
- Kalichman SC, Ramachandran B, Catz S: Adherence to combination antiretroviral therapies in HIV patients of low health literacy. J Gen Int Med. 1999, 14 (5): 267-273. 10.1046/j.1525-1497.1999.00334.x.View ArticleGoogle Scholar
- Schillinger D, Grumbach K, Piette J, Wang F, Osmond D, Daher C, Palacios J, Sullivan GD, Bindman AB: Association of health literacy with diabetes outcomes. JAMA. 2002, 288 (4): 475-482. 10.1001/jama.288.4.475.View ArticleGoogle Scholar
- Williams MV, Davis T, Parker RM, Weiss BD: The role of health literacy in patient-physician communication. Family Med. 2002, 34 (5): 383-389.Google Scholar
- Roter DL, Hall JA, Katz NR: K: patient-physician communication: a descriptive summary of the literature. Patient Educ Couns. 1988, 12 (2): 99-119. 10.1016/0738-3991(88)90057-2.View ArticleGoogle Scholar
- Ngo-Metzger Q, Massagli MP, Clarridge BR, et al: Linguistic and cultural barriers to care: perspectives of Chinese and Vietnamese immigrants. J Gen Intern Med. 2003, 18: 44-52. 10.1046/j.1525-1497.2003.20205.x.View ArticleGoogle Scholar
- Trinh-Shevrin C, Islam NS, Rey MJ: Asian American communities and health. 2009, San Francisco: John Wiley & Sons, IncGoogle Scholar
- Schmidley D: The foreign-born population in the United States: March 2002. 2003, Washington, DC: Current Population Reports, 20-539.Google Scholar
- Weech-Maldonado R, Morales LS, Elliott M, Spritzer K, Marshall G, Hays RD: Race/ethnicity, language, and patients’ assessments of care in Medicaid managed care. Health Serv Res. 2003, 38 (3): 789-808. 10.1111/1475-6773.00147.View ArticleGoogle Scholar
- Divi C, Koss RG, Schmaltz SP, Loeb JM: Language proficiency and adverse events in US hospitals: a pilot study. Int J Quality Health Care. 2007, 19 (2): 60-67. 10.1093/intqhc/mzl069.View ArticleGoogle Scholar
- Hampers LC, McNulty JE: Professional interpreters and bilingual physicians in a pediatric emergency department: effect on resource utilization. Arch Pediatr Adolesc Med. 2002, 156 (11): 1108-1113. 10.1001/archpedi.156.11.1108.View ArticleGoogle Scholar
- John-Baptiste A, Naglie G, Tomlinson G, et al: The effect of English language proficiency on length of stay and in-hospital mortality. J Gen Int Med. 2004, 19 (3): 221-228. 10.1111/j.1525-1497.2004.21205.x.View ArticleGoogle Scholar
- Mollica RF, Kirschner KE, Ngo-Metzger Q: The mental health challenges of immigration. Oxford textbook of community mental health. Edited by: Thornicroft G, Szmukler G, Mueser KT, Drake RE. 2011, Oxford: Oxford University Press, 95-103.Google Scholar
- Harvard Program in Refugee Trauma: http://www.hprt-cambridge.org.
- Sareen J, Cox BJ, Stein MB, Afifi TO, Fleet C, Asmundson GJ: Physical and mental comorbidity, disability, and suicidal behavior associated with posttraumatic stress disorder in a large community sample. Psych Med. 2007, 69 (3): 242-248. 10.1097/PSY.0b013e31803146d8.View ArticleGoogle Scholar
- Sledjeski EM, Speisman B, Dierker L: Does number of lifetime traumas explain the relationship between PTSD and chronic medical conditions? Answers from the national comorbidity survey-replication (NCS-R). J Behav Med. 2008, 31: 341-349. 10.1007/s10865-008-9158-3.View ArticleGoogle Scholar
- Mollica RF, McInnes K, Pham T, Smith Fawzi MC, Murphy E, Lin L: The dose-effect relationships between torture and psychiatric symptoms in Vietnamese ex-political detainees and a comparison group. J Nerv Mental Dis. 1998, 186 (9): 543-553. 10.1097/00005053-199809000-00005.View ArticleGoogle Scholar
- Mollica RF, Cardozo BL, Osofsky HJ, Raphael B, Ager A, Salama P: Mental health in complex emergencies. Lancet. 2004, 364 (9450): 2058-2067. 10.1016/S0140-6736(04)17519-3.View ArticleGoogle Scholar
- Marshall GN, Schell TL, Elliott MN, Berthold SM, Chun CA: Mental health of Cambodian refugees 2 decades after resettlement in the United States. JAMA. 2005, 294 (5): 571-579. 10.1001/jama.294.5.571.View ArticleGoogle Scholar
- Cassano P, Fava M: Depression and public health: an overview. J Psych Res. 2002, 53 (4): 849-857. 10.1016/S0022-3999(02)00304-5.View ArticleGoogle Scholar
- Culpepper L: Depression and chronic medical illness: diabetes as a model. Psychiatric Annals. 2002, 32 (9): 528-534.View ArticleGoogle Scholar
- Bergin R, Fors U: Interactive simulation of patients-an advanced tool for student-activated learning in medicine & healthcare. Comp Ed. 2003, 40 (4): 361-376. 10.1016/S0360-1315(02)00167-7.View ArticleGoogle Scholar
- Fors U, Muntean V, Botezatu M, Zary N: Cross-cultural use and development of virtual patients. Med Teach. 2009, 31 (8): 732-738. 10.1080/01421590903124724.View ArticleGoogle Scholar
- Lave J: Cognition in practice: mind, mathematics and culture in everyday life (learning in doing). 1988, New York: Cambridge University PressView ArticleGoogle Scholar
- Quinn C: Engaging learning: designing e-learning simulation games. 2005, San Francisco: PfeifferGoogle Scholar
- Sansone C, Morgan C: Intrinsic motivation and education: competence in context. Motication Emotion. 1992, 16: 249-270. 10.1007/BF00991654.View ArticleGoogle Scholar
- Dede C, Whitehouse P, Brown-L’Bahy T: Designing and studying learning experiences that use multiple interactive media to bridge distance and time. Glass CVG. Edited by: Current Perspectives on Applied Information Technologies. 2002, Greenwhich, CT: Information Age Press, Distance Education, Vol. 1Google Scholar
- Triola M, Feldman H, Kalet AL, Zabar S, Kachur EK, Gillespie C, et al: A randomized trial of teaching clinical skills using virtual and live standardized patients. J Gen Int Med. 2006, 21 (5): 424-429. 10.1111/j.1525-1497.2006.00421.x.View ArticleGoogle Scholar
- Cook DA, Triola MM: Virtual patients: a critical literature review and proposed next steps. Med Ed. 2009, 43 (4): 303-311. 10.1111/j.1365-2923.2008.03286.x.View ArticleGoogle Scholar
- Perron NJ, Perneger T, Kolly V, Dao MD, Somer J, Hudelson P: Use of a computer-based simulated consultation tool to assess whether doctors explore sociocultural factors during patient evaluation. J Eval Clin Pract. 2009, 15 (6): 1190-1195. 10.1111/j.1365-2753.2009.01302.x.View ArticleGoogle Scholar
- Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, Satava RM: Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Annals surg. 2002, 236 (4): 458-463. 10.1097/00000658-200210000-00008. discussion 463–454View ArticleGoogle Scholar
- Gesundheit N, Brutlag P, Youngblood P, Gunning WT, Zary N, Fors U: The use of virtual patients to assess the clinical skills and reasoning of medical students: initial insights on student acceptance. Med Teach. 2009, 31 (8): 739-742. 10.1080/01421590903126489.View ArticleGoogle Scholar
- Botezatu M, Hult H, Tessma MK, Fors UG: Virtual patient simulation for learning and assessment: superior results in comparison with regular course exams. Med teach. 2010, 32 (10): 845-850. 10.3109/01421591003695287.View ArticleGoogle Scholar
- Gutierrez-Maldonado J, Peñaloza C, Jarne A, Talarn A, Andres A, Aguilar A, Ferrer M: ITNG’10 Proceedings of the 2010 seventh International Conference on Information Technology: New Generations. A training program of differential diagnosis skills based on virtual reality and artificial intelligence. 2010, Washington DC: IEEE Computer Society, 1298-1299.Google Scholar
- Gutierrez-Maldonado J, Alsina I, Ferrer M, Aguilar A: Virtual reality and artificial intelligence to train abilities of diagnosis in psychology and psychiatry. Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications. Edited by: Luca J, Weippl E. 2008, Chesapeake, VA: AACE, 5871-5876.Google Scholar
- Kenny P, Parsons TD, Gratch J, Leuski A, Rizzo AA: Intelligent virtual agents. Virtual patients for clinical therapist skills training. 2007, Berlin Heidelberg: Springer, 197-210.Google Scholar
- Kenny P, Parsons TD, Gratch J, Rizzo AA: Proceedings of the 8th International Conference on Intelligent Virtual Agents. Evaluation of Justina: a virtual patient with PTSD. 2008, Berlin Heidelberg: Springer, 394-408.Google Scholar
- Pantziaras I, Courteille O, Mollica R, Fors U, Ekblad S: Perceived usefulness and educational potentials of virtual clinical encounters in transcultural psychiatry: a pilot study. Int J Med Ed. 2012, 3: 132-140.View ArticleGoogle Scholar
- Institute of Medicine: Primary care: America’s health in a new era. 1996, Washington, DC: National Academy PressGoogle Scholar
- Barrett JE, Barrett JA, Oxman TE, Gerber PD: The prevalence of psychiatric disorders in a primary care practice. Archives Gen Psych. 1988, 45 (12): 1100-1106. 10.1001/archpsyc.1988.01800360048007.View ArticleGoogle Scholar
- Taubman-Ben-Ari ORJ, Feldman D, Vaturi R: Post-traumatic stress disorder in the primary care medical setting. Psychol Med. 2001, 31: 555-560.View ArticleGoogle Scholar
- Pignone MP, Gaynes BN, Rushton JL, Burchell CM, Orleans CT, Mulrow CD, Lohr KN: Screening for depression in adults: a summary of the evidence for the U.S. preventive services task force. Annals Int Med. 2002, 136 (10): 765-10.7326/0003-4819-136-10-200205210-00013.View ArticleGoogle Scholar
- Magruder KMFB, Knapp RG, Davis L: Prevalence of post-traumatic stress disorder in veterans’ affairs primary health care clinics. Gen Hospital Psych. 2005, 27 (3): 169-179. 10.1016/j.genhosppsych.2004.11.001.View ArticleGoogle Scholar
- Ekblad S, Roth G: Diagnosing PTSD among multicultural patients in a Stockholm psychiatric clinic. J Nervous Mental Dis. 1997, 2: 102-107.View ArticleGoogle Scholar
- Miranda J, Duan N, Sherbourne C, et al: Improving care for minorities: can quality improvement interventions improve care and outcomes for depressed minorities? Results of a randomized controlled trial. Health Serv Res. 2003, 38 (2): 613-630. 10.1111/1475-6773.00136.View ArticleGoogle Scholar
- Feldman R: Primary health care for refugees and asylum seekers: a review of the literature and a framework for services. Public Health. 2006, 120 (9): 809-816. 10.1016/j.puhe.2006.05.014.View ArticleGoogle Scholar
- Alim TN, Graves E, Mellman TA, Aigbogun N, Gray E, Lawson W, Charney DS: Trauma exposure, posttraumatic stress disorder and depression in an African-American primary care population. J Nat Med Assoc. 2006, 98 (10): 1630-1636.Google Scholar
- Winberg TM, Hedman L: Student attitudes toward learning, level of pre-knowledge and instruction type in a computer-simulation: effects on flow experiences and perceived learning outcomes. Instruct Sci. 2008, 36 (4): 269-287. 10.1007/s11251-007-9030-9.View ArticleGoogle Scholar
- Moorthy K, Munz Y, Sarker SK, Darzi A: Objective assessment of technical skills in surgery. BMJ. 2003, 327 (7422): 1032-1037. 10.1136/bmj.327.7422.1032.View ArticleGoogle Scholar
- Graneheim UH, Lundman B: Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Ed Today. 2004, 24 (2): 105-112. 10.1016/j.nedt.2003.10.001.View ArticleGoogle Scholar
- Siebens H: The domain management model-a tool for teaching and management of older adults in emergency departments. Acad Emerg Med. 2005, 12 (2): 162-168.Google Scholar
- Engel GL: The need for a new medical model: a challenge for biomedicine. Science. 1977, 196 (4286): 129-136. 10.1126/science.847460.View ArticleGoogle Scholar
- Felix U: E-learning pedagogy in the third millennium: the need for combining social and cognitive constructivist approaches. reCALL. 2005, 17 (1): 85.View ArticleGoogle Scholar
- Lein C, Wills CE: Using patient-centered interviewing skills to manage complex patient encounters in primary care. J Am Acad Nurse Pract. 2007, 19: 215-220. 10.1111/j.1745-7599.2007.00217.x.View ArticleGoogle Scholar
- Botezatu M, Hult H, Fors UG: Virtual patient simulation: what do students make of it? A focus group study. BMC Med Ed. 2010, 10: 91-10.1186/1472-6920-10-91.View ArticleGoogle Scholar
- Botezatu M, Hult H, Tessma M, Fors U: As time goes by: stakeholder opinions on the implementation of a virtual patient simulation system. Med Teach. 2010, 32 (11): 509-516. 10.3109/0142159X.2010.519066.View ArticleGoogle Scholar
- Kalyuga S, Rikers R, Paas F: Educational implications of expertise reversal effects in learning and performance of complex cognitive and sensorimotor skills. Educ Psychol Rev. 2012, 24: 313-337. 10.1007/s10648-012-9195-x.View ArticleGoogle Scholar
- Bergin R, Youngblood Y, Ayers M, Boberg J, Bolander K, Courteille O, Dev P, Hindbeck H, Stringer J, Thalme A, Fors UGH: Interactive simulated patient–experiences with collaborative e-learning in medicine. J Ed Comp Res. 2003, 29 (3): 387-400. 10.2190/UT9B-F3E7-3P75-HPK5.View ArticleGoogle Scholar
- Parsons TD, Kenny P, Ntuen CA, Pataki CS, Pato MT, Rizzo AA, St-George C, Sugar J: Objective structured clinical interview training using a virtual human patient. Studies Health Technol Inform. 2008, 132: 357.Google Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6920/13/110/prepub