“It's in My language”: A Case Study on Multilingual mHealth Application for Immigrant Populations With Limited English Proficiency (2024)

We recruited 5 participants through referrals from the IHCD Center. Our inclusion criteria included patients 1) whose preferred language was Spanish and had LEP and 2) who owned a personal mobile phone and had previous experience using a patient portal. To ascertain that the patients met the inclusion criteria, staff from IHCD reviewed the patients’ preferred language in the electronic health record (EHR) system and asked the patients about their English proficiency. All participants gave verbal consent to participate at the beginning of the study. Each participant was compensated with a $75 Amazon gift card.

All sessions were conducted in-person in a quiet room in the hospital's outpatient facility. We met the participants after their scheduled visit at the clinic site for their convenience. We did not use remote usability testing since previous research with IHCD had shown that remote usability testing excluded participants without internet or personal computer access.

The study protocol was first developed in English and translated into Spanish. The UX designer conducted a mock session with the interpreter from IHCD using the translated protocol. After the mock session, the interpreter and another bilingual staff provided revisions to the protocol to culturally adapt its content to the target population.

Each session comprised a semi-structured interview and usability testing and lasted approximately one hour. During the session, the interpreter moderated in Spanish and interpreted the participants’ responses into English for the UX designer as shown in Figure 1. In this way, the UX designer could get immediate responses from the participants. While the interpreter primarily used the study protocol to facilitate the session, the UX designer asked follow-up questions in English when the participants gave responses beyond the scope of the protocol. In such cases, the interpreter relayed the questions for the participant.

During the first part of the session, we conducted a semi-structured interview to collect the participants’ attitudinal data, such as their comfort with using technology, previous experiences with using a patient portal, perceived value of using a patient portal, and experiences with needing assistance in using the portal (e.g., have you ever used a patient portal app to interact with your doctor or hospital, have you ever received help using the patient portal?). The purpose of these questions was to understand the existing barriers that LEP patients confront in using a patient portal.

During the second part of the session, we conducted usability testing on a prototype of a mobile patient portal application with a Spanish translation feature (Figure 2) to collect behavioral data on their interactions with multilingual translation. We utilized this prototype to measure task success and elicit responses from the participants about the usability issues they identified. The prototype was installed on the institution's mobile test device and given to the participants. The participants were then given tasks to complete and asked to articulate their thoughts and impressions as they completed them, known as the “think-aloud protocol.” Participants were asked to complete three tasks: Enter their username and password, turn on biometric settings, and turn on translation settings on the patient portal mobile application.

As shown in Fig. 2, the prototype introduced the translation setting after the user logs in so that the system can register the language preference with an authenticated account. Therefore, the login screen was only available in English. Immediately after the participant completed logging in, a modal to opt-in to biometric login was displayed with the following copy: “Tired of logging in? Enable Face ID to unlock MyMSK,” with options, “Not now” or “Okay.” Once the participant selected one of the options, their preference for biometric login was saved. Lastly, they were asked to select in which language they wanted to use the portal.

We recorded the sessions and transcribed the English responses that the interpreter conveyed during the sessions verbatim. The interpreter who helped moderate the sessions reviewed the transcripts and made minor revisions.

The UX designer and the UX researcher used an inductive thematic analysis approach [29] to analyze our qualitative data to identify patterns throughout the interview transcripts. They read the transcripts to become familiar with the data and created codes, and iteratively refined themes developed from the codes that corresponded with broader patterns in the data.

For the usability testing, the UX designer assessed whether the participants successfully completed tasks, if there were more than 2 unsuccessful attempts to complete the tasks, and if they accurately comprehended the results of the tasks they completed. The UX designer used the interpreted responses from the think-aloud protocol to determine the results.

All participants were Hispanic, owned a personal mobile device, and had previous experience using a patient portal. All participants did not speak or read English well and spoke Spanish as their primary language, except for P1, who spoke and read English somewhat well. All participants were between the ages of 50 and 59 and identified as female, except for one participant, who identified as male.

For task 1, two participants had difficulty with logging in, due to the English labels that required English proficiency. Their experience with logging in to the prototype was similar to their experiences with logging in to other English websites, which required support from their caregivers or English-proficient peers. P2 noted, “The experience that I have with other websites has been amazing because I just get my kids. I don't do it myself. They help me for it. And if I'm out on the street, I ask for help.” P5 also mentioned that his caregiver “logs in for me because I'm not very good with computers” and called out that an auto-save of login credentials or biometric login would enhance usability.

All participants selected “Okay” on the modal to opt-in to biometric login, but many didn't make an informed selection or understood the outcome of their selection. Three participants instinctively selected “Okay” to proceed to the next page. When we asked them about their familiarity with biometric login, 4 participants had familiarity with using their fingerprint or face for authentication. P3 had experience with using her fingerprint for authentication but did not understand the term “Face ID.” P1 preferred to use a combination of username and password, a knowledge-based authentication, favoring active memory retrieval.

Four participants perceived greater ease of use after the patient portal interface was translated into their language. During the first two tasks, when the prototype did not offer a language translation option, the participants needed help understanding the labels in English. For the first task, as seen in Fig. 2, the layout and the requested fields in the login screen followed the standard convention, but two participants misinterpreted the ‘username’ field as a field for entering their name. For the second task, the modal to opt-in to biometric login was in English, and many participants immediately tapped “Okay” to proceed with the next steps without acknowledging the expected outcome of the selection.

On the other hand, their comprehension improved once the static labels within the interface were translated into their language. The participants also expressed increased trust and confidence in the information presented to them:

“Sometimes it's scary and we are scared to put our information there, like they will just steal information and if you don't understand it because it's in English. But if I see it in Spanish, then I'm sure of what I'm reading and why I'm giving that information.” (P4)

When participants were asked about what they liked the most about the prototype, they emphasized the independence and self-efficacy gained from the translated patient portal app:

“I don't need to call my daughter for this. I like it a lot. I like it because, first of all, it's my language (…) I think it (portal app) will help me to be a little more independent. I would like that. So that I don't have to bother them” (P5).

P3, who had more challenges with digital literacy, didn't exhibit an increased likelihood to use the translated portal in comparison to the portal in English. When she was asked about her experience using the prototype, she said, “It was the same. Confusing. Because first you see it in English, and then it goes to Spanish.” She addressed that the information density and small font sizes contribute to the perceived difficulty.

The caregivers with English proficiency, who were the participants’ children, had a significant role in mediating patient-provider communication in addition to other caregiving responsibilities. For example, P2 was experiencing a language barrier with her provider and delegated all patient-provider communication to her caregiver:

“I don't communicate. I have a little card and if I need to tell the doctor something, I just call my daughter. And my daughter calls him because she speaks English.”

We were also interested in how caregivers with English proficiency contribute to non-English speaking patients’ adoption of patient portals. All study participants shared that they had difficulty with using websites that are not in Spanish and relied on third-party translation applications or their English-proficient caregivers to overcome the language barrier. For example, P3, who was also a user of MyChart, the Epic system's patient portal application, heavily relied on her daughter for setup. P3 said,

“Every time that I used the patient portal, I received help. (…) My daughter's the one who took care of everything because for me at the beginning it was very difficult. (…) so she said, ‘No, mom, I'm gonna do everything.’”

Most participants didn't live with their caregivers to assist with these tasks on demand. P2 expressed frustration about the difficulty with installing the mobile portal application, and the lack of support to meet the onboarding needs. She said, “I was told that my daughter has to download it on the computer to be able to put on my phone or something like that. But I told that my daughter doesn't live with me, and she has her own work.”

Despite the varying English proficiency of immigrant populations, they are often only represented by race and ethnicity in HCI research. Among health technology research on older adults, only three studies reported both race/ethnicity and primary language, as stated by Harrington et al. [14]. In the systematic review on mHealth technology research for vulnerable populations by Stowell et.al [28], 37.62% of studies did not report the participants’ primary language. These reviews indicate a lack of thorough evaluation regarding the participant's language proficiency, and a lack of standardization in capturing language proficiency in health technology research. The Joint Commission mandates health organizations to collect the patient's preferred language to determine the language that the patient wishes to communicate health information [7]. Similarly, studies on health technology should capture the participant's preferred language and report if they were using the technology in their preferred language when evaluating their usability.

To reduce the language barrier experienced by users with LEP, prior studies have examined the use of translation technologies, such as Google translate, in healthcare settings from the perspective of the care team [1, 23, 24]. However, our case study is the first to specifically focus on patients with LEP and leverage their feedback to assess the effects of translated interfaces in health technology use.

Language proficiency is a social determinant of health that should not be overlooked and there are opportunities for HCI studies to expand upon health disparities research on individuals with LEP. While operational and financial constraints limit the availability of on-site interpreters, the use of multilingual health technology presents an opportunity for HCI intervention to empower this user group.

Given the large portion of individuals with LEP in the U.S., HCI practitioners should not assume that the user is proficient in the language provided in the user interface. Jakob Nielsen's 10 Usability Heuristics address that the terminology in the user's interface should ‘speak the users’ language’ [21]. Aside from effective content strategy, he asserts that the dialogues should ideally be in the user's native language. Yet, when evaluating this heuristic in practice, designers often assume the user is proficient in the interface's language. Such assumptions and gaps in awareness can yield risks addressed in Sin's digital design marginalization (DDM) framework [27]. The unintended marginalization of individuals with LEP in the design process can undermine their needs, especially language concordance, and exacerbate the digital divide. For instance, our study showed that even a simple login page that follows standard design is prone to user errors when the user does not speak the interface's language. Without testing this flow with the individuals with LEP, we would not have identified the significance of offering translation settings in the login page.

Unlike other HCI studies on immigrant populations, the UX designer and the researcher of our study were not proficient in the language of this user group. Therefore, conducting research on individuals with LEP required additional preparation and resources. We moderated the study with a professionally trained interpreter who has expertise in this population and conducted mock sessions before the actual study. This step was crucial to not only train the interpreter who had no experience with moderating usability testing, but also to refine the script based on their knowledge of intercultural and interlingual nuances. Our case study demonstrates that collaboration with ethnic community-based organizations (CBOs) that understand the immigrant populations’ cultural norms and language implications is critical for an effective study involving participants with LEP [10, 30].

Our findings align with prior research on patient portals that usability barriers such as difficulties in typing in username or password may hinder the portal adoption of vulnerable populations [32]. We initially designed the user flow to introduce the translation settings after the user logs into the portal to associate the user's language preference with their authenticated account. However, our study revealed that a language barrier contributes to a usability barrier at any moment in the user's journey. Multilingual translation should be presented as early as possible to reduce the usability barriers caused by LEP. By opting into using the patient portal in their preferred language at the beginning of the enrollment process, patients with LEP will be able to accurately comprehend the authentication options presented to them and successfully set up the patient portal based on their preferences. In doing so, patients with LEP will require less support from caregivers to access the patient portal [26].

Our mobile portal app promotes biometric authentication, a user-friendly solution increasingly adopted by mHealth services [8]. However, there is little known about the digital literacy of individuals with LEP on biometric authentication solutions and their perception of this solution, which this study also explored. The different authentication preferences of study participants, ranging from biometric authentication to knowledge-based authentication, suggest that patient portals should refrain from a one-size-fits-all approach and provide a diverse set of authentication methods to accommodate the varying needs of patients [20].