Print this page Email this page Users Online: 59 | Click here to view old website
Home About us Editorial Board Search Current Issue Archives Submit Article Author Instructions Contact Us Login 

 Table of Contents  
Year : 2018  |  Volume : 31  |  Issue : 1  |  Page : 17-24

Assessing the hidden curriculum for the care of patients with limited english proficiency: An instrument development

1 Division of General Internal Medicine, Massachusetts General Hospital; Harvard Medical School, Boston, Massachusetts, USA
2 Department of Health Professions Education, MGH Institute for Health Professions, Boston, Massachusetts, USA
3 Department of Medicine, Mount Sinai Hospital, New York, USA
4 Department of Medicine, Brigham and Women's Hospital; Harvard Medical School, Boston, Massachusetts, USA

Date of Web Publication14-Aug-2018

Correspondence Address:
Alexander R Green
Massachusetts General Hospital, Division of General Internal Medicine, Boston, Massachusetts 02114
Login to access the Email id

Source of Support: None, Conflict of Interest: None

DOI: 10.4103/1357-6283.239042


Background: Patients with limited English proficiency (LEP) are a growing population in the United States at risk for disparities in quality and safety of care. Medical student competency to care for patients with LEP is impacted by a hidden curriculum (HC) that undermines the learning experience; yet to date, there is no way to measure it. Thus, we designed an instrument to assess this HC. Methods: Based on findings from previous qualitative work and input from medical students and experts in LEP and psychometrics, we developed a 23-item survey with four domains. We e-mailed this to 3rd and 4th year students from two medical schools in the US. We conducted principal axis factoring to determine the instrument's construct validity. Only items with a factor loading ≥0.50 were retained. Results: We obtained 111 complete responses. Twenty-two of the 23 original items were retained. Four factors/components emerged, which did not support the original proposed domains. Three factors loaded on a mix of role modeling, and learning environment, structural, and organizational variables, while the fourth factor retained two role modeling items. Based on the factor extraction solution, we restructured the instrument into three domains: role modeling, demonstration of effective systems, and consequences of structural barriers for patients with LEP (Cronbach's alpha: 0.81–0.95, total variance accounted for 53.7%). Discussion: The results led us to reassess the domain structure to create an instrument representing students' perceptions and context. Our instrument, the LEP-HC, will allow medical educators to investigate a specific and important HC and improve teaching about care of patients with LEP.

Keywords: Hidden curriculum, informal curriculum, limited English proficiency, language barriers, and medical education

How to cite this article:
Green AR, Rosu C, Kenison T, Nze C. Assessing the hidden curriculum for the care of patients with limited english proficiency: An instrument development. Educ Health 2018;31:17-24

How to cite this URL:
Green AR, Rosu C, Kenison T, Nze C. Assessing the hidden curriculum for the care of patients with limited english proficiency: An instrument development. Educ Health [serial online] 2018 [cited 2022 Jan 23];31:17-24. Available from:

  Background Top

Effective communication is an essential component of providing safe, high-quality clinical care. Unfortunately, for nearly 10% of the United States' residents who are not proficient in English, effective communication is not the norm.[1],[2] It has been well documented that communication barriers faced by people with limited English proficiency (LEP) lead to higher rates of misunderstanding of diagnosis, treatment, and follow-up plans,[3],[4],[5],[6],[7] inappropriate use of medications,[8] lack of informed consent for surgical procedures,[9] high rates of serious adverse events,[10],[11],[12],[13] and a lower quality health-care experience than their English-proficient counterparts.[14],[15],[16],[17],[18] Inadequate training of health professional students to work with patients with language barriers may contribute to these disparities, giving rise to a kind of “perfect storm” with a growing LEP population and limited number of physicians trained to care for them. While some medical schools address this topic within the curriculum, most of the practical, clinical learning, both positive and negative, occurs beyond the scope of the formal curriculum, in what has been termed the hidden curriculum (HC).

The HC has been defined as “the set of influences that function at the level of organizational structure and culture including, for example, implicit rules to survive the institution such as customs, rituals, and taken-for-granted aspects.”[19] There is extensive literature about the HC, particularly as it pertains to loss of idealism, emotional neutralization, change of ethical integrity, and acceptance of hierarchy among medical students.[19] One study also validated an instrument to measure the HC concerning patient-centered care.[20] There is, however, a paucity of literature about the HC for the care of patients with LEP and we know of no validated measure. This study represents phase 2 of a three-phase study to explore, measure, and change the HC in medical education for the care of patients with LEP. In Phase 1, we carried out a qualitative study that identified and characterized four components of the HC for medical and nursing students when caring for patients with LEP: role modeling, system factors, learning environment, and organizational culture.[21] Students described experiencing indifference to issues of language and effective communication among their supervisors (a “null curriculum”) and frustration with ineffective systems for caring patients with LEP. For phase 2, our goal was to use the phase 1 findings to develop and validate a survey instrument to assess medical students' perceptions of the informal curriculum/HC with respect to the care of patients with LEP. We will refer to this as the LEP-HC instrument. Ideally, this tool could be used to assess specific clinical rotations, compare among sites, and monitor the impact of interventions. With minor modifications, it could also be used in sites where English is not the dominant language.

  Methods Top

Sampling and survey

Based on findings from previous qualitative interviews with students about their clinical experiences learning to care for patients with LEP,[21] we developed a set of 23 survey items, organized according to four broad HC domains: (1) Role modeling, (2) structural factors, (3) learning environment, and (4) organizational culture. Items were scored on a 5-point Likert-type scale with anchors of 1 = always and 5 = never, as well as 1 = strongly agree and 5 = strongly disagree. We created both positively and negatively worded items to minimize extreme response bias and acquiescent bias. Negative items were reverse coded to facilitate analysis. Other survey items identified the hospital and department of the most recent clerkship, which was designated as the one being evaluated for the survey, the estimated percentage of patients with LEP at the site as experienced by the student, and the medical students' characteristics, including age, gender, race/ethnicity, first language, year in medical school, and the field of medicine they intend to enter. We labeled this the LEP-HC instrument. We pilot tested the items by conducting cognitive interviews with two medical students in order to refine the survey instrument. The students completed the survey within approximately 10 min. We then instructed these students to retake the survey and provide real-time feedback on any unclear instructions, intent of questions or response options, awkward wording, discomfort with a question, incorrect assumptions, and any other feedback that came to mind. Two study staff members (ARG and TCK) observed the sessions and took notes. Three experts in survey design and health care for patients with LEP also reviewed our draft instrument. Using these approaches, we revised the survey, clarifying the wording for several questions, removing nine questions that could not be clarified or were redundant, and produced a refined draft.

The study sample was limited to 3rd and 4th year medical students from Harvard Medical School and Case Western Reserve University, who had completed at least one clinical clerkship. Using each school's established e-mail list serve, we invited students to participate in a 10-min self-administered, online survey to characterize their experience in learning to care for patients with LEP during their most recent clerkship. We then sent an e-mail reminder at 1-week intervals for a total of 3 weeks. We did not offer any incentive to complete the survey and participation was voluntary, anonymous, and students were assured that the survey would have no impact on grades or evaluations. Each participant electronically acknowledged informed consent before starting the survey. The study protocol was approved by the Institutional Review Board at the Harvard Medical School and Case Western Reserve University. We used Qualtrics survey software (Qualtrics, Provo, UT) to administer the online survey and collect the data.

Statistical analysis

We conducted basic descriptive analyses to establish the distribution and frequencies of each variable. We removed sixty participants who did not answer any of the main outcome questions.

We then sought to determine whether the four domains of the LEP-HC instrument could be measured by items and scored as scales that were reliable yet relatively independent. We conducted principal axis factoring (PAF) to determine the construct validity of the LEP instrument which comprised role modeling, structural factors, learning environment, and organizational culture. PAF assesses the pattern of correlation among a set of variables to identify the least number of factors which can account for the shared (overlapping) variance (cross loading cutoff <0.35).

The factors are comprised by the items that reflect the construct of the hypothesized subscale. Data screening indicated no assumption violations and the Kaiser–Meyer–Olkin of 0.80 indicated that the data were suitable for PAF. Similarly, Bartlett's test of sphericity was significant (P < 0.001), indicating sufficient correlation between the variable to proceed with the analysis. Four factors were extracted using PAF in order to account for 53.7% variation. The decision to retain four factors was confirmed using parallel analysis. We applied promax (oblique) rotation to allow for correlated factors. For data with a clear factor structure like our conceptual model, samples of 100 cases are considered sufficient for a correct estimation of the number of factors.[22],[23] Due to our sample size, only items with a factor loading ≥0.50 were retained, as this indicates that the variable does have a strong impact on the factor once the other variables have been accounted for.[24] New subscales were constructed based on the exploratory factor analysis. The internal consistency of each subscale was assessed using Cronbach's alpha, and Pearson's correlations were conducted to assess discriminate validity, i.e., r < 0.80.

We used IBM SPSS (Chicago, IL, USA) version 23.0 for all analyses.

  Results Top

We obtained 171 responses from eligible medical students. We excluded sixty responses from participants who opened the link but decided not to take the survey and did not provide demographic information for comparison. This left a total of 111 eligible responses. The demographic characteristics of the sample are shown in [Table 1]. Females and 4th year students were overrepresented compared to the class demographics overall. The mean scores for items from the LEP-HC instrument broken down by clerkship type are shown in [Table 2].
Table 1: Demographic Characteristics of Participating Medical Students

Click here to view
Table 2: Mean scores for LEP-HC Items by Clerkship

Click here to view

Four components emerged that accounted for 53.7% of the variance, with respective component loading ranging from 0.53 to 0.98 [Table 3]. The four-factor promax-rotated structure did not support the proposed inventory. Three factors loaded significantly on a mix of the original role modeling, structural factors, learning environment, and organizational culture variables, while the fourth one retained only two of the role modeling initial variables. We removed one item (“residents were more likely than attending physicians to assure effective communication with patients with LEP”) because in retrospect it was not a direct measure of the HC. This left 22 of the 23 original items.
Table 3: Summary of Exploratory Factor Analysis Results for LEP – HC*

Click here to view

New subscales were constructed based on the factor extraction solution: negative role modeling, positive and indifferent role modeling, demonstration of effective systems for interpreter services, and consequences of structural barriers to care for LEP patients. Each of the new subscales possessed excellent internal reliability as assessed by coefficient alpha (alpha1 = 0.85, alpha2 = 0.81, alpha3 = 0.85, and alpha4= 0.91) [Table 4].
Table 4: Internal consistency* and reliability** of the limited English proficiency-hidden curriculum domains

Click here to view

After careful consideration, we removed two more items from the final instrument. One of these–residents being open to medical students' concerns about patients due to language barriers–did not fit the new structure (which did not include an educational environment domain) and we realized that it was too dependent on the student having these concerns in the first place. The other-residents emphasizing that effective communication with patients with LEP-was an organizational priority also did not fit the new structure, which did not have an organizational culture domain. For the final instrument, we recombined all role modeling items back into one domain. The revised three-domain structure of the LEP-HC instrument is shown in Appendix 1.

  Discussion Top

Our study is the first to create and assess construct validity for a tool to measure the HC/informal curriculum for medical students as they learn to care for patients with LEP. Haidet et al. developed the C3 tool [20],[25] to measure the HC with respect to patient-centered care specifically, which heavily informed our work. Many other studies have explored the HC in other contexts,[19],[26],[27],[28] setting the groundwork in this area. Our tool, the LEP-HC, will now allow medical educators to investigate a specific HC that has been minimally explored in medical education – the implicit (and not so implicit) messages and lessons that medical students receive about caring for a very vulnerable patient group that cannot communicate well in English. The tool could also be adapted for broader international use to assess HC for the care of patients who do not speak the dominant language, though additional validation studies would be recommended.

The LEP-HC has gone through many rounds of development and revision to maximize face validity and content validity. We first gained confidence in our instrument after receiving input from survey design experts who helped us refine it, avoiding misleading questions and redundancy. Cognitive testing was informative and led to some significant revisions in wording and item selection. Our factor analysis was very strong in terms of loading, clearly indicating similar patterns of responses associated with the themes defined by the four extracted factors. The results led us to reassess both the four themes from our qualitative study and our proposed organization of survey items to create an instrument that better represents students' perceptions and context.

The first and most important result from the factor analysis was the lack of loading of organizational culture as a unique factor. Looking over our qualitative findings in the context of the results of the factor analysis, we recognized that medical students may not have the same perspective on organizational culture as practicing physicians or other more permanent members of the health-care team do. The total amount of time they spend at a particular institution is relatively short and their exposure to the culture of the organization is limited both by this and by the people to whom they have significant contact (mostly residents, a few attending physicians, and a limited number of other staff). Thus, if organizational culture has an effect on students' perceptions of the HC for patients with LEP, it is likely mediated by the role modeling of residents (and perhaps attending physicians) and the learning environment they establish with respect to this issue. We believe that this likely explains why the organizational culture domain did not hold together and was removed from the final instrument.

Structural barriers to caring for patients with LEP loaded together relatively well. However, some items that we had not initially intended as structural barriers also loaded with this component. For example, “Making indirect comments or gestures indicating frustration about communicating with patients with LEP” was intended as a measure of indirect negative role modeling; however, it loaded with the structural barriers' component. In retrospect, this was likely perceived by the students as a sign of frustration with the structural barriers that the students found so prevalent. We reorganized the instrument accordingly.

We also found that negative role modeling loaded separately from other aspects of role modeling (positive and indifferent role modeling). We originally grouped positive, indifferent, and negative role modeling together for the creation of the instrument. It is clear from these results that students perceive these as different constructs. We did not expect that positive and indifferent role modeling would load together. We rationalized that perhaps these represent expected (and accepted) behavior while negative role modeling was clearly aberrant. While we recognize these as separate constructs based on our results, for simplicity and flow of the survey, we recombined them into one overall role modeling domain.

Learning environment items loaded together but not as a separate factor. Instead, they were part of a more complex factor that combined learning environment with positive role modeling and organizational culture items. On reviewing these items, it is likely that medical students considered positive characteristics of their supervisors and staff with respect to LEP patient care more so than an educational environment.[29] Role models clearly influence the learning environment and the students' perceptions of the organizational culture [30] and this might further explain this factor loading.

We included response data (mean scores) for each of the original survey items broken down by clinical rotation in order to provide some context for understanding the instrument. These were not intended to be viewed as representative or generalizable data, but they do provide an idea of the patterns we observed. For example, medical students in surgical clerkships gave relatively low scores regarding their experience in learning to care for patients with LEP (a more negative HC overall). Based on our discussions with medical students and prior assumptions, we would have predicted this, and this does help to support the external validity of the instrument. However, we did not perform statistical analyses to compare these as this was not the intent of the study.

Our study has several potential limitations. It is possible that the qualitative study on which we based our survey [21] missed themes that may have had a significant impact on the proposed LEP-HC dimensions. The sample size was also relatively small and not randomly selected, thus limiting the study's statistical power and generalizability. However, as discussed in the methods section, 100 is an acceptable sample for our purposes and led to very strong factor loading and important changes to the instrument. As a validation study, we were not looking to draw conclusions about students' perceptions of the HC at their schools or more broadly and our results should not be viewed in this way. Furthermore, it is possible that the LEP-HC instrument would not measure the same HC constructs that we intended in other schools with different clerkship experiences and environments. The fact that our factor loading was quite strong helps to mitigate this however. Finally, we did not create the LEP-HC as a scale, but rather as an instrument, with three major domains. Further research is needed to determine how the LEP-HC instrument could be used as a scale with a single composite value.

  Conclusion Top

Our goal with this study was to use the findings from our previous qualitative work to create and determine construct validity of an instrument to assess medical students' perceptions of the HC with respect to the care of patients with LEP during specific clerkships. We intended that the LEP-HC would be used in medical education research to study the HC for patients with language barriers and drive further exploration and understanding of this important topic. We also hope that medical educators will use it to assess the HC in their own clerkships in order to provide feedback to clerkship directors, raise awareness, and catalyze efforts to improve these highly formative experiences.


The authors would like to thank Tomi Jun, MD, and Christen Cuevas, MD, for their help with this study and also Elyse R. Park, PhD, MPH, for reviewing the manuscript.

Financial support and sponsorship

This project was funded by the Arnold P. Gold Foundation

Conflicts of interest

There are no conflicts of interest.

  References Top

Ryan C. Language Use in the United States: 2011. American Community Survey Reports 2013;22:1-16.  Back to cited text no. 1
Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001.  Back to cited text no. 2
Diamond LC, Schenker Y, Curry L, Bradley EH, Fernandez A. Getting by: Underuse of interpreters by resident physicians. J Gen Intern Med 2009;24:256-62.  Back to cited text no. 3
Price-Wise G. Language, culture, and medical tragedy: The Case of Willie Ramirez. Health Affairs Blog; November 19 2008. DOI: 10.1377/hblog20081119.000463  Back to cited text no. 4
Baker DW, Parker RM, Williams MV, Coates WC, Pitkin K. Use and effectiveness of interpreters in an emergency department. JAMA 1996;275:783-8.  Back to cited text no. 5
Wilson E, Chen AH, Grumbach K, Wang F, Fernandez A. Effects of limited English proficiency and physician language on health care comprehension. J Gen Intern Med 2005;20:800-6.  Back to cited text no. 6
Coren JS, Filipetto FA, Weiss LB. Eliminating barriers for patients with limited English proficiency. J Am Osteopath Assoc 2009;109:634-40.  Back to cited text no. 7
Gandhi TK, Burstin HR, Cook EF, Puopolo AL, Haas JS, Brennan TA, et al. Drug complications in outpatients. J Gen Intern Med 2000;15:149-54.  Back to cited text no. 8
Schenker Y, Wang F, Selig SJ, Ng R, Fernandez A. The impact of language barriers on documentation of informed consent at a hospital with on-site interpreter services. J Gen Intern Med 2007;22 Suppl 2:294-9.  Back to cited text no. 9
Divi C, Koss RG, Schmaltz SP, Loeb JM. Language proficiency and adverse events in US hospitals: A pilot study. Int J Qual Health Care 2007;19:60-7.  Back to cited text no. 10
Cohen AL, Rivara F, Marcuse EK, McPhillips H, Davis R. Are language barriers associated with serious medical events in hospitalized pediatric patients? Pediatrics 2005;116:575-9.  Back to cited text no. 11
John-Baptiste A, Naglie G, Tomlinson G, Alibhai SM, Etchells E, Cheung A, et al. The effect of English language proficiency on length of stay and in-hospital mortality. J Gen Intern Med 2004;19:221-8.  Back to cited text no. 12
Quan K, Lynch C. The High Costs of Language Barriers in Medical Malpractice. Berkeley: National Health Law Program; 2010.  Back to cited text no. 13
Woloshin S, Bickell NA, Schwartz LM, Gany F, Welch HG. Language barriers in medicine in the United States. JAMA 1995;273:724-8.  Back to cited text no. 14
Weech-Maldonado R, Elliott MN, Morales LS, Spritzer K, Marshall GN, Hays RD, et al. Health plan effects on patient assessments of Medicaid managed care among racial/ethnic minorities. J Gen Intern Med 2004;19:136-45.  Back to cited text no. 15
Baker DW, Hayes R, Fortier JP. Interpreter use and satisfaction with interpersonal aspects of care for Spanish-speaking patients. Med Care 1998;36:1461-70.  Back to cited text no. 16
Carrasquillo O, Orav EJ, Brennan TA, Burstin HR. Impact of language barriers on patient satisfaction in an emergency department. J Gen Intern Med 1999;14:82-7.  Back to cited text no. 17
Morales LS, Cunningham WE, Brown JA, Liu H, Hays RD. Are Latinos less satisfied with communication by health care providers? J Gen Intern Med 1999;14:409-17.  Back to cited text no. 18
Lempp H, Seale C. The hidden curriculum in undergraduate medical education: Qualitative study of medical students' perceptions of teaching. BMJ 2004;329:770-3.  Back to cited text no. 19
Haidet P, Kelly PA, Chou C; Communication, Curriculum, and Culture Study Group. Characterizing the patient-centeredness of hidden curricula in medical schools: Development and validation of a new measure. Acad Med 2005;80:44-50.  Back to cited text no. 20
Kenison TC, Madu A, Krupat E, Ticona L, Vargas IM, Green AR, et al. Through the veil of language: Exploring the hidden curriculum for the care of patients with limited English proficiency. Acad Med 2017;92:92-100.  Back to cited text no. 21
Gorsuch RL. Factor Analysis. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 2014.  Back to cited text no. 22
Kline P. An Easy Guide to Factor Analysis. New York: Rutledge; 1994.  Back to cited text no. 23
Hair JF, Black B, Babin B, Anderson RE, Tatham RL. Multivariate Data Analysis. 6th ed. Upper Saddle River, NJ: Pearson; 2006.  Back to cited text no. 24
Haidet P, Kelly PA, Bentley S, Blatt B, Chou CL, Fortin AH 6th, et al. Not the same everywhere. Patient-centered learning environments at nine medical schools. J Gen Intern Med 2006;21:405-9.  Back to cited text no. 25
Hafferty FW, Franks R. The hidden curriculum, ethics teaching, and the structure of medical education. Acad Med 1994;69:861-71.  Back to cited text no. 26
Hafferty FW, Gaufberg EH, O'Donnell JF. The role of the hidden curriculum in “on doctoring” courses. AMA J Ethics 2015;17:130-9.  Back to cited text no. 27
Turbes S, Krebs E, Axtell S. The hidden curriculum in multicultural medical education: The role of case examples. Acad Med 2002;77:209-16.  Back to cited text no. 28
Cruess SR, Cruess RL, Steinert Y. Role modelling – Making the most of a powerful teaching strategy. BMJ 2008;336:718-21.  Back to cited text no. 29
Humphrey HJ. Mentoring in Academic Medicine (ACP Teaching in Medicine Series). Philadelphia, PA: American College of Physicians; 2010.  Back to cited text no. 30


  [Table 1], [Table 2], [Table 3], [Table 4]

This article has been cited by
1 How Language Barriers Influence Provider Workload for Home Health Care Professionals: A Secondary Analysis of Interview Data
Allison Squires,Sarah Miner,Eva Liang,Maichou Lor,Chenjuan Ma,Amy Witkoski Stimpfel
International Journal of Nursing Studies. 2019; : 103394
[Pubmed] | [DOI]


Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
Access Statistics
Email Alert *
Add to My List *
* Registration required (free)

  In this article
Article Tables

 Article Access Statistics
    PDF Downloaded427    
    Comments [Add]    
    Cited by others 1    

Recommend this journal