Assessing socioeconomic bias in machine learning algorithms in health care: A case study of the HOUSES index

Young J. Juhn, Euijung Ryu, Chung Il Wi, Katherine S. King, Momin Malik, Santiago Romero-Brufau, Chunhua Weng, Sunghwan Sohn, Richard R. Sharp, John D. Halamka

Research output: Contribution to journalArticlepeer-review


Objective: Artificial intelligence (AI) models may propagate harmful biases in performance and hence negatively affect the underserved. We aimed to assess the degree to which data quality of electronic health records (EHRs) affected by inequities related to low socioeconomic status (SES), results in differential performance of AI models across SES. Materials and Methods: This study utilized existing machine learning models for predicting asthma exacerbation in children with asthma. We compared balanced error rate (BER) against different SES levels measured by HOUsing-based SocioEconomic Status measure (HOUSES) index. As a possible mechanism for differential performance, we also compared incompleteness of EHR information relevant to asthma care by SES. Results: Asthmatic children with lower SES had larger BER than those with higher SES (eg, ratio = 1.35 for HOUSES Q1 vs Q2-Q4) and had a higher proportion of missing information relevant to asthma care (eg, 41% vs 24% for missing asthma severity and 12% vs 9.8% for undiagnosed asthma despite meeting asthma criteria). Discussion: Our study suggests that lower SES is associated with worse predictive model performance. It also highlights the potential role of incomplete EHR data in this differential performance and suggests a way to mitigate this bias. Conclusion: The HOUSES index allows AI researchers to assess bias in predictive model performance by SES. Although our case study was based on a small sample size and a single-site study, the study results highlight a potential strategy for identifying bias by using an innovative SES measure.

Original languageEnglish (US)
Pages (from-to)1142-1151
Number of pages10
JournalJournal of the American Medical Informatics Association
Issue number7
StatePublished - Jul 1 2022


  • algorithmic bias
  • artificial intelligence
  • electronic health records
  • social determinants of health

ASJC Scopus subject areas

  • Health Informatics


Dive into the research topics of 'Assessing socioeconomic bias in machine learning algorithms in health care: A case study of the HOUSES index'. Together they form a unique fingerprint.

Cite this