A Systematic Review of ‘Fair’ AI Model Development for Image Classification and Prediction

Ramon Correa, Mahtab Shaan, Hari Trivedi, Bhavik Patel, Leo Anthony G. Celi, Judy W. Gichoya, Imon Banerjee

Research output: Contribution to journalReview articlepeer-review


Purpose: The new challenge in Artificial Intelligence (AI) is to understand the limitations of models to reduce potential harm. Particularly, unknown disparities based on demographic factors could encrypt currently existing inequalities worsening patient care for some groups. Methods: Following PRISMA guidelines, we present a systematic review of ‘fair’ deep learning modeling techniques for natural and medical image applications which were published between year 2011 to 2021. Our search used Covidence review management software and incorporates articles from PubMed, IEEE, and ACM search engines and three reviewers independently review the manuscripts. Results: Inter-rater agreement was 0.89 and conflicts were resolved by obtaining consensus between three reviewers. Our search initially retrieved 692 studies but after careful screening, our review included 22 manuscripts that carried four prevailing themes; ‘fair’ training dataset generation (4/22), representation learning (10/22), model disparity across institutions (5/22) and model fairness with respect to patient demographics (3/22). We benchmark the current literature regarding fairness in AI-based image analysis and highlighted the existing challenges. We observe that often discussion regarding fairness are limited to analyzing existing bias without further establishing methodologies to overcome model disparities. Conclusion: Based on the current research trends, exploration of adversarial learning for demographic/camera/institution agnostic models is an important direction to minimize disparity gaps for imaging. Privacy preserving approaches also present encouraging performance for both natural and medical image domain.

Original languageEnglish (US)
Pages (from-to)816-827
Number of pages12
JournalJournal of Medical and Biological Engineering
Issue number6
StatePublished - Dec 2022


  • Fairness in AI
  • Image classification
  • Medical imaging
  • Natural imaging

ASJC Scopus subject areas

  • Biomedical Engineering


Dive into the research topics of 'A Systematic Review of ‘Fair’ AI Model Development for Image Classification and Prediction'. Together they form a unique fingerprint.

Cite this