Application of a bayesian graded response model to characterize areas of disagreement between clinician and patient grading of symptomatic adverse events

Thomas M. Atkinson, Bryce B. Reeve, Amylou C. Dueck, Antonia V. Bennett, Tito R. Mendoza, Lauren J. Rogak, Ethan Basch, Yuelin Li

Research output: Contribution to journalArticlepeer-review

1 Scopus citations


Background: Traditional concordance metrics have shortcomings based on dataset characteristics (e.g., multiple attributes rated, missing data); therefore it is necessary to explore supplemental approaches to quantifying agreement between independent assessments. The purpose of this methodological paper is to apply an Item Response Theory (IRT)-based framework to an existing dataset that included unidimensional clinician and multiple attribute patient ratings of symptomatic adverse events (AEs), and explore the utility of this method in patient-reported outcome (PRO) and health-related quality of life (HRQOL) research. Methods: Data were derived from a National Cancer Institute-sponsored study examining the validity of a measurement system (PRO-CTCAE) for patient self-reporting of AEs in cancer patients receiving treatment (N = 940). AEs included 13 multiple attribute patient-reported symptoms that had corresponding unidimensional clinician AE grades. A Bayesian IRT Model was fitted to calculate the latent grading thresholds between raters. The posterior mean values of the model-fitted item responses were calculated to represent model-based AE grades obtained from patients and clinicians. Results: Model-based AE grades showed a general pattern of clinician underestimation relative to patient-graded AEs. However, the magnitude of clinician underestimation was associated with AE severity, such that clinicians’ underestimation was more pronounced for moderate/very severe model-estimated AEs, and less so with mild AEs. Conclusions: The Bayesian IRT approach reconciles multiple symptom attributes and elaborates on the patterns of clinician-patient non-concordance beyond that provided by traditional metrics. This IRT-based technique may be used as a supplemental tool to detect and characterize nuanced differences in patient-, clinician-, and proxy-based ratings of HRQOL and patient-centered outcomes. Trial registration: NCT01031641. Registered 1 December 2009.

Original languageEnglish (US)
Article number56
JournalJournal of Patient-Reported Outcomes
StatePublished - 2018


  • Clinician-patient agreement
  • Item response theory
  • Neoplasms
  • Patient-reported outcomes

ASJC Scopus subject areas

  • Health Informatics
  • Health Information Management


Dive into the research topics of 'Application of a bayesian graded response model to characterize areas of disagreement between clinician and patient grading of symptomatic adverse events'. Together they form a unique fingerprint.

Cite this