Automated CT angiography collateral scoring in anterior large vessel occlusion stroke: A multireader study

Mohamed Sobhi Jabal, David F. Kallmes, George Harston, Norbert Campeau, Kara Schwartz, Steven Messina, Carrie Carr, John Benson, Jason Little, Alex Nagelschneider, Ajay Madhavan, Deena Nasr, Sherry Braksick, James Klaas, Eugene Scharf, Cem Bilgin, Waleed Brinjikji

Research output: Contribution to journalArticlepeer-review


Background: Computed tomography (CT) angiography collateral score (CTA-CS) is an important clinical outcome predictor following mechanical thrombectomy for ischemic stroke with large vessel occlusion (LVO). The present multireader study aimed to evaluate the performance of e-CTA software for automated assistance in CTA-CS scoring. Materials and Methods: Brain CTA images of 56 patients with anterior LVO were retrospectively processed. Twelve readers of various clinical training, including junior neuroradiologists, senior neuroradiologists, and neurologists graded collateral flow using visual CTA-CS scale in two sessions separated by a washout period. Reference standard was the consensus of three expert readers. Duration of reading time, inter-rater reliability, and statistical comparison of readers’ performance metrics were analyzed between the e-CTA assisted and unassisted sessions. Results: e-CTA assistance resulted in significant increase in mean accuracy (58.6% to 67.5%, p = 0.003), mean F1 score (0.574 to 0.676, p = 0.002), mean precision (58.8% to 68%, p = 0.007), and mean recall (58.7% to 69.9%, p = 0.002), especially with slight filling deficit (CTA-CS 2 and 3). Mean reading time was reduced across all readers (103.4 to 59.7 s, p = 0.001), and inter-rater agreement in CTA-CS assessment was increased (Krippendorff's alpha 0.366 to 0.676). Optimized occlusion laterality detection was also noted with mean accuracy (92.9% to 96.8%, p = 0.009). Conclusion: Automated assistance for CTA-CS using e-CTA software provided helpful decision support for readers in terms of improving scoring accuracy and reading efficiency for physicians with a range of experience and training backgrounds and leading to significant improvements in inter-rater agreement.

Original languageEnglish (US)
JournalInterventional Neuroradiology
StateAccepted/In press - 2023


  • artificial intelligence
  • computed tomography angiography
  • Decision support system
  • ischemic stroke
  • machine learning

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging
  • Clinical Neurology
  • Cardiology and Cardiovascular Medicine


Dive into the research topics of 'Automated CT angiography collateral scoring in anterior large vessel occlusion stroke: A multireader study'. Together they form a unique fingerprint.

Cite this