Impact of e-ASPECTS software on the performance of physicians compared to a consensus ground truth: a multi-reader, multi-case study

Background: The Alberta Stroke Program Early CT Score (ASPECTS) is used to quantify the extent of injury to the brain following acute ischemic stroke (AIS) and to inform treatment decisions. The e-ASPECTS software uses artificial intelligence to automatically process non-contrast CT (NCCT) brain scans and generate an ASPECTS. This study (Mayo Clinic) aimed to evaluate the impact of e-ASPECTS (Brainomix, Oxford, UK) on the performance of US physicians compared to a consensus ground truth.

Methods: The study used a multi-reader, multi-case design. A total of 10 US board-certified physicians (neurologists and neuroradiologists) scored 54 NCCT brain scans of patients with AIS affecting the MCA territory, each on two occasions: once with and once without reference to the e-ASPECTS software, in random order. Agreement with a reference standard was evaluated with and without software support.

Results: The area under the curve improved significantly from 0.81 to 0.83 (p = 0.028) with e-ASPECTS support. Agreement with the reference standard improved: Cohen's kappa from 0.60 to 0.65, and the case-based weighted kappa from 0.70 to 0.81.

Conclusion: Decision support with the e-ASPECTS software significantly improves the accuracy of ASPECTS scoring, even by expert US neurologists and neuroradiologists.