Validate: Trustworthy AI to Improve Acute Stroke Outcomes

Artificial intelligence (AI)-powered prognostic tools and clinical decision support systems have the ability to predict the outcome of certain diseases based on a multitude of patient data at high speed, facilitating decisions by healthcare professionals.

In acute ischemic stroke, the overall treatment effect and population-wide outcome benefit of IV thrombolysis and mechanical thrombectomy are well established. However, the outcome still differs significantly for individual patients, where some are eligible for treatment but may have catastrophic outcomes. Multiple prognostic variables and their combination in a single patient make it difficult to predict individual outcomes after stroke treatment. We aim to validate an AI prognostic tool to provide accurate outcome prediction in patients with acute ischemic stroke in a prospective observational follow-up study.

Hypothesis AI-based models can be applied in real-time in acute stroke patients and provide an early accurate prediction of their outcome.

Methodology The study complies two phases. Phase 1: retrospective study. While technological readiness will be achieved for the clinical validation study further model refinement on heterogeneous data will be performed based on existing models that have been developed on extensive high-quality medical data. VALIDATE will analyse retrospective databases from the 3 clinical sites involved in the study to test and validate the previously generated AI models. Encrypted data of all acute ischemic stroke patients admitted to the centres during 2018-2021 period will fed the AI models to validate the model's accuracy comparing the outcomes predicted by the AI modelling with those of the actual patients. The dataset includes demographics, baseline clinical characteristics, risk factors, neuroimaging data, acute treatments, clinical evaluation (National Institute of Health Stroke Scale (NIHSS)), functional evaluation at 3-6 months (mRS), patient reported outcome measures (PROMs), etc. The interaction between these data sets and the AI models will be done through a federated learning procedure, that is, the data will be analyzed on our hospital servers, and they will not be transferred to any other center.

Grading the contribution of the progressively complex diagnostic procedures to the AI models and establishing a set of the minimum relevant variables for the AI model able to accurately predict functional outcome will also be achieved.

Phase 3: Prospective multicenter observational shadowing study. consecutive acute stroke patients admitted to 3 high-volume comprehensive stroke centres will be evaluated. All patients will receive the usual stroke work-up and standard of care treatment according to the treating physicians. A shadow clinical researcher with no interaction with the treating physicians will recollect in vivo the data required by the AI modelling. These data will be obtained by filling of clinical data through an app and by connection with the electronical medical record of the patient to obtain additional baseline and neuroimaging data. The real outcomes of the patients will be measured through clinician and patient reported outcomes measurements (CROMs and PROMs), and they will be compared with the estimated outcomes according to the AI model. PROMS after 7 days, 1 and 3 months will be obtained through the implementation of an innovative nudging-based digital platform (NORA) to improve patient-clinician communication and follow-up. An electronic case report form (eCRF) will be designed to recollect key process indicators (KPI) and CROMs, which will be integrated in the NORA platform.

The sample size calculation has been based on the results of a clinical dataset of consecutive code stroke patients admitted to Hospital Vall d'Hebron during the year 2020. It has been used as an example of the usual mRS distribution at 3 months in a cohort of consecutive acute stroke patients.

In a test for agreement between two raters using the Kappa statistic, a sample size of 182 subjects achieves 95% power to detect a true Kappa value of 0,7 in a test of H0: Kappa = κ0 vs. H1: Kappa ≠ κ0 when there are three categories with frequencies equal to 0,58, 0,34, and 0,08. This power calculation is based on a significance level of 0,05 and a minimum acceptable kappa (κ0) of 0.6 (moderate agreement) and an expected kappa (κ1) of 0.8 (substantial agreement). Assuming a drop-out rate of 20% for the 90-day follow-up the Dropout-Inflated Expected Enrolment (DIEE) Number would be 218 patients with acute ischemic stroke.