Training size predictably improves machine learning-based epileptic seizure forecasting from wearables

Abstract

Objective: Wrist-worn wearable devices that monitor autonomous nervous system function and movement have shown promise in providing non-invasive, broadly applicable seizure forecasts that increase in accuracy with larger training size. Nevertheless, challenges related to missing validation, small number of enrolled patients, insufficient training data, and lack of patient seizure cycles data hinder its clinical implementation. Here we sought to prospectively validate a previously implemented seizure forecasting algorithm using a larger cohort of pediatric patients with epilepsy (pwe), improve it by including information on seizure cycles, and (3) assess the utility of precise power-laws to predict performance as a function of dataset size.

Methods: We used video-EEG recordings from 166 pwe as ground-truth for seizures, recorded electrodermal activity (EDA), peripheral body temperature (TEMP), blood volume pulse (BVP), accelerometery (ACC) and applied a deep neural LSTM network model (NN) on these data along with information on 24-hour cycles to forecast seizures in a leave-one-subject-out cross validation. Evaluations were made using improvement over chance (IoC) and the Brier skill score (BSS), which measured the improvement of the NN Brier score compared to the Brier score of a rate-matched random (RMR) forecast.

Results: Performance quantified by IoC and BSS increased with training data following precise power-law scaling laws, thereby exceeding prior reported performance levels from smaller datasets. Including information on 24-hour seizure cycles further improved performance. For the largest training set we achieved significant IoC in 68% of pwe, an IoC of 27.3% and a BSS of 0.087.

Interpretation: Our results validate a previous forecast approach and indicate that performance improves predictably as a function of dataset size following power-law scaling.