Performance evaluation of neural networks and support-vector regression for lesion-symptom mapping in cerebral small vessel disease

Abstract

Introduction

Lesion-symptom mapping (LSM) is used to capture the impact of lesions on cognitive performance while accounting for their location in the brain and is highly relevant for vascular cognitive impairment [1], [2]. Current LSM methods only consider a single lesion type and single cognitive score at a time. Neural networks (NNs) allow for multiple inputs (lesions) and outputs (cognitive domains), which might take interrelations of vascular lesions and cognitive subscores into account. Explainable AI (XAI) can be used to compute attribution maps reflecting which image locations are deemed important for a NN, even at an individual level. We explore the feasibility of NNs and XAI for LSM by comparing two NNs with current gold standard, support vector regression (SVR) [3], [4].

Methods

White matter hyperintensity segmentations from 821 patients in the TRACE-VCI dataset were used to develop a simulation study similar to [3]. Three regions of interest (ROIs) were defined within the lesion prevalence map. Lesion volume fractions within each ROI were calculated and summed to create an artificial cognitive score with a known source location. A linear NN, a convolutional NN (CNN), and SVR were used to predict the artificial scores and determine responsible ROI locations. Predictive performance was quantified using the coefficient of determination, while ROI identification was evaluated using the precision-recall metric based on the attribution maps of each method: SVR's β-map, the linear NN's weight map, and the CNN XAI saliency map. The XAI saliency map was computed by occluding parts of the image: the predicted outcome changes considerably for relevant locations and remains unchanged when background is occluded [5].

Results

SVR and both NNs have similar predictive performance, all reaching R^2>0.9 (Fig. 1). However, attribution maps (Fig. 2) show differences in ROI location determination, which is reflected by the precision-recall curves. The curves in Fig. 3 show that SVR has overall better precision and recall (AUC=0.761), followed by the CNN (AUC=0.582) and the linear NN (AUC=0.203).

Discussion

In this first exploration, the CNN with XAI did not outperform SVR, but it proved able to detect relevant lesion locations, thereby showing potential for LSM.