Identifying homogenous patient subgroups using transformer based hierarchical clustering of heterogeneous Mixed-Modality medical data

Abstract

Objective

Patients are highly heterogeneous, with varying needs and responses to treatment. Identifying clinically homogenous patient subgroups is critical to improve personalized care. Patient records are often heterogeneous, may include multiple modalities which conventionally require separate data processing considerations, and are often incomplete, leading to difficulties in identifying meaningful clusters of patients.

Methods

We introduce a Med-ROAR, a transformer-based Random Order AutoRegressive (ROAR) embedding model for medical data. Med-ROAR hierarchically clusters data by encoding it into hierarchical discrete embeddings using a modified self-attention operation to facilitate random order mixed modality autoregressive modeling. This allows the model to accept arbitrary mixes of record types without special considerations. We compare our method’s clustering effectiveness to standard agglomerative clustering using 147,469 individuals diagnosed with Autism Spectrum Disorder (ASD). We also evaluate its use on data with mixed modalities and its resilience to missing information using 50,458 clinical records from Intensive Care Unit (ICU) patients which include both tabular and time-series components.

Results

We demonstrate that Med-ROAR is more likely to discover more cohesive high-level clusters than distance-based methods like agglomerative clustering. Our exploratory analysis of the autism data identifies clinically meaningful patterns of phenotypes within ASD. We identify homogenous, but atypical, patient subgroups within the ASD population. We also demonstrate Med-ROAR’s effectiveness in clustering patients using mixes of both tabular and time series clinical records from ICU patients. We demonstrate that Med-ROAR can predict patient subgroups even using incomplete, preliminary information collected shortly after admission.

Conclusion

Med-ROAR is a flexible hierarchical clustering technique which learns to cluster patients based on learned high-level semantic similarities rather than rule-based metrics. It can accept whatever patient data may be available without modification to the underlying model architecture. The data modalities which Med-ROAR can accept are primarily constrained by computational resources, rather than architectural limitations.