Extracting seizure control metrics from clinic notes of patients with epilepsy: A natural language processing approach

IF 2 4区医学 Q3 CLINICAL NEUROLOGY

Epilepsy Research Pub Date : 2024-09-10 DOI:10.1016/j.eplepsyres.2024.107451

{"title":"Extracting seizure control metrics from clinic notes of patients with epilepsy: A natural language processing approach","authors":"","doi":"10.1016/j.eplepsyres.2024.107451","DOIUrl":null,"url":null,"abstract":"<div><h3>Objectives</h3><p>Monitoring seizure control metrics is key to clinical care of patients with epilepsy. Manually abstracting these metrics from unstructured text in electronic health records (EHR) is laborious. We aimed to abstract the date of last seizure and seizure frequency from clinical notes of patients with epilepsy using natural language processing (NLP).</p></div><div><h3>Methods</h3><p>We extracted seizure control metrics from notes of patients seen in epilepsy clinics from two hospitals in Boston. Extraction was performed with the pretrained model RoBERTa_for_seizureFrequency_QA, for both date of last seizure and seizure frequency, combined with regular expressions. We designed the algorithm to categorize the timing of last seizure (“today”, “1–6 days ago”, “1–4 weeks ago”, “more than 1–3 months ago”, “more than 3–6 months ago”, “more than 6–12 months ago”, “more than 1–2 years ago”, “more than 2 years ago”) and seizure frequency (“innumerable”, “multiple”, “daily”, “weekly”, “monthly”, “once per year”, “less than once per year”). Our ground truth consisted of structured questionnaires filled out by physicians. Model performance was measured using the areas under the receiving operating characteristic curve (AUROC) and precision recall curve (AUPRC) for categorical labels, and median absolute error (MAE) for ordinal labels, with 95 % confidence intervals (CI) estimated via bootstrapping.</p></div><div><h3>Results</h3><p>Our cohort included 1773 adult patients with a total of 5658 visits with reported seizure control metrics, seen in epilepsy clinics between December 2018 and May 2022. The cohort average age was 42 years old, the majority were female (57 %), White (81 %) and non-Hispanic (85 %). The models achieved an MAE (95 % CI) for date of last seizure of 4 (4.00–4.86) weeks, and for seizure frequency of 0.02 (0.02–0.02) seizures per day.</p></div><div><h3>Conclusions</h3><p>Our NLP approach demonstrates that the extraction of seizure control metrics from EHR is feasible allowing for large-scale EHR research.</p></div>","PeriodicalId":11914,"journal":{"name":"Epilepsy Research","volume":null,"pages":null},"PeriodicalIF":2.0000,"publicationDate":"2024-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Epilepsy Research","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0920121124001669","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"CLINICAL NEUROLOGY","Score":null,"Total":0}

引用次数: 0

Abstract

Objectives

Monitoring seizure control metrics is key to clinical care of patients with epilepsy. Manually abstracting these metrics from unstructured text in electronic health records (EHR) is laborious. We aimed to abstract the date of last seizure and seizure frequency from clinical notes of patients with epilepsy using natural language processing (NLP).

Methods

We extracted seizure control metrics from notes of patients seen in epilepsy clinics from two hospitals in Boston. Extraction was performed with the pretrained model RoBERTa_for_seizureFrequency_QA, for both date of last seizure and seizure frequency, combined with regular expressions. We designed the algorithm to categorize the timing of last seizure (“today”, “1–6 days ago”, “1–4 weeks ago”, “more than 1–3 months ago”, “more than 3–6 months ago”, “more than 6–12 months ago”, “more than 1–2 years ago”, “more than 2 years ago”) and seizure frequency (“innumerable”, “multiple”, “daily”, “weekly”, “monthly”, “once per year”, “less than once per year”). Our ground truth consisted of structured questionnaires filled out by physicians. Model performance was measured using the areas under the receiving operating characteristic curve (AUROC) and precision recall curve (AUPRC) for categorical labels, and median absolute error (MAE) for ordinal labels, with 95 % confidence intervals (CI) estimated via bootstrapping.

Results

Our cohort included 1773 adult patients with a total of 5658 visits with reported seizure control metrics, seen in epilepsy clinics between December 2018 and May 2022. The cohort average age was 42 years old, the majority were female (57 %), White (81 %) and non-Hispanic (85 %). The models achieved an MAE (95 % CI) for date of last seizure of 4 (4.00–4.86) weeks, and for seizure frequency of 0.02 (0.02–0.02) seizures per day.

Conclusions

Our NLP approach demonstrates that the extraction of seizure control metrics from EHR is feasible allowing for large-scale EHR research.

查看原文本刊更多论文

从癫痫患者的门诊记录中提取发作控制指标：自然语言处理方法

目标监测癫痫发作控制指标是癫痫患者临床治疗的关键。从电子健康记录（EHR）中的非结构化文本中手动抽取这些指标非常费力。我们的目标是使用自然语言处理（NLP）从癫痫患者的临床笔记中抽取最后一次发作的日期和发作频率。我们使用预训练模型 RoBERTa_for_seizureFrequency_QA，结合正则表达式对最后一次发作日期和发作频率进行了提取。我们设计的算法可对最后一次发作的时间（"今天"、"1-6 天前"、"1-4 周前"、"1-3 个多月前"、"3-6 个多月前"、"6-12 个多月前"、"1-2 年多前"、"2 年多前"）和发作频率（"无数次"、"多次"、"每天"、"每周"、"每月"、"每年一次"、"每年少于一次"）进行分类。我们的基本事实由医生填写的结构化问卷组成。对于分类标签，我们使用接收操作特征曲线下面积（AUROC）和精确召回曲线（AUPRC）来衡量模型性能；对于序数标签，我们使用中位绝对误差（MAE）来衡量模型性能，并通过引导法估算出 95% 的置信区间（CI）。结果我们的队列包括 2018 年 12 月至 2022 年 5 月期间在癫痫诊所就诊的 1773 名成年患者，他们共就诊 5658 次，报告了癫痫发作控制指标。队列平均年龄为 42 岁，大多数为女性（57%）、白人（81%）和非西班牙裔（85%）。模型在最后一次癫痫发作日期为 4 (4.00-4.86) 周和癫痫发作频率为每天 0.02 (0.02-0.02) 次方面达到了 MAE (95 % CI)。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Epilepsy Research 医学-临床神经学

CiteScore

0.10

自引率

4.50%

发文量

143

审稿时长

62 days

期刊介绍： Epilepsy Research provides for publication of high quality articles in both basic and clinical epilepsy research, with a special emphasis on translational research that ultimately relates to epilepsy as a human condition. The journal is intended to provide a forum for reporting the best and most rigorous epilepsy research from all disciplines ranging from biophysics and molecular biology to epidemiological and psychosocial research. As such the journal will publish original papers relevant to epilepsy from any scientific discipline and also studies of a multidisciplinary nature. Clinical and experimental research papers adopting fresh conceptual approaches to the study of epilepsy and its treatment are encouraged. The overriding criteria for publication are novelty, significant clinical or experimental relevance, and interest to a multidisciplinary audience in the broad arena of epilepsy. Review articles focused on any topic of epilepsy research will also be considered, but only if they present an exceptionally clear synthesis of current knowledge and future directions of a research area, based on a critical assessment of the available data or on hypotheses that are likely to stimulate more critical thinking and further advances in an area of epilepsy research.