The "Artificial Intelligence Statistician": Utilizing Generative Artificial Intelligence to Select an Appropriate Model and Execute Network Meta-Analyses.
Tim Reason, Yunchou Wu, Cheryl Jones, Emma Benbow, Kasper Johannesen, Bill Malcolm
{"title":"The \"Artificial Intelligence Statistician\": Utilizing Generative Artificial Intelligence to Select an Appropriate Model and Execute Network Meta-Analyses.","authors":"Tim Reason, Yunchou Wu, Cheryl Jones, Emma Benbow, Kasper Johannesen, Bill Malcolm","doi":"10.1016/j.jval.2025.08.001","DOIUrl":null,"url":null,"abstract":"<p><strong>Objectives: </strong>This exploratory study aimed to develop a large language model (LLM)-based process to automate components of network meta-analysis (NMA), including model selection, analysis, output evaluation, and results interpretation. Automating these tasks with LLMs can enhance efficiency, consistency, and scalability in health economics and outcomes research, while ensuring that analyses adhere to established guidelines required by health technology assessment agencies. Improvements in efficiency and scalability may potentially become relevant as the European Union Health Technology Assessment Regulation comes into force, given anticipated analysis requirements and timelines.</p><p><strong>Methods: </strong>Using Claude 3.5 Sonnet (V2), a process was designed to automate statistical model selection, NMA output evaluation, and results interpretation based on an \"analysis-ready\" data set. Validation was assessed by replicating examples from the National Institute for Health and Care Excellence Technical Support Document (TSD2), replicating results of non-Decision Support Unit-published NMAs, and generating comprehensive outputs (eg, heterogeneity, inconsistency, and convergence).</p><p><strong>Results: </strong>The automated LLM-based process produced accurate results. Compared with TSD2 examples, differences were minimal, within expectations (given differences in sampling frameworks used), and comparable to those observed between estimates produced by the R vignettes against TSD2. Similar consistency was noted for non-Decision Support Unit-published NMA examples. Additionally, the LLM process generated and interpreted comprehensive NMA outputs.</p><p><strong>Conclusions: </strong>This exploratory study demonstrates the feasibility of LLMs to automate key components of NMAs, determining the requisite NMA framework based only on input data. Further exploring these capabilities could clarify their role in streamlining NMA workflows.</p>","PeriodicalId":23508,"journal":{"name":"Value in Health","volume":" ","pages":""},"PeriodicalIF":6.0000,"publicationDate":"2025-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Value in Health","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1016/j.jval.2025.08.001","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ECONOMICS","Score":null,"Total":0}
引用次数: 0
Abstract
Objectives: This exploratory study aimed to develop a large language model (LLM)-based process to automate components of network meta-analysis (NMA), including model selection, analysis, output evaluation, and results interpretation. Automating these tasks with LLMs can enhance efficiency, consistency, and scalability in health economics and outcomes research, while ensuring that analyses adhere to established guidelines required by health technology assessment agencies. Improvements in efficiency and scalability may potentially become relevant as the European Union Health Technology Assessment Regulation comes into force, given anticipated analysis requirements and timelines.
Methods: Using Claude 3.5 Sonnet (V2), a process was designed to automate statistical model selection, NMA output evaluation, and results interpretation based on an "analysis-ready" data set. Validation was assessed by replicating examples from the National Institute for Health and Care Excellence Technical Support Document (TSD2), replicating results of non-Decision Support Unit-published NMAs, and generating comprehensive outputs (eg, heterogeneity, inconsistency, and convergence).
Results: The automated LLM-based process produced accurate results. Compared with TSD2 examples, differences were minimal, within expectations (given differences in sampling frameworks used), and comparable to those observed between estimates produced by the R vignettes against TSD2. Similar consistency was noted for non-Decision Support Unit-published NMA examples. Additionally, the LLM process generated and interpreted comprehensive NMA outputs.
Conclusions: This exploratory study demonstrates the feasibility of LLMs to automate key components of NMAs, determining the requisite NMA framework based only on input data. Further exploring these capabilities could clarify their role in streamlining NMA workflows.
期刊介绍：
Value in Health contains original research articles for pharmacoeconomics, health economics, and outcomes research (clinical, economic, and patient-reported outcomes/preference-based research), as well as conceptual and health policy articles that provide valuable information for health care decision-makers as well as the research community. As the official journal of ISPOR, Value in Health provides a forum for researchers, as well as health care decision-makers to translate outcomes research into health care decisions.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
