Evelio González-González, Iván Castilla-Rodríguez, Joel Aday Dorta-Hernández
{"title":"Ontology-driven generation of parameters for health technology assessment models: a prompt engineering study.","authors":"Evelio González-González, Iván Castilla-Rodríguez, Joel Aday Dorta-Hernández","doi":"10.1017/S0266462326103754","DOIUrl":null,"url":null,"abstract":"<p><strong>Objectives: </strong>Ontologies support transparent and reproducible conceptual modeling in Health Technology Assessment (HTA), but their population remains resource-intensive and reliant on expert input. This study evaluates the feasibility, reliability, and methodological implications of using generative artificial intelligence (GenAI) to populate ontology individuals for HTA applications.</p><p><strong>Methods: </strong>A factorial experimental framework was developed using the Ontology for Simulation Modeling (OSDi) and three HTA-relevant use cases of varying complexity. Two GenAI systems were evaluated under multiple experimental conditions, including prompting strategy, serialization format, and provision of supporting information. Generated ontology individuals were validated by an HTA expert and assessed across four quality dimensions: consistency, relevance, completeness, and adequacy. Multivariate and regression analyses were conducted to examine the effects of experimental factors on quality outcomes and hallucination likelihood.</p><p><strong>Results: </strong>GenAI systems successfully generated ontology individuals across use cases, although performance varied by quality dimension and experimental condition. Iterative prompting significantly improved completeness, while serialization format strongly influenced reliability, with Turtle serialization associated with substantially lower hallucination likelihood compared with XML. Other factors showed dimension-specific effects, highlighting the multidimensional nature of ontology quality. Errors occurred more frequently in structurally complex ontology components, suggesting a relationship between ontological complexity and generative performance.</p><p><strong>Conclusions: </strong>GenAI-assisted ontology population can enhance the efficiency and scalability of HTA conceptual modeling, enhancing the agility of HTA agencies in exploratory phases. Its effective use requires structured prompting, appropriate representation formats, and expert validation. Further research should evaluate its impact on HTA decision modeling workflows and validation frameworks.</p>","PeriodicalId":14467,"journal":{"name":"International Journal of Technology Assessment in Health Care","volume":" ","pages":"e47"},"PeriodicalIF":3.0000,"publicationDate":"2026-04-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13161931/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Technology Assessment in Health Care","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1017/S0266462326103754","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"HEALTH CARE SCIENCES & SERVICES","Score":null,"Total":0}
引用次数: 0
Abstract
Objectives: Ontologies support transparent and reproducible conceptual modeling in Health Technology Assessment (HTA), but their population remains resource-intensive and reliant on expert input. This study evaluates the feasibility, reliability, and methodological implications of using generative artificial intelligence (GenAI) to populate ontology individuals for HTA applications.
Methods: A factorial experimental framework was developed using the Ontology for Simulation Modeling (OSDi) and three HTA-relevant use cases of varying complexity. Two GenAI systems were evaluated under multiple experimental conditions, including prompting strategy, serialization format, and provision of supporting information. Generated ontology individuals were validated by an HTA expert and assessed across four quality dimensions: consistency, relevance, completeness, and adequacy. Multivariate and regression analyses were conducted to examine the effects of experimental factors on quality outcomes and hallucination likelihood.
Results: GenAI systems successfully generated ontology individuals across use cases, although performance varied by quality dimension and experimental condition. Iterative prompting significantly improved completeness, while serialization format strongly influenced reliability, with Turtle serialization associated with substantially lower hallucination likelihood compared with XML. Other factors showed dimension-specific effects, highlighting the multidimensional nature of ontology quality. Errors occurred more frequently in structurally complex ontology components, suggesting a relationship between ontological complexity and generative performance.
Conclusions: GenAI-assisted ontology population can enhance the efficiency and scalability of HTA conceptual modeling, enhancing the agility of HTA agencies in exploratory phases. Its effective use requires structured prompting, appropriate representation formats, and expert validation. Further research should evaluate its impact on HTA decision modeling workflows and validation frameworks.
期刊介绍：
International Journal of Technology Assessment in Health Care serves as a forum for the wide range of health policy makers and professionals interested in the economic, social, ethical, medical and public health implications of health technology. It covers the development, evaluation, diffusion and use of health technology, as well as its impact on the organization and management of health care systems and public health. In addition to general essays and research reports, regular columns on technology assessment reports and thematic sections are published.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
