BMC Medical Research Methodology最新文献

{"title":"Correction: Grey literature scoping review: a synthesis of the application of participatory methodologies in underrepresented groups at an elevated risk of dementia.","authors":"Katrina Messiha, Nicole Thomas, Carol Brayne, Danielle M Agnello, Lea R Delfmann, Maria Giné-Garriga, Sonia Lippke, John Downey","doi":"10.1186/s12874-025-02614-1","DOIUrl":"10.1186/s12874-025-02614-1","url":null,"abstract":"","PeriodicalId":9114,"journal":{"name":"BMC Medical Research Methodology","volume":"25 1","pages":"160"},"PeriodicalIF":3.9,"publicationDate":"2025-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144511499","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Comparison of total event analysis and first event analysis in relation to heterogeneity in cardiovascular trials.","authors":"Shun-Fu Lee, Chinthanie Ramasundarahettige, Hertzel C Gerstein, William F McIntyre, John Eikelboom, Martin J O'Donnell, Yueci Zhou, Shrikant I Bangdiwala, Lehana Thabane","doi":"10.1186/s12874-025-02593-3","DOIUrl":"10.1186/s12874-025-02593-3","url":null,"abstract":"<p><strong>Background: </strong>In cardiovascular (CV) trials, analyzing the total number of events, rather than just time-to-first event, enhances understanding of participants' health. Adapting Cox models to account for between-subject heterogeneity in multiple events and understanding its impact plays crucial roles in total event analysis.</p><p><strong>Method: </strong>This study compares effect sizes from first event and total event analyses in three cardiovascular trials: ORIGIN (N = 12,537, median follow-up of 6.2 years), COMPASS (N = 18,278, median follow-up of 1.8 years), TRANSCEND (N = 5,926, median follow-up of 1.1 years). It also examines the impact of heterogeneity, measured by the negative binomial overdispersion parameter. Treatment effects were assessed using the Cox model for first events and the negative binomial (NB), Andersen-Gill (AG), Prentice-Williams-Peterson (PWP), Wei-Lin-Weissfeld (WLW), and Lin-Wei-Yang-Ying (LWYY) models for total events. Hazard ratios (HRs) or risk ratios (RRs), 95% confidence intervals (CIs), and CI widths were reported. The risk ratio applies to negative binomial. The first composite was consisted of myocardial infarction (MI), stroke, cardiovascular death. Simulations assessed Type I error, power, and mean squared error across the different approaches.</p><p><strong>Results: </strong>In ORIGIN, the incidence per 100 years increased from 2.9 to 3.8 for the first composite with a heterogeneity of 2.4. The HR or RR for the first composite was 1.03 (95% CI, 0.94-1.12, CI width = 0.18) using Cox, 1.01 (95% CI, 0.92-1.11, CI width = 0.19) for NB, 1.01 (95% CI, 0.94-1.09, CI width = 0.15) for AG, 1.02 (95% CI, 0.94-1.10, CI width = 0.16) for PWP total, 1.01 (95% CI, 0.94-1.09, CI width = 0.15) for PWP gap, 1.03 (95% CI, 0.94-1.12, CI width = 0.18) for WLW and 1.01 (95% CI, 0.92-1.11, CI width = 0.19) for LWYY. Similar trends were observed in other studies. Our simulation results showed that total event approaches had approximately 5% higher power than the Cox model, though power declined exponentially across all methods with increasing heterogeneity. Among the total event methods, AG, PWP gap, and LWYY demonstrated better power, with AG and LWYY also achieving the smallest mean squared error (MSE).</p><p><strong>Conclusions: </strong>High heterogeneity arises when a small number of patients experience a disproportionately large number of events. This effect is more pronounced when the overall event incidence is low and few patients experience any events. The effect size and CI width stayed consistent with low heterogeneity across different approaches. Power decreased with high heterogeneity. The AG and LWYY approaches slightly outperformed the other approaches.</p><p><strong>Clinical trial registration: </strong>ORIGIN (NCT00069784), COMPASS (NCT01776424), TRANSCEND (NCT00153101).</p>","PeriodicalId":9114,"journal":{"name":"BMC Medical Research Methodology","volume":"25 1","pages":"159"},"PeriodicalIF":3.9,"publicationDate":"2025-06-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12147344/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144257322","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"VAS value set still has its own application value compare with TTO value set of EQ-5D-3L in Chinese population.","authors":"Lin Zhuo, Siyuan Gao, Rui Jin, Lang Zhuo, Xiuying Wang","doi":"10.1186/s12874-025-02609-y","DOIUrl":"10.1186/s12874-025-02609-y","url":null,"abstract":"<p><strong>Objective: </strong>To develop a Chinese visual analogue scale (VAS) value set for Europe Quality of Life Questionnaire (EQ-5D-3L) and compare it with the time trade-off (TTO) value set which was constructed based on the same sample.</p><p><strong>Methods: </strong>An adapted Measurement and Valuation of Health (MVH) protocol was applied with VAS method. EQ-5D-3L was used in face-to-face interviews conducted by trained interviewers with participants selected via multi-stage stratified clustered random sample. Fifteen hypothetical health statuses (11 random states in MVH protocol, plus full health, severe problems for all dimensions, unconscious, and death) were assigned for assessment individually. Ordinary least square (OLS), general least square (GLS), and weighted least square (WLS) models were constructed. Five categories of indices, including quality of original data, distribution of rescaled values, goodness of fit of models, distribution of predicted values, and dimensions order were adopted to compare between Chinese VAS and TTO value sets.</p><p><strong>Results: </strong>All 5,939 participants aged 15 and over were completely interviewed; 5,884 eligible participants were included in constructing models. Model 2 was the best for having 4 out of 7 indices of goodness of fit. Comparing with the TTO value set, Adjusted R-square of Model2 increased from 0.354 to 0.670;the mean absolute error decreased from 0.0838 to 0.0319; and the Pearson correlation coefficient between predicted and mean values increased from0.8989 to 0.9837. Model 2 gave uniformly lower values than Chinese TTO value set. VAS method had a higher responsive rate, less inconsistency, lower skewed values, and better goodness of fit values.</p><p><strong>Conclusions: </strong>We recommend VAS value set, Model 2, as the reference of scoring algorithm when using EQ-5D-3L in large scale survey of Chinese population.</p>","PeriodicalId":9114,"journal":{"name":"BMC Medical Research Methodology","volume":"25 1","pages":"158"},"PeriodicalIF":3.9,"publicationDate":"2025-06-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12147354/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144257323","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Power and sample size calculation for non-inferiority trials with treatment switching in intention-to-treat analysis comparing RMSTs.","authors":"Austin Shih, Chih-Yuan Hsu, Yu Shyr","doi":"10.1186/s12874-025-02604-3","DOIUrl":"10.1186/s12874-025-02604-3","url":null,"abstract":"<p><strong>Background: </strong>Difference in Restricted Mean Survival Time (DRMST) has attracted attention and is increasingly used in non-inferiority (NI) trials because of its superior power in detecting treatment effects compared to hazard ratio. However, when treatment switching (also known as crossover) occurs, the widely used intention-to-treat (ITT) analysis can underpower or overpower NI trials.</p><p><strong>Methods: </strong>We propose a simulation-based approach, named nifts, to calculate powers and determine the necessary sample size to achieve a desired power for non-inferiority trials that allow treatment switching, in ITT analysis using DRMST.</p><p><strong>Results: </strong>The nifts approach offers three options for a non-inferiority margin, assumes three entry patterns and generalized gamma distributions for event time, incorporates two distributions for dropout censoring, and provides five distribution options for switching. Real-world and simulated examples are used to illustrate the proposed method and examine how switching probability, switching time, the relative effectiveness of treatments, allocation ratio, entry patterns, and event time distribution influence powers and sample sizes. nifts adjusts the non-inferiority margins intended for NI trials without treatment switching to accommodate the presence of treatment switching in the designs. With the adjusted margins, the type I errors are well-controlled. The ratios of sample sizes with treatment switching to those without switching are close to 1, indicating no significant change in power at sample sizes without switching when using adjusted margins. The performance on power and sample sizes is not sensitive to the choice of switch time distributions.</p><p><strong>Conclusions: </strong>This simulation-based approach provides power and sample size calculation in NI trials with treatment switching, when comparing the RMSTs of two treatment groups in ITT analysis. With its comprehensive parameter settings, nifts will be useful for designing NI trials that allow for treatment switching. nifts is freely available at https://github.com/cyhsuTN/nifts .</p>","PeriodicalId":9114,"journal":{"name":"BMC Medical Research Methodology","volume":"25 1","pages":"157"},"PeriodicalIF":3.9,"publicationDate":"2025-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12144734/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144246397","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Agreement and utility of coded primary and secondary care data for long-term follow-up of clinical trial outcomes.","authors":"Ariel Wang, Anna E Seeley, Matthew R Sydes, Nicholas Jones, Simon de Lusignan, Fd Richard Hobbs, Richard J McManus, Marney Williams, James P Sheppard","doi":"10.1186/s12874-025-02606-1","DOIUrl":"10.1186/s12874-025-02606-1","url":null,"abstract":"<p><strong>Background: </strong>Whilst interest in efficient trial design has grown with the use of electronic health records (EHRs) to collect trial outcomes, practical challenges remain. Commonly raised concerns often revolve around data availability, data quality and issues with data validation. This study aimed to assess the agreement between data collected on clinical trial participants from different sources to provide empirical evidence on the utility of EHRs for follow-up in randomised controlled trials (RCTs).</p><p><strong>Methods: </strong>This retrospective, participant-level data utility comparison study was undertaken using data collected as part of a UK primary care-based, randomised controlled trial (OPTiMISE). The primary outcome measure was the recording of all-cause hospitalisation or mortality within 3 years post-randomisation and was assessed across (1) Coded primary care data; (2) Coded-plus-free-text primary care data; and (3) Coded secondary care and mortality data. Agreement levels across data sources were assessed using Fleiss' Kappa (K). Kappa statistics were interpreted using an established framework, categorising agreement strength as follows: <0 (poor), 0.00-0.20 (slight), 0.21-0.40 (fair), 0.41-0.60 (moderate), 0.61-0.80 (substantial), and 0.81-1.00 (almost perfect) agreement. The impact of using different data sources to determine trial outcomes was assessed by replicating the trial's original analyses.</p><p><strong>Results: </strong>Almost perfect agreement was observed for mortality outcome across the three data sources (K = 0.94, 95%CI 0.91-0.98). Fair agreement (weak consistency) was observed for hospitalisation outcomes, including all-cause hospitalisation or mortality (K = 0.35, 95%CI 0.28-0.42), emergency hospitalisation (K = 0.39, 95%CI 0.33-0.46), and hospitalisation or mortality due to cardiovascular disease (K = 0.32, 95%CI 0.19-0.45). The overall trial results remained consistent across data sources for the primary outcome, albeit with varying precision.</p><p><strong>Conclusion: </strong>Significant discrepancies according to data sources were observed in recording of secondary care outcomes. Investigators should be cautious when choosing which data source(s) to use to measure outcomes in trials. Future work on linking participant-level data across healthcare settings should consider the variations in diagnostic coding practices. Standardised definitions for outcome measures when using healthcare systems data and using data from different data sources for cross-checking and verification should be encouraged.</p>","PeriodicalId":9114,"journal":{"name":"BMC Medical Research Methodology","volume":"25 1","pages":"156"},"PeriodicalIF":3.9,"publicationDate":"2025-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12144831/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144246395","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"DAGSLAM: causal Bayesian network structure learning of mixed type data and its application in identifying disease risk factors.","authors":"Yuanyuan Zhao, Jinzhu Jia","doi":"10.1186/s12874-025-02582-6","DOIUrl":"10.1186/s12874-025-02582-6","url":null,"abstract":"<p><strong>Background: </strong>Identifying and understanding disease risk factors is crucial in epidemiology, particularly for chronic and noncommunicable diseases that often have complex interrelationships. Traditional statistical methods struggle to capture these complexities, necessitating more sophisticated analytical frameworks. Bayesian networks and directed acyclic graphs (DAGs) provide powerful tools for exploring the complex relationships between variables. However, existing DAG structure learning algorithms still have limitations in handling mixed-type data (including continuous and discrete variables), which restricts their practical utility. Therefore, developing DAG structure learning methods that can effectively handle mixed data is highly important for obtaining an in-depth understanding of disease risk factors and pathogenic mechanisms.</p><p><strong>Methods: </strong>This study proposes an extension of the NOTEARS algorithm, termed DAGSLAM, which is designed for Bayesian network structure learning with mixed-type data. The algorithm integrates continuous and categorical variables through a tailored loss function, enhancing its applicability to real-world epidemiological datasets.</p><p><strong>Results: </strong>Extensive simulations were conducted across eight distinct scenarios, specifically, variations in the number of nodes, changes in the proportion of categorical nodes, different sample sizes, levels of categorical nodes, variations in edge sparsity, adjustments to the weight scale, different graph types, and diverse noise distributions. These scenarios demonstrate that DAGSLAM consistently outperforms existing methods such as HC, TABU, mDAG, and DAGBagM across key metrics, including precision, recall, F1 score, and structural Hamming distance (SHD). Furthermore, the robustness of DAGSLAM is validated through its application to the National Health and Nutrition Examination Survey (NHANES) dataset, revealing critical causal relationships among risk factors for CHD and diabetes.</p><p><strong>Conclusions: </strong>DAGSLAM provides a powerful and scalable tool for uncovering causal relationships in complex disease networks, with significant implications for risk factor identification and public health research.</p>","PeriodicalId":9114,"journal":{"name":"BMC Medical Research Methodology","volume":"25 1","pages":"154"},"PeriodicalIF":3.9,"publicationDate":"2025-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12142989/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144246396","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Adherence to STARD guidelines in miRNA diagnostic accuracy studies for hepatocellular carcinoma: a systematic review.","authors":"Shuaiyang Wang, Mei Wu, Bei Xie, Xingyuan Ma, Jing Li, Bin Ma, Linjing Li","doi":"10.1186/s12874-025-02610-5","DOIUrl":"10.1186/s12874-025-02610-5","url":null,"abstract":"<p><strong>Background: </strong>Accumulating evidence demonstrates that miRNAs exhibit enhanced diagnostic sensitivity and specificity compared to the conventional hepatocellular carcinoma (HCC) biomarker alpha-fetoprotein (AFP), particularly showing promising clinical utility in early-stage HCC detection. However, insufficient transparency in methodology reporting compromises the validity assessment of these findings and hinders their translational applications. In this study, we investigated the adherence to the STARD criteria in studies investigating the diagnostic accuracy of miRNA for liver cancer. In addition, the quality of reporting was examined to identify factors influencing the quality of reporting.</p><p><strong>Method: </strong>A comprehensive search strategy was performed on the PubMed, EMBASE, Cochrane Library, and Web of Science, as of March 30, 2023. Clinical trials investigating the diagnostic value of miRNA for HCC diagnosis were included in the analysis. The eligible studies were checked against adherence to the STARD criteria. Factors determining quality of studies were evaluated through subgroup analyses. All statistical analyses were performed using SPSS (varsion25.0).</p><p><strong>Results: </strong>Sixty-two eligible studies, all published between 2010 and 2022, were eventually included in the final analysis. The results revealed a moderate overall adherence to STARD 2015, with an average of 12.1 (52.6%) of the 23 items reported. Subgroup analysis revealed that adherence to the STARD 2015 varied among countries (USA 58.7%;Egypt 46.5%)(p<0.05).</p><p><strong>Conclusion: </strong>The quality of studies reporting the diagnostic accuracy of miRNAs in HCC was average and did not increase after the publication of STARD 2015. Our findings rise awareness regarding the need to improve STARD standards, implying that more journals need to include STARD standards for relevant manuscripts. In addition, editorial and peer approval procedures should adopt measures that will aim to improve reporting quality.</p>","PeriodicalId":9114,"journal":{"name":"BMC Medical Research Methodology","volume":"25 1","pages":"155"},"PeriodicalIF":3.9,"publicationDate":"2025-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12142931/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144246394","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Value-adaptive clinical trial designs for efficient delivery of publicly funded trials - a discussion of methods, case studies, opportunities and challenges.","authors":"Laura Flight, Alan Brennan, Stephen E Chick, Martin Forster, Steven Julious, Puvan Tharmanathan","doi":"10.1186/s12874-025-02566-6","DOIUrl":"10.1186/s12874-025-02566-6","url":null,"abstract":"<p><strong>Background: </strong>Value-adaptive designs for clinical trials are a novel set of emerging methods for delivering greater value for clinical research. There is increasing interest in using them within publicly funded health systems. A value-adaptive design permits 'in progress' changes to be made to the trial according to criteria which reflect its overall value to the healthcare system, including the cost-effectiveness of the technologies under investigation, the cost of running the trial and the total health benefit delivered to patients. These trial designs offer the potential to explicitly balance the costs and benefits of adaptive clinical trials with the health economic benefits expected for populations that are affected by any subsequent health technology adoption decisions. They may also improve the expected value of learning from the budget that is spent within a trial.</p><p><strong>Main body: </strong>This paper introduces value-adaptive designs for publicly funded clinical trials. It discusses the idea of delivering 'value for money' in health technology assessment, what is meant by being 'value-adaptive' and the key features that characterise these designs. The methodology behind one kind of value-adaptive design - the value-based sequential model of a two-armed clinical trial proposed by Chick et al. (2017) - is described and illustrated using three retrospective case studies from the United Kingdom. The paper concludes by reviewing a range of perspectives provided by stakeholders, together with our own thoughts, on the practical opportunities and changes required for implementing a value-adaptive approach.</p><p><strong>Conclusions: </strong>Value-adaptive clinical trial designs offer the potential to align health research funding allocations with population health economic goals. Many of the systems required to deploy value-adaptive designs within a publicly funded health system already exist and, with increased application, experience, and refinement they have the potential to deliver improved value for money.</p>","PeriodicalId":9114,"journal":{"name":"BMC Medical Research Methodology","volume":"25 1","pages":"153"},"PeriodicalIF":3.9,"publicationDate":"2025-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12139198/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144233166","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"An extension of the Spiegelhalter-Knill-Jones method for continuous covariates in clinical decision making.","authors":"Bart K M Jacobs, Tafadzwa Maseko, Lutgarde Lynen, Aquiles Rodrigo Henriquez-Trujillo, Jozefien Buyze","doi":"10.1186/s12874-025-02591-5","DOIUrl":"10.1186/s12874-025-02591-5","url":null,"abstract":"<p><strong>Background: </strong>There is still demand for algorithms that can be used at the point of care, especially when dealing with events that do not present with a single obvious clinical indicator. The Spiegelhalter-Knill-Jones (SKJ) method is an approach for the development of a clinical score that focuses on the effect size of predictors, which is more relevant in settings where events may be rare or data is scarce. However, it does require predictors to be binary or dichotomised.</p><p><strong>Methods: </strong>We developed an extension of the Spiegelhalter-Knill-Jones method that can include continuous variables and added additional features that make it more useful in a variety of settings. We illustrated our method on two historical datasets dealing with viral failure in HIV patients in Cambodia. We used area under the curve (AUC) and risk classification improvement (RCI) as metrics to evaluate the performance of resulting predictions scores and risk classifications.</p><p><strong>Results: </strong>All new features worked as intended. Scoring systems developed with the new method outperformed an earlier application of a classic version of SKJ method on the training dataset, while no significant difference was found on any of the performance measures in the test dataset.</p><p><strong>Conclusions: </strong>This extension provides a useful tool for clinical decision-making that is much more flexible than the original version of SKJ, and can be applied in a variety of settings.</p>","PeriodicalId":9114,"journal":{"name":"BMC Medical Research Methodology","volume":"25 1","pages":"152"},"PeriodicalIF":3.9,"publicationDate":"2025-06-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12131495/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144214909","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Behavior of test specificity under an imperfect gold standard: findings from a simulation study and analysis of real-world oncology data.","authors":"Mark S Walker, Lukas Slipski, Yanina Natanzon","doi":"10.1186/s12874-025-02603-4","DOIUrl":"10.1186/s12874-025-02603-4","url":null,"abstract":"<p><strong>Background: </strong>Gold standards used in validation of new tests may be imperfect, with sensitivity or specificity less than 100%. The impact of imperfection in a gold standard on measured test attributes has been demonstrated formally, but its relevance in real-world oncology research may not be well understood.</p><p><strong>Methods: </strong>This simulation study examined the impact of imperfect gold standard sensitivity on measured test specificity at different levels of condition prevalence for a hypothetical real-world measure of death. The study also evaluated real-world oncology datasets with a linked National Death Index (NDI) dataset, to examine the measured specificity of a death indicator at levels of death prevalence that matched the simulation. The simulation and real-world data analysis both examined measured specificity of the death indicator at death prevalence ranging from 50 to 98%. To isolate the effects of death prevalence and imperfect gold standard sensitivity, the simulation assumed a test with perfect sensitivity and specificity, and with perfect gold standard specificity. However, gold standard sensitivity was modeled at values from 90 to 99%.</p><p><strong>Results: </strong>Results of the simulation showed that decreasing gold standard sensitivity was associated with increasing underestimation of test specificity, and that the extent of underestimation increased with higher death prevalence. Analysis of the real-world data yielded findings that closely matched the simulation pattern. At 98% death prevalence, near-perfect gold standard sensitivity (99%) still resulted in suppression of specificity from the true value of 100% to the measured value of < 67%.</p><p><strong>Conclusions: </strong>New validation research, and review of existing validation studies, should consider the prevalence of the conditions assessed by a measure, and the possible impact on sensitivity and specificity of an imperfect gold standard.</p>","PeriodicalId":9114,"journal":{"name":"BMC Medical Research Methodology","volume":"25 1","pages":"151"},"PeriodicalIF":3.9,"publicationDate":"2025-05-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12125893/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144186552","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}