Jojanneke Drogt, Megan Milota, Anne van den Brink, Karin Jongsma
{"title":"Ethical guidance for reporting and evaluating claims of AI outperforming human doctors","authors":"Jojanneke Drogt, Megan Milota, Anne van den Brink, Karin Jongsma","doi":"10.1038/s41746-024-01255-w","DOIUrl":"10.1038/s41746-024-01255-w","url":null,"abstract":"Claims of AI outperforming medical practitioners are under scrutiny, as the evidence supporting many of these claims is not convincing or transparently reported. These claims often lack specificity, contextualization, and empirical grounding. In this comment, we offer constructive ethical guidance that can benefit authors, journal editors, and peer reviewers when reporting and evaluating findings in studies comparing AI to physician performance. The guidance provided here forms an essential addition to current reporting guidelines for healthcare studies using machine learning.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-10-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01255-w.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142362894","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yosra Magdi Mekki, Osman Hassan Ahmed, Dyllan Powell, Amy Price, H. Paul Dijkstra
{"title":"Games Wide Open to athlete partnership in building artificial intelligence systems","authors":"Yosra Magdi Mekki, Osman Hassan Ahmed, Dyllan Powell, Amy Price, H. Paul Dijkstra","doi":"10.1038/s41746-024-01261-y","DOIUrl":"10.1038/s41746-024-01261-y","url":null,"abstract":"The integration of artificial intelligence (AI) in sports medicine is opening new frontiers for athlete health and performance, aligning with the spirit of the Paris 2024 Olympic Games slogan, “Games Wide Open.”","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01261-y.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142360071","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Rosanna Tarricone, Francesco Petracca, Hannah-Marie Weller
{"title":"“Towards harmonizing assessment and reimbursement of digital medical devices in the EU through mutual learning”","authors":"Rosanna Tarricone, Francesco Petracca, Hannah-Marie Weller","doi":"10.1038/s41746-024-01263-w","DOIUrl":"10.1038/s41746-024-01263-w","url":null,"abstract":"Digital medical devices (DMDs) present unique opportunities in their regulation and reimbursement. A dynamic landscape of DMD assessment frameworks is emerging within the European Union, with five clusters of prevailing approaches identified. Despite notable gaps in maturity levels, cross-country learning effects are becoming prevalent. We expect more countries, both within the EU and beyond, to follow the steps of current frontrunners, hence expediting the harmonization process.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01263-w.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142361793","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Karin Slater, Paul N. Schofield, James Wright, Paul Clift, Anushka Irani, William Bradlow, Furqan Aziz, Georgios V. Gkoutos
{"title":"Talking about diseases; developing a model of patient and public-prioritised disease phenotypes","authors":"Karin Slater, Paul N. Schofield, James Wright, Paul Clift, Anushka Irani, William Bradlow, Furqan Aziz, Georgios V. Gkoutos","doi":"10.1038/s41746-024-01257-8","DOIUrl":"10.1038/s41746-024-01257-8","url":null,"abstract":"Deep phenotyping describes the use of standardised terminologies to create comprehensive phenotypic descriptions of biomedical phenomena. These characterisations facilitate secondary analysis, evidence synthesis, and practitioner awareness, thereby guiding patient care. The vast majority of this knowledge is derived from sources that describe an academic understanding of disease, including academic literature and experimental databases. Previous work indicates a gulf between the priorities, perspectives, and perceptions held by different healthcare stakeholders. Using social media data, we develop a phenotype model that represents a public perspective on disease and compare this with a model derived from a combination of existing academic phenotype databases. We identified 52,198 positive disease-phenotype associations from social media across 311 diseases. We further identified 24,618 novel phenotype associations not shared by the biomedical and literature-derived phenotype model across 304 diseases, of which we considered 14,531 significant. Manifestations of disease affecting quality of life, and concerning endocrine, digestive, and reproductive diseases were over-represented in the social media phenotype model. An expert clinical review found that social media-derived associations were considered similarly well-established to those derived from literature, and were seen significantly more in patient clinical encounters. The phenotype model recovered from social media presents a significantly different perspective than existing resources derived from biomedical databases and literature, providing a large number of associations novel to the latter dataset. We propose that the integration and interrogation of these public perspectives on the disease can inform clinical awareness, improve secondary analysis, and bridge understanding and priorities across healthcare stakeholders.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01257-8.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142329429","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Overcoming biases of individual level shopping history data in health research","authors":"Anya Skatova","doi":"10.1038/s41746-024-01231-4","DOIUrl":"10.1038/s41746-024-01231-4","url":null,"abstract":"Novel sources of population data, especially administrative and medical records, as well as the digital footprints generated through interactions with online services, present a considerable opportunity for advancing health research and policymaking. An illustrative example is shopping history records that can illuminate aspects of population health by scrutinizing extensive sets of everyday choices made in the real world. However, like any dataset, these sources possess specific limitations, including sampling biases, validity issues, and measurement errors. To enhance the applicability and potential of shopping data in health research, we advocate for the integration of individual-level shopping data with external datasets containing rich repositories of longitudinal population cohort studies. This strategic approach holds the promise of devising innovative methodologies to address inherent data limitations and biases. By meticulously documenting biases, establishing validated associations, and discerning patterns within these amalgamated records, researchers can extrapolate their findings to encompass population-wide datasets derived from national supermarket chain. The validation and linkage of population health data with real-world choices pertaining to food, beverages, and over-the-counter medications, such as pain relief, present a significant opportunity to comprehend the impact of these choices and behavioural patterns associated with them on public health.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01231-4.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142329454","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Xiaodi Liu, Yingnan Liu, Mong Li Lee, Wynne Hsu, Ming Han Lincoln Liow
{"title":"Identifying who are unlikely to benefit from total knee arthroplasty using machine learning models","authors":"Xiaodi Liu, Yingnan Liu, Mong Li Lee, Wynne Hsu, Ming Han Lincoln Liow","doi":"10.1038/s41746-024-01265-8","DOIUrl":"10.1038/s41746-024-01265-8","url":null,"abstract":"Identifying and preventing patients who are not likely to benefit long-term from total knee arthroplasty (TKA) would decrease healthcare expenditure significantly. We trained machine learning (ML) models (image-only, clinical-data only, and multimodal) among 5720 knee OA patients to predict postoperative dissatisfaction at 2 years. Dissatisfaction was defined as not achieving a minimal clinically important difference in postoperative Knee Society knee and function scores (KSS), Short Form-36 Health Survey [SF-36, divided into a physical component score (PCS) and mental component score (MCS)], and Oxford Knee Score (OKS). Compared to image-only models, both clinical-data only and multimodal models achieved superior performance at predicting dissatisfaction measured by AUC, clinical-data only model: KSS 0.888 (0.866–0.909), SF-PCS 0.836 (0.812–0.860), SF-MCS 0.833 (0.812–0.854), and OKS 0.806 (0.753–0.859); multimodal model: KSS 0.891 (0.870–0.911), SF-PCS 0.832 (0.808–0.857), SF-MCS 0.835 (0.811–0.856), and OKS 0.816 (0.768–0.863). Our findings highlighted that ML models using clinical or multimodal data were capable to predict post-TKA dissatisfaction.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01265-8.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142351024","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Katharina Wenderott, Jim Krups, Fiona Zaruchas, Matthias Weigl
{"title":"Effects of artificial intelligence implementation on efficiency in medical imaging—a systematic literature review and meta-analysis","authors":"Katharina Wenderott, Jim Krups, Fiona Zaruchas, Matthias Weigl","doi":"10.1038/s41746-024-01248-9","DOIUrl":"10.1038/s41746-024-01248-9","url":null,"abstract":"In healthcare, integration of artificial intelligence (AI) holds strong promise for facilitating clinicians’ work, especially in clinical imaging. We aimed to assess the impact of AI implementation for medical imaging on efficiency in real-world clinical workflows and conducted a systematic review searching six medical databases. Two reviewers double-screened all records. Eligible records were evaluated for methodological quality. The outcomes of interest were workflow adaptation due to AI implementation, changes in time for tasks, and clinician workload. After screening 13,756 records, we identified 48 original studies to be incuded in the review. Thirty-three studies measured time for tasks, with 67% reporting reductions. Yet, three separate meta-analyses of 12 studies did not show significant effects after AI implementation. We identified five different workflows adapting to AI use. Most commonly, AI served as a secondary reader for detection tasks. Alternatively, AI was used as the primary reader for identifying positive cases, resulting in reorganizing worklists or issuing alerts. Only three studies scrutinized workload calculations based on the time saved through AI use. This systematic review and meta-analysis represents an assessment of the efficiency improvements offered by AI applications in real-world clinical imaging, predominantly revealing enhancements across the studies. However, considerable heterogeneity in available studies renders robust inferences regarding overall effectiveness in imaging tasks. Further work is needed on standardized reporting, evaluation of system integration, and real-world data collection to better understand the technological advances of AI in real-world healthcare workflows. Systematic review registration: Prospero ID CRD42022303439, International Registered Report Identifier (IRRID): RR2-10.2196/40485.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01248-9.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142351023","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yan-Bing Huang, Li Lin, Xin-Yu Li, Bo-Zhu Chen, Lu Yuan, Hui Zheng
{"title":"An indirect treatment comparison meta-analysis of digital versus face-to-face cognitive behavior therapy for headache","authors":"Yan-Bing Huang, Li Lin, Xin-Yu Li, Bo-Zhu Chen, Lu Yuan, Hui Zheng","doi":"10.1038/s41746-024-01264-9","DOIUrl":"10.1038/s41746-024-01264-9","url":null,"abstract":"Cognitive behavioral therapy (CBT) is effective for headache disorders. However, it is unclear whether the emerging digital CBT is noninferior to face-to-face CBT. An indirect treatment comparison (ITC) meta-analysis was conducted to assess the relative effects between them using standard mean differences (SMDs). Effective sample size (ESS) and required sample size (RSS) were calculated to demonstrate the robustness of the results. Our study found that digital CBT had a similar effect on headache frequency reduction (SMD, 0.12; 95%CI, −2.45 to 2.63) compared with face-to-face CBT. The ESS had 84 participants, while the RSS had 466 participants to achieve the same power as a non-inferior head-to-head trial. Digital CBT is as effective as face-to-face CBT in preventing headache disorders. Due to the heterogeneity (I2 = 94.5%, τ2 = 1.83) and the fact that most of the included studies were on migraine prevention, further head-to-head trials are warranted.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-09-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01264-9.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142329189","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Tamsin J. Robb, Yinan Liu, Braden Woodhouse, Charlotta Windahl, Daniel Hurley, Grant McArthur, Stephen B. Fox, Lisa Brown, Parry Guilford, Alice Minhinnick, Christopher Jackson, Cherie Blenkiron, Kate Parker, Kimiora Henare, Rose McColl, Bianca Haux, Nick Young, Veronica Boyle, Laird Cameron, Sanjeev Deva, Jane Reeve, Cristin G. Print, Michael Davis, Uwe Rieger, Ben Lawrence
{"title":"Blending space and time to talk about cancer in extended reality","authors":"Tamsin J. Robb, Yinan Liu, Braden Woodhouse, Charlotta Windahl, Daniel Hurley, Grant McArthur, Stephen B. Fox, Lisa Brown, Parry Guilford, Alice Minhinnick, Christopher Jackson, Cherie Blenkiron, Kate Parker, Kimiora Henare, Rose McColl, Bianca Haux, Nick Young, Veronica Boyle, Laird Cameron, Sanjeev Deva, Jane Reeve, Cristin G. Print, Michael Davis, Uwe Rieger, Ben Lawrence","doi":"10.1038/s41746-024-01262-x","DOIUrl":"10.1038/s41746-024-01262-x","url":null,"abstract":"We introduce a proof-of-concept extended reality (XR) environment for discussing cancer, presenting genomic information from multiple tumour sites in the context of 3D tumour models generated from CT scans. This tool enhances multidisciplinary discussions. Clinicians and cancer researchers explored its use in oncology, sharing perspectives on XR’s potential for use in molecular tumour boards, clinician-patient communication, and education. XR serves as a universal language, fostering collaborative decision-making in oncology.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-09-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01262-x.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142329248","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Soo Bin Yoon, Jipyeong Lee, Hyung-Chul Lee, Chul-Woo Jung, Hyeonhoon Lee
{"title":"Comparison of NLP machine learning models with human physicians for ASA Physical Status classification","authors":"Soo Bin Yoon, Jipyeong Lee, Hyung-Chul Lee, Chul-Woo Jung, Hyeonhoon Lee","doi":"10.1038/s41746-024-01259-6","DOIUrl":"10.1038/s41746-024-01259-6","url":null,"abstract":"The American Society of Anesthesiologist’s Physical Status (ASA-PS) classification system assesses comorbidities before sedation and analgesia, but inconsistencies among raters have hindered its objective use. This study aimed to develop natural language processing (NLP) models to classify ASA-PS using pre-anesthesia evaluation summaries, comparing their performance to human physicians. Data from 717,389 surgical cases in a tertiary hospital (October 2004–May 2023) was split into training, tuning, and test datasets. Board-certified anesthesiologists created reference labels for tuning and test datasets. The NLP models, including ClinicalBigBird, BioClinicalBERT, and Generative Pretrained Transformer 4, were validated against anesthesiologists. The ClinicalBigBird model achieved an area under the receiver operating characteristic curve of 0.915. It outperformed board-certified anesthesiologists with a specificity of 0.901 vs. 0.897, precision of 0.732 vs. 0.715, and F1-score of 0.716 vs. 0.713 (all p <0.01). This approach will facilitate automatic and objective ASA-PS classification, thereby streamlining the clinical workflow.","PeriodicalId":19349,"journal":{"name":"NPJ Digital Medicine","volume":null,"pages":null},"PeriodicalIF":12.4,"publicationDate":"2024-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s41746-024-01259-6.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142328616","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}