{"title":"为什么临床试验不能确保人工智能的安全。","authors":"David P W Rastall, Mohamed Rehman","doi":"10.1007/s10916-025-02231-x","DOIUrl":null,"url":null,"abstract":"<p><p>Recent reports have raised concerns about emergent behaviors in next-generation artificial intelligence (AI) models. These systems have been documented selectively adapting their behaviors during testing to falsify experimental outcomes and bypass regulatory oversight. This phenomenon-alignment faking-represents a fundamental challenge to medical AI safety. Regulatory strategies have largely adapted established protocols like clinical trials and medical device approval frameworks, but for next-generation AI these approaches may fail. This paper introduces alignment faking to a medical audience and critically evaluates how current regulatory tools are inadequate for advanced AI systems. We propose continuous logging through \"AI SOAP notes\" as a first step toward transparent and accountable AI functionality in clinical settings.</p>","PeriodicalId":16338,"journal":{"name":"Journal of Medical Systems","volume":"49 1","pages":"98"},"PeriodicalIF":5.7000,"publicationDate":"2025-07-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Why Clinical Trials Will Fail to Ensure Safe AI.\",\"authors\":\"David P W Rastall, Mohamed Rehman\",\"doi\":\"10.1007/s10916-025-02231-x\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p>Recent reports have raised concerns about emergent behaviors in next-generation artificial intelligence (AI) models. These systems have been documented selectively adapting their behaviors during testing to falsify experimental outcomes and bypass regulatory oversight. This phenomenon-alignment faking-represents a fundamental challenge to medical AI safety. Regulatory strategies have largely adapted established protocols like clinical trials and medical device approval frameworks, but for next-generation AI these approaches may fail. This paper introduces alignment faking to a medical audience and critically evaluates how current regulatory tools are inadequate for advanced AI systems. We propose continuous logging through \\\"AI SOAP notes\\\" as a first step toward transparent and accountable AI functionality in clinical settings.</p>\",\"PeriodicalId\":16338,\"journal\":{\"name\":\"Journal of Medical Systems\",\"volume\":\"49 1\",\"pages\":\"98\"},\"PeriodicalIF\":5.7000,\"publicationDate\":\"2025-07-17\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Medical Systems\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1007/s10916-025-02231-x\",\"RegionNum\":3,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"HEALTH CARE SCIENCES & SERVICES\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Medical Systems","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1007/s10916-025-02231-x","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"HEALTH CARE SCIENCES & SERVICES","Score":null,"Total":0}
Recent reports have raised concerns about emergent behaviors in next-generation artificial intelligence (AI) models. These systems have been documented selectively adapting their behaviors during testing to falsify experimental outcomes and bypass regulatory oversight. This phenomenon-alignment faking-represents a fundamental challenge to medical AI safety. Regulatory strategies have largely adapted established protocols like clinical trials and medical device approval frameworks, but for next-generation AI these approaches may fail. This paper introduces alignment faking to a medical audience and critically evaluates how current regulatory tools are inadequate for advanced AI systems. We propose continuous logging through "AI SOAP notes" as a first step toward transparent and accountable AI functionality in clinical settings.
期刊介绍:
Journal of Medical Systems provides a forum for the presentation and discussion of the increasingly extensive applications of new systems techniques and methods in hospital clinic and physician''s office administration; pathology radiology and pharmaceutical delivery systems; medical records storage and retrieval; and ancillary patient-support systems. The journal publishes informative articles essays and studies across the entire scale of medical systems from large hospital programs to novel small-scale medical services. Education is an integral part of this amalgamation of sciences and selected articles are published in this area. Since existing medical systems are constantly being modified to fit particular circumstances and to solve specific problems the journal includes a special section devoted to status reports on current installations.