Pushpak Pati, Sofia Karkampouna, Francesco Bonollo, Eva Compérat, Martina Radić, Martin Spahn, Adriano Martinelli, Martin Wartenberg, Marianna Kruithof-de Julio, Marianna Rapsomaniki
{"title":"Accelerating histopathology workflows with generative AI-based virtually multiplexed tumour profiling","authors":"Pushpak Pati, Sofia Karkampouna, Francesco Bonollo, Eva Compérat, Martina Radić, Martin Spahn, Adriano Martinelli, Martin Wartenberg, Marianna Kruithof-de Julio, Marianna Rapsomaniki","doi":"10.1038/s42256-024-00889-5","DOIUrl":"10.1038/s42256-024-00889-5","url":null,"abstract":"Understanding the spatial heterogeneity of tumours and its links to disease initiation and progression is a cornerstone of cancer biology. Presently, histopathology workflows heavily rely on hematoxylin and eosin and serial immunohistochemistry staining, a cumbersome, tissue-exhaustive process that results in non-aligned tissue images. We propose the VirtualMultiplexer, a generative artificial intelligence toolkit that effectively synthesizes multiplexed immunohistochemistry images for several antibody markers (namely AR, NKX3.1, CD44, CD146, p53 and ERG) from only an input hematoxylin and eosin image. The VirtualMultiplexer captures biologically relevant staining patterns across tissue scales without requiring consecutive tissue sections, image registration or extensive expert annotations. Thorough qualitative and quantitative assessment indicates that the VirtualMultiplexer achieves rapid, robust and precise generation of virtually multiplexed imaging datasets of high staining quality that are indistinguishable from the real ones. The VirtualMultiplexer is successfully transferred across tissue scales and patient cohorts with no need for model fine-tuning. Crucially, the virtually multiplexed images enabled training a graph transformer that simultaneously learns from the joint spatial distribution of several proteins to predict clinically relevant endpoints. We observe that this multiplexed learning scheme was able to greatly improve clinical prediction, as corroborated across several downstream tasks, independent patient cohorts and cancer types. Our results showcase the clinical relevance of artificial intelligence-assisted multiplexed tumour imaging, accelerating histopathology workflows and cancer biology. VirtualMultiplexer is a generative AI tool that produces realistic multiplexed immunohistochemistry images from tissue biopsies. The generated images could be used to improve clinical predictions, enhancing histopathology workflows and accelerating cancer research.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":"6 9","pages":"1077-1093"},"PeriodicalIF":18.8,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s42256-024-00889-5.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142158783","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Chengdong Ma, Aming Li, Yali Du, Hao Dong, Yaodong Yang
{"title":"Efficient and scalable reinforcement learning for large-scale network control","authors":"Chengdong Ma, Aming Li, Yali Du, Hao Dong, Yaodong Yang","doi":"10.1038/s42256-024-00879-7","DOIUrl":"10.1038/s42256-024-00879-7","url":null,"abstract":"The primary challenge in the development of large-scale artificial intelligence (AI) systems lies in achieving scalable decision-making—extending the AI models while maintaining sufficient performance. Existing research indicates that distributed AI can improve scalability by decomposing complex tasks and distributing them across collaborative nodes. However, previous technologies suffered from compromised real-world applicability and scalability due to the massive requirement of communication and sampled data. Here we develop a model-based decentralized policy optimization framework, which can be efficiently deployed in multi-agent systems. By leveraging local observation through the agent-level topological decoupling of global dynamics, we prove that this decentralized mechanism achieves accurate estimations of global information. Importantly, we further introduce model learning to reinforce the optimal policy for monotonic improvement with a limited amount of sampled data. Empirical results on diverse scenarios show the superior scalability of our approach, particularly in real-world systems with hundreds of agents, thereby paving the way for scaling up AI systems. Applying large-scale AI systems to multi-agent scenarios in real-world settings is challenging. The authors propose a decentralized model-based policy optimization framework to enable scalable decision-making.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":"6 9","pages":"1006-1020"},"PeriodicalIF":18.8,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s42256-024-00879-7.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142123712","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Shayne Longpre, Robert Mahari, Anthony Chen, Naana Obeng-Marnu, Damien Sileo, William Brannon, Niklas Muennighoff, Nathan Khazam, Jad Kabbara, Kartik Perisetla, Xinyi (Alexis) Wu, Enrico Shippole, Kurt Bollacker, Tongshuang Wu, Luis Villa, Sandy Pentland, Sara Hooker
{"title":"A large-scale audit of dataset licensing and attribution in AI","authors":"Shayne Longpre, Robert Mahari, Anthony Chen, Naana Obeng-Marnu, Damien Sileo, William Brannon, Niklas Muennighoff, Nathan Khazam, Jad Kabbara, Kartik Perisetla, Xinyi (Alexis) Wu, Enrico Shippole, Kurt Bollacker, Tongshuang Wu, Luis Villa, Sandy Pentland, Sara Hooker","doi":"10.1038/s42256-024-00878-8","DOIUrl":"10.1038/s42256-024-00878-8","url":null,"abstract":"The race to train language models on vast, diverse and inconsistently documented datasets raises pressing legal and ethical concerns. To improve data transparency and understanding, we convene a multi-disciplinary effort between legal and machine learning experts to systematically audit and trace more than 1,800 text datasets. We develop tools and standards to trace the lineage of these datasets, including their source, creators, licences and subsequent use. Our landscape analysis highlights sharp divides in the composition and focus of data licenced for commercial use. Important categories including low-resource languages, creative tasks and new synthetic data all tend to be restrictively licenced. We observe frequent miscategorization of licences on popular dataset hosting sites, with licence omission rates of more than 70% and error rates of more than 50%. This highlights a crisis in misattribution and informed use of popular datasets driving many recent breakthroughs. Our analysis of data sources also explains the application of copyright law and fair use to finetuning data. As a contribution to continuing improvements in dataset transparency and responsible use, we release our audit, with an interactive user interface, the Data Provenance Explorer, to enable practitioners to trace and filter on data provenance for the most popular finetuning data collections: www.dataprovenance.org . The Data Provenance Initiative audits over 1,800 text artificial intelligence (AI) datasets, analysing trends, permissions of use and global representation. It exposes frequent errors on several major data hosting sites and offers tools for transparent and informed use of AI training data.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":"6 8","pages":"975-987"},"PeriodicalIF":18.8,"publicationDate":"2024-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s42256-024-00878-8.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142091222","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"What is in your LLM-based framework?","authors":"","doi":"10.1038/s42256-024-00896-6","DOIUrl":"10.1038/s42256-024-00896-6","url":null,"abstract":"To maintain high standards in clarity and reproducibility, authors need to clearly mention and describe the use of GPT-4 and other large language models in their work.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":"6 8","pages":"845-845"},"PeriodicalIF":18.8,"publicationDate":"2024-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s42256-024-00896-6.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142091142","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A step forward in tracing and documenting dataset provenance","authors":"Nicholas Vincent","doi":"10.1038/s42256-024-00884-w","DOIUrl":"10.1038/s42256-024-00884-w","url":null,"abstract":"Training data are crucial for advancements in artificial intelligence, but many questions remain regarding the provenance of training datasets, license enforcement and creator consent. Mahari et al. provide a set of tools for tracing, documenting and sharing AI training data and highlight the importance for developers to engage with metadata of datasets.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":"6 8","pages":"848-849"},"PeriodicalIF":18.8,"publicationDate":"2024-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142091186","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Emanuele Zappala, Antonio Henrique de Oliveira Fonseca, Josue Ortega Caro, Andrew Henry Moberly, Michael James Higley, Jessica Cardin, David van Dijk
{"title":"Learning integral operators via neural integral equations","authors":"Emanuele Zappala, Antonio Henrique de Oliveira Fonseca, Josue Ortega Caro, Andrew Henry Moberly, Michael James Higley, Jessica Cardin, David van Dijk","doi":"10.1038/s42256-024-00886-8","DOIUrl":"10.1038/s42256-024-00886-8","url":null,"abstract":"Nonlinear operators with long-distance spatiotemporal dependencies are fundamental in modelling complex systems across sciences; yet, learning these non-local operators remains challenging in machine learning. Integral equations, which model such non-local systems, have wide-ranging applications in physics, chemistry, biology and engineering. We introduce the neural integral equation, a method for learning unknown integral operators from data using an integral equation solver. To improve scalability and model capacity, we also present the attentional neural integral equation, which replaces the integral with self-attention. Both models are grounded in the theory of second-kind integral equations, where the indeterminate appears both inside and outside the integral operator. We provide a theoretical analysis showing how self-attention can approximate integral operators under mild regularity assumptions, further deepening previously reported connections between transformers and integration, as well as deriving corresponding approximation results for integral operators. Through numerical benchmarks on synthetic and real-world data, including Lotka–Volterra, Navier–Stokes and Burgers’ equations, as well as brain dynamics and integral equations, we showcase the models’ capabilities and their ability to derive interpretable dynamics embeddings. Our experiments demonstrate that attentional neural integral equations outperform existing methods, especially for longer time intervals and higher-dimensional problems. Our work addresses a critical gap in machine learning for non-local operators and offers a powerful tool for studying unknown complex systems with long-range dependencies. Integral equations are used in science and engineering to model complex systems with non-local dependencies; however, existing traditional and machine-learning-based methods cannot yield accurate or efficient solutions in several complex cases. Zappala and colleagues introduce a neural-network-based method that can learn an integral operator and its dynamics from data, demonstrating higher accuracy or scalability compared with several state-of-the-art methods.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":"6 9","pages":"1046-1062"},"PeriodicalIF":18.8,"publicationDate":"2024-08-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s42256-024-00886-8.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142090186","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yi Zhong, Gaozheng Li, Ji Yang, Houbing Zheng, Yongqiang Yu, Jiheng Zhang, Heng Luo, Biao Wang, Zuquan Weng
{"title":"Learning motif-based graphs for drug–drug interaction prediction via local–global self-attention","authors":"Yi Zhong, Gaozheng Li, Ji Yang, Houbing Zheng, Yongqiang Yu, Jiheng Zhang, Heng Luo, Biao Wang, Zuquan Weng","doi":"10.1038/s42256-024-00888-6","DOIUrl":"10.1038/s42256-024-00888-6","url":null,"abstract":"Unexpected drug–drug interactions (DDIs) are important issues for both pharmaceutical research and clinical applications due to the high risk of causing severe adverse drug reactions or drug withdrawals. Many deep learning models have achieved high performance in DDI prediction, but model interpretability to reveal the underlying causes of DDIs has not been extensively explored. Here we propose MeTDDI—a deep learning framework with local–global self-attention and co-attention to learn motif-based graphs for DDI prediction. MeTDDI achieved competitive performance compared with state-of-the-art models. Regarding interpretability, we conducted extensive assessments on 73 drugs with 13,786 DDIs and MeTDDI can precisely explain the structural mechanisms for 5,602 DDIs involving 58 drugs. Besides, MeTDDI shows potential to explain complex DDI mechanisms and mitigate DDI risks. To summarize, MeTDDI provides a new perspective on exploring DDI mechanisms, which will benefit both drug discovery and polypharmacy for safer therapies for patients. A transformer-based approach that predicts drug–drug interactions in polypharmacy has been shown, which also identifies perpetrator drugs and the chemical mechanisms causing the interactions. The method could facilitate high-throughput optimization of drug combinations and mitigate adverse drug–drug interaction risks.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":"6 9","pages":"1094-1105"},"PeriodicalIF":18.8,"publicationDate":"2024-08-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142085189","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Davide Carnevali, Limei Zhong, Esther González-Almela, Carlotta Viana, Mikhail Rotkevich, Aiping Wang, Daniel Franco-Barranco, Aitor Gonzalez-Marfil, Maria Victoria Neguembor, Alvaro Castells-Garcia, Ignacio Arganda-Carreras, Maria Pia Cosma
{"title":"A deep learning method that identifies cellular heterogeneity using nanoscale nuclear features","authors":"Davide Carnevali, Limei Zhong, Esther González-Almela, Carlotta Viana, Mikhail Rotkevich, Aiping Wang, Daniel Franco-Barranco, Aitor Gonzalez-Marfil, Maria Victoria Neguembor, Alvaro Castells-Garcia, Ignacio Arganda-Carreras, Maria Pia Cosma","doi":"10.1038/s42256-024-00883-x","DOIUrl":"10.1038/s42256-024-00883-x","url":null,"abstract":"Cellular phenotypic heterogeneity is an important hallmark of many biological processes and understanding its origins remains a substantial challenge. This heterogeneity often reflects variations in the chromatin structure, influenced by factors such as viral infections and cancer, which dramatically reshape the cellular landscape. To address the challenge of identifying distinct cell states, we developed artificial intelligence of the nucleus (AINU), a deep learning method that can identify specific nuclear signatures at the nanoscale resolution. AINU can distinguish different cell states based on the spatial arrangement of core histone H3, RNA polymerase II or DNA from super-resolution microscopy images. With only a small number of images as the training data, AINU correctly identifies human somatic cells, human-induced pluripotent stem cells, very early stage infected cells transduced with DNA herpes simplex virus type 1 and even cancer cells after appropriate retraining. Finally, using AI interpretability methods, we find that the RNA polymerase II localizations in the nucleoli aid in distinguishing human-induced pluripotent stem cells from their somatic cells. Overall, AINU coupled with super-resolution microscopy of nuclear structures provides a robust tool for the precise detection of cellular heterogeneity, with considerable potential for advancing diagnostics and therapies in regenerative medicine, virology and cancer biology. Cellular phenotypic heterogeneity is a key determinant of biological functions and is challenging to identify. A deep learning method that recognizes specific nuclear signatures is discussed, which can identify cellular heterogeneity and differentiate between various cell states using a small amount of super-resolution microscopy data.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":"6 9","pages":"1021-1033"},"PeriodicalIF":18.8,"publicationDate":"2024-08-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s42256-024-00883-x.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142085188","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Isabelle Augenstein, Timothy Baldwin, Meeyoung Cha, Tanmoy Chakraborty, Giovanni Luca Ciampaglia, David Corney, Renee DiResta, Emilio Ferrara, Scott Hale, Alon Halevy, Eduard Hovy, Heng Ji, Filippo Menczer, Ruben Miguez, Preslav Nakov, Dietram Scheufele, Shivam Sharma, Giovanni Zagni
{"title":"Factuality challenges in the era of large language models and opportunities for fact-checking","authors":"Isabelle Augenstein, Timothy Baldwin, Meeyoung Cha, Tanmoy Chakraborty, Giovanni Luca Ciampaglia, David Corney, Renee DiResta, Emilio Ferrara, Scott Hale, Alon Halevy, Eduard Hovy, Heng Ji, Filippo Menczer, Ruben Miguez, Preslav Nakov, Dietram Scheufele, Shivam Sharma, Giovanni Zagni","doi":"10.1038/s42256-024-00881-z","DOIUrl":"10.1038/s42256-024-00881-z","url":null,"abstract":"The emergence of tools based on large language models (LLMs), such as OpenAI’s ChatGPT and Google’s Gemini, has garnered immense public attention owing to their advanced natural language generation capabilities. These remarkably natural-sounding tools have the potential to be highly useful for various tasks. However, they also tend to produce false, erroneous or misleading content—commonly referred to as hallucinations. Moreover, LLMs can be misused to generate convincing, yet false, content and profiles on a large scale, posing a substantial societal challenge by potentially deceiving users and spreading inaccurate information. This makes fact-checking increasingly important. Despite their issues with factual accuracy, LLMs have shown proficiency in various subtasks that support fact-checking, which is essential to ensure factually accurate responses. In light of these concerns, we explore issues related to factuality in LLMs and their impact on fact-checking. We identify key challenges, imminent threats and possible solutions to these factuality issues. We also thoroughly examine these challenges, existing solutions and potential prospects for fact-checking. By analysing the factuality constraints within LLMs and their impact on fact-checking, we aim to contribute to a path towards maintaining accuracy at a time of confluence of generative artificial intelligence and misinformation. Large language models (LLMs) present challenges, including a tendency to produce false or misleading content and the potential to create misinformation or disinformation. Augenstein and colleagues explore issues related to factuality in LLMs and their impact on fact-checking.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":"6 8","pages":"852-863"},"PeriodicalIF":18.8,"publicationDate":"2024-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142022075","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}