Nature Machine Intelligence最新文献

筛选
英文 中文
Wing-strain-based flight control of flapping-wing drones through reinforcement learning 通过强化学习实现基于翼应变的拍翼无人机飞行控制
IF 18.8 1区 计算机科学
Nature Machine Intelligence Pub Date : 2024-09-20 DOI: 10.1038/s42256-024-00893-9
Taewi Kim, Insic Hong, Sunghoon Im, Seungeun Rho, Minho Kim, Yeonwook Roh, Changhwan Kim, Jieun Park, Daseul Lim, Doohoe Lee, Seunggon Lee, Jingoo Lee, Inryeol Back, Junggwang Cho, Myung Rae Hong, Sanghun Kang, Joonho Lee, Sungchul Seo, Uikyum Kim, Young-Man Choi, Je-sung Koh, Seungyong Han, Daeshik Kang
{"title":"Wing-strain-based flight control of flapping-wing drones through reinforcement learning","authors":"Taewi Kim, Insic Hong, Sunghoon Im, Seungeun Rho, Minho Kim, Yeonwook Roh, Changhwan Kim, Jieun Park, Daseul Lim, Doohoe Lee, Seunggon Lee, Jingoo Lee, Inryeol Back, Junggwang Cho, Myung Rae Hong, Sanghun Kang, Joonho Lee, Sungchul Seo, Uikyum Kim, Young-Man Choi, Je-sung Koh, Seungyong Han, Daeshik Kang","doi":"10.1038/s42256-024-00893-9","DOIUrl":"10.1038/s42256-024-00893-9","url":null,"abstract":"Although drone technology has advanced rapidly, replicating the dynamic control and wind-sensing abilities of biological flight is still beyond reach. Biological studies reveal that insect wings are equipped with mechanoreceptors known as campaniform sensilla, which detect complex aerodynamic loads critical for flight agility. By leveraging robotic experiments designed to mimic these biological systems, we confirm that wing strain provides crucial information about the drone’s attitude angle, as well as the direction and velocity of the wind. We introduce a wing-strain-based flight controller that employs the aerodynamic forces exerted on a flapping drone’s wings to deduce vital flight data such as attitude and airflow without accelerometers and gyroscopic sensors. The present work spans five key experiments: initial validation of the wing strain sensor system for state information provision, control in a single degree of freedom movement environment with changing winds, control in a two degrees of freedom movement environment for gravitational attitude adjustment, a test for position control in windy conditions and a demonstration of precise flight path manipulation in a windless condition using only wing strain sensors. We have successfully demonstrated control of a flapping drone in various environments using only wing strain sensors, with the aid of a reinforcement-learning-driven flight controller. The demonstrated adaptability to environmental shifts will be beneficial across varied applications, from gust resistance to wind-assisted flight for autonomous flying robots. Inspired by mechanoreceptors on flying insects, a flapping-wing drone that makes use of strain sensors on the wings and reinforcement-learning-based flight control has been developed. The drone can fly in various unsteady environments, including in windy conditions.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":"6 9","pages":"992-1005"},"PeriodicalIF":18.8,"publicationDate":"2024-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s42256-024-00893-9.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142273337","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Zero-shot transfer of protein sequence likelihood models to thermostability prediction 蛋白质序列似然模型在热稳定性预测中的零点转移
IF 18.8 1区 计算机科学
Nature Machine Intelligence Pub Date : 2024-09-20 DOI: 10.1038/s42256-024-00887-7
Shawn Reeves, Subha Kalyaanamoorthy
{"title":"Zero-shot transfer of protein sequence likelihood models to thermostability prediction","authors":"Shawn Reeves, Subha Kalyaanamoorthy","doi":"10.1038/s42256-024-00887-7","DOIUrl":"10.1038/s42256-024-00887-7","url":null,"abstract":"Protein sequence likelihood models (PSLMs) are an emerging class of self-supervised deep learning algorithms that learn probability distributions over amino acid identities conditioned on structural or evolutionary context. Recently, PSLMs have demonstrated impressive performance in predicting the relative fitness of variant sequences without any task-specific training, but their potential to address a central goal of protein engineering—enhancing stability—remains underexplored. Here we comprehensively analyse the capacity for zero-shot transfer of eight PSLMs towards prediction of relative thermostability for variants of hundreds of heterogeneous proteins across several quantitative datasets. PSLMs are compared with popular task-specific stability models, and we show that some PSLMs have competitive performance when the appropriate statistics are considered. We highlight relative strengths and weaknesses of PSLMs and examine their complementarity with task-specific models, specifically focusing our analyses on stability-engineering applications. Our results indicate that all PSLMs can appreciably augment the predictions of existing methods by integrating insights from their disparate training objectives, suggesting a path forward in the stagnating field of computational stability prediction. Stabilization of proteins is a key task in protein engineering; however, current methods to predict mutant stability face a number of limitations. Reeves and Kalyaanamoorthy study the performance of self-supervised protein sequence likelihood models for stability prediction and find that combining them with task-specific supervised models can lead to appreciable practical gains.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":"6 9","pages":"1063-1076"},"PeriodicalIF":18.8,"publicationDate":"2024-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142273333","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
An end-to-end recurrent compressed sensing method to denoise, detect and demix calcium imaging data 用于钙成像数据去噪、检测和去混合的端到端循环压缩传感方法
IF 18.8 1区 计算机科学
Nature Machine Intelligence Pub Date : 2024-09-19 DOI: 10.1038/s42256-024-00892-w
Kangning Zhang, Sean Tang, Vivian Zhu, Majd Barchini, Weijian Yang
{"title":"An end-to-end recurrent compressed sensing method to denoise, detect and demix calcium imaging data","authors":"Kangning Zhang, Sean Tang, Vivian Zhu, Majd Barchini, Weijian Yang","doi":"10.1038/s42256-024-00892-w","DOIUrl":"10.1038/s42256-024-00892-w","url":null,"abstract":"Two-photon calcium imaging provides large-scale recordings of neuronal activities at cellular resolution. A robust, automated and high-speed pipeline to simultaneously segment the spatial footprints of neurons and extract their temporal activity traces while decontaminating them from background, noise and overlapping neurons is highly desirable to analyse calcium imaging data. Here we demonstrate DeepCaImX, an end-to-end deep learning method based on an iterative shrinkage-thresholding algorithm and a long short-term memory neural network to achieve the above goals altogether at a very high speed and without any manually tuned hyperparameter. DeepCaImX is a multi-task, multi-class and multi-label segmentation method composed of a compressed sensing-inspired neural network with a recurrent layer and fully connected layers. The neural network can simultaneously generate accurate neuronal footprints and extract clean neuronal activity traces from calcium imaging data. We trained the neural network with simulated datasets and benchmarked it against existing state-of-the-art methods with in vivo experimental data. DeepCaImX outperforms existing methods in the quality of segmentation and temporal trace extraction as well as processing speed. DeepCaImX is highly scalable and will benefit the analysis of mesoscale calcium imaging. Extracting time traces and spatial footprints of single neurons from population calcium imaging data presents challenges. Zhang et al. introduce a deep learning method that efficiently segments neuronal footprints and extracts activity traces from these data. The method surpasses existing approaches in both quality and speed, providing a robust tool for large-scale neuronal circuit analysis.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":"6 9","pages":"1106-1118"},"PeriodicalIF":18.8,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142245662","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Pre-training with fractional denoising to enhance molecular property prediction 利用分数去噪进行预训练,提高分子特性预测能力
IF 18.8 1区 计算机科学
Nature Machine Intelligence Pub Date : 2024-09-18 DOI: 10.1038/s42256-024-00900-z
Yuyan Ni, Shikun Feng, Xin Hong, Yuancheng Sun, Wei-Ying Ma, Zhi-Ming Ma, Qiwei Ye, Yanyan Lan
{"title":"Pre-training with fractional denoising to enhance molecular property prediction","authors":"Yuyan Ni, Shikun Feng, Xin Hong, Yuancheng Sun, Wei-Ying Ma, Zhi-Ming Ma, Qiwei Ye, Yanyan Lan","doi":"10.1038/s42256-024-00900-z","DOIUrl":"10.1038/s42256-024-00900-z","url":null,"abstract":"Deep learning methods have been considered promising for accelerating molecular screening in drug discovery and material design. Due to the limited availability of labelled data, various self-supervised molecular pre-training methods have been presented. Although many existing methods utilize common pre-training tasks in computer vision and natural language processing, they often overlook the fundamental physical principles governing molecules. In contrast, applying denoising in pre-training can be interpreted as an equivalent force learning, but the limited noise distribution introduces bias into the molecular distribution. To address this issue, we introduce a molecular pre-training framework called fractional denoising, which decouples noise design from the constraints imposed by force learning equivalence. In this way, the noise becomes customizable, allowing for incorporating chemical priors to substantially improve the molecular distribution modelling. Experiments demonstrate that our framework consistently outperforms existing methods, establishing state-of-the-art results across force prediction, quantum chemical properties and binding affinity tasks. The refined noise design enhances force accuracy and sampling coverage, which contribute to the creation of physically consistent molecular representations, ultimately leading to superior predictive performance. Denoising methods introduce useful priors in pre-training methods for molecular property prediction, but chemically unaware noise can lead to inaccurate predictions in downstream tasks. A molecular pre-training framework that uses fractional denoising to improve molecular distribution modelling is proposed, resulting in better predictions in various property prediction tasks.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":"6 10","pages":"1169-1178"},"PeriodicalIF":18.8,"publicationDate":"2024-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142236693","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Sparse learned kernels for interpretable and efficient medical time series processing 用于可解释和高效医学时间序列处理的稀疏学习核
IF 18.8 1区 计算机科学
Nature Machine Intelligence Pub Date : 2024-09-18 DOI: 10.1038/s42256-024-00898-4
Sully F. Chen, Zhicheng Guo, Cheng Ding, Xiao Hu, Cynthia Rudin
{"title":"Sparse learned kernels for interpretable and efficient medical time series processing","authors":"Sully F. Chen, Zhicheng Guo, Cheng Ding, Xiao Hu, Cynthia Rudin","doi":"10.1038/s42256-024-00898-4","DOIUrl":"10.1038/s42256-024-00898-4","url":null,"abstract":"Rapid, reliable and accurate interpretation of medical time series signals is crucial for high-stakes clinical decision-making. Deep learning methods offered unprecedented performance in medical signal processing but at a cost: they were compute intensive and lacked interpretability. We propose sparse mixture of learned kernels (SMoLK), an interpretable architecture for medical time series processing. SMoLK learns a set of lightweight flexible kernels that form a single-layer sparse neural network, providing not only interpretability but also efficiency, robustness and generalization to unseen data distributions. We introduce parameter reduction techniques to reduce the size of SMoLK networks and maintain performance. We test SMoLK on two important tasks common to many consumer wearables: photoplethysmography artefact detection and atrial fibrillation detection from single-lead electrocardiograms. We find that SMoLK matches the performance of models orders of magnitude larger. It is particularly suited for real-time applications using low-power devices, and its interpretability benefits high-stakes situations. Deep learning excels in medical signal processing but lacks interpretability. An efficient, interpretable architecture that matches the performance of larger models at orders of magnitude fewer parameters in tasks common to wearable devices has been proposed.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":"6 10","pages":"1132-1144"},"PeriodicalIF":18.8,"publicationDate":"2024-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142236674","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Realizing full-body control of humanoid robots 实现仿人机器人的全身控制
IF 18.8 1区 计算机科学
Nature Machine Intelligence Pub Date : 2024-09-11 DOI: 10.1038/s42256-024-00891-x
Guangliang Li, Randy Gomez
{"title":"Realizing full-body control of humanoid robots","authors":"Guangliang Li, Randy Gomez","doi":"10.1038/s42256-024-00891-x","DOIUrl":"10.1038/s42256-024-00891-x","url":null,"abstract":"Using deep reinforcement learning, flexible skills and behaviours emerge in humanoid robots, as demonstrated in two recent reports.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":"6 9","pages":"990-991"},"PeriodicalIF":18.8,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142166408","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Author Correction: Integrated structure prediction of protein–protein docking with experimental restraints using ColabDock 作者更正:利用 ColabDock 对带有实验约束的蛋白质-蛋白质对接进行综合结构预测
IF 18.8 1区 计算机科学
Nature Machine Intelligence Pub Date : 2024-09-10 DOI: 10.1038/s42256-024-00905-8
Shihao Feng, Zhenyu Chen, Chengwei Zhang, Yuhao Xie, Sergey Ovchinnikov, Yi Qin Gao, Sirui Liu
{"title":"Author Correction: Integrated structure prediction of protein–protein docking with experimental restraints using ColabDock","authors":"Shihao Feng, Zhenyu Chen, Chengwei Zhang, Yuhao Xie, Sergey Ovchinnikov, Yi Qin Gao, Sirui Liu","doi":"10.1038/s42256-024-00905-8","DOIUrl":"10.1038/s42256-024-00905-8","url":null,"abstract":"","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":"6 10","pages":"1270-1270"},"PeriodicalIF":18.8,"publicationDate":"2024-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s42256-024-00905-8.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142487192","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Accelerating histopathology workflows with generative AI-based virtually multiplexed tumour profiling 利用基于生成式人工智能的虚拟多重肿瘤特征分析加速组织病理学工作流程
IF 18.8 1区 计算机科学
Nature Machine Intelligence Pub Date : 2024-09-09 DOI: 10.1038/s42256-024-00889-5
Pushpak Pati, Sofia Karkampouna, Francesco Bonollo, Eva Compérat, Martina Radić, Martin Spahn, Adriano Martinelli, Martin Wartenberg, Marianna Kruithof-de Julio, Marianna Rapsomaniki
{"title":"Accelerating histopathology workflows with generative AI-based virtually multiplexed tumour profiling","authors":"Pushpak Pati, Sofia Karkampouna, Francesco Bonollo, Eva Compérat, Martina Radić, Martin Spahn, Adriano Martinelli, Martin Wartenberg, Marianna Kruithof-de Julio, Marianna Rapsomaniki","doi":"10.1038/s42256-024-00889-5","DOIUrl":"10.1038/s42256-024-00889-5","url":null,"abstract":"Understanding the spatial heterogeneity of tumours and its links to disease initiation and progression is a cornerstone of cancer biology. Presently, histopathology workflows heavily rely on hematoxylin and eosin and serial immunohistochemistry staining, a cumbersome, tissue-exhaustive process that results in non-aligned tissue images. We propose the VirtualMultiplexer, a generative artificial intelligence toolkit that effectively synthesizes multiplexed immunohistochemistry images for several antibody markers (namely AR, NKX3.1, CD44, CD146, p53 and ERG) from only an input hematoxylin and eosin image. The VirtualMultiplexer captures biologically relevant staining patterns across tissue scales without requiring consecutive tissue sections, image registration or extensive expert annotations. Thorough qualitative and quantitative assessment indicates that the VirtualMultiplexer achieves rapid, robust and precise generation of virtually multiplexed imaging datasets of high staining quality that are indistinguishable from the real ones. The VirtualMultiplexer is successfully transferred across tissue scales and patient cohorts with no need for model fine-tuning. Crucially, the virtually multiplexed images enabled training a graph transformer that simultaneously learns from the joint spatial distribution of several proteins to predict clinically relevant endpoints. We observe that this multiplexed learning scheme was able to greatly improve clinical prediction, as corroborated across several downstream tasks, independent patient cohorts and cancer types. Our results showcase the clinical relevance of artificial intelligence-assisted multiplexed tumour imaging, accelerating histopathology workflows and cancer biology. VirtualMultiplexer is a generative AI tool that produces realistic multiplexed immunohistochemistry images from tissue biopsies. The generated images could be used to improve clinical predictions, enhancing histopathology workflows and accelerating cancer research.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":"6 9","pages":"1077-1093"},"PeriodicalIF":18.8,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s42256-024-00889-5.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142158783","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Efficient and scalable reinforcement learning for large-scale network control 用于大规模网络控制的高效可扩展强化学习
IF 18.8 1区 计算机科学
Nature Machine Intelligence Pub Date : 2024-09-03 DOI: 10.1038/s42256-024-00879-7
Chengdong Ma, Aming Li, Yali Du, Hao Dong, Yaodong Yang
{"title":"Efficient and scalable reinforcement learning for large-scale network control","authors":"Chengdong Ma, Aming Li, Yali Du, Hao Dong, Yaodong Yang","doi":"10.1038/s42256-024-00879-7","DOIUrl":"10.1038/s42256-024-00879-7","url":null,"abstract":"The primary challenge in the development of large-scale artificial intelligence (AI) systems lies in achieving scalable decision-making—extending the AI models while maintaining sufficient performance. Existing research indicates that distributed AI can improve scalability by decomposing complex tasks and distributing them across collaborative nodes. However, previous technologies suffered from compromised real-world applicability and scalability due to the massive requirement of communication and sampled data. Here we develop a model-based decentralized policy optimization framework, which can be efficiently deployed in multi-agent systems. By leveraging local observation through the agent-level topological decoupling of global dynamics, we prove that this decentralized mechanism achieves accurate estimations of global information. Importantly, we further introduce model learning to reinforce the optimal policy for monotonic improvement with a limited amount of sampled data. Empirical results on diverse scenarios show the superior scalability of our approach, particularly in real-world systems with hundreds of agents, thereby paving the way for scaling up AI systems. Applying large-scale AI systems to multi-agent scenarios in real-world settings is challenging. The authors propose a decentralized model-based policy optimization framework to enable scalable decision-making.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":"6 9","pages":"1006-1020"},"PeriodicalIF":18.8,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s42256-024-00879-7.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142123712","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A large-scale audit of dataset licensing and attribution in AI 对人工智能中的数据集许可和归属进行大规模审计
IF 18.8 1区 计算机科学
Nature Machine Intelligence Pub Date : 2024-08-30 DOI: 10.1038/s42256-024-00878-8
Shayne Longpre, Robert Mahari, Anthony Chen, Naana Obeng-Marnu, Damien Sileo, William Brannon, Niklas Muennighoff, Nathan Khazam, Jad Kabbara, Kartik Perisetla, Xinyi (Alexis) Wu, Enrico Shippole, Kurt Bollacker, Tongshuang Wu, Luis Villa, Sandy Pentland, Sara Hooker
{"title":"A large-scale audit of dataset licensing and attribution in AI","authors":"Shayne Longpre, Robert Mahari, Anthony Chen, Naana Obeng-Marnu, Damien Sileo, William Brannon, Niklas Muennighoff, Nathan Khazam, Jad Kabbara, Kartik Perisetla, Xinyi (Alexis) Wu, Enrico Shippole, Kurt Bollacker, Tongshuang Wu, Luis Villa, Sandy Pentland, Sara Hooker","doi":"10.1038/s42256-024-00878-8","DOIUrl":"10.1038/s42256-024-00878-8","url":null,"abstract":"The race to train language models on vast, diverse and inconsistently documented datasets raises pressing legal and ethical concerns. To improve data transparency and understanding, we convene a multi-disciplinary effort between legal and machine learning experts to systematically audit and trace more than 1,800 text datasets. We develop tools and standards to trace the lineage of these datasets, including their source, creators, licences and subsequent use. Our landscape analysis highlights sharp divides in the composition and focus of data licenced for commercial use. Important categories including low-resource languages, creative tasks and new synthetic data all tend to be restrictively licenced. We observe frequent miscategorization of licences on popular dataset hosting sites, with licence omission rates of more than 70% and error rates of more than 50%. This highlights a crisis in misattribution and informed use of popular datasets driving many recent breakthroughs. Our analysis of data sources also explains the application of copyright law and fair use to finetuning data. As a contribution to continuing improvements in dataset transparency and responsible use, we release our audit, with an interactive user interface, the Data Provenance Explorer, to enable practitioners to trace and filter on data provenance for the most popular finetuning data collections: www.dataprovenance.org . The Data Provenance Initiative audits over 1,800 text artificial intelligence (AI) datasets, analysing trends, permissions of use and global representation. It exposes frequent errors on several major data hosting sites and offers tools for transparent and informed use of AI training data.","PeriodicalId":48533,"journal":{"name":"Nature Machine Intelligence","volume":"6 8","pages":"975-987"},"PeriodicalIF":18.8,"publicationDate":"2024-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.nature.com/articles/s42256-024-00878-8.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142091222","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信