{"title":"This Microtubule Does Not Exist: Super-Resolution Microscopy Image Generation by a Diffusion Model.","authors":"Alon Saguy, Tav Nahimov, Maia Lehrman, Estibaliz Gómez-de-Mariscal, Iván Hidalgo-Cenalmor, Onit Alalouf, Ashwin Balakrishnan, Mike Heilemann, Ricardo Henriques, Yoav Shechtman","doi":"10.1002/smtd.202400672","DOIUrl":null,"url":null,"abstract":"<p><p>Generative models, such as diffusion models, have made significant advancements in recent years, enabling the synthesis of high-quality realistic data across various domains. Here, the adaptation and training of a diffusion model on super-resolution microscopy images are explored. It is shown that the generated images resemble experimental images, and that the generation process does not exhibit a large degree of memorization from existing images in the training set. To demonstrate the usefulness of the generative model for data augmentation, the performance of a deep learning-based single-image super-resolution (SISR) method trained using generated high-resolution data is compared against training using experimental images alone, or images generated by mathematical modeling. Using a few experimental images, the reconstruction quality and the spatial resolution of the reconstructed images are improved, showcasing the potential of diffusion model image generation for overcoming the limitations accompanying the collection and annotation of microscopy images. Finally, the pipeline is made publicly available, runnable online, and user-friendly to enable researchers to generate their own synthetic microscopy data. This work demonstrates the potential contribution of generative diffusion models for microscopy tasks and paves the way for their future application in this field.</p>","PeriodicalId":229,"journal":{"name":"Small Methods","volume":" ","pages":"e2400672"},"PeriodicalIF":10.7000,"publicationDate":"2024-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Small Methods","FirstCategoryId":"88","ListUrlMain":"https://doi.org/10.1002/smtd.202400672","RegionNum":2,"RegionCategory":"材料科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"CHEMISTRY, PHYSICAL","Score":null,"Total":0}
引用次数: 0
Abstract
Generative models, such as diffusion models, have made significant advancements in recent years, enabling the synthesis of high-quality realistic data across various domains. Here, the adaptation and training of a diffusion model on super-resolution microscopy images are explored. It is shown that the generated images resemble experimental images, and that the generation process does not exhibit a large degree of memorization from existing images in the training set. To demonstrate the usefulness of the generative model for data augmentation, the performance of a deep learning-based single-image super-resolution (SISR) method trained using generated high-resolution data is compared against training using experimental images alone, or images generated by mathematical modeling. Using a few experimental images, the reconstruction quality and the spatial resolution of the reconstructed images are improved, showcasing the potential of diffusion model image generation for overcoming the limitations accompanying the collection and annotation of microscopy images. Finally, the pipeline is made publicly available, runnable online, and user-friendly to enable researchers to generate their own synthetic microscopy data. This work demonstrates the potential contribution of generative diffusion models for microscopy tasks and paves the way for their future application in this field.
Small MethodsMaterials Science-General Materials Science
CiteScore
17.40
自引率
1.60%
发文量
347
期刊介绍:
Small Methods is a multidisciplinary journal that publishes groundbreaking research on methods relevant to nano- and microscale research. It welcomes contributions from the fields of materials science, biomedical science, chemistry, and physics, showcasing the latest advancements in experimental techniques.
With a notable 2022 Impact Factor of 12.4 (Journal Citation Reports, Clarivate Analytics, 2023), Small Methods is recognized for its significant impact on the scientific community.
The online ISSN for Small Methods is 2366-9608.