Marimba: A Python framework for structuring and processing FAIR scientific image datasets

IF 2.4 4区计算机科学 Q2 COMPUTER SCIENCE, SOFTWARE ENGINEERING

SoftwareX Pub Date : 2025-07-03 DOI:10.1016/j.softx.2025.102251

Christopher J. Jackett , Kevin Barnard , Franziska Althaus , Nicolas Mortimer , David Webb , Candice Untiedt , Aaron Tyndall , Ian Jameson , Bec Gorton , Carlie Devine , Joanna Strzelecki , Peter H. Thrall , Ben Scoulding

{"title":"Marimba: A Python framework for structuring and processing FAIR scientific image datasets","authors":"Christopher J. Jackett , Kevin Barnard , Franziska Althaus , Nicolas Mortimer , David Webb , Candice Untiedt , Aaron Tyndall , Ian Jameson , Bec Gorton , Carlie Devine , Joanna Strzelecki , Peter H. Thrall , Ben Scoulding","doi":"10.1016/j.softx.2025.102251","DOIUrl":null,"url":null,"abstract":"<div><div>The rapid advancement of scientific imaging technologies has created significant challenges in managing large-scale image datasets while maintaining compliance with FAIR (Findable, Accessible, Interoperable, and Reusable) data principles. We present Marimba, an open-source Python framework for structuring, processing, and packaging scientific image datasets. Marimba enhances data management through unified workflow processing, automated metadata embedding, efficient data handling, and standardized dataset packaging while integrating with the image FAIR Digital Object (iFDO) metadata standard. The framework's capabilities were evaluated through four diverse marine case studies involving multi-instrument microscopy, automated plankton imagery, deep-sea coral surveys, and historical image digitization. Marimba successfully processed datasets ranging from thousands to hundreds of thousands of images and videos, demonstrating robust performance and scalability. Marimba's modular architecture enables customization for specific research requirements while ensuring consistent data management practices. Results demonstrate Marimba's potential to advance scientific image data management by improving workflow efficiency, data quality, and adherence to FAIR principles throughout the research data lifecycle.</div></div>","PeriodicalId":21905,"journal":{"name":"SoftwareX","volume":"31 ","pages":"Article 102251"},"PeriodicalIF":2.4000,"publicationDate":"2025-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"SoftwareX","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2352711025002183","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, SOFTWARE ENGINEERING","Score":null,"Total":0}

引用次数: 0

Abstract

The rapid advancement of scientific imaging technologies has created significant challenges in managing large-scale image datasets while maintaining compliance with FAIR (Findable, Accessible, Interoperable, and Reusable) data principles. We present Marimba, an open-source Python framework for structuring, processing, and packaging scientific image datasets. Marimba enhances data management through unified workflow processing, automated metadata embedding, efficient data handling, and standardized dataset packaging while integrating with the image FAIR Digital Object (iFDO) metadata standard. The framework's capabilities were evaluated through four diverse marine case studies involving multi-instrument microscopy, automated plankton imagery, deep-sea coral surveys, and historical image digitization. Marimba successfully processed datasets ranging from thousands to hundreds of thousands of images and videos, demonstrating robust performance and scalability. Marimba's modular architecture enables customization for specific research requirements while ensuring consistent data management practices. Results demonstrate Marimba's potential to advance scientific image data management by improving workflow efficiency, data quality, and adherence to FAIR principles throughout the research data lifecycle.

查看原文本刊更多论文

Marimba：用于构建和处理FAIR科学图像数据集的Python框架

科学成像技术的快速发展为管理大规模图像数据集带来了重大挑战，同时保持符合FAIR（可查找、可访问、可互操作和可重用）数据原则。我们介绍了Marimba，一个用于构建、处理和包装科学图像数据集的开源Python框架。Marimba通过统一的工作流处理、自动元数据嵌入、高效的数据处理和标准化的数据集打包来增强数据管理，同时集成了图像FAIR数字对象（iFDO）元数据标准。通过四个不同的海洋案例研究，包括多仪器显微镜、自动浮游生物成像、深海珊瑚调查和历史图像数字化，对该框架的能力进行了评估。Marimba成功处理了从数千到数十万个图像和视频的数据集，展示了强大的性能和可扩展性。Marimba的模块化架构可以定制特定的研究需求，同时确保一致的数据管理实践。结果表明，通过提高工作流程效率、数据质量和在整个研究数据生命周期中遵守公平原则，Marimba有潜力推进科学图像数据管理。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

SoftwareX COMPUTER SCIENCE, SOFTWARE ENGINEERING-

CiteScore

5.50

自引率

2.90%

发文量

184

审稿时长

9 weeks

期刊介绍： SoftwareX aims to acknowledge the impact of software on today''s research practice, and on new scientific discoveries in almost all research domains. SoftwareX also aims to stress the importance of the software developers who are, in part, responsible for this impact. To this end, SoftwareX aims to support publication of research software in such a way that: The software is given a stamp of scientific relevance, and provided with a peer-reviewed recognition of scientific impact; The software developers are given the credits they deserve; The software is citable, allowing traditional metrics of scientific excellence to apply; The academic career paths of software developers are supported rather than hindered; The software is publicly available for inspection, validation, and re-use. Above all, SoftwareX aims to inform researchers about software applications, tools and libraries with a (proven) potential to impact the process of scientific discovery in various domains. The journal is multidisciplinary and accepts submissions from within and across subject domains such as those represented within the broad thematic areas below: Mathematical and Physical Sciences; Environmental Sciences; Medical and Biological Sciences; Humanities, Arts and Social Sciences. Originating from these broad thematic areas, the journal also welcomes submissions of software that works in cross cutting thematic areas, such as citizen science, cybersecurity, digital economy, energy, global resource stewardship, health and wellbeing, etcetera. SoftwareX specifically aims to accept submissions representing domain-independent software that may impact more than one research domain.