pyMDMA: Multimodal data metrics for auditing real and synthetic datasets

Abstract

Data auditing plays a critical role in ensuring the reliability and robustness of machine learning models. Existing repositories often lack comprehensive validation across modalities and clear metric categorization. This inconsistency can lead to confusion and hinder effective dataset evaluation and model benchmarking. pyMDMA introduces an open-source library that unifies auditing metrics for time series, tabular, and image data, proposing a structured taxonomy to clarify their purpose. The library serves as a centralized resource for researchers and practitioners, promoting robust dataset assessment. This open-source initiative fosters community-driven contributions, advancing data auditing practices and making them more accessible to a wider audience. Currently, the library includes 48 metric implementations across the data modalities.