Advancing research software engineering with AI: a research framework

Abstract

Research software has become a central pillar of scientific discovery, yet its engineering quality, sustainability, and reproducibility vary widely across projects. At the same time, advances in artificial intelligence (AI), particularly generative AI (GenAI), are rapidly transforming how software is developed. While these tools promise productivity gains, their broader impact on research software engineering practices remains poorly understood at scale. In this study, we present a large-scale empirical analysis of AI-assisted research software engineering. We analyzed 1,510 open-source research software repositories retrieved from Zenodo using the IEEE Taxonomy 2025 top-level categories (598 query terms), restricted to records labeled Software and created after November 2022 (post-GenAI emergence), with duplicate and incomplete entries removed. To distinguish archival dissemination from active development, we separate Zenodo-only artifacts from records linked to evolving GitHub repositories and enrich the latter with repository-level development indicators. Our analysis integrates multiple dimensions, including software engineering maturity (e.g., documentation, automation, testing, and releases), FAIRness for research software (FAIR4RS metadata indicators), inferred AI and GenAI usage, and operational signals related to AIOps and MLOps practices. Based on these indicators, we propose and empirically ground a quadrant-based model that characterizes research software development modes along the axes of engineering maturity and AI integration. The results show that AI-assisted practices are increasingly present in research software, but their adoption remains uneven and often decoupled from established engineering disciplines. Repositories classified as AI4RSE exhibit longer active lifespans, stronger maintenance signals, and higher FAIR alignment than exploratory or informally developed projects. At the same time, a substantial fraction of Zenodo artifacts represent archival snapshots rather than evolving software, highlighting the importance of interpreting engineering indicators in light of dissemination intent. This work contributes (i) a large-scale empirical characterization based on 1,510 repositories of AI-assisted research software development, (ii) an integrated analytical framework combining software engineering, FAIRness, AI usage, and operational practices, and (iii) evidence-based insights into the opportunities and challenges of responsible and sustainable AI4RSE. Together, these contributions provide a foundation for future research and practical guidance on integrating AI into research software engineering.