Predicting the truck factor in a software repository using machine learning

IF 4.3 2区计算机科学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS

Information and Software Technology Pub Date : 2025-05-09 DOI:10.1016/j.infsof.2025.107765

Ahmed El Cheikh Ammar, Sukru Eraslan, Yeliz Yesilada

{"title":"Predicting the truck factor in a software repository using machine learning","authors":"Ahmed El Cheikh Ammar, Sukru Eraslan, Yeliz Yesilada","doi":"10.1016/j.infsof.2025.107765","DOIUrl":null,"url":null,"abstract":"<div><h3>Context:</h3><div>The Truck or Bus factor is a metric that evaluates which developers would cause the development process in a software project to decelerate should they get removed (or hit by a truck/bus). Measuring the truck factor in software development is complex due to the many variables involved. Several algorithms have been developed to address this. However, they suffer from the fact that they tend to tunnel vision on code-centric metrics such as commits made by a developer. While such a feature is important in assessing the contribution of a developer, it does not tell the whole story behind a contribution.</div></div><div><h3>Objective:</h3><div>This paper aims to consider a comprehensive set of version control system (VCS) features, including those that have not yet been investigated in the literature, with Machine Learning (ML) to predict Truck Factor.</div></div><div><h3>Method:</h3><div>We examine what features existing algorithms utilize and then design a feature set that addresses various coding-based metrics, collaborative behaviors, developer activity patterns, and the broader technological context of a project. Afterwards, multiple supervised ML models with different algorithms, such as Random Forest, Naive Bayes, etc., are designed to utilize this feature set to predict the key contributors in GitHub repositories, ultimately computing the truck factor, and then these ML models are compared with the literature.</div></div><div><h3>Results:</h3><div>Random Forest with hypertuned parameters and an aggregated model of hypertuned Random Forest and Naive Bayes with priors achieve the best performance, with mean F1-Scores of 84.1% and 86.4%, respectively. These models outperform existing algorithms except one of them, which lagged slightly behind in terms of precision.</div></div><div><h3>Conclusion:</h3><div>Our research addresses the limitations of existing work by investigating a wider range of VCS features and developing a supervised ML model to predict the truck factor, which demonstrates robust identification of true Truck Factor members.</div></div>","PeriodicalId":54983,"journal":{"name":"Information and Software Technology","volume":"184 ","pages":"Article 107765"},"PeriodicalIF":4.3000,"publicationDate":"2025-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Information and Software Technology","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0950584925001041","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}

引用次数: 0

Abstract

Context:

The Truck or Bus factor is a metric that evaluates which developers would cause the development process in a software project to decelerate should they get removed (or hit by a truck/bus). Measuring the truck factor in software development is complex due to the many variables involved. Several algorithms have been developed to address this. However, they suffer from the fact that they tend to tunnel vision on code-centric metrics such as commits made by a developer. While such a feature is important in assessing the contribution of a developer, it does not tell the whole story behind a contribution.

Objective:

This paper aims to consider a comprehensive set of version control system (VCS) features, including those that have not yet been investigated in the literature, with Machine Learning (ML) to predict Truck Factor.

Method:

We examine what features existing algorithms utilize and then design a feature set that addresses various coding-based metrics, collaborative behaviors, developer activity patterns, and the broader technological context of a project. Afterwards, multiple supervised ML models with different algorithms, such as Random Forest, Naive Bayes, etc., are designed to utilize this feature set to predict the key contributors in GitHub repositories, ultimately computing the truck factor, and then these ML models are compared with the literature.

Results:

Random Forest with hypertuned parameters and an aggregated model of hypertuned Random Forest and Naive Bayes with priors achieve the best performance, with mean F1-Scores of 84.1% and 86.4%, respectively. These models outperform existing algorithms except one of them, which lagged slightly behind in terms of precision.

Conclusion:

Our research addresses the limitations of existing work by investigating a wider range of VCS features and developing a supervised ML model to predict the truck factor, which demonstrates robust identification of true Truck Factor members.

查看原文本刊更多论文

使用机器学习预测软件存储库中的卡车因素

上下文：卡车或公共汽车因素是一个度量标准，用于评估哪些开发人员在被移除（或被卡车/公共汽车击中）时会导致软件项目中的开发过程减速。由于涉及到许多变量，测量软件开发中的卡车因素是复杂的。已经开发了几种算法来解决这个问题。然而，他们的缺点是，他们倾向于在以代码为中心的指标（如开发人员所做的提交）上狭隘地看待问题。虽然这样的特性在评估开发人员的贡献时很重要，但它并不能说明贡献背后的全部情况。目的：本文旨在考虑一套全面的版本控制系统（VCS）功能，包括那些尚未在文献中进行研究的功能，并使用机器学习（ML）来预测卡车因子。方法：我们检查现有算法利用的特性，然后设计一个特性集，处理各种基于编码的度量、协作行为、开发人员活动模式和项目的更广泛的技术背景。然后，设计多个不同算法的有监督ML模型，如Random Forest、Naive Bayes等，利用该特征集预测GitHub存储库中的关键贡献者，最终计算卡车因子，然后将这些ML模型与文献进行比较。结果：参数超调随机森林和超调随机森林与具有先验的朴素贝叶斯的聚合模型的性能最好，平均F1-Scores分别为84.1%和86.4%。这些模型优于现有的算法，只有其中一个在精度方面稍微落后。结论：我们的研究解决了现有工作的局限性，研究了更广泛的VCS特征，并开发了一个有监督的ML模型来预测卡车因子，该模型证明了对真正卡车因子成员的鲁棒性识别。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Information and Software Technology 工程技术-计算机：软件工程

CiteScore

9.10

自引率

7.70%

发文量

164

审稿时长

9.6 weeks

期刊介绍： Information and Software Technology is the international archival journal focusing on research and experience that contributes to the improvement of software development practices. The journal''s scope includes methods and techniques to better engineer software and manage its development. Articles submitted for review should have a clear component of software engineering or address ways to improve the engineering and management of software development. Areas covered by the journal include: • Software management, quality and metrics, • Software processes, • Software architecture, modelling, specification, design and programming • Functional and non-functional software requirements • Software testing and verification & validation • Empirical studies of all aspects of engineering and managing software development Short Communications is a new section dedicated to short papers addressing new ideas, controversial opinions, "Negative" results and much more. Read the Guide for authors for more information. The journal encourages and welcomes submissions of systematic literature studies (reviews and maps) within the scope of the journal. Information and Software Technology is the premiere outlet for systematic literature studies in software engineering.