How Much Undocumented Knowledge is there in Agile Software Development?: Case Study on Industrial Project Using Issue Tracking System and Version Control System

Abstract

In agile software development projects, software engineers prioritize implementation over documentation to eliminate needless documentation. Is the cost of missing documentation greater than the cost of producing unnecessary or unused documentation? Even without these documents, software engineers maintain other software artifacts, such as tickets in an issue tracking system (ITS) or source code committed to a version control system (VCS). Do these artifacts contain the necessary knowledge? In this paper, we examine undocumented knowledge in an agile software development project at NTT. For our study, we collected 159 commit logs in a VCS and 102 tickets in the ITS from the three-month period of the project. We propose a ticket-commit network chart (TCC) that visually represents time-series commit activities along with filed issue tickets. We also implement a tool to generate the TCC using both commit log and ticket data. Our study revealed that in 16% of all commits, software engineers committed source code to the VCS without a corresponding issue ticket in the ITS. Had these commits been based on individual issue tickets, these "unissued" tickets would have accounted for 20% of all tickets. Software users and requirements engineers also evaluated the contents of these commits and found that 42% of the "unissued" tickets were required for software operation and 23% of those were required for requirements modification.