Hierarchical multi-label classification for concrete defects: An industrial case study at Vermeg

Abstract

Effective software defects management is particularly advantageous for companies that rely on service solutions, as it helps minimize risks and improve resolution monitoring. Many existing approaches have aimed at tracking, locating and classifying defects more reliably. When it comes to practice, we have found that defects are inherently organized in hierarchies based on class inclusion. Taking advantage of these established taxonomies, we report in this paper our experience of deploying a hierarchical multi-label defects classification approach, within a development team in a banking and finance software company. Overall, we gathered over 2000 defect reports, coming from their agile reporter calculation engine. The collected dataset was then balanced and enriched with synthetically generated defects using Meta’s LLM Llama 3.1 405B. Key findings reveal that our approach not only provides better interpretability of overlapping categories but also results in significantly better performance results than traditional flat feedforward neural networks and transformers-based large language models.