Hybrid approach for multilevel multi-class requirement classification: Impact of stop-word removal and data augmentation

Abstract

Requirement classification in software engineering is essential for effective development. Automating this process reduces human effort and enhances decision-making. Previous studies experimented with machine learning and deep learning models to classify requirements. This novel research fills that gap by evaluating transformer-based models and a proposed Hybrid Stacked Model for multilevel, multi-class classification task. To address the limitations of existing software requirement datasets (imbalanced dataset, insufficient granularity, real world examples), we combined instances from the PROMISE_exp dataset, PURE corpus, and 20 manually collected software requirement specifications (SRS) documents using a Boolean keyword search to create a multilevel, multi-class dataset. These 3072 combined requirements are organized into a two-level hierarchy: Level 1 (functional (FR)/non-functional (NFR)); Level 2 (FRs: core functional (CFR)/derived functional (DFR)/system integration (SI)/external dependency (ED); NFRs: product (PR)/organizational (OR)/external (ER)). We applied BERT-based context-aware text augmentation to address class imbalance by expanding the dataset to 3343 instances. This study also investigates the effects of domain-specific stopword removal and text augmentation on model performance. Results show that text augmentation boosts accuracy by 0.2–3.76% across all models. Stopword removal enhances precision and recall by reducing noise, but it slightly lowers overall accuracy due to the loss of some semantic cues. The proposed Hybrid Stacked Model outperformed all pre-trained transformer models, achieving the highest accuracy of 96.77% at Level 1 and 83.06% at Level 2. A statistical t-test confirms the significance of these improvements. These findings emphasize the importance of hybrid models and domain-specific data preprocessing in enhancing requirement classification, with practical implications for automating early-stage software engineering tasks.