Identification of Software Problem Report Types Using Multiclass Classification

Abstract

Users often experience failures, have problems, or have further requests with regard to the software they use. Software companies provide customer care service or customer support to handle such issues or problems which sometimes can be resolved right away and sometimes have to be forwarded to responsible persons. Efficiency of problem handling is very important to software companies to maintain customer satisfaction. This paper reports a case of a software company in Thailand whose derivatives trading software is used by a large number of broker companies and their customers. The software company has experienced problems where the reported software problems are classified incorrectly and hence are directed to the wrong persons and have to be reclassified. Assigning the problem reports to the responsible persons in a timely and correct manner is crucial especially for the nature of the trading software. This paper presents a multiclass classification method to classify 11 problem report types that are found in this trading software. Machine learning algorithms that are applied include Multinomial Naïve Bayes, Linear SVC, Random Forest, and Logistic Regression, and consider both lexical features and metadata of the problem reports. In an experiment, Linear SVC performed best, having the F1 score of 91.69% and accuracy of 91.79% when using unigram and trigram features of the problem report text which is written in Thai and English. The paper presents a support tool for classifying new problem reports and providing a dashboard of the problems found in this derivatives trading software for the software team to manage its maintenance.