Sentiment Analysis of User Feedback on Business Processes

Abstract

Business Process Management (BPM) is an established discipline that uses business processes for organizing the operations of an enterprise. The enterprises that embrace BPM continuously analyze their processes and improve them to achieve competitive edge. Consequently, a plethora of studies have developed contrasting approaches to analyze business processes. These approach vary from examining event logs of process-aware information systems to employing the data warehousing technology for analyzing the execution logs of business processes. In contrast to these classical approaches, this work proposes to combine two prominent domains, BPM and Natural Language Processing, for analyzing business processes. In particular, this study has proposed to perform sentiment analysis of end-user feedback on business processes to assess the satisfaction level of end-users. More specifically, firstly, a structured approach is used to develop a corpus of over 7000 user-feedback sentences. Secondly, these feedback sentences are annotated at three levels of classification, where, the first-level classification determines the relevance of a sentence to the process. Whereas, the second-level classifies the relevant sentences across four process performance dimensions, time, cost, quality and flexibility, and the third-level classifies the sentences into positive, negative, or neutral sentiments. Finally, 78 experiments are performed to determine the effectiveness of six supervised learning techniques and one state-of-the-art deep learning technique for the automatic classification of user feedback sentences at three levels of classifications. The results show that deep learning technique is most effective for the classification tasks.