Extracting emotion topics from blog sentences: use of voting from multi-engine supervised classifiers

Abstract

This paper presents a supervised multi-engine classifier approach followed by voting to identify emotion topic(s) from English blog sentences. Manual annotation of the English blog sentences in the training set has shown a satisfactory agreement with kappa (κ) measure of 0.85 and MASI (Measure of Agreement on Set-valued Items) measure of 0.82 for emotion topic spans. The baseline system based on object related dependency relations includes the topic oriented thematic roles present in the verb based syntactic frame of the sentences. In contrast, the supervised approach consists of three classifiers, Conditional Random Field (CRF), Support Vector Machine (SVM) and a Fuzzy Classifier (FC). The important features are incorporated based on the ablation study of all features and Information Gain Based Pruning (IGBP) on the development set. One or more emotion topics associated with focused target span are identified based on the majority voting of the classifiers. The supervised multi-engine classifier system has been evaluated with average F-scores of 70.51% and 90.44% for emotion topic and target span identification respectively on 500 gold standard test sentences and has outperformed the baseline system.