Fine-grained Emotion Classification: Class Imbalance Effects on Classifier Performance

We explore a set of machine learning experiments in fine-grained emotion classification to test different proportion of positive and negative samples in the training data with the goal to examine if class imbalance affects classifier performance. The class distribution in a tweet corpus (EmoTweet-28) labelled with 28 emotion categories varies significantly with the largest category (happiness) occurring 11.5% and the smallest category occurring only 0.2%. For each emotion category, there are far more negative examples than positive examples. Unlike conventional wisdom, downsampling the data in our skewed corpus did not improve classifier performance. However, we found that increasing the negative examples in the training data leads to lower recall but higher precision. Demonstrating how the ratio of positive and negative examples in the training data affect the performance of emotion classifiers is the main contribution of this study.