What are the emotions of developers towards deep learning documentation? — An exploratory study on Stack Overflow posts

Abstract

Context:

Non native machine learning and deep learning (DL) developers face several challenges in using DL frameworks owing to the issues persistent in DL documentation. However, there are no studies that explore the reasons for issues in documentation.

Objective:

Investigating the underlying emotions in developer discussions on documentation could help in identifying reasons for issues in documentation. Hence, in this study, we analyse emotions of Stack Overflow posts corresponding to documentation of DL frameworks.

Methodology:

We identify relevant deep-learning related tags using integrated snowballing approach and extract 159.2K posts related to DL. We then identify documentation related posts among these using keyword matching approach, which resulted in 13,572 DL documentation related posts. We use Random Forest Classifier to build six emotion classifier models based on Gold Label Dataset for emotions. We then classify the extracted posts into each of the six emotions — Anger, Fear, Love, Joy, Sadness and Surprise using the classifier models, and curate the results.

Results:

We observe a large expression of anger and sadness, with more than half of posts having ‘yolo’ and ‘activation-function’ tags exhibiting these emotions, while Love emotion is predominantly present in posts with ‘theano’ tag. During our analysis, we observed that 40% of ‘Body’ and ‘Answer’ posts exhibited anger and sadness emotions.

Conclusion:

Our study reveals the large presence of Anger, Fear and Sadness emphasizing the need to improve DL framework documentation. Specifically, maintainers of the ‘yolo’ and ‘matcaffe’ libraries could improve their documentation, as the corresponding posts exhibit more of Anger and Sadness.