Feasibility analysis of machine learning for performance-related attributional statements

Abstract

We investigate the feasibility of machine learning methods for attributional content and framing analysis in corporate reporting. We test the performance of five widely-used supervised machine learning classifiers (naïve Bayes, logistic regression, support vector machines, random forests, decision trees) in a top-down three-level hierarchical setting to (1) identify performance-related statements; (2) detect attributions in these; and (3) classify the content of the attributional statements. The training set comprises manually coded statements from a corpus of management commentary reports of listed companies. The attributions include both intra- and inter-sentential attributional statements. The results show that for both intra- and inter-sentential attributions, F1-scores of our most accurate classifier (i.e., support vector machines) vary in the range of 76% up to 94%, depending on the identification, detection and classification levels and the content characteristics of attributions. Additionally, we assess the hierarchical performance of classifiers, providing insights into a more holistic classification process for attributional statements. Overall, our results show how machine learning methods may facilitate narrative disclosure analysis by providing a more efficient way to detect and classify performance-related attributional statements. Our findings contribute to the accounting and management literature by providing a basis for implementing machine learning methodologies for research investigating attributional behavior and related impression management.