More than just sentiment: Using social, cognitive, and behavioral information of social media to predict stock markets with artificial intelligence and big data

Abstract

Digital transformation offers unprecedented opportunities to access data on hard-to-measure social aspects. In this digital era, social media platforms have become critical data sources for the social sciences. This study moves beyond traditional finance assumptions of “perfect information,” “rational humans,” and “isolated individuals” by analyzing retail investor behavior using Twitter data. It adopts a human model characterized by incomplete information, bounded rationality, and the influence of social and emotional factors. Tweets shared between January 1, 2012, and February 28, 2020, were collected. A GRU-based context classifier achieved 98% accuracy in identifying tweets related to Borsa Istanbul (BIST). Sentiment classification using a BERT model achieved 91% accuracy for positive and negative classes. Relationships between Twitter-obtained features and BIST indices were analyzed using machine learning methods such as linear regression, Lasso regression, random forest, and XGBoost. The analysis revealed that 91% of the change in the opening value, 63% of the change in trading volume, and 67% in volatility of the BIST 100 index could be attributed to cognitive, behavioral, and social features gleaned from tweets.