Comparing and mapping difference indices of debate quality on Twitter

Abstract

Albeit the measurement of debate quality is not a new endeavour, this paper raises two research questions for which we still have limited knowledge: What are important and reliable indicators of debate quality on social media? How does debate quality relate to individual factors on social media? First, we empirically analysed how two well-established discourse’ quality indices (the DQI and CC index) correlate to each other using a random sample of 1000 tweets selected from the full history of tweets written by Swiss elected politicians between 2011 and 2021. While the sample was automatically coded for CC using LIWC, we manually annotated the tweets according to an adapted version of the DQI for social media texts. Second, we conducted a correspondence analysis to investigate the relations between these dimensions, additional debate quality features, as well as individual political factors. Results show a good correlation between both indices (r up to 0.46), while also highlighting their respective weaknesses. Furthermore, the results highlight the necessity to include alternative dimensions of debate quality (such as emotion and inclusive or exclusive views) to enhance future measurements of debate quality in the realm of social media.