A Quest for Transparent and Reproducible Text-Mining Methodologies in Computational Social Science

Abstract

We thank the editorial board for the opportunity to discuss our methodological contribution in a symposium dialogue as well as the two commentators for their inspiring and challenging comments. We are especially delighted that the commentators agree on the relevance of analyzing the dynamics of manifest and latent meanings in big data using different textmining tools in general and for map analysis in particular. According to our reading, the commentators focused on quality criteria, namely, two different but highly relevant aspects of transparency in research processes. Laura K. Nelson (this volume, pp. 139–143) focused on transparency in the context of research foci and analytical steps in a text-analysis project to ensure the reproducibility of results, whereas Burt L. Monroe (this volume, pp. 132–139) focused on transparency regarding data inspection and thus the credibility of results. We structured our rejoinder as follows: First, we draw on selected aspects Nelson and Monroe posed that we believe they consider to be most important to reflect on transparency in the context of big data and text-mining tools. Second, because quality criteria such as transparency do not exist in isolation, we complement the discussion on quality by adding general issues regarding overarching textmining methodology. Finally, we conclude by providing a prospect for further establishment of big data analysis in the social sciences.