Validation of the Story World Absorption Scale through annotation of online book reviews

In this paper we present our attempt at validating a self-report instrument developed in the field of empirical literary studies to capture absorption experiences, namely the Story World Absorption Scale (SWAS, Kuijpers, et al. 2014). We used the SWAS as the foundation for a tag set, that targets mentions of absorption in online book reviews. Online book reviews posted on social media platforms are a relatively new form of reader testimonials that can be of use to researchers from different disciplines to investigate reading experience and evaluation, as well as social discourse about reading. This paper discusses the annotation tag set, which was developed through an iterative process, presented alongside a series of inter-annotator agreement studies that show the validity of our annotation process. Finally, it will discuss the validation and reconceptualization of the Story World Absorption construct, where we consider instances of systematic disagreement during annotation and discuss new categories that we added to the tag set that indicate areas where absorption theory may need to be refined.