Spatio-temporal Conditioned Language Models

Abstract

The ubiquitous availability of mobile devices with GPS capabilities and the popularity of social media platforms have created a rich source for textual data with spatio-temporal information. Also, other domains like crime incident description and search engine queries, can provide spatio-temporal textual data. These data sources can be used to discover space-time related insights of human behavior. This work focuses on modeling text that is associated with a particular time and place. We extend the traditional language modeling task from natural language processing to language modeling under spatio-temporal conditions. This task definition allows us to use the same evaluation framework used in language modeling. A model for spatio-temporal text data representation should be able to capture the patterns that guide how text is generated in a spatio-temporal context. We aim to develop neural network models for language modeling conditioned on spatio-temporal variables with the ability to capture properties such as: neighborhood, periodicity and hierarchy.