Identifying and Translating Subjective Content Descriptions Among Texts

Abstract

An agent pursuing a task may work with a corpus of documents as a reference library. Subjective content descriptions (SCDs) provide additional data that add value in the context of the agent’s task. In the pursuit of documents to add to the corpus, an agent may come across new documents where content text and SCDs from another agent are interleaved and no distinction can be made unless the agent knows the content from somewhere else. Therefore, this paper presents a hidden Markov model-based approach to identify SCDs in a new document where SCDs occur inline among content text. Additionally, we present a dictionary selection approach to identify suitable translations for content text and SCDs based on [Formula: see text]-grams. We end with a case study evaluating both approaches based on simulated and real-world data.