Hardness-Aware Truth Discovery in Social Sensing Applications

This paper develops a new principled framework to solve a hardness-aware truth discovery problem in social sensing applications. Social sensing has emerged as a new application paradigm where a large crowd of social sensors (humansor devices on their behalf) are recruited to or voluntarily report observations about the physical environment at scale. These observations may be either true or false, and hence are viewed as binary claims. A fundamental problem in social sensing applications lies in ascertaining the correctness of claims and the reliability of data sources. We refer to this problem as truth discovery. Significant efforts were made to address the truth discovery problem, but an important dimension of the problem has not been fully exploited: hardness of claims (how challenging a claim is to be made). A common assumption made in the previous work is that they assumed all claims areof the same degree of hardness. However, in real world social sensing applications, simply ignoring the hardness differences between claims could easily lead to suboptimal truth discovery results. In this paper, we develop a new hardness-aware truth discovery scheme that explicitly considers different hardness degrees of claims into a rigorous analytical framework. The new truth discovery scheme solves a maximum likelihood estimation problem to determine both the claim correctness and the source reliability. We compare our hardness-aware scheme with the state-of-the-art baselines through three real world case studies(Baltimore Riots, Paris Attack and Oregon Shootings, all in 2015) using Twitter data feeds. The evaluation results showed that our new scheme outperforms all compared baselines and significantly improves the truth discovery accuracy in social sensing applications.