Enhancing decision-making by leveraging human intervention in large-scale sensor networks

Abstract

Extensive deployment of sensor networks in recent years has led to the generation of large volumes of data. One approach to processing such large volumes of data is to rely on parallelized approaches based on architectures such as MapReduce. However, fully-automated processing without human intervention is error prone. Supporting human involvement in processing pipelines of data in a variety of contexts such as warfare, cyber security, threat monitoring, and malware analysis leads to improved decision-making. Although this kind of human-machine collaboration seems straightforward, involving a human operator into an automated processing pipeline presents some challenges. For example, due to the asynchronous nature of the human intervention, care must be taken to ensure that once a user-made correction or assertion is introduced, all necessary adjustment and reprocessing is performed. In addition, to make the best use of limited resources and processing capabilities, reprocessing of data in light of such corrections must be minimized. This paper introduces an innovative approach for human-machine integration in decisionmaking for large-scale sensor networks that rely on the popular Hadoop MapReduce framework.