Context-aware data driven sensor data analysis: With application to H2S concentration prediction in urban drainage networks

This paper presents a context-aware data-driven approach for the analysis of big data from sensors. Different from conventional methods, this approach incorporates exogenous variables or contextual information that influences the dynamic behaviour of the monitored system. In the context of water distribution systems, for example, key system variables including water demand variations and pressure are significantly affected by factors like time of day, the day of the week, unusual events, seasonal variations and weather conditions. This contextual information creates dynamic relationships between water demand and pressure, which are critical for understanding system behaviour. Specifically, the context-aware method will use present and past observed values from sensors (which are normally time-series data recording the system’s dynamic behaviour), in addition to also including contextual information regarding the spatial context (e.g., the correlation between the values of different sensors) and temporal context (e.g., correlation between observed values and days of the week and time of the day). The method is applied to the prediction of Hydrogen Sulphide (H₂S) concentration in a real-world urban drainage network, based on the analysis of big real-time data sets from different sensors. Although the datasets are variables with non-uniform time intervals, uncertainties, and faulty data, the context-aware method identifies the correlations among different datasets to predict the concentration of H₂S with high accuracy (R² > 0.92; RMSE = 0.029). The method is also proven robust for a Deep Neural Networks approach.