HRL-D3: High resolution and lightweight defective data detection for IoT data integrity

Due to the limited storage capacity of Internet of Things (IoT) devices, the use of third-party cloud storage service is an integral part of IoT based systems. Ensuring data integrity in cloud storage services is paramount for maintaining the safety and trustworthiness of the data generated and consumed by IoT applications. While verifying data integrity through defective data detection, the number of False Positives and False Negatives should be fewer so that the resolution is higher. However, increasing the resolution also incurs an increase in metadata for integrity verification and results in higher storage overhead. This paper proposes High Resolution and Lightweight Defective Data Detection (HRL-D3) for IoT data integrity with a short verification time, low storage overhead and minimal computational cost. HRL-D3 introduces 1) the use of Merkle Hash Tree and the novel concept of Intermediate Hash for enabling faster Data Integrity Verification (DIV) and higher resolution, and 2) an Adaptive Data Chunking Algorithm for balancing the trade-off between resolution and storage overhead. Our security analysis examined the risks of potential attacks to HRL-D3, and outlined the prevention provided by the proposed solution as well as the mitigation through an operational workaround. A Proof of Concept implementation HRL-D3 was evaluated and demonstrated its effectiveness in balancing the trade off between the resolution and the storage overhead tradeoff as well as achieving low-latency DIV.