Security-aware RPL: Designing a novel objective function for risk-based routing with rank evaluation

Routing Protocol for Low-Power and Lossy Networks (RPL) is a widely used routing protocol in the Internet of Things (IoT) environments due to its suitability for resource-constrained devices and lossy network conditions. However, the inherent vulnerabilities in RPL pose significant security risks to IoT deployments. To address these challenges, we propose a security-aware routing approach that modifies objective functions (OFs) in terms of risk and vulnerability aspects. Among these vulnerabilities, rank attacks are particularly critical, as they exploit RPL’s core mechanism for route optimization by manipulating rank values to disrupt network performance. Detecting such attacks is crucial, as they can lead to suboptimal routing, increased energy consumption, and network instability, severely impacting IoT operations. To the best of our knowledge, it is the first approach to making the RPL algorithm security-aware by conducting a risk- and vulnerability-focused routing. In our approach, a step-by-step vulnerability-oriented security model is applied. First, we establish the IoT topology using communication range metrics to connect devices. We identify and analyze potential security vulnerabilities in RPL using established databases such as the National Vulnerability Database (NVD) and the Common Vulnerability Scoring System (CVSS). Based on these analyses, a set of OFs is defined to guide RPL routing decisions. The OF formulation incorporates factors such as CVSS values, vulnerability severity, and network topology. Risk levels are measured at device, path, and network-levels, leveraging these OFs. The proposed security-aware routing procedure dynamically adapts routing behavior based on the defined OFs, integrating risk assessment mechanisms into the routing process. This enables the protocol to prioritize routes with lower security risks while avoiding those vulnerable to potential attacks. Additionally, rank attacks are detected by identifying malicious nodes that manipulate rank values. To evaluate the proposed method, comparisons are made with existing procedures regarding running time and asymptotic complexity. The results demonstrate better performance in parent selection and rank attack detection, highlighting the effectiveness of our approach in enhancing the security of RPL-based IoT networks.