KoNKS: konsensus-style network koordinate system

Abstract

A network coordinate system [7, 14, 15] assigns virtual coordinates (network positions) to every node in the network. These coordinates are assigned so that the coordinate distance between two nodes reflects the real network distance between those two nodes. This allows any peer in the sytem to accurately estimate the network distance between any pair of nodes, without having the pair of nodes contact each other. Network coordinate systems' ability to predict the network latency between arbitrary pairs of nodes can be used in many applications: finding the closest node to download content from in a content distribution network or route to in a peer-to-peer system [18], reducing inter-ISP communication [5, 13], reducing the amount of state stored in routers [1], performing byzantine leader elections [6], and detecting Sybil attackers [3, 8]. Current network coordinate systems have been shown to have good accuracy in predicting network distances, low processing and communication overhead, and fast convergence to stable positions. More recent papers have improved on the earlier designs by providing coordinate stability under churn and convergence under measurement uncertainty [2, 7, 11, 12]. However, it has also been shown [10] that those network coordinate systems are not secure, in the sense that a malicious peer in the network can report randomly chosen coordinates or maliciously delay responses to disrupt the network coordinate system. The fake reported coordinates or round-trip time (RTT) causes the nodes in the system to incorrectly update their coordinates. This renders the network latency prediction useless because the coordinate distance between two nodes will not reflect the real network distance between the two nodes. Moreover, the adversary could "lie" about its coordinates so that the coordinate distance between itself and a targeted node is smaller than the real network distance. In some applications, the adversary will then be more likely to be contacted or picked as a peer to download content from. Several schemes [9, 16, 17, 19, 20] have been developed to protect network coordinate systems against the attacks in [10], where malicious peers report randomly chosen coordinates, report random but consistent coordinates, or add random delay in their messages to other peers. These schemes can be categorized into anomaly/outlier detection [9, 20], reputation system [16], and distributed reputation systems [17, 19]; all of them were shown to effectively mitigate the known attacks. Recently, however, a new type of attack [4] -- the frog-boiling attack -- was introduced, and it was shown that some of these schemes fail to protect against this attack. The frog-boiling attacker reports small but consistent lies that are not detected by any of the security mechanisms, but which cumulatively introduce unacceptable errors; for example, it was shown that this technique can randomly partition an overlay using a secure network coordinate system [20]. One of the issues is that the current secure schemes aimed only to "patch" against the known attacks. This could lead to an arms race where new attacks which they did not consider, bypass existing security mechanisms, resulting in new improved schemes to defend against the new attack, and so on. To avoid this arms race, we evaluate a network coordinate system in terms of an explicit security goal -- an invariant that should hold despite the presence and actions of an attacker -- under a concrete threat model that states what resources the attacker can marshall. The two goals are 1) an attacker's influence on either the network distance or coordinate distance between two honest nodes is limited, and 2) the coordinate distance between a malicious peer and an honest peer cannot be smaller than the true network distance between these two nodes. The first goal limits an attacker's influence on honest nodes' coordinates while the second goal prevents an attacker from appearing closer than it actually is. Our main contribution is describing a completely decentralized network coordinate system, KoNKS, which is secure under our stated security model. KoNKS -- consensus-style network coordinate system -- modifies the objective function that each peer follows to update its coordinates. In current network coordinate systems, a peer's goal is to minimize the sum of the prediction errors for all of its neighbors. In contrast, using KoNKS, a peer's goal is to minimize the number of neighbors whose individual relative error is unacceptable -- KoNKS puts an upper bound on each neighbor's relative error. The relative error determines how accurate the coordinate system is, thus when there are no attackers, minimizing the sum of errors should lead to more accurate distance predictions. However, minimizing the sum of prediction errors allows each neighbor to have a significant influence on the position of its peers. This is one of the reasons why the frog-boiling attack works. For example, a malicious neighbor could craft a lie so that its coordinate distance to the peer is much smaller than the measured network distance. In response, the peer would make a significant change to its coordinate because that update seemed to give the minimum total prediction error, even though it adds significant prediction error to every other neighbor. This example cannot happen in KoNKS because every neighbor of a peer has the same amount of influence on that peer. In a way, KoNKS peers achieve consensus among their neighbors: each neighbor "votes" for a region in which the peer should reside, and the network position with the most "votes" from the neighbors is the one that KoNKS chooses. A malicious neighbor can still choose its reported coordinates and add delay to its RTT, but the push that lie has on the peer is limited, as the latter will have to satisfy its other neighbors as well. At every update, the peer takes into consideration each of its neighbors' relative error. We argue that KoNKS is secure because 1) a malicious node's influence on the coordinate distance between two honest nodes is limited, and 2) a malicious node cannot appear closer than it actually is because its relative error will be higher than the imposed threshold. We show that KoNKS is as accurate as Vivaldi [7], one of the most popular decentralized network coordinate system (Vivaldi is implemented in Vuze [18] and is the basis for previous "secure" network coordinate systems [9, 16, 17, 20]), and is secure against all the current attacks, including the network-partition frog-boiling attack. More specifically, KoNKS puts an upper bound on the amount of influence an adversary can have on the honest nodes. For example, 10% of attackers can partition a network using KoNKS only so much before their lies do not have any effect anymore because they are outside of the threshold, or the other honest neighbors' influence equals the malicious neighbors' influence. KoNKS with no attack can achieve a median relative error as low as 12%, which is comparable to Vivaldi's median relative error of 10%. Moreover, KoNKS incurs a very low overhead, similar to Vivaldi as coordinates can be piggybacked on top of application messages. The processing overhead of each node updating its coordinates is also very small.