OL-BaS: Efficient data load-balancing for skewed data with approximate load statistics

Data skew can significantly deteriorate query performance in distributed systems. More concretely, when the data is range partitioned, usually it may be unequally distributed across the partitions. When tuples are inserted and deleted continuously, some of these data shall be moved from the hot nodes to the least loaded ones in order to satisfy the storage balance requirement. These movements have an important impact in terms of maintaining the load statistics related to each node, such as the partition boundaries and the load size. Efficient solutions from the state-of-art that address the data skew problem require global load statistics with a cost of O(log n) messages. In this paper, we propose an efficient online load-balancing algorithm for the range-partitioned data. Our solution is based on the fuzzy image (FZIM) concept. The basic idea about the FZIM is that both clients and nodes have an approximate knowledge about the effective partition statistics. They can nevertheless locate any data with almost the same efficiency as using exact partition statistics. Furthermore, maintaining load distribution statistics do not require exchanging additional messages as opposed to the cost of efficient solutions from the state-of-art (which requires at least O(log n) messages).