ZIDX: A Generic Framework for Random Access to BGP Records in Compressed MRT Datasets

Abstract

This paper presents an approach, called ZIDX, to decompress a desired chunk of a GZIP compressed large data file without decompressing the prefix of the file. For a GZIP compressed data file that is sorted by an attribute, ZIDX also allows a user to implement random seek to the desired location in the compressed file to retrieve and decompress a desired data block. We use ZIDX to implement efficient search and retrieval of BGP routing data in GZIP compressed files stored locally or fetched over the public Internet (from the RIPE RIS BGP archival repository). Our evaluation results show that ZIDX incurs minimal storage overhead and provides significant time and bandwidth savings in retrieving BGP records of interest from GZIP compressed BGP data files stored both locally and remotely.