Optimal XOR hashing for a linearly distributed address lookup in computer networks

Hashing algorithms have been widely adopted to provide a fast address look-up process which involves a search through a large database to find a record associated with a given key. Modern examples include address-lookup in network routers for a forwarding outgoing link, rule-matching in intrusion detection systems comparing incoming packets with a large database, etc. Hashing algorithms involve transforming a key inside each target data to a hash value hoping that the hashing would render the database a uniform distribution with respect to this new hash value. When the database are already key-wise uniformly distributed, any regular hashing algorithm would easily lead to perfectly uniform distribution after the hashing. On the other hand, if records in the database are instead not uniformly distributed, then different hashing functions would lead to different performance. This paper addresses the case when such distribution follows a natural negative linear distribution, which is found to approximate distributions in many various applications. For this distribution, we derive a general formula for calculating the distribution variance produced by any given non-overlapped bit-grouping XOR hashing function. Such a distribution variance from the hashing directly translates to performance variations in searching. In this paper, the best XOR hashing function is determined for any given key size and any given hashing target size.