Finding hash collisions using MPI on HPC clusters

Abstract

In cryptography, a hash function is a very important cryptographic primitive with a wide range of applications. There are three required properties for a good hash function, i.e., collision, pre-image, and second pre-image resistance. In this paper, we try to contest these properties on a popular and widely used hash function called MD5 - and its two simplified versions that we made. The birthday attack technique was used to test MD5's general collision resistance, while the brute force method was used in the search for pre-image and second pre-image collisions. We calculated the Hamming distance to monitor the progress in our search for a collision; the smaller the Hamming distance the better. Our input domain for the MD5 hash function consisted of hexadecimal bit-strings and strategically generated ASCII character strings. Since finding hash collisions demands much more computing power and storage, we wrote C parallel programs in conjunction with the Message Passing Interface (MPI) library that runs over multiple processors / cores in the heavily used CUNY HPC cluster called Penzias. Multiple search / sort / merge algorithms were tested, not only to reduce time and space complexities, but also to improve performance. Hash distributions, numerous arbitrary meaningless and a few meaningful collisions were found.