Count your bits: fingerprint benchmarking to assess broad chemical space representation

Abstract

Quantifying molecular similarity is a cornerstone of cheminformatics, underpinning applications from virtual screening and nearest-neighbor search to chemical space visualization and the evaluation of machine-learning predictions. Yet, despite widespread use of Tanimoto comparisons of 2D fingerprints, the practical behavior of these similarity measures depends strongly on the fingerprint type, representation (binary vs. count), and whether fingerprints are folded into fixed-length vectors. In this work, our main goal is to provide a broad and systematic comparison of common fingerprint types, including dictionary-based, circular (Morgan/FCFP), path-based (RDKit), topological-distance-based (Atom Pair), hybrid distance-encoded (MAP4), torsion, LINGO, and Avalon, together with several representation variants. We benchmark these fingerprints across multiple large and heterogeneous datasets using complementary evaluation tasks that probe specificity, score distributions, compound-size dependence, top-k ranking agreement, and compare fingerprint similarities to a graph-based reference. Across benchmarks, count and often log-count variants generally improve specificity and structural alignment, while folding-induced bit collisions can strongly distort similarities for high-occupancy fingerprints, making unfolded variants particularly important for RDKit and often necessary for MAP4 on heterogeneous datasets. To support reproducible benchmarking and future extensions, we also introduce the open-source Python library chemap, which provides unified computation of folded, unfolded, and frequency-folded fingerprints together with optimized similarity calculations.

Scientific contribution

We introduce a multi-criteria benchmarking framework for molecular fingerprints that goes beyond classical virtual-screening retrieval tests by evaluating specificity (fingerprint duplicates), score behavior (including compound-size dependence), ranking agreement, and neighborhood structure on large, heterogeneous small-molecule datasets. Our results reveal that common default settings, especially folding for high-occupancy fingerprints, can introduce severe bit-collision artifacts, and that count (often log-count) and unfolded variants substantially improve specificity and agreement with structure-based references. We release the open-source Python library chemap to standardize these fingerprint variants and enable reproducible, extensible benchmarking for future fingerprint development.