Case count metric for comparative analysis of entity resolution results.

Abstract

Introduction: Entity resolution (ER) systems often produce different clustering outcomes when applied to the same dataset, especially when parameters, algorithms, or system configurations change. However, in many real-world settings, the true linking structure is unknown, making traditional accuracy-based evaluation difficult.

Methods: This paper presents the Case Count Metric System (CCMS), a process and software system for comparing two cluster ER outcomes without requiring a truth set. CCMS classifies how each cluster from the first ER process is transformed by the second process into four mutually exclusive cases: unchanged, merged, partitioned, or overlapping.

Results: CCMS produces aggregate case counts, singleton summaries, and per-cluster transformation details to support diagnostic analysis. Example applications using synthetic demographic data and an industrial materials dataset show that CCMS can identify how clustering outcomes change under parameter adjustments and alternative ER systems.

Discussion: CCMS provides a practical and interpretable method for comparing ER clustering results when labeled ground truth is unavailable. By distinguishing between over-linking, under-linking, and more complex cluster reorganizations, CCMS offers more actionable insight than single-value similarity measures and supports both research analysis and operational ER evaluation.