SIER: An Efficient Entity Resolution Mechanism Combining SNM and Iteration

Abstract

With the rapid increase of data, entity resolution (ER) faces two challenges: high quality and high performance. Correspondingly, current work focuses on iteration-based entity resolution or sorted neighborhood (SNM) - based entity resolution. The former iteratively merges similar records to acquire higher precision and recall. The latter only compares the records within the same sliding window to maintain higher performance. However, they are at the cost of either sacrificing efficiency or result quality. In this paper, we present an entity resolution mechanism combining SNM and iteration (called SIER). Unlike traditional approaches, SIER can fully exploit the advantages of SNM and iteration. Also a two-stage entity matching algorithm is proposed. In the first stage, the records are initially matched based on sliding window. In the second stage, the matching result is rectified iteratively to improve the quality of the result. The experiments demonstrate the feasibility and effectiveness of our method.