Maximizing diversity in k-pattern set mining through constraint programming and entropy

Abstract

Extracting diverse and frequent closed itemsets from large datasets is a core challenge in pattern mining, with significant implications across domains such as fraud detection, recommendation systems, and machine learning. Existing approaches often lack flexibility and efficiency, and struggle with initial itemset selection bias and redundancy. This paper addresses these research gaps by introducing a compact and modular constraint programming model that formalizes the search for diverse patterns. Our approach incorporates a novel global constraint derived from a relaxed Overlap diversity measure, using tighter lower and upper bounds to improve filtering capabilities. Unlike traditional methods, we leverage an entropy-based optimization framework that combines joint entropy maximization with top-k pattern mining to identify the maximally k-diverse pattern set. Our approach ensures more comprehensive and informative pattern discovery by minimizing redundancy and promoting pattern diversity. Extensive experiments validate the effectiveness of the proposed method, demonstrating significant performance gains and superior pattern quality compared to state-of-the-art approaches. Implemented in both sequential and parallel versions, the framework offers an efficient and adaptable solution for anytime pattern mining tasks in various domains.