RPriori: A Divide-and-Conquer Frequent Itemset Mining Algorithm With Improved Time-Space Efficiency on Large Transactional Data

Abstract

Frequent itemset mining (FIM) is a fundamental task in data mining with applications ranging from market basket analysis and recommendation systems to fraud detection and bioinformatics. However, mining frequent itemsets from massive datasets remains computationally challenging due to high execution time and memory consumption. Traditional algorithms such as Apriori suffer from exponential candidate generation and repeated database scans, FP-Growth incurs high memory overhead from tree construction, and Equivalence Class Transformation (ECLAT) faces expensive tid-list intersections. To address these limitations, we propose RPriori, a novel recursive divide-and-conquer algorithm that operates on the item-space instead of the transaction-space. RPriori recursively partitions the set of items and terminates early when a subset is found frequent, thereby eliminating candidate generation and avoiding global data structures. It performs localized support counting and merges only verified frequent subsets, resulting in O (NMlogM) time complexity and O (M) space complexity. Extensive experiments on six benchmark datasets, namely, Bakery, Chess, Mushroom, Retail, Accident, and T10I4D100K, demonstrate that RPriori consistently outperforms Apriori, FP-Growth, ECLAT, and TL-DIC. At low support thresholds, RPriori achieves up to 91% reduction in runtime and 93% reduction in memory usage, confirming its superior scalability and efficiency for both sparse and dense datasets.