A simple yet effective difficulty-aware bucketed fine-tuning strategy for LLM-based recommendation

Abstract

Recommendation systems have been widely researched and applied in the industry. Recently, with the advancement of large language models (LLMs), there has been much research on LLM-based recommendation models (LLM-RMs), yielding outstanding performance compared with traditional recommendation models. In many LLM-RMs, supervised fine-tuning plays a pivotal role in enhancing model performance. However, existing research primarily focuses on enriching the information of the fine-tuning data, to enable LLMs to learn specific capabilities effectively. In this paper, we study the fine-tuning for LLM-RMs from a different perspective. Specifically, we propose a Difficulty-aware Bucketed Fine-tuning (DBF) strategy to replace the random sampling approach commonly used in existing research. The main philosophy of the proposed DBF strategy is to fine-tune an LLM from easy samples to difficult samples, which simulates the progressive learning process of humans. First, we propose a metric to measure the difficulty of fine-tuning samples based on three aspects: category entropy, category consistency, and category similarity. Based on the proposed metric, we design the bucketed tuning strategy that considers both intra-bucket and inter-bucket difficulty. In addition, we propose a Coarse-to-fine-grained Prompting LLM-based Recommendation Model, CP4Rec, which adopts a two-step reasoning process for making recommendations. We conduct extensive experiments on four real-world benchmark datasets, and results demonstrate that CP4Rec, fine-tuned with the proposed DBF strategy, outperforms the state-of-the-art LLM-RMs across these datasets. Experimental results also highlight the importance of using the buckets in the fine-tuning process to prevent performance degradation. The implementation code is available at https://anonymous.4open.science/r/CP4Rec1.