From "people also ask" to clarifying questions for conversational search using parameter-efficient fine-tuning and prompt engineering.

Abstract

Conversational search systems enable natural, coherent dialogues between users and search systems to satisfy the user information need. The best of such systems improve user experience by asking clarifying questions (CQs) to resolve ambiguity in user queries. However, generating CQs that are both relevant and conversationally appropriate remains a significant challenge. To this end, we use Google's "People Also Ask" (PAA) feature, grounding our questions in real-life user search behavior. We identify the most suitable methodology for this task by conducting an extensive comparative analysis of two methods: Parameter-Efficient Fine-Tuning (PEFT) and prompt engineering, and evaluate these using three state-of-the-art, lightweight language models. Furthermore, we introduce two new, human-annotated test sets derived from various conversational search datasets. Our results demonstrate that PEFT consistently and significantly outperforms all prompt engineering approaches across all models and test sets, and is statistically comparable to traditional full fine-tuning. These findings suggest that for the PAA-to-CQ task, investing in a high-quality dataset for efficient fine-tuning is a more reliable path to achieving high-quality, stylistically consistent outputs than relying on in-context learning alone.