EXPRESS: New Tools, New Rules: A Practical Guide to Effective and Responsible GenAI Use for Surveys and Experiments Research

Abstract

Generative Artificial Intelligence (GenAI) tools based on Large Language Models (LLMs) are quickly reshaping how researchers conduct surveys and experiments. From reviewing the literature and designing instruments, to administering studies, coding data, and interpreting results, these tools offer substantial opportunities to improve research productivity and advance methodology. Yet with this potential comes a critical challenge: researchers often use these systems without fully understanding how they work. This article aims to provide a practical guide for effective and responsible GenAI use in primary research. We begin by explaining how GenAI systems operate, highlighting the gap between their intuitive interfaces and the underlying model architectures. We then examine different use cases throughout the research process, both the opportunities and associated risks at each stage. Throughout our review, we provide flexible tips for best practice and rules for effective and responsible GenAI use, particularly in areas pertaining to ensuring the validity of GenAI coded responses. In doing so, we hope to help researchers integrate GenAI into their workflows in a transparent, rigorous, and ethically sound manner. Our accompanying website (questionableresearch.ai) provides supporting materials, including reproducible coding templates in R and SPSS and sample pre-registrations.