Development and Validation of the ICAP GenAI Scale to Measure How Graduate Students Integrate Generative AI Into Academic Research

Abstract

Generative artificial intelligence (GenAI) has profoundly reshaped how graduate students conceptualise, design and conduct academic research in higher education. While current metrics predominantly focus on technology acceptance and usage patterns, they often neglect the diverse cognitive engagements in graduate students' integration of GenAI across academic research activities. This study developed and validated a novel scale, grounded in the Interactive-Constructive-Active-Passive (ICAP) framework, to measure the integration of GenAI in academic research by graduate students. The scale development followed rigorous procedures beginning with a systematic literature review to create initial items, which were refined through expert reviews and pilot testing. Subsequently, two samples were created using data from 1216 Chinese graduate students across five disciplines (Arts and humanities, Social sciences, Science, Engineering, Medicine). For Sample 1, item-total correlation analysis and exploratory factor analysis were conducted, revealing four distinct factors: Passive, Active, Constructive and Interactive. Sample 2 was used for confirmatory factor analysis and validity testing. The finalised 27-item ICAP GenAI Scale exhibited excellent model fit, high reliability, robust construct validity and demographic invariance. This empirically validated tool not only advances our understanding of human-AI collaboration in academic research but also has significant implications for enhancing the research capabilities and higher-order thinking of graduate students in the AI-driven era.