When collaborating turns into dishonesty: A data-driven heuristic comparing human and AI collaborators

With respect to AI writing technologies (AIWT), we pose three foundational questions about academic dishonesty. First, do writing instructors and students perceive differences between AI agents and human agents in classroom scenarios? Second, to what extent are writing instructor and student perceptions are aligned? Third, what types of writing scenarios are perceived as academic dishonesty? Answering these questions provides a baseline of comparison not only for future studies of AIWT collaboration but also contextualizes perceptions of human-to-human collaboration. We report on a large-scale experimental survey study that answers these questions using item response theory (IRT). Our findings demonstrate that while there are differences between AI and human agents of collaborations, writing instructors and students are generally aligned in their perceptions. Using a Rasch model, we find that academic dishonesty operates along a spectrum of textual production. Regardless of whether the collaborating agent is human or AI, the more an agent produces text, the more this collaboration is perceived as academic dishonesty. Conversely, the less text that is produced, the less this scenario is perceived as academically dishonest. In our discussion, we provide a data-driven heuristic to guide instructors and administrators.