Towards a Criteria-Based Approach to Selecting Human-AI Interaction Mode

Abstract

Artificial intelligence (AI) tools are now prevalent in many knowledge work industries. As AI becomes more capable and interactive, there is a growing need for guidance on how to employ AI most effectively. The A²C framework (Tariq, Chhetri, Nepal & Paris, 2024) distinguishes three decision-making modes for engaging AI: automation (AI completes a task, including decision/action), augmentation (AI supports human to decide) and collaboration (iterative interaction between human and AI). However, selecting the appropriate mode for a specific application is not always straightforward. The goal of the present study was to compile and trial a simple set of criteria to support recommendations about appropriate AI mode for a given application. Drawing on human factors and computer science literature, we identified key criteria related to elements of the task, worker experience and support needs. From these criteria we built a scoring rubric with recommendation for A²C AI mode. As a preliminary test of this approach, we applied the criteria to cognitive task analysis (CTA) outputs from three case studies within the science domain—genome annotation, biological collections curation and protein crystallization—which provided insights into worker decision points, challenges and expert strategies. This paper describes the method for connecting CTA to A²C, reflecting on the challenges and future directions.