Biased decisions of Large Language Models (LLM): Computer control agents and the decoy effect

Abstract

Large Language Models (LLMs) are increasingly embedded into Computer Control Agents (CCAs), enabling them to act autonomously on behalf of users. While prior research has examined behavioral biases in LLM-generated text, little is known about whether such biases and heuristics extend to automated actions. This study investigates whether CCAs exhibit the decoy effect — a well-documented bias in human decision making, where the presence of an inferior alternative shifts preferences towards a target option. Using the open-source framework browser-use, 3600 simulations across five canonical decision scenarios with six state-of-the-art LLMs are analyzed. Results show an aggregate decoy effect where CCAs select the target more often when a decoy is present (43.4%) than when it is absent (38.0%; $p<.001$), consistent with bounded rationality observed in humans. However, the magnitude and direction of the effect vary across models, with some displaying no significant bias. These findings extend evidence of heuristics in LLM behavior from text outputs to autonomous actions, highlighting both opportunities and risks for deploying CCAs in decision-making contexts. Implications for digital nudging, system design, and the use of LLMs as simulations in behavioral research are discussed.