Exploring the Impact of Automation Bias and Complacency on Individual Criminal Responsibility for War Crimes

Abstract

Abstract With advancing technology, complex decision-making in warfare, including targeting, is increasingly assisted by machines. Although involving humans in decision-making is often seen as a safeguard against machine errors, it does not always prevent them. Machines can make incorrect determinations or delay them when time is critical. In these cases, human operators, influenced by automation bias (excessive trust in machines’ determinations, despite the availability of contradicting or different information from other sources) or complacency (excessive trust in machines’ determinations, leading to reduced vigilance), may fail to recognize machine errors, potentially resulting in conduct amounting to a war crime. Considering the role of automation bias and complacency in the determination of the criminal responsibility of systems’ operators is crucial, especially, for understanding the accountability framework for war crimes involving autonomous weapon systems (AWS). By exploring how automation bias and complacency affect the determination of criminal responsibility for humans who operate AWS, this article offers insights for lawmakers at the national and international levels to understand complexities and effectively shape legislative responses with respect to criminal responsibility. This article also examines automation bias and complacency in psychology, their relevance in military operations employing AWS, and their potential to exonerate human operators from criminal responsibility. It concludes by advocating for legislative, organizational, and technical measures to counteract automation bias and complacency.