Balancing Security and Privacy: Web Bot Detection, Privacy Challenges, and Regulatory Compliance under the GDPR and AI Act.

Abstract

This paper presents a comprehensive analysis of web bot activity, exploring both offensive and defensive perspectives within the context of modern web infrastructure. As bots play a dual role-enabling malicious activities like credential stuffing and scraping while also facilitating benign automation-distinguishing between humans, good bots, and bad bots has become increasingly critical. We examine the technical challenges of detecting web bots amidst large volumes of benign traffic, highlighting the privacy risks involved in monitoring users at scale. Additionally, the study dives into the use of Privacy Enhancing Technologies (PETs) to strike a balance between bot detection and user privacy. These technologies provide innovative approaches to minimising data exposure while maintaining the effectiveness of bot-detection mechanisms. Furthermore, we explore the legal and ethical considerations associated with bot detection, mapping the technical solutions to the regulatory frameworks set forth by the EU General Data Protection Regulation (GDPR) and the Artificial Intelligence Act (AI Act). By analysing these regulatory constraints, we provide insights into how organisations can ensure compliance while maintaining robust bot defence strategies, fostering a responsible approach to cybersecurity in a privacy-conscious world.