CTFAgent: An LLM-powered Agent for CTF Challenge Solving

Abstract

Capture-the-Flag (CTF) competitions play an important role in the cybersecurity landscape by simulating realistic attack and defense scenarios and offering diverse categories of challenges. This diversity demands flexible reasoning and adaptive problem-solving, which traditional automation tools struggle to provide, as they are typically designed for specific tasks. Large Language Models (LLMs) with their vast knowledge and strong reasoning capabilities, present a promising approach to overcome these limitations. In this work, we propose CTFAgent, an LLM-powered agent featuring a new plan-and-execute paradigm with a stateful task tree for long-horizon reasoning. To handle diverse challenges, CTFAgent integrates challenge-specific prompting and specialized tools for multimodal analysis and concrete operations. The agent comprises two modes: a fully automated mode and a human-in-the-loop (HITL) mode, which incorporates human operational support for tool execution beyond the automation. Evaluated on challenges from PicoCTF with GPT-4o, Gemini-2.5-Pro and DeepSeek-V3, CTFAgent outperforms 88% of human teams in its automated mode. This performance rises significantly in HITL mode, where it surpasses approximately 94% of teams. These results demonstrate that CTFAgent can effectively solve a wide range of complex tasks, highlighting the potential of LLM-powered agents to advance autonomous cybersecurity solutions.