AI Captain: Conversational mission planning and execution system for autonomous surface vehicles

Abstract

Interactive, high-level mission planning for autonomous ships remains an underexplored topic. We present AI Captain, a large language model-based (LLM) proof-of-concept mission planning and execution system that can replan according to commands from a human operator in the loop. The system hierarchically combines a high-level LLM planner with a low-level control for commonsense reasoning and action, while interacting with the operator through a conversational user interface. We validate the feasibility of the system on a real vehicle in the field, demonstrating the ability to plan, re-plan, and handle scenarios that were not explicitly programmed beforehand. The results highlight how LLMs can facilitate both high-level contextual decision-making and human-AI teaming in maritime operations.