Development and Adaptation of Robotic Vision in the Real World: The Challenge of Door Detection

Abstract

Autonomous service robots are becoming increasingly common in human-centric, long-term deployments in unstructured indoor environments. Robotic vision is a crucial capability, enabling robots to perceive and interpret high-level environmental features from visual input. While data-driven approaches based on deep learning have advanced the capabilities of vision systems, applying these techniques in real robotic scenarios still presents unique methodological challenges. Conventional datasets often do not represent the object categories that a service robot needs to detect. More importantly, state-of-the-art models struggle to address the demanding perception constraints faced by service robots, posing the need for adaptations to the specific environments in which the robots operate. We devise a method that addresses these challenges by leveraging photorealistic simulations to create synthetic visual datasets from a robot's perspective. This approach balances data quality with acquisition costs, enabling the training of deep, general-purpose detectors tailored for service robots. We then demonstrate the benefits of qualifying a general detector for the domain in which the robot is deployed, studying the trade-off between data-acquisition efforts and performance improvement. We evaluate our method using a representative selection of prominent deep-learning object detectors for the challenge of recognizing, in real time, the presence and traversability of doorways. This task, which we refer to as door detection, is fundamental to numerous significant robotic tasks, such as tracking the changing topology of dynamic environments. We conduct an extensive experimental campaign in the field, considering different real-world setups while emulating the typical challenges encountered in long-term deployments of service robots. Our key findings demonstrate that simulation and qualification techniques can significantly reduce costs associated with domain adaptation for service robots. While simulation allows embedding the robot's perspective during the training of end-to-end robotic vision modules, qualification is essential to improve their robustness over challenging detection instances, thus reaching the performance level typically required by realistic long-term deployments of service robots.