SimLiquid: A Simulation-Based Liquid Perception Pipeline for Robot Liquid Manipulation

Abstract

Transparent liquid volume estimation is crucial for robot manipulation tasks, such as pouring. However, estimating the volume of transparent liquids is a challenging problem. Most existing methods primarily focus on data collection in the real world, and the sensors are fixed to the robot body for liquid volume estimation. These approaches limit both the timeliness of the research process and the flexibility of perception. In this paper, we present SimLiquid20k, a high-fidelity synthetic data set for liquid volume estimation, and propose a YOLO-based multi-task network trained on fully synthetic data for estimating the volume of transparent liquids. Extensive experiments demonstrate that our method can effectively transfer from simulation to the real world. In scenarios involving changes in background, viewpoint, and container variations, our approach achieves an average error of 5% in real-world volume estimation. In addition, our work conducts two application experiments integrating with GPT-4, showcasing the potential of our method in service robotics. The accompanying videos and supporting Information are available at https://simliquid.github.io/.