Learning real-time stereo vergence control

Abstract

Online learning robotic systems have many desirable properties. This work contributes a reinforcement learning framework for learning a time-constrained closed-loop control policy. The task is to verge the two cameras of a stereo vision system to foveate on the same world feature, within a limited number of perception-action cycles. Online learning is beneficial in at least the following ways: 1) the control parameters are optimized with respect to the characteristics of the environment actually encountered during operation; 2) visual feedback contributes to the choice of the best control action at every step in a multi-step control policy; 3) no initial calibration or explicit modeling of system parameters is required; and 4) the system can be made to adapt to non-stationary environments. Our vergence system provides a running estimate of the resulting verge quality that can be exploited by a real-time scheduler. It is shown to perform superior to two hand-calibrated vergence policies.