Virtual Reality Calibration and Preview/Predictive Displays for Telerobotics

A virtual reality (VR) calibration technique of matching a virtual environment of simulated three-dimensional (3-D) graphic models with actual camera views of the remote site task environment has been developed. This VR calibration enables high-fidelity preview/predictive displays with calibrated graphics overlay on live video. Reliable and accurate calibration is achieved by operator-interactive camera calibration and object localization procedures based on new linear/nonlinear least-squares algorithms that can handle multiple-camera views. Since the object pose becomes known through the VR calibration, the operator can now effectively use the semiautomatic computer-generated trajectory mode in addition to the manual teleoperation mode. The developed VR calibration technique and the resultant high fidelity preview/predictive displays were successfully utilized in a recent JPL/NASA-GSFC (Jet Propulsion Laboratory/Goddard Space Flight Center) telerobotic servicing demonstration. Preview/predictive displays were very useful for both noncontact and contact tasks, providing an effective VR interface with immediate visual prediction/verification to the operator. The positioning alignment accuracy achieved using four-camera views in inserting a tool into the ORU hole was 0.51 cm on the average with a 1.07 cm maximum error at 95% confidence level. Results also indicate that the object localization with two well-chosen, e.g., near orthogonal camera views, could be nearly as accurate as that with four-camera views.