Real-time 3D marker tracking with a WIIMOTE stereo vision system: Application to robotic throwing

Abstract

In this paper we describe the use of a standard game console joystick, namely the Nintendo WIIMOTE, for an active real-time 3D marker tracking. We show the ease of applicability of inexpensive and robust standard game controllers for 3D object tracking, e.g. to track an infrared source in 3D space. Recovering the 3D information using stereo vision is still one of the major research areas in computer vision and has given rise to a great deal of literature in the recent past. In this paper we present the method for calibrating a WIIMOTE stereo pair without knowing any parameters of the build-in infrared cameras in advance. The results are two matrices which includes both the intrinsic and extrinsic parameters for left and right cameras. The comparison between the stereo and the mono WIIMOTE tracking system is presented. Furthermore, to demonstrate the use of the WIIMOTE stereo system we considered the task of throwing a ball with robotic hand, to the target identified with an infrared source. The throwing task was divided into two separate parts: the tracking part and the throwing part.