Pose Estimation of Household Objects Using RGB-D-NIR Camera

Pose estimation is a technique used to predict the pose of an object, i.e., its orientation and its position. This technique is needed by the robot to pick up objects that are placed somewhere. In general, this technique is developed with visual input from a camera. The challenge that is usually faced in this technique is the lighting conditions that are not fixed, where the stability of the estimation results in this environment is highly expected. In this study, we first use an RGB-D-NIR camera to provide color (RGB), depth (D), and near-infrared (NIR) inputs which are expected to complement each other. Second, by utilizing data from the camera, we combine it with Guided Filtering Fusion and Deep learning methods. As a preliminary result of the study, we conducted experiments in normal and dark lightning conditions on household objects, with various input combinations; and obtained quite accurate results in dark conditions.