Real-time object tracking for the robot-based nanohandling in a scanning electron microscope

Abstract

In this paper, current research work on an automated nanohandling station using mobile microrobots is presented. For the automated positioning of mobile microrobots a closed-loop control system is necessary, usually using data from pose sensors for the degrees of freedom (DOF) of the microrobot are needed. Mobile microrobots with piezo slip-stick actuation mostly do not have internal pose sensors to determine a global pose. This paper focuses on the continuous pose estimation (tracking) of mobile microrobots by external visual sensors. One possibility for fast pose estimation is the application of video cameras in combination with image processing algorithms as global sensors. However, for pose estimation with accuracy in the nanometer range high-resolution sensors are necessary. In consideration of resolution, image acquisition time and depth of focus a scanning electron microscope (SEM) is a powerful sensor for high-resolution pose estimation of a microrobot. On the other hand, the use of a SEM requires high demands on the image processing. High update rates of the pose data for the robot control require a short image acquisition time of the SEM images. As a result, the image noise increases as frame averaging or averaging of the detector signal is time consuming. This paper presents two approaches to tracking a micro-object in a SEM image stream. First, a cross-correlation algorithm is described, which enables pose estimation (x, y, ϕ) in extremely noised images in real-time. Afterwards object tracking with active contours is presented. This approach allows real-time tracking with more than 3 DOF by using shape spaces, instead of defining large model sets as it is necessary for correlation-based pattern matching.