Intruder Detection and Tracking using 77GHz FMCW Radar and Camera Data

Target detection and tracking using optical and radar sensors have many applications in surveillance. As the optical sensor helps to visualize the target and the radar can provide its range and velocity, their combination results in useful information for continuous monitoring and coherence. This paper presents a radar-camera experimental setup to detect and track intruders in a restricted area. A real-time experiment with different target speeds and various radar cross-sections(RCS) (1. A person running, 2. A cyclist, and 3. A motorcyclist). We deployed a 77GHz IWR1642BOOST FMCW (Frequency Modulated Continuous Wave) radar module as a radar unit and a phone camera with an aperture of f/1.79 as an optical sensor. The data collected from the radar and camera sensor are applied to detection and tracking modules to obtain target tracks. The radar provides the observations of range, Doppler, and angle information. These observations are used to estimate the state of the target via extended Kalman filtering(EKF), dBscan clustering, and global nearest neighbor(GNN) association, followed by track maintenance. The optical sensor provides video frames as input and output tracks via foreground detection, blob analysis, motion-based detection, Kalman filtering, and track maintenance. The experimental result shows that combining radar and optical sensors accomplishes tracking accuracy and coherence in target detection and tracking.