Calibration-Free Gaze Estimation by Combination with Hand and Facial Features Detection for Interactive Advertising Display

Abstract

In recent years, in addition to droplet infection, hand infection is also one of the major transmission routes of the new crown epidemic. Therefore, smart advertising machines that can be operated without hand contact are an important research topic. Gaze estimation is a technology that can identify the direction of gaze, which can infer the customer’s visual attention and convert it into interactive input information. Gaze estimation has great potential in contactless interactions as it allows systems to detect gaze focus and display relevant information/advertisements based on customer interests. The proposed design methodology is divided into three steps: face/hand objects detection by YOLO, threshold estimation by feature points, and gaze region detection by SVM classifier. The experimental results show that the average FPS with YOLO-based model reaches 15 when the number of filters is reduced to a quarter and the input size is set to 416x416 pixels. In the case of 4-block gaze regions, the YOLO-based model maintains a good enough accuracy that is up to 90%, and the experimental results reveals that the proposed expectation by adding hand features can effectively raise the accuracy of gaze estimation.