Trustworthy Artificial Intelligence and Automatic Morse Code Based Communication Recognition with Eye Tracking

Abstract

Morse code is one of the oldest communication techniques and used in telecommunication systems. Morse code can be transmitted as a visual signal by using reflections or with the help of flashlights, but it can also be used as a non-detectable form of communication by using the tapping of fingers or even blinking of eyes. In this paper, we develop a computer vision based approach that automatically characterizes the characters conveyed wherein a person can communicate to system or another person through Morse code with eye gestures. We can decode this visual eye tracking based language with the help of our automatic computer vision driven method. Our approach uses a normal webcam to detect the gestures made by the eyes and are interpreted as dots and dashes. These dots and dashes are used to represent the Morse code-based words. Image processing techniques-based blink and pupil detectors are employed. Blink detector helps us to detect a blink and the time that took for each blink. A blink that takes 2 to 4 seconds is acknowledged as a dot whereas a blink that takes more than 4 seconds is represented as a dash. The pupil detector helps us to detect the movement of the pupils, and if pupils move towards right with respect to a person then it is acknowledged as next letter and if the pupils are moved towards left with respect to a person then it is acknowledged as next word. In this way, we decode the Morse code which will be communicated using eyes and establish a non-detectable communication between a person and an automatic system. Our experimental results on an unconstrained visual scene with preliminary greeting words indicate the promise of an automatic eye tracking based system with success rate of 98.25% that can be of use in non-verbal communications.