Poster: TapSnoop -- Inferring Tapstrokes from Listening to Tap Sound on Mobile Devices

Mobile device users tap a touch-screen for entering sensitive information such as passwords and PIN numbers, and many works have proposed an attack model snooping such tapstrokes especially with the use of built-in sensors [1, 2, 3]. These studies raise the serious security concerns with the following attack scenario. A malicious application runs in the foreground as a normal chatting application, collecting a training set of sensor data generated from tapstrokes. While a user types her credit card number for purchasing something on a shopping application, it sneakingly takes sensor streams in the background and infers the tapped number by comparing the streams with the training data. However, in practice, the existing works have shown a limited inference accuracy, due to the following reasons. First, the intensity of tapstrokes is typically much low, resulting in a subtle change on sensor data. Second, mobile devices generally come with small on-screen keyboards where keys are very close to each other. Thus, it is essential to perform fine-grained tapstroke localization. Third, each mobile device has its own hardware characteristics with regard to screen’s size and thickness, as well as built-in sensor’s sensitivity. This inherently leads to different characteristics of tapstrokes for different devices. Last, smartphone users can use their devices in various places with different noise levels, while moving around. Therefore, it should be able to infer tapstrokes robustly against the environmental changes.