Let's Grab a Drink: Teacher-Student Learning for Fluid Intake Monitoring using Smart Earphones

Abstract

This paper shows the feasibility of fluid intake estimation using earphone sensors, which are gaining in popularity. Fluid consumption estimation has a number of healthcare-related applications in tracking dehydration and overhydration which can be connected to issues in fatigue, irritability, high blood pressure, kidney stones, etc. Therefore, accurate tracking of hydration levels not only has direct benefits to users in preventing such disorders but also offers diagnostic information to healthcare providers. Towards this end, this paper employs a voice pickup microphone that captures body vibrations during fluid consumption directly from skin contact and body conduction. This results in the extraction of stronger signals while being immune to ambient environmental noise. However, the main challenge for accurate estimation is the lack of availability of large-scale training datasets to train machine learning models (ML). To address the challenge, this paper designs robust ML models based on techniques in data augmentation and semi-supervised learning. Extensive user study with 12 users shows a per-swallow volume estimation accuracy of 3.35 mL (≈ 19.17% error) and a cumulative error of 3.26% over an entire bottle, while being robust to body motion, container type, liquid temperature, sensor position, etc. The ML models are implemented on smartphones with low power consumption and latency.