A tablet-type acoustic digital pen that presents pen strokes with music tones

Abstract

This study presents a tablet-type acoustic digital pen that presents pen strokes in handwriting with music tones. An acoustic digital pen for the visually impaired has been previously proposed, and it is shown that the pen-tip position (sound localization) can be identified by continuously changing the volume and acoustic frequency in correspondence with the pen strokes in space. In the tablet-type acoustic digital pen proposed in this study, we change the musical scale in the horizontal direction and the octave in the vertical direction on a spatially discretized 2D grid. By enabling the grid to be adjusted to the desired size using the pinch in/out operation on the tablet touch screen, it is possible to efficiently practice writing letters of various scales in a balanced manner. Furthermore, by changing the music tone (frequency distortion) in accordance with the pen touch at character feature points, such as the start point, via the end point of each pen stroke, the proposed design eases learning of the Tome, Hane, and Harai accents.