Application of visual speech synthesis in therapy of auditory verbal hallucinations

Abstract

In this article the application of a visual speech synthesis system in the therapy of auditory verbal hallucinations is presented. In the demonstrated proof-of-concept implementation an animated talking head driven by a synthetic voice converses with the patient. The proposed system integrates the XFace face model, synthetic voice generated by text-to-speech systems available on the market and the custom automatic lip synchronization algorithm. The article describes the signal processing algorithms used in the proposed system and opinions gathered from twelve patients after the treatment process. Most of the patients admitted that they changed their attitude to the hallucinations thanks to the avatar based therapy, and they were also satisfied with the audio video content presented during the therapy.