Assistive method for people with hearing disability-music visualisation

Abstract

Scientists have been in contact for centuries for interaction between different human sensory effects. The same roots of colors and sounds were shown by the Descartes light theory of the seventeenth century. Accordingly, the light and color perception were seen through the changes in the state of the ether, as well as the pressure of ether on the eye, just as hearing would be the pressure of the air vortex on the eardrum [1]. The analogy approach has given new impetus to Newton's color and light theory, as it has assigned a musical tone to the lines of colors on a chart of color charts. He considered that the seven colors of the spectrum could correspond to the seven voices of the diatonic scale [2]. Color can depend on the sound, pitch, volume, and other characteristics of the sound [3, 4]. Previously, in the studies with scientifically non-synesthetic subjects, each participant had to mark certain emotional dimensions on a scale, e.g. sad, cheerful, calm-angry, etc. Then participants had to choose a color that best suited the piece. The results demonstrated that emotions serve as an intermediate link in the association of music and colors [5, 6, 7]. Based on the above, the idea was that perhaps hearingimpaired, deaf people could be given a kind of “musical experience” by implementing an application that, by means of the colors assigned to the sounds and sounds through a visible dynamic, triggers the emotions they have induced. During the selection of the method and the appropriate technology, the purpose of developing is a visualization application which targets modern VR tools and 3D development environments, and adapts the technical toolkit of the 21st century, and of course, based on mathematical analysis [8, 9].