Synesthetic enrichment of mobile photography

This paper presents a novel approach to enrich mobile photography. To this end, images framed by the camera are analyzed in realtime to assess composition and exposure features, as well as to detect the presence of human subjects. Information resulting from this analysis serves to give users aesthetic cues before they take the photograph, helping to improve the quality of the final images captured. The main novelty of this system is the use of synesthetic feedback: a musical composition that maps aspects resulting from the realtime analysis of images is composed and played back to photographers. A user study shows that musical attributes can be controlled programmatically to provide effective information about the visual modality. The augmented synesthetic dimension has the potential to create a more compelling experience to the users of mobile photography.