A Web-based Real-Time Kinect Application for Gestural Interaction with Virtual Musical Instruments

Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion Pub Date : 2018-09-12 DOI:10.1145/3243274.3243297

Athanasia Zlatintsi, P. Filntisis, Christos Garoufis, A. Tsiami, Kosmas Kritsis, Maximos A. Kaliakatsos-Papakostas, Aggelos Gkiokas, V. Katsouros, P. Maragos

{"title":"A Web-based Real-Time Kinect Application for Gestural Interaction with Virtual Musical Instruments","authors":"Athanasia Zlatintsi, P. Filntisis, Christos Garoufis, A. Tsiami, Kosmas Kritsis, Maximos A. Kaliakatsos-Papakostas, Aggelos Gkiokas, V. Katsouros, P. Maragos","doi":"10.1145/3243274.3243297","DOIUrl":null,"url":null,"abstract":"We present a web-based real-time application that enables gestural interaction with virtual instruments for musical expression. Skeletons of the users are tracked by a Kinect sensor, while the performance of the virtual instruments is accomplished using gestures inspired from their corresponding physical counterparts. The application supports the virtual performance of an air guitar and an upright bass, as well as a more abstract conductor-like performance with two instruments, while collaborative playing of two or more players is also allowed. The multimodal virtual interface of our application, which includes 3D avatars, allows users, even if not musically educated, to engage in innovative interactive musical activities, while its web-based architecture improves its accessibility and performance. The application was qualitatively evaluated by 13 users, in terms of its usability and enjoyability, among others, accomplishing high ratings and positive feedback.","PeriodicalId":129628,"journal":{"name":"Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion","volume":"114 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2018-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"9","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3243274.3243297","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 9

Abstract

We present a web-based real-time application that enables gestural interaction with virtual instruments for musical expression. Skeletons of the users are tracked by a Kinect sensor, while the performance of the virtual instruments is accomplished using gestures inspired from their corresponding physical counterparts. The application supports the virtual performance of an air guitar and an upright bass, as well as a more abstract conductor-like performance with two instruments, while collaborative playing of two or more players is also allowed. The multimodal virtual interface of our application, which includes 3D avatars, allows users, even if not musically educated, to engage in innovative interactive musical activities, while its web-based architecture improves its accessibility and performance. The application was qualitatively evaluated by 13 users, in terms of its usability and enjoyability, among others, accomplishing high ratings and positive feedback.

查看原文本刊更多论文

基于网络的实时Kinect手势与虚拟乐器交互应用

我们提出了一个基于网络的实时应用程序，可以与虚拟乐器进行手势交互，用于音乐表达。用户的骨架由Kinect传感器跟踪，而虚拟乐器的表演则是通过从相应的物理乐器中获得灵感的手势来完成的。该应用程序支持空气吉他和直立低音的虚拟表演，以及使用两种乐器进行更抽象的指挥表演，同时也允许两个或更多玩家合作演奏。我们的应用程序的多模态虚拟界面，其中包括3D化身，允许用户，即使没有音乐教育，参与创新的互动音乐活动，而其基于web的架构提高了其可访问性和性能。该应用程序由13名用户进行了定性评估，包括其可用性和可享受性等，获得了很高的评价和积极的反馈。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion

自引率

0.00%

发文量