{"title":"Deep Augmented Performers: A New Ensemble Performance System by Fusion of Melody Morphing and Body Movements","authors":"Hiroyasu Miura, M. Hamanaka","doi":"10.1145/3474349.3480201","DOIUrl":"https://doi.org/10.1145/3474349.3480201","url":null,"abstract":"The fusion of music-information processing and human physical functions will enable new musical experiences to be created. We developed an interactive music system called “Deep Augmented Performers,” which provides users with the experience of conducting a musical performance. This system converts music arranged using melody morphing into electrical muscle stimulation (EMS) to control the body movements of multiple performers. The melodies used in the system are divided into segments, and each segment has multiple variations of melodies. The user can interactively control the performers, thus the actual performance.","PeriodicalId":265194,"journal":{"name":"Adjunct Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117063675","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Harnessing Disagreement to Create AI-Powered Systems That Reflect Our Values","authors":"Mitchell L. Gordon","doi":"10.1145/3474349.3477590","DOIUrl":"https://doi.org/10.1145/3474349.3477590","url":null,"abstract":"How do we build artificial intelligence systems that reflect our values? Competing potential values we may want to choose from are, at their core, made up of disagreements between individual people. But while the raw datasets that most ML systems rely on are made up of individuals, today’s approaches to building AI typically abstract the individuals out of the pipeline. My thesis contributes a set of algorithms and interactive systems that re-imagine the pipeline for designing and evaluating AI systems, requiring that we deal with competing values in an informed and intentional way. I start from the insight that at each stage of the pipeline, we need to treat individuals as the key unit of operation, rather than the abstractions or aggregated pseudo-humans in use by today’s approaches. I instantiate this insight to address two problems that arise from today’s AI pipeline. First, evaluation metrics produce actively misleading scores about the extent to which some people’s values are being reflected. I introduce a mathematical transformation that more closely aligns metrics with the values and methods of user-facing performance measures. Second, the resulting AI systems either surreptitiously choose which values to listen to without input from users, or simply present several different outcomes to users without mechanisms to help them select an outcome grounded in their values. I propose a new interaction paradigm for deploying classifiers, asking users to compose a jury consisting of the individual people they’d like their classifiers’ decisions to come from.","PeriodicalId":265194,"journal":{"name":"Adjunct Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125132400","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dishita G. Turakhia, Andrew Wong, Yini Qi, Lotta-Gili Blumberg, Yoonji Kim
{"title":"Designing Adaptive Tools for Motor Skill Training","authors":"Dishita G. Turakhia, Andrew Wong, Yini Qi, Lotta-Gili Blumberg, Yoonji Kim","doi":"10.1145/3474349.3480205","DOIUrl":"https://doi.org/10.1145/3474349.3480205","url":null,"abstract":"We demonstrate the design of adaptive tools for motor skill training that use shape change to automatically vary task difficulty based on a learner’s performance. Studies [1] have shown that automatically-adaptive tools lead to significantly higher learning gains when compared to non-adaptive and manually-adaptive tools. We demonstrate the use of Adapt2Learn [2] - a toolkit that supports designers in building adaptive training tools. Adapt2Learn auto-generates an algorithm that converts a learner’s performance data into adaptation states during motor skill training. This algorithm, that maintains the training difficulty at the ’optimal challenge point’, can be uploaded to the micro-controller to convert several shape-changing tools into adaptive tools for motor skill training. We demonstrate 7 prototypes of adaptive tools for motor-skill learning to show applications in sports, music, rehabilitation, and accessibility.","PeriodicalId":265194,"journal":{"name":"Adjunct Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology","volume":"122 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125920770","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Assisting with Voluntary Pinch Force Control by Using Electrical Muscle Stimulation and Active Bio-Acoustic Sensing","authors":"Arinobu Niijima, Yuki Kubo","doi":"10.1145/3474349.3480214","DOIUrl":"https://doi.org/10.1145/3474349.3480214","url":null,"abstract":"We present a novel wearable system for assisting with voluntary pinch force control that requires no devices on the fingers. We use active bio-acoustic sensing to estimate voluntary pinch force with piezo elements attached to the back of the hand and electrical muscle stimulation to the forearm to control involuntary pinch force in a closed-loop system. We conducted a user study with eight participants to investigate whether the system could assist with pinch force control under target forces of 3 N, 6 N, or 9 N. When they tried to get pinch force closer to each target force with our system, the medians of the absolute errors were 0.8 N, 1.0 N, and 2.8 N, respectively. When they tried to do it without our system, they were 0.9 N, 1.4 N, and 2.4 N, respectively.","PeriodicalId":265194,"journal":{"name":"Adjunct Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126945795","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"OpenNFCSense: Open-Source Library for NFCSense","authors":"Rong-Hao Liang","doi":"10.1145/3474349.3480196","DOIUrl":"https://doi.org/10.1145/3474349.3480196","url":null,"abstract":"OpenNFCSense is an open-source library for NFCSense, a system for detecting the movements of near-field communication (NFC) tags using a commodity, low-cost RC522 NFC reader. With a user-defined tag profile, the users can use its application programming interface (API) to obtain the NFC tagged objects’ motion speed, motion frequency, and motion type while recognizing these tagged objects. Since NFC tags support straightforward augmentation to existed physical objects, such as modular building blocks and mechanical toys, OpenNFCSense offers an easy and safe method for rapid prototyping tangible user interfaces for designers.","PeriodicalId":265194,"journal":{"name":"Adjunct Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology","volume":"124 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122694253","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"HMK: Head-Mounted-Keyboard for Text Input in Virtual or Augmented Reality","authors":"Wahyu Hutama, Hikari Harashima, Hironori Ishikawa, Hiroyuki Manabe","doi":"10.1145/3474349.3480195","DOIUrl":"https://doi.org/10.1145/3474349.3480195","url":null,"abstract":"Text input is essential in a variety of uses in virtual and augmented reality (VR and AR). We present HMK: an effective text input method that mounts split keyboards on the left and right side of the head mounted display (HMD). Users who can touch-type are able to type using HMK by relying on their familiarity with the normal QWERTY keyboard. We develop custom keycaps to make it easier to find the home position. A study with three participants shows that users retain most of their normal keyboard typing skills. The participants achieved, on average, 34.7 words per minute (WPM) by the end of three days of use, retaining 81 percent of their regular entry speed.","PeriodicalId":265194,"journal":{"name":"Adjunct Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129336747","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"IntoTheVideos: Exploration of Dynamic 3D Space Reconstruction From Single Sports Videos","authors":"Xiaotong Chen, Misha Sra","doi":"10.1145/3474349.3480215","DOIUrl":"https://doi.org/10.1145/3474349.3480215","url":null,"abstract":"We present IntoTheVideos, a novel system that takes a sports video and reconstructs the 3D space (e.g., basketball court, football field) along with the highlighted event (e.g., making a basket, scoring a touchdown) to enable viewers to experience the sports entertainment with more freedom. Users can move around in the reconstructed 3D space and explore the event freely from different angles and distances as well as view the players in 3D. With this system, we hope to offer users a new option to enjoy sports entertainment beyond passively watching video footage potentially leading to more active fan engagement and participation.","PeriodicalId":265194,"journal":{"name":"Adjunct Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129894379","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"FaceMe: An Augmented Reality Social Agent Game for Facilitating Children's Learning about Emotional Expressions","authors":"Jiajia Li, Zixia Zheng, Xiemin Wei, Guanyun Wang","doi":"10.1145/3474349.3480216","DOIUrl":"https://doi.org/10.1145/3474349.3480216","url":null,"abstract":"Children with autism spectrum disorder (ASD) experience dysfunctional emotional development leading to negative effects on their social communication. Although interventions are effective in helping children with ASD improve their social skills over time, they have been found to lack the essential ability to engage children in a real social environment. In this paper, we present “FaceMe,” which is an augmented reality (AR) system that uses a virtual agent and a set of tangible toolkits to teach children with ASD about six basic emotions and improve their emotional and communication skills. On the basis of the pilot data, the results suggest that children, especially those with ASD, were willing to socialize with the virtual agent and understand more emotional states. It is hoped that FaceMe can be used as a tool to provide assistance to children with ASD, as well as a way for future interface system design to support emotional development in children.","PeriodicalId":265194,"journal":{"name":"Adjunct Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology","volume":"172 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124036593","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"ViObject: A Smartwatch-based Object Recognition System via Vibrations","authors":"Wenqiang Chen, Daniel Bevan, J. Stankovic","doi":"10.1145/3474349.3480182","DOIUrl":"https://doi.org/10.1145/3474349.3480182","url":null,"abstract":"Today, in order to start an interaction with most digital technology, we must perform some sort of action to indicate our intention, such as shaking a computer’s mouse to wake it or pressing a coffee maker’s start button for your morning cup of coffee. Our system aims to help remove these currently necessary ”trigger actions” and aims to support an array of applications to create borderless and fluid interactions between the technological world and our own. Our system as has the potential for application within the world of accessible technology as well. The system we propose is a method of identifying held objects via a smartwatch’s accelerometer and gyroscope sensors. A preview demo video is available at https://youtu.be/1YCTzvjcJ18.","PeriodicalId":265194,"journal":{"name":"Adjunct Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology","volume":"61 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121772745","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Vishnu Nair, Shao-en Ma, Hannah Huddleston, Karen Lin, Mason Hayes, Matthew Donnelly, Ricardo E. Gonzalez, Yicheng He, Brian A. Smith
{"title":"Towards a Generalized Acoustic Minimap for Visually Impaired Gamers","authors":"Vishnu Nair, Shao-en Ma, Hannah Huddleston, Karen Lin, Mason Hayes, Matthew Donnelly, Ricardo E. Gonzalez, Yicheng He, Brian A. Smith","doi":"10.1145/3474349.3480177","DOIUrl":"https://doi.org/10.1145/3474349.3480177","url":null,"abstract":"Video games created for visually impaired players (VIPs) remain inequivalent to those created for sighted players. Sighted players use minimaps within games to learn how their surrounding environment is laid out, but there is no effective analogue to the minimap for visually impaired players. A major accessibility challenge is to create a generalized, acoustic (non-visual) version of the minimap for VIPs. To address this challenge, we develop and investigate four acoustic minimap techniques which represent a breadth of ideas for how an acoustic minimap might work: a companion smartphone app, echolocation, a directional scanner, and a simple menu. Each technique is designed to communicate information about the area around the player within a game world, providing functionality analogous to a visual minimap but in acoustic form. We close by describing a user study that we are performing with these techniques to investigate the factors that are important in the design of acoustic minimap tools.","PeriodicalId":265194,"journal":{"name":"Adjunct Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology","volume":"65 3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130867633","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}