Kajetan Enge, A. Rind, Michael Iber, Robert Höldrich, W. Aigner
{"title":"It’s about Time: Adopting Theoretical Constructs from Visualization for Sonification","authors":"Kajetan Enge, A. Rind, Michael Iber, Robert Höldrich, W. Aigner","doi":"10.1145/3478384.3478415","DOIUrl":"https://doi.org/10.1145/3478384.3478415","url":null,"abstract":"Both sonification and visualization convey information about data by effectively using our human perceptual system, but their ways to transform the data could not be more different. The sonification community has demanded a holistic perspective on data representation, including audio-visual analysis, several times during the past 30 years. A design theory of audio-visual analysis could be a first step in this direction. An indispensable foundation for this undertaking is a terminology that describes the combined design space. To build a bridge between the domains, we adopt two of the established theoretical constructs from visualization theory for the field of sonification. The two constructs are the spatial substrate and the visual mark. In our model, we choose time to be the temporal substrate of sonification. Auditory marks are then positioned in time, such as visual marks are positioned in space. The proposed definitions allow discussing visualization and sonification designs as well as multi-modal designs based on a common terminology. While the identified terminology can support audio-visual analytics research, it also provides a new perspective on sonification theory itself.","PeriodicalId":173309,"journal":{"name":"Proceedings of the 16th International Audio Mostly Conference","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126855322","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
M. Wanderley, Travis J. West, Josh Rohs, Eduardo A. L. Meneses, Christian Frisson
{"title":"The IDMIL Digital Audio Workbench: An interactive online application for teaching digital audio concepts","authors":"M. Wanderley, Travis J. West, Josh Rohs, Eduardo A. L. Meneses, Christian Frisson","doi":"10.1145/3478384.3478397","DOIUrl":"https://doi.org/10.1145/3478384.3478397","url":null,"abstract":"The Input Devices and Music Interaction Laboratory (IDMIL) Digital Audio Workbench (DAWb) is a web application designed for experimentation with key concepts in digital audio theory with interactive visualizations of each stage of the Analog-to-Digital Conversion (ADC) and Digital-to-Analog Conversion (DAC) processes. By experimenting with the simulation settings, numerous key concepts in digital signal theory can be illustrated, such as aliasing, quantization, critical sampling, anti-aliasing filtering and dithering. The interactive interface allows the simulation to be explored freely; users can modify parameters and examine the resulting signals visually through numerous graphs or listen to the resulting signals. The workbench has been extensively used during the 200-level Introduction to Digital Audio course at McGill University in Fall 2020.","PeriodicalId":173309,"journal":{"name":"Proceedings of the 16th International Audio Mostly Conference","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122272997","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Composing the Network with Streams","authors":"Ian Clester, Jason Freeman","doi":"10.1145/3478384.3478416","DOIUrl":"https://doi.org/10.1145/3478384.3478416","url":null,"abstract":"We present Aleatora, an early-stage framework for building compositions from lazy, effectful streams. Aleatora’s streams, which may be combined by sequential, parallel, or functional composition, are well-suited to expressing interactive and aleatoric musical compositions. Aleatora includes a networking module which aids in writing compositions for the Internet of Sounds using network data sources such as OSC, external APIs, and Internet sound repositories (e.g. Freesound). This paper describes the design and implementation of Aleatora and demonstrates how it can facilitate weaving external input sources, such as network streams, into compositions.","PeriodicalId":173309,"journal":{"name":"Proceedings of the 16th International Audio Mostly Conference","volume":"55 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132147551","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Sunflower: an environment for standardized communication of IoMusT","authors":"Rômulo Vieira, F. Schiavoni","doi":"10.1145/3478384.3478414","DOIUrl":"https://doi.org/10.1145/3478384.3478414","url":null,"abstract":"The Internet of Musical Things (IoMusT) area, although recent, has well-defined aspects concerning musical practice via the network. However, several challenges are also present, from those related to musical and artistic practice, even those dealing with environmental and social issues. From a computational point of view, the main dilemmas revolve around the lack of resources to deal with heterogeneity and the lack of standard in the communication of the devices that make up this scenario. Therefore, this paper presents Sunflower, a tool inspired by the Pipes-and-Filters architecture that allows communication between different objects, and focuses on its usage protocol. Its layered structure is also presented, showing the types of data, messages, and musical things present in each one of them. After all, the tests and results that certify to the functionality of this environment are demonstrated.","PeriodicalId":173309,"journal":{"name":"Proceedings of the 16th International Audio Mostly Conference","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123992636","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Similarity Analysis of Visual Sketch-based Search for Sounds","authors":"Lars Engeln, N. Le, M. Mcginity, Rainer Groh","doi":"10.1145/3478384.3478423","DOIUrl":"https://doi.org/10.1145/3478384.3478423","url":null,"abstract":"Searching through a large audio database for a specific sound can be a slow and tedious task with detrimental effects on creative workflow. Listening to each sample is time consuming, while textual descriptions or tags may be insufficient, unavailable or simply unable to meaningfully capturing certain sonic qualities. This paper explores the use of visual sketches that express the mental model associated with a sound to accelerate the search process. To achieve this, a study was conducted to collect data on how 30 people visually represent sound, by providing hand-sketched visual representations for a range of 30 different sounds. After augmenting the data to a sparse set of 855 samples, two different autoencoder were trained. The one finds similar sketches in latent space and delivers the associated audio files. The other one is a multimodal autoencoder combining both visual and sonic cues in a common feature space but lacks on having no audio input for the search task. These both were then used to implement and discuss a visual query-by-sketch search interface for sounds.","PeriodicalId":173309,"journal":{"name":"Proceedings of the 16th International Audio Mostly Conference","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117004363","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Technological and Methodological Ecosystem for Dynamic Virtual Acoustics in Telematic Performance Contexts","authors":"Rory Hoy, D. Nort","doi":"10.1145/3478384.3478425","DOIUrl":"https://doi.org/10.1145/3478384.3478425","url":null,"abstract":"This paper presents the design and development of a technological ecosystem which facilitates research at the intersection of Virtual Acoustics, Networking, and Telematic Music. Building upon existing robust software packages, we integrate and extend this work in the context of exploring the effect of telematically-shared dynamic virtual acoustic spaces and expressive movement trajectories of player sound source positions upon live play contexts. Through analysis of impulse responses generated by a distributable measurement kit, remote players are able to share these spaces with their collaborators. Supplementary tools developed as part of this research also extend established telematic software solutions, allowing for easier use of network based musical communication systems for both performers and technicians. A network topology of interconnected telematic solutions is described, allowing flexibility between various internet based audio platforms. Applications which challenge and contain these developments are discussed, including a large-scale musical ensemble, web based musical performance, and an ongoing study into collaborative telematic improvisation.","PeriodicalId":173309,"journal":{"name":"Proceedings of the 16th International Audio Mostly Conference","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123680437","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Leveraging Compatibility and Diversity in Computational Music Mashup Creation","authors":"Gonçalo Bernardo, Gilberto Bernardes","doi":"10.1145/3478384.3478424","DOIUrl":"https://doi.org/10.1145/3478384.3478424","url":null,"abstract":"In this paper, we advance a multimodal optimization music mashup creation model for loop recombination at scale. The motivation to pursue such a model is to 1) tackle current scalability limitations in state-of-the-art (brute force) models while enforcing the 2) compatibility, i.e., recombination quality, of audio loops, and 3) a pool of diverse solutions that can accommodate personal user preferences or promote different musical styles. To this end, we adopt the Artificial Immune System (AIS) opt-aiNet algorithm to efficiently compute a population of compatible and diverse mashups from loop recombinations. Optimal mashups result from local minima in a feature space that objectively represents harmonic and rhythmic compatibility. We implemented our model as a prototype application named Mixmash-AIS, and conducted an objective evaluation that tackles three dimensions: loop recombination compatibility, mashups diversity, and computational model efficiency. The conducted evaluation compares the proposed system to a standard genetic algorithm (GA) and a brute force (BF) approach. While the GA stands as the most efficient algorithm, its poor results in terms of compatibility reinforce the primacy of the AIS opt-aiNet in efficiently finding optimal compatible loop mashups. Furthermore, the AIS opt-aiNet showed to promote a diverse mashup population, outperforming both GA or BF approaches.","PeriodicalId":173309,"journal":{"name":"Proceedings of the 16th International Audio Mostly Conference","volume":"31 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123451354","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Sonification of Planetary Orbits in Asteroid Belts","authors":"M. Quinton, I. Mcgregor, D. Benyon","doi":"10.1145/3478384.3478390","DOIUrl":"https://doi.org/10.1145/3478384.3478390","url":null,"abstract":"This study investigates the design and evaluation of a sonification designed to detect any planets orbiting within an asteroid belt of an exosolar system. The interface was designed for an astronomer who studies this phenomenon. User centered design methods were applied to create an accurate sonification of the data that could allow the astronomer to perceive possible planetary movements within an asteroid belt. The sonification was developed over three stages: A requirements gathering exercise inquiring about the data that the astronomer uses in her work. A design and development stage based on the findings of the requirements gathering and the third stage, an evaluation of the sonification design. The sonification effectively allowed the astronomer to immediately detect a planet orbiting within an asteroid belt. Multiple parameter mappings provide richer auditory stimuli that are more semantical to the user. The use of more familiar, natural sounding sound design led to a clearer comprehension of the dataset. The use of spatial mapping and movement allowed for immediate identification and understanding of the planet's course through the asteroid belt.","PeriodicalId":173309,"journal":{"name":"Proceedings of the 16th International Audio Mostly Conference","volume":"107 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127670781","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"SOURCE: a Freesound Community Music Sampler","authors":"F. Font","doi":"10.1145/3478384.3478388","DOIUrl":"https://doi.org/10.1145/3478384.3478388","url":null,"abstract":"SOURCE is an open-source hardware music sampler powered by Freesound’s collection of near 500k Creative Commons sounds contributed by a community of thousands of people around the world. SOURCE provides a hardware interface with Freesound and implements different methods to search and load sounds into the sampler. We see SOURCE as a proof of concept device that can be used as and extendable base system on top of which further research on the interaction between hardware devices and online large sound collections can be carried out. This paper describes the architecture of SOURCE and the different ways in which it interacts with Freesound. Even though we have not carried out a formal evaluation of SOURCE our informal tests show that SOURCE can be successfully integrated into a live music performance setup, and influence the creative process in interesting ways by being able to quickly generate new rich sound palettes that would otherwise be difficult to create with traditional hardware music samplers.","PeriodicalId":173309,"journal":{"name":"Proceedings of the 16th International Audio Mostly Conference","volume":"67 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127454206","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Atmosphéries and the poetics of the in situ: the role and impact of sensors in data-to-sound transposition installations","authors":"Anne Despond, Nicolas Reeves, Vincent Cusson","doi":"10.1145/3478384.3478422","DOIUrl":"https://doi.org/10.1145/3478384.3478422","url":null,"abstract":"Atmosphéries is a research-creation program which produced a series of art installations originally called “Cloud Harps”, that has known multiple transformations since their first instantiation in 1997. They are made of sculptural wooden “buffets” integrating different sensors that probe the sky and the surroundings to collect various atmospheric data. An internal process tuned by a human composer then maps these data to different audio parameters from a custom-made synthesis environment to create continuous sound and musical sequences. The present paper addresses conceptual and technological considerations about real-time sensing of clouds in a sonic-artistic installation context. Particular attention is given to the limitations and artefacts produced by such specialized sensors, and how those can be embraced to remain consistent with the spatio-temporal aspects of in situ works.","PeriodicalId":173309,"journal":{"name":"Proceedings of the 16th International Audio Mostly Conference","volume":" 10","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"113950709","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}