Christopher Tauchmann, Johannes Daxenberger, Margot Mieskes
{"title":"The Influence of Input Data Complexity on Crowdsourcing Quality","authors":"Christopher Tauchmann, Johannes Daxenberger, Margot Mieskes","doi":"10.1145/3379336.3381499","DOIUrl":"https://doi.org/10.1145/3379336.3381499","url":null,"abstract":"Crowdsourcing has a huge impact on data gathering for NLP tasks. However, most quality control measures rely on data aggregation methods which are only employed after the crowdsourcing process and thus cannot deal with different worker qualifications during data gathering. This is time-consuming and cost-ineffective because some datapoints might have to be re-labeled or discarded. Training workers and distributing work according to worker qualifications beforehand helps to overcome this limitation. We propose a setup that accounts for input data complexity and allows only a set of workers that successfully completed tasks of rising complexity to continue work on more difficult subsets. Like this, we are able to train workers and at the same time exclude unqualified workers. In initial experiments, our method achieves higher agreement with four annotations by qualified crowd workers compared to five annotations from random crowd workers on the same dataset.","PeriodicalId":335081,"journal":{"name":"Proceedings of the 25th International Conference on Intelligent User Interfaces Companion","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-03-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115122113","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Tara Tressel, Claudel Rheault, Masha Krol, Chris Tyler
{"title":"An Interactive Approach to Bias Identification in a Machine Teaching Task","authors":"Tara Tressel, Claudel Rheault, Masha Krol, Chris Tyler","doi":"10.1145/3379336.3381501","DOIUrl":"https://doi.org/10.1145/3379336.3381501","url":null,"abstract":"Supervised machine learning requires labelled data examples to train models, and those examples often come from humans who may not be experts in artificial intelligence (i.e., \"AI\"). Currently, many resources are devoted to these labelling tasks; a majority of which are outsourced by companies to reduce costs, and oversight on such tasks can be cumbersome. Concurrently, biases in machine learning models and human cognition are a growing concern in applications of AI. In this paper, we present a machine teaching platform for non-AI experts that leverages interactive data exploration approaches to identify algorithmic and human (e.g., cognitive) biases. Our main objective is to understand how data exploration and explainability might impact the machine teacher (i.e., data labeller) and their understanding of AI, subsequently improving model performance, all while reducing potential bias concerns.","PeriodicalId":335081,"journal":{"name":"Proceedings of the 25th International Conference on Intelligent User Interfaces Companion","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-03-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116829776","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Pan: Conversational Agent for Criminal Investigations","authors":"S. Hepenstal, Leishi Zhang, N. Kodagoda, B. Wong","doi":"10.1145/3379336.3381463","DOIUrl":"https://doi.org/10.1145/3379336.3381463","url":null,"abstract":"We present an early prototype conversational agent (CA), called Pan, for retrieving information to support criminal investigations. Our approach tackles the issue of algorithmic transparency, which is critical in unpredictable, high risk, and high consequence domains. We present a novel method to flexibly model CA intentions and provide transparency of attributes that is underpinned with human recognition. We propose that Pan can be used for experimentation to probe analyst requirements and to evaluate the effectiveness of our explanation structure.","PeriodicalId":335081,"journal":{"name":"Proceedings of the 25th International Conference on Intelligent User Interfaces Companion","volume":"53 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-03-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133646261","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Measuring Mental Effort via Entropy in VR","authors":"Daniel Reinhardt, J. Hurtienne, Carolin Wienrich","doi":"10.1145/3379336.3381493","DOIUrl":"https://doi.org/10.1145/3379336.3381493","url":null,"abstract":"Recognizing changes in users' experienced mental effort is a perennial interest in human-computer interaction research particularly in the design of intelligent user interfaces built to adapt to different levels of mental effort. With virtual reality (VR) applications, for example, many measures of mental workload (e.g., secondary tasks) are highly intrusive and can distort what is being measured. In this paper we investigate the entropy of controller movements as an indicator of mental effort that can be measured unobtrusively. We report a proof-of-concept study that manipulates the experienced mental effort using the popular e-crossing task. As expected, the results show that entropy is increased for people with higher mental effort than for people with lower mental effort and that there is a positive relationship with NASA-TLX scores, the benchmark questionnaire for mental effort. Thus, intelligent user interfaces become capable of detecting mental effort in VR on the basis of controller entropy and could recognize when users need assistance in their decision making.","PeriodicalId":335081,"journal":{"name":"Proceedings of the 25th International Conference on Intelligent User Interfaces Companion","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-03-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124896124","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Fabio Catania, G. D. Luca, Nicola Bombaci, Erica Colombo, Pietro Crovari, E. Beccaluva, F. Garzotto
{"title":"Musical and Conversational Artificial Intelligence","authors":"Fabio Catania, G. D. Luca, Nicola Bombaci, Erica Colombo, Pietro Crovari, E. Beccaluva, F. Garzotto","doi":"10.1145/3379336.3381479","DOIUrl":"https://doi.org/10.1145/3379336.3381479","url":null,"abstract":"Music production software often has complex interfaces and needs the user to know the basic musical know-how. In this paper, we present a conversational agent that allows creating music in a simplified way through voice-based interaction. Indeed, our agent can be configured and customized with simple and natural voice commands. In addition, it has some typically human cognitive skills to produce music: it listens to the user while singing a song and generates a melody by discovering and copying the patterns of her/his human voice. Technologically, the system is empowered by Google Dialogflow for conversation management and uses an advanced technique called abstract melody for music production. This Musical and Conversational Artificial Intelligence is an actual innovation since it does not require any preliminary knowledge about music and, consequently, includes professionals, but also children, beginners, and people with physical disease.","PeriodicalId":335081,"journal":{"name":"Proceedings of the 25th International Conference on Intelligent User Interfaces Companion","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-03-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126441514","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Saeed Amal, Mustafa Adam, Peter Brusilovsky, Einat Minkov, Zef Segal, T. Kuflik
{"title":"Demonstrating Personalized Multifaceted Visualization of People Recommendation to Conference Participants","authors":"Saeed Amal, Mustafa Adam, Peter Brusilovsky, Einat Minkov, Zef Segal, T. Kuflik","doi":"10.1145/3379336.3381455","DOIUrl":"https://doi.org/10.1145/3379336.3381455","url":null,"abstract":"We demonstrate an intelligent, personalized, multifaceted visualization of people recommendation using a personalized 2D entities graph and a word cloud for exploration by the user. This visualization aims to show non-trivial connections, e.g., those that the user may had forgotten about, but they are interesting and relevant. Since entities we are linked to are part of our lives (and profile), they help to understand who we are and what are we interested in. We adapt the typed entity-relation graph (profile) concept as introduced by [1] and based on this presentation we visualize the entity profile. In this demonstration, the users, as case study are the participants of IUI'20, will be able to explore their own personalized entities graph based on entities and relations that the system harvest about them (after getting their approval), from the web for finding interesting connections that they may meet in the context of this conference.","PeriodicalId":335081,"journal":{"name":"Proceedings of the 25th International Conference on Intelligent User Interfaces Companion","volume":"89 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-03-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121442023","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"What's in it for me?: Augmenting Recommended Learning Resources with Navigable Annotations","authors":"Sahan Bulathwela, S. Kreitmayer, M. Pérez-Ortiz","doi":"10.1145/3379336.3381457","DOIUrl":"https://doi.org/10.1145/3379336.3381457","url":null,"abstract":"This paper introduces an interface that enables the user to quickly identify relevant fragments within multiple long documents. The proposed method relies on a machine-generated layer of annotations that reveals the coverage of topics per fragment and document. To illustrate how the annotations double as a tool for preview as well as navigation, an example application is presented in the form of a personalised learning system that recommends relevant fragments of video lectures according to user's history. Potential implications of this approach for lifelong learning are discussed. We argue that this approach is generally applicable to recommender and information retrieval systems, across multiple knowledge domains and document types.","PeriodicalId":335081,"journal":{"name":"Proceedings of the 25th International Conference on Intelligent User Interfaces Companion","volume":"67 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-03-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117014054","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An Adaptive Agent-Based Interface for Personalized Health Interventions","authors":"S. Mozgai, Arno Hartholt, A. Rizzo","doi":"10.1145/3379336.3381467","DOIUrl":"https://doi.org/10.1145/3379336.3381467","url":null,"abstract":"This demo introduces a novel mHealth application with an agent-based interface designed to collect multimodal data with passive sensors native to popular wearables (e.g., Apple Watch, FitBit, and Garmin) as well as through user self-report. This mHealth application delivers personalized and adaptive multimedia content via smartphone application specifically tailored to the user in the interdependent domains of physical, cognitive, and emotional health via novel adaptive logic-based algorithms while employing behavior change techniques (e.g., goal-setting, barrier identification, etc.). A virtual human coach leads all interactions to improve adherence.","PeriodicalId":335081,"journal":{"name":"Proceedings of the 25th International Conference on Intelligent User Interfaces Companion","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-03-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122427784","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Evaluation of Educational Graph Creation Tool based on Natural Mathematical Description","authors":"Tetsuo Fukui","doi":"10.1145/3379336.3381483","DOIUrl":"https://doi.org/10.1145/3379336.3381483","url":null,"abstract":"The procedure used to input an equation and define a graph using existing tools remains unnatural and troublesome for novice students. To address this shortcoming, we propose a graph creation tool based on a natural mathematical description. In this study, we improved the predictive conversion speed of the mathematical input interface used in the previous graph creation tool. The results of a performance comparing test showed that the mean task time when using the proposed tool was approximately 1.2--1.7 times faster than that using GeoGebra.","PeriodicalId":335081,"journal":{"name":"Proceedings of the 25th International Conference on Intelligent User Interfaces Companion","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-03-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122831079","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Giving Faces to Data: Creating Data-Driven Personas from Personified Big Data","authors":"Soon-Gyo Jung, Joni O. Salminen, B. Jansen","doi":"10.1145/3379336.3381465","DOIUrl":"https://doi.org/10.1145/3379336.3381465","url":null,"abstract":"Creating personas from large amounts of online data is useful but difficult with manual methods. To address this difficulty, we present Automatic Persona Generation (APG), which is an implementation of a methodology for quantitatively generating data-driven personas from online social media data. APG is functional, and it is deployed with several organizations in multiple industry verticals. APG employs a scalable web front-end user interface and robust back-end database framework processing tens of millions of user interactions with tens of thousands of online digital products across multiple online platforms, including Facebook, Google Analytics, and YouTube. APG identifies audience segments that are both distinct and impactful for an organization to create persona profiles. APG enhances numerical social media data with relevant human attributes, such as names, photos, topics, etc. Here, we discuss the architecture development and central system features. Overall, APG can benefit organizations distributing content via online platforms or with online content that relates to commercial products. APG is unique in its algorithmic approach to processing social media data for customer insights. APG can be found online at https://persona.qcri.org.","PeriodicalId":335081,"journal":{"name":"Proceedings of the 25th International Conference on Intelligent User Interfaces Companion","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-03-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115401688","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}