{"title":"IORapp: An R tool for Inter-Observer Reliability Assessment of Time and Motion data","authors":"Stefano Guidi, M. Tanzini, Johanna I. Westbrook","doi":"10.1145/3452853.3452890","DOIUrl":"https://doi.org/10.1145/3452853.3452890","url":null,"abstract":"Direct observational time and motion studies, in which one or more observers continuously observe and record an individual's activities over a certain period of time, can provide valuable insights into work and communication patterns and assist in identification of areas to intervene to support safe and effective work. These studies have increased in number and sophistication, also due to the availability of software tools for PDA to support the observers. A key methodological issue for these studies is the assessment of Inter-Observer Reliability (IOR). IOR is crucial for ensuring reliability of data collection in which multiple observers are involved. In workflow time studies the multivariate, time-stamped and ordered nature of the data limits the applicability of traditional inter-rater reliability measures, and makes this assessment challenging. Measures such as Cohen's Kappa, in fact, are only applicable to one variable at the time, so that high K scores for one aspect can be achieved even if two observers disagree substantially on other variables which are the object of their observation. Secondly, computing these measures first requires either matching pairs of datapoints from different observers viewing the same entity, a problem that cannot be done with perfect certainty, or restructuring the data into fixed-length time window sequences. No single method can address all the different aspects on which observers in time and motion studies can disagree, so one should adopt a composite method. We developed a set of functions and a Shiny app to assist this process for data collected with the Work Observation Method By Activity Timing (WOMBAT) method. The app allows the loading of data from pairs/groups of observers and computes a wide set of agreement measures on both matched data and on time window data. These measures are presented in a dashboard along with interactive visualizations of the observers’ data, and can be saved and plotted over time in a different section.","PeriodicalId":334884,"journal":{"name":"Proceedings of the 32nd European Conference on Cognitive Ergonomics","volume":"90 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115831760","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Christian Remy, G. Harboe, J. Frich, M. M. Biskjaer, P. Dalsgaard
{"title":"Challenges and Opportunities in the Design of Digital Distributed Affinity Diagramming Tools","authors":"Christian Remy, G. Harboe, J. Frich, M. M. Biskjaer, P. Dalsgaard","doi":"10.1145/3452853.3452871","DOIUrl":"https://doi.org/10.1145/3452853.3452871","url":null,"abstract":"Affinity diagramming is an oft-used sense-making technique in design research and practice to analyze qualitative data, utilizing a large number of sticky notes on walls. Over the past two decades, several digital tools have been tried and tested to augment or even replace the physical affinity diagramming process. Even so, the analog process usually prevails. We developed an online collaboration tool specifically tailored toward affinity diagramming to explore the challenges and opportunities of such a system in the particular case where the distributed teams do not have access to co-located settings. Here, we present a user experience study of five groups (dyads) of students based on a one-hour diagramming task under remote observation, followed by semi-structured interviews. Our study contributes three distinct insights to inform future work, namely that digital affinity diagrams 1) reduce the awareness of co-participants’ actions, 2) provide fewer cues about ownership and use than physical diagrams, and 3) save time, improve manipulation, and overview. We end with a discussion of the challenges and opportunities for the design of digital tools for distributed teams involved in sense-making tasks.","PeriodicalId":334884,"journal":{"name":"Proceedings of the 32nd European Conference on Cognitive Ergonomics","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115674402","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
P. Marti, O. Parlangeli, Annamaria Recupero, Matteo Sirizzotti, Stefano Guidi
{"title":"Touching virtual objects in mid-air: a study on shape recognition","authors":"P. Marti, O. Parlangeli, Annamaria Recupero, Matteo Sirizzotti, Stefano Guidi","doi":"10.1145/3452853.3452875","DOIUrl":"https://doi.org/10.1145/3452853.3452875","url":null,"abstract":"Mid-air haptic technology allows designers to build touchless interfaces incorporating tactile feedback. In the present study, 17 students of the University of Siena were engaged in an experiment where they had to recognize shapes (a circle, a square, or a point) presented one at a time as mid-air haptic stimuli. The task was to discriminate if a stimulus (e.g. round stimulus on the palm of the hand) was compatible with an image (e.g. a circle) or a word (e.g. “circle”) displayed on a screen. The results indicate that only the “point” stimulus can lead to appreciable recognition performance, both in terms of accuracy and in relation to the time needed for the identification. No learning-based performance improvements were found. These results suggest that the haptic feedback provided by the mid-air technology used in the experiment is informative but difficult to discriminate and therefore it should be carefully designed in particular when associated to tasks requiring accuracy and speed of recognition.","PeriodicalId":334884,"journal":{"name":"Proceedings of the 32nd European Conference on Cognitive Ergonomics","volume":"22 2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130697203","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Exploring the Relationship Between Physical Presence, User Experience, and Task Parameters in Robotic Telepresence","authors":"V. Kaptelinin, Patrik Björnfot, K. Danielsson","doi":"10.1145/3452853.3452874","DOIUrl":"https://doi.org/10.1145/3452853.3452874","url":null,"abstract":"This paper reports a study of pilots of a robotic telepresence system, exploring the relationship between the pilots’ experience of physical presence, overall user experience, and task parameters. The pilots (N=12) performed a set of tasks characterized by different Task Focus (Process vs. Outcome) and Navigation Difficulty (Low vs. High); their experience was assessed by using a set of subjective rating scales. It was found that the experience of being physically present in a remote location positively correlated with self-reported gaming skills, objective indicators of navigation efficiency, as well as feelings of being safe and relaxed. No significant effects of task focus and navigation difficulty were found.","PeriodicalId":334884,"journal":{"name":"Proceedings of the 32nd European Conference on Cognitive Ergonomics","volume":"175 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130240616","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Elsa Yousfi, S. Malin, Lynda Halit, Stéphane Roger, Ebru Dogan
{"title":"Driver experience and safety during manual intervention in a simulated automated vehicle: Influence of longer time margin allowed by connectivity","authors":"Elsa Yousfi, S. Malin, Lynda Halit, Stéphane Roger, Ebru Dogan","doi":"10.1145/3452853.3452888","DOIUrl":"https://doi.org/10.1145/3452853.3452888","url":null,"abstract":"Deployment of personal automated vehicles is an on-going process. However, automated systems are expected to rely on human control and cooperation for a consequent amount of time. Thus, it is essential to (1) design carefully the interaction between the automated system and the driver and, (2) ensure proper trust and acceptance of the system. One way to do that is by providing drivers with longer time margins to prepare to intervene in driving in response to a takeover request. Extended sensing and prediction enabled by vehicle-to-infrastructure (V2I) connectivity might help to reach this goal. The experiment presented here aimed at investigating potential benefits of V2I on the interaction between the driver and the automated vehicle. In this study, carried out on a driving simulator, participants experienced two driving scenarios; one with an automated vehicle connected by V2I and one with a basic non-connected automated vehicle. Each scenario encompassed a takeover request caused by a roadwork zone. The vehicle connected with V2I allowed an anticipated takeover request. Those scenarios aimed at comparing (1) the safety of the driver intervention (2) driver's experience and mental workload during manual takeover and, (3) subjective evaluation of the system, with the connected and the basic automated vehicle. Results revealed a positive effect of V2I on the quality of driver's safety intervention in response to a takeover request and on the drivers’ subjective evaluation of the system and experience during the manual intervention.","PeriodicalId":334884,"journal":{"name":"Proceedings of the 32nd European Conference on Cognitive Ergonomics","volume":"90 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126423232","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
M. Landoni, T. Huibers, Emiliana Murgia, Mohammad Aliannejadi, M. S. Pera
{"title":"Somewhere over the Rainbow: Exploring the Sense for Relevance in Children","authors":"M. Landoni, T. Huibers, Emiliana Murgia, Mohammad Aliannejadi, M. S. Pera","doi":"10.1145/3452853.3452885","DOIUrl":"https://doi.org/10.1145/3452853.3452885","url":null,"abstract":"We explore the facets of relevance that guide children when assessing materials retrieved by search engines when looking for information in the classroom. We involved children in a collaborative exercise and asked them to design innovative icons to point their peers towards useful results. We also asked them to complete a survey meant to capture explicit motivators guiding their design. This resulted in a rich set of metaphors. Analysis of the emerging metaphors is what allowed us to identify and discuss the many interpretations of relevance children naturally assign to resources they find in response to school-related information discovery tasks.","PeriodicalId":334884,"journal":{"name":"Proceedings of the 32nd European Conference on Cognitive Ergonomics","volume":"64 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127728849","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
S. Ermini, Dario Bernabini, G. Burresi, Martino Lorusso, A. Rizzo
{"title":"Human-Centered Retrofitting","authors":"S. Ermini, Dario Bernabini, G. Burresi, Martino Lorusso, A. Rizzo","doi":"10.1145/3452853.3452889","DOIUrl":"https://doi.org/10.1145/3452853.3452889","url":null,"abstract":"Many of the essential elements of the Internet of Things (IoT) and the Industrial Internet of Things (IIoT) are driving the transition to Industry 4.0. It is important for companies and their staff to be flexible in accepting this change and to be ready to upgrade their equipment, structures and processes. In this paper, taking as a case study the Smart Retrofitting process performed applying a Design Thinking approach at a foundry in Livorno, Tuscany, we analyze the relationship between man and machine in the industrial sector and how the introduction of data collections devices, sensors and Machine Learning algorithms in production processes represents an opportunity to redesign this relationship. The solutions proposed aim to allow the plant to increase efficiency and decrease machinery fault. The main contribution of this work is to show how a XX century production machinery can be upgraded through human-centered Retrofitting focusing on specific problems to produce rapid and low-cost technical improvements, as well as the way in which the worker approaches their new or traditional tasks.","PeriodicalId":334884,"journal":{"name":"Proceedings of the 32nd European Conference on Cognitive Ergonomics","volume":"86 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116238003","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. Frich, M. M. Biskjaer, Christian Remy, Lindsay MacDonald Vermeulen, P. Dalsgaard
{"title":"Revisiting the Role of User Research in Design Creativity","authors":"J. Frich, M. M. Biskjaer, Christian Remy, Lindsay MacDonald Vermeulen, P. Dalsgaard","doi":"10.1145/3452853.3452857","DOIUrl":"https://doi.org/10.1145/3452853.3452857","url":null,"abstract":"Despite the claimed relevance of user research for design, we know little about how it impacts creativity specifically. Our objective is to establish a foundation for answering this question. We review empirical findings from related domains to examine how the value of user research is linked to design creativity.","PeriodicalId":334884,"journal":{"name":"Proceedings of the 32nd European Conference on Cognitive Ergonomics","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131868197","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Empowering resilient crisis response training through purposefully designed boundary objects in a simulation-gaming exercise approach: Supporting ad-hoc team interaction","authors":"B. Johansson, J. V. Laere, P. Berggren","doi":"10.1145/3452853.3452879","DOIUrl":"https://doi.org/10.1145/3452853.3452879","url":null,"abstract":"This paper reports on experiences of creating resilience-empowering simulation-gaming exercises concerning disruptions in the payment system. The aim of the exercises was to utilize gaming-simulation as a training environment where teams of practitioners can learn to develop in-depth understanding of systemic interaction (i.e., cascading effects of disruptions) and learn how to develop collaborative resilience across many different critical infrastructures. Interaction between different stakeholders in the payment system, such as business owners, banks, municipality crisis managers are considered as the foundation for building collaborative resilience. Designing the exercises in such a way that they encourage and support such interactions are therefore seen as a primary design goal. This paper describes lessons learned from the process of conducting the 17 exercises in terms of creating a well-balance simulation and an immersive experience.","PeriodicalId":334884,"journal":{"name":"Proceedings of the 32nd European Conference on Cognitive Ergonomics","volume":"61 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131020679","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The effect of future self-continuity on delaying immediate gratification. A virtual reality experiment","authors":"Valeria Faralla, A. Innocenti, Stefano Baraldi, Luca Lusuardi, Maurizio Masini, Vincenzo Santalucia, Diletta Scaruffi, Matteo Sirizzotti","doi":"10.1145/3452853.3452893","DOIUrl":"https://doi.org/10.1145/3452853.3452893","url":null,"abstract":"Empirical evidence shows that individuals have present-biased preferences, namely they put more weight on the present than on the future and their preferences are dynamically inconsistent. The lack of connectedness to the future self was proposed by Parfit (1971) as a possible explanation of this behavior. In our experiment we study the effect of experiencing future self-continuity in a CAVE automatic virtual environment on delaying immediate gratification. Participants choose between sooner or later monetary options and we find that they decide to differ with higher probability after virtual reality exposure.","PeriodicalId":334884,"journal":{"name":"Proceedings of the 32nd European Conference on Cognitive Ergonomics","volume":"47 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125468884","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}