Ben W. Morrison, Kathryn Bergin, Joshua N. Kelson, Natalie M. V. Morrison, J. Innes, Gregory Zelic, Yeslam Al‐Saggaf, Manoranjan Paul
{"title":"Decision Support Systems (DSSs) ‘In the Wild’: The Factors That Influence Users’ Acceptance of DSSs in Naturalistic Settings","authors":"Ben W. Morrison, Kathryn Bergin, Joshua N. Kelson, Natalie M. V. Morrison, J. Innes, Gregory Zelic, Yeslam Al‐Saggaf, Manoranjan Paul","doi":"10.1177/15553434231191385","DOIUrl":"https://doi.org/10.1177/15553434231191385","url":null,"abstract":"A richer approach to studying Decision Support System (DSS) interactions is required to understand and predict the nature of actual use in the workplace. We used questionnaire and interview techniques to examine workers’ experiences relating to DSS use in naturalistic settings. We aimed to: 1) Reveal what workers perceive to be the most important factors when deciding whether to accept support from a DSS and 2) Elicit patterns that emerge from DSS users’ recounted experiences using the systems, which may impact their future use. Current and prospective DSS users ( N = 93) from numerous industries responded to a questionnaire relating to the factors they perceive to influence their use of DSSs. Subsequently, a retrospective interview protocol was employed to investigate the experiences of a subset of DSS users ( N = 10). The questionnaire results underscore a range of factors considered to be very important to the acceptance of DSSs (i.e. decision quality; decision importance; decision risk; historical accuracy; decision accountability; and system comprehension). Further, a series of interconnected themes relating to workers’ use of DSSs were identified from the interview transcripts using thematic analysis. We discuss how these issues may impact workers’ intentions to use DSSs in the workplace, and advocate for the use of naturalistic decision-making techniques to study technology acceptance.","PeriodicalId":46342,"journal":{"name":"Journal of Cognitive Engineering and Decision Making","volume":" ","pages":""},"PeriodicalIF":2.0,"publicationDate":"2023-08-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47496716","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Failure to Grasp Automation Failure","authors":"G. Skraaning, G. Jamieson","doi":"10.1177/15553434231189375","DOIUrl":"https://doi.org/10.1177/15553434231189375","url":null,"abstract":"Automation failure is a key construct in human-automation interaction research. Yet the paucity of exposition on this construct has led to confusion about what sorts of failures are suitable for testing predictions of human performance in response to automation failure. We illustrate here how overly narrow or broad definitions of automation failure limit the explanatory power of human performance models in a way that is not obviously reasoned. We then review three aviation safety events that challenge the overly narrow definition. Reflecting on those events and other observations, we propose an initial taxonomy of automation failure and other automation-related human performance challenges. We conclude by pointing out the utility of the taxonomy for advancing human-automation interaction research.","PeriodicalId":46342,"journal":{"name":"Journal of Cognitive Engineering and Decision Making","volume":" ","pages":""},"PeriodicalIF":2.0,"publicationDate":"2023-07-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47325957","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mohamad El Iskandarani, Jad A. Atweh, Shannon P. D. McGarry, S. L. Riggs, N. Moacdieh
{"title":"Does It MultiMatch? What Scanpath Comparison Tells us About Task Performance in Teams","authors":"Mohamad El Iskandarani, Jad A. Atweh, Shannon P. D. McGarry, S. L. Riggs, N. Moacdieh","doi":"10.1177/15553434231171484","DOIUrl":"https://doi.org/10.1177/15553434231171484","url":null,"abstract":"Teamwork and collaboration form the cornerstones of organizational performance and success. It is important to understand how the attention allocation of team members is linked to performance. One approach to studying attention allocation in a team context is to compare the scanpath similarity of two people working in teams and to explore the link between scanpath similarity and team performance. In this study, participants were recruited to work in pairs on an unmanned aerial vehicle (UAV) task that included low and high workload conditions. An eye tracker was used to collect the eye movements of both participants in each team. The scanpaths of two teammates were compared in low and high workload conditions using MultiMatch, an established scanpath comparison algorithm. The obtained scanpath similarity values were correlated with performance measures of response time and accuracy. Several MultiMatch measures showed significant strong correlations across multiple dimensions, providing insight into team behavior and attention allocation. The results suggested that the more similar each team member’s scanpath is, the better their performance. Additional research and consideration of experimental variables will be necessary to further understand how best to use MultiMatch for scanpath similarity assessment in complex domains.","PeriodicalId":46342,"journal":{"name":"Journal of Cognitive Engineering and Decision Making","volume":"17 1","pages":"294 - 309"},"PeriodicalIF":2.0,"publicationDate":"2023-06-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42675384","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Carrie Reale, Megan E Salwei, Laura G Militello, Matthew B Weinger, Amanda Burden, Christen Sushereba, Laurence C Torsher, Michael H Andreae, David M Gaba, William R McIvor, Arna Banerjee, Jason Slagle, Shilo Anders
{"title":"Decision-Making During High-Risk Events: A Systematic Literature Review.","authors":"Carrie Reale, Megan E Salwei, Laura G Militello, Matthew B Weinger, Amanda Burden, Christen Sushereba, Laurence C Torsher, Michael H Andreae, David M Gaba, William R McIvor, Arna Banerjee, Jason Slagle, Shilo Anders","doi":"10.1177/15553434221147415","DOIUrl":"10.1177/15553434221147415","url":null,"abstract":"<p><p>Effective decision-making in crisis events is challenging due to time pressure, uncertainty, and dynamic decisional environments. We conducted a systematic literature review in PubMed and PsycINFO, identifying 32 empiric research papers that examine how trained professionals make naturalistic decisions under pressure. We used structured qualitative analysis methods to extract key themes. The studies explored different aspects of decision-making across multiple domains. The majority (19) focused on healthcare; military, fire and rescue, oil installation, and aviation domains were also represented. We found appreciable variability in research focus, methodology, and decision-making descriptions. We identified five main themes: (1) decision-making strategy, (2) time pressure, (3) stress, (4) uncertainty, and (5) errors. Recognition-primed decision-making (RPD) strategies were reported in all studies that analyzed this aspect. Analytical strategies were also prominent, appearing more frequently in contexts with less time pressure and explicit training to generate multiple explanations. Practitioner experience, time pressure, stress, and uncertainty were major influencing factors. Professionals must adapt to the time available, types of uncertainty, and individual skills when making decisions in high-risk situations. Improved understanding of these decisional factors can inform evidence-based enhancements to training, technology, and process design.</p>","PeriodicalId":46342,"journal":{"name":"Journal of Cognitive Engineering and Decision Making","volume":"17 2","pages":"188-212"},"PeriodicalIF":2.2,"publicationDate":"2023-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10564111/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41215741","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ross Owen Phillips, Hossein Baharmand, Nico Vandaele, Catherine Decouttere, Lise Boey
{"title":"How Can Authorities Support Distributed Improvisation During Major Crises? A Study of Decision Bottlenecks Arising During Local COVID-19 Vaccine Roll-Out.","authors":"Ross Owen Phillips, Hossein Baharmand, Nico Vandaele, Catherine Decouttere, Lise Boey","doi":"10.1177/15553434221125092","DOIUrl":"10.1177/15553434221125092","url":null,"abstract":"<p><p>Despite the increased importance attributed to distributed improvisation in major crises, few studies investigate how central authorities can promote a harmonic, coordinated national response while allowing for distributed autonomy and improvisation. One idea implicit in the literature is that central authorities could help track and tackle common decision bottlenecks as they emerge across \"improvising\" local authorities as a result of shared, dynamic external constraints. To explore this idea we map central functions needed to roll-out vaccines to local populations and identify and classify bottlenecks to decision-making by local authorities managing COVID-19 vaccine roll-out in Norway. We found five bottlenecks which emerged as vaccine roll-out progressed, three of which could feasibly have been addressed by changing the local authorities' external constraints as the crisis developed. While the national crisis response strategy clearly allowed for distributed improvisation, our overall findings suggest that there is potential for central authorities to address external constraints in order to ease common bottlenecks as they emerge across local authorities responding to the crisis. More research is to explore alternative centralized response strategies and assess how well they effectively balance centralized and distributed control. The study contributes to the growing literature examining the interaction between local and centralized response in crisis management.</p>","PeriodicalId":46342,"journal":{"name":"Journal of Cognitive Engineering and Decision Making","volume":"17 1","pages":"166-187"},"PeriodicalIF":2.2,"publicationDate":"2023-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9659687/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46967841","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"What we got Here, is a Failure to Coordinate: Implicit and Explicit Coordination in Air Combat","authors":"H. Mansikka, K. Virtanen, D. Harris","doi":"10.1177/15553434231179566","DOIUrl":"https://doi.org/10.1177/15553434231179566","url":null,"abstract":"Air combat is the ultimate test for teamwork, as teams of fighter pilot (or flights), must coordinate their actions in a highly complex, hostile, dynamic and time critical environment. Flights can coordinate their actions using communication, that is, explicitly, or by relying on team situation awareness (SA), that is, implicitly. This paper examines how these two forms of coordination are associated with performance when prosecuting or evading an attack in simulated air combat. This was done by investigating the flights’ team SA, number of SA-related communication acts and performance in these two types of critical events during air combat. The results exhibit a quadratic dependence between team SA and communication. The rate of change of SA-related communication frequency with respect to change of team SA was negative: communication was needed to build team SA, but once an appropriate level of team SA was established, fewer communications were required. If, however, team SA deteriorated the number of SA communication acts increased. However, during time critical events, the flights did not always have enough time to coordinate their actions verbally. If the flights’ team SA in such situations was low, the flights’ explicit coordination attempts were not sufficient to avoid poor performance.","PeriodicalId":46342,"journal":{"name":"Journal of Cognitive Engineering and Decision Making","volume":"17 1","pages":"279 - 293"},"PeriodicalIF":2.0,"publicationDate":"2023-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44306134","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kyana H. J. van Eijndhoven, Travis J. Wiltshire, Elwira A. Hałgas, J. Gevers
{"title":"A Computational Approach to Examining Team Coordination Breakdowns During Crisis Situations","authors":"Kyana H. J. van Eijndhoven, Travis J. Wiltshire, Elwira A. Hałgas, J. Gevers","doi":"10.1177/15553434231156417","DOIUrl":"https://doi.org/10.1177/15553434231156417","url":null,"abstract":"During crisis situations, teams are more prone to coordination breakdowns that are characterized by a temporary, diminished ability to function effectively as a team. However, team research currently lacks robust approaches for identifying transitions from effective team functioning to coordination breakdowns. With the current study, we aimed to develop such robust approaches, and to deepen our understanding of how team coordination dynamics across various physiological signals reflect coordination breakdowns. Consequently, we used audiovisual data from four-person teams involved in a stressful collaborative game task to manually identify coordination breakdowns. Next, we set out to computationally identify coordination breakdowns by applying continuous measures of team coordination (windowed synchronization coefficient and multidimensional recurrence quantification analysis) to photoplethysmogram and electrodermal activity data obtained during the task, and identifying transitions therein with change point and nonlinear prediction algorithms. We found that our computational coordination breakdown identification approaches can identify up to 96% of the manually identified coordination breakdowns although our results also show that the precision of our approaches falls far behind. Our findings contribute theoretically and methodologically to the systematic investigation of coordination breakdowns, which may ultimately facilitate the support of teams in responding to and mitigating negative consequences of crisis situations.","PeriodicalId":46342,"journal":{"name":"Journal of Cognitive Engineering and Decision Making","volume":"17 1","pages":"256 - 278"},"PeriodicalIF":2.0,"publicationDate":"2023-02-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46503899","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Three Player Interactions in Urban Settings: Design Challenges for Autonomous Vehicles","authors":"D. Nathanael, V. Papakostopoulos","doi":"10.1177/15553434231155032","DOIUrl":"https://doi.org/10.1177/15553434231155032","url":null,"abstract":"An observational analysis of crossing episodes between two intersecting vehicles, in which a third road user clearly affected its evolution, was conducted in an attempt to identify (i) recurring patterns of informal coordination among road users and (ii) traffic situational invariances that may inform AV prediction algorithms. The term BLOCK-EXPLOITING is introduced to describe a driver’s exploitation of situational opportunities to gain priority often contrary to regulatory provisions, but favouring overall traffic efficiency. Video-data from an urban stop-controlled intersection were analysed through the lens of joint systems theory using a phenomenological framework developed in this study. Four generic types of BLOCK-EXPLOITING were identified (i.e. covering, ghost-covering, piggybacking, sneaking). Covering and ghost-covering led to minimal or no delays while piggybacking and sneaking, although abusive to other drivers, still only resulted in 1.99 to 3.33 sec delay. It is advocated that BLOCK-EXPLOITING can be socially acceptable. Proposed design challenges for AVs in mixed traffic include the ability to (i) distinguish BLOCK-EXPLOITING from errant driving, (ii) recognise to whom a ‘space-offering’ is addressed, and (iii) assess the appropriateness or abusiveness of a BLOCK-EXPLOITING action. Finally, this study brings to fore very short-time span joint-activity coordination requirements among diverse agents unknown to each other.","PeriodicalId":46342,"journal":{"name":"Journal of Cognitive Engineering and Decision Making","volume":"17 1","pages":"236 - 255"},"PeriodicalIF":2.0,"publicationDate":"2023-02-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44038123","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Alexandre Marois, Katherine Labonté, D. Lafond, Heather F. Neyedli, S. Tremblay
{"title":"Cognitive and Behavioral Impacts of Two Decision-Support Modes for Judgmental Bootstrapping","authors":"Alexandre Marois, Katherine Labonté, D. Lafond, Heather F. Neyedli, S. Tremblay","doi":"10.1177/15553434231153311","DOIUrl":"https://doi.org/10.1177/15553434231153311","url":null,"abstract":"The Cognitive Shadow is a decision-support system that uses policy capturing to model human operators’ judgment policies and provide online predictions of their decisions. The system can provide support in reaction to a decision mismatch (shadowing mode) or proactively (recommendation mode). The goal of this study was to compare these two modes of operation in their ability to effectively model and support decision-making and to examine impacts on information processing, workload, and trust. Participants took part in an aircraft threat evaluation simulation without decision support or with the Cognitive Shadow (either shadowing or recommendation mode). Dwell time was collected over different areas of the user interface. While the recommendation mode had no advantage over the control group, the shadowing mode resulted in greater human and model accuracy. This mode led to longer dwell time over the parameters zone presenting key information for decision-making. These benefits were maintained even after the tool was removed. Workload was unaffected by the mode, and while trust was initially higher in the recommendation mode, it quickly became equivalent between both modes, overall supporting shadowing as the better configuration for cognitive assistance. Results are discussed in terms of decision processes, operators support, and automation bias.","PeriodicalId":46342,"journal":{"name":"Journal of Cognitive Engineering and Decision Making","volume":"17 1","pages":"215 - 235"},"PeriodicalIF":2.0,"publicationDate":"2023-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42767362","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
K. Bennett, Christopher Edman, Dylan G. Cravens, Natalie C. Jackson
{"title":"Decision Support for Flexible Manufacturing Systems: Application of the Cognitive Systems Engineering and Ecological Interface Design Approach","authors":"K. Bennett, Christopher Edman, Dylan G. Cravens, Natalie C. Jackson","doi":"10.1177/15553434221118976","DOIUrl":"https://doi.org/10.1177/15553434221118976","url":null,"abstract":"We applied the cognitive systems engineering (CSE)/ecological interface design (EID) approach to the work domain of flexible manufacturing systems (FMS). Work domain analyses (WDA) and activity analyses were conducted. Previous concerns regarding the suitability of CSE for FMS are addressed. Principles of EID (i.e., direct perception, direct manipulation, and visual momentum) were applied in designing an ecological interface for a flexible manufacturing system (EcoFlex). The critical links between the products of a WDA and the content of an ecological interface (a key innovation of EID) are made explicit. The evaluations of this interface were very positive across a wide variety of traditional and novel dependent variables (please see the companion manuscript). The overall success of this research program clearly demonstrates that the CSE/EID approach is capable of being applied to FMS. Contributions of this manuscript include both theoretical insights (defining properties of the FMS work domain, concrete examples of the principles of EID, critical links between CSE and EID, resolution of concerns about the overall framework) and practical applications (specific design solutions for FMS and similar work domains, and successful examples of the CSE/EID approach which can be studied and leveraged by students).","PeriodicalId":46342,"journal":{"name":"Journal of Cognitive Engineering and Decision Making","volume":"17 1","pages":"99 - 119"},"PeriodicalIF":2.0,"publicationDate":"2023-01-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42689309","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}