Human Factors and Simulation最新文献

{"title":"Characterizing Complexity: A Multidimensional Approach to Digital Control Room Display Research","authors":"Kelly Dickerson, John Grasso, Heather Watkins, Niav Hughes","doi":"10.54941/ahfe1003563","DOIUrl":"https://doi.org/10.54941/ahfe1003563","url":null,"abstract":"Complexity can be characterized at numerous levels; physical, perceptual, and cognitive features all influence the overall complexity of an informational display. The Human Performance Test Facility (HPTF) at the U.S. Nuclear Regulatory Commission (NRC) develops lightweight simulator studies to examine the workload induced by various control room-related tasks in expert and non-expert populations. During the initial development of the lightweight simulator, cognitive complexity was defined based on the number of elements in each control panel. While the number of items roughly maps onto information density, it is only one of several features contributing to display complexity. This study is a follow-up to the original complexity evaluation and includes an initial characterization of the perceptual complexity of a set of control panels in their original (i.e., unmodified) and modified (for cognitive complexity reduction) forms. To assess perceptual complexity, a 3-dimensional approach was developed. The control panel displays were assessed using common measures of physical complexity (e.g., edge congestion, clutter, symmetry), performance-based measures (reaction time and accuracy for target identification), and subjective impressions using a survey adapted from a similar FAA assessment of air traffic controller workstation display complexity. Overall, the results suggested that clutter and symmetry were associated with target identification performance; participants interacting with high symmetry-low clutter displays identified target controls faster than those interacting with low symmetry-high clutter displays. Survey results tended to follow the same pattern as the physical and performance-based results; however, these patterns were not statistically significant, likely due to the small sample size. These initial results are a promising indication that the physical and performance-based measures were valid for assessing display complexity and that they are sensitive to differences in complexity, even with smaller samples. The physical and performance-based measures may be good candidates for human factors validation of future system designs - they are quick and easy to administer while providing a holistic sense of display perceptual complexity. Like other types of surveys, surveys for display complexity often require large samples to detect meaningful differences between groups. System designers and other stakeholders may want to consider alternative strategies, such as physical system measurement and characterization using performance-based methods if the user base is small or designs are in the early stages of development, requiring quick answers and an iterative approach to evaluation.","PeriodicalId":102446,"journal":{"name":"Human Factors and Simulation","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128582640","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Behavioral indicators - an approach for assessing nuclear control room operators’ excessive cognitive workload?","authors":"Per Øivind Braarud, Giovanni Pignoni","doi":"10.54941/ahfe1003565","DOIUrl":"https://doi.org/10.54941/ahfe1003565","url":null,"abstract":"Cognitive workload that deteriorates the control room team’s performance is a central topic for human-technology design and evaluation. However, while stated as an essential research topic, the literature provides few studies investigating the excessive cognitive workload of complex dynamic human-system work. Multiple techniques have been developed to sample workload. Still, they all struggle to determine the nature of excessive workload, capturing change but leaving the interpretation to the investigator. To advance the measurement of excessive cognitive workload of complex work, this paper proposes to investigate behavioral indicators. Behavioral-based methods differ from performance measures as they concentrate on the operator's behavior rather than the outcome of the actions. The information embedded in the operator’s behavior may not directly reflect the outcome of the task. The paper proposes indicator categories in terms of task prioritization, work practices and low-level behavior. The approach implies developing an understanding of how control room teams adapt to and manage task load and how operators are affected by high workload – for the identification of indicators, and for the development and validation of measures from these cognitive workload indicators. The paper presents an initial review of simulator studies identifying adaption such as down-prioritizing secondary tasks, reducing attention to global process overview, asking for or providing team support on task demand, reducing verification of work, and delayed response in communication. Furthermore, we briefly consider the technical and staffing requirements necessary to support these measures.","PeriodicalId":102446,"journal":{"name":"Human Factors and Simulation","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129269654","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Assessment of pilots' training efficacy as a safety barrier in the context of Enhanced Flight Vision Systems (EFVS)","authors":"Dimitrios Ziakkas, K. Pechlivanis, Brian G. Dillman","doi":"10.54941/ahfe1003568","DOIUrl":"https://doi.org/10.54941/ahfe1003568","url":null,"abstract":"Aviation and air travel have always been among the businesses at the forefront of technological advancement throughout history. Both the International Air Transportation Authority's (IATA) Technology Roadmap (IATA, 2019) and the European Aviation Safety Agency's (EASA) Artificial Intelligence (AI) roadmap (EASA, 2020) propose an outline and assessment of ongoing technological prospects that change the aviation environment with the implementation of AI from the initial phases. New technology increased the operational capabilities of airplanes in adverse weather. An enhanced flight vision system (EFVS) is a piece of aircraft equipment that captures and displays a scene image for the pilot, allowing for improved scene and object detection. Moreover, an EFVS is a device that enhances the pilot's vision to the point where it is superior to natural sight. An EFVS has a display for the pilot, which can be a head-mounted display or a head-up display, and image sensors such as a color camera, infrared camera, or radar. A combined vision system can be made by combining an EFVS with a synthetic vision system. A forward-looking infrared camera, also known as an enhanced vision system (EVS), and a Head-Up Display (HUD) are used to form the EFVS. Two aircraft types can house an EFVS: fixed-wing (airplane) and rotary-wing (helicopter).Several operators argue that the use of Enhanced Flight Vision Systems (EFVS) may be operated without the prior approval of the competent authority, assuming that the flight procedures, equipment, and pilot safety barriers are sufficiently robust. This research aims to test pilots' readiness levels with no or little exposure to EFVS to use such equipment (EASA, 2020). Moreover, the Purdue simulation center aims to validate this hypothesis. The Purdue human systems integration team is developing a test plan that could be easily incorporated into the systems engineering test plan to implement Artificial Intelligence (AI) in aviation training globally and evaluate the results. Based on guidelines from the International Air Transport Association (IATA), the Purdue University School of Aviation and Transportation Technology (SATT) professional flying program recognizes technical and nontechnical competencies. Furthermore, the Purdue Virtual Reality research roadmap is focused on the certification process (FAA, EASA), implementation of an AI training syllabus following a change management approach, and introduction of AI standardization principles in the global AI aviation ecosystem.","PeriodicalId":102446,"journal":{"name":"Human Factors and Simulation","volume":"319 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123203384","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Developing Confidence in Machine Learning Results","authors":"Jessica Baweja, Brett A. Jefferson, C. Fallon","doi":"10.54941/ahfe1003576","DOIUrl":"https://doi.org/10.54941/ahfe1003576","url":null,"abstract":"As the field of deep learning has emerged in recent years, the amount of knowledge and expertise that data scientists are expected to absorb and maintain has correspondingly increased. One of the challenges experienced by data scientists working with deep learning models is developing confidence in the accuracy of their approach and the resulting findings. In this study, we conducted semi-structured interviews with data scientists at a national laboratory to understand the processes that data scientists use when attempting to develop their models and the ways that they gain confidence that the results they obtained were accurate. These interviews were analysed to provide an overview of the techniques currently used when working with machine learning (ML) models. Opportunities for collaboration with human factors researchers to develop new tools are identified.","PeriodicalId":102446,"journal":{"name":"Human Factors and Simulation","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127476209","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Toward a Systems Framework Coupling Safety Culture, Risk Perception, and Hazard Recognition for the Mining Industry","authors":"Leonard Brown, Ngan Pham, J. Burgess","doi":"10.54941/ahfe1001493","DOIUrl":"https://doi.org/10.54941/ahfe1001493","url":null,"abstract":"The United States mining industry has made steady progress to improve worker safety and reduce injuries. Despite these gains, the industry remains largely reactive in its approach to health and safety. There remains a primary focus on lagging indicators, such as the numbers of injuries, hours lost, and hazards found at the worksite. To facilitate a more proactive approach, new methods are needed to evaluate hazardous conditions and unsafe behaviors. This work explores the relationships among mine workers’ hazard recognition abilities, the individual’s perception of risk, and the safety culture of the mining workplace. We have conducted a literature review to identify key factors and analytical models in industries where health and safety are a major consideration, including construction, manufacturing, mining, and transportation. Our analysis considered both process-oriented frameworks, such as Systems Thinking approaches, and statistical methods, including Structural Equation Modeling (SEM). A meta-model was then developed to aggregate and examine key factors and potential causal relationships. We discuss the creation of this meta-model, identifying notable structural characteristics and hypotheses for future confirmatory analysis. Use cases are then outlined, including descriptive, evaluative, and generative applications.","PeriodicalId":102446,"journal":{"name":"Human Factors and Simulation","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129162698","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The Role of Artificial Theory of Mind in Supporting Human-Agent Teaming Interactions","authors":"Jessica Williams, Rhyse Bendell, S. Fiore, F. Jentsch","doi":"10.54941/ahfe1003561","DOIUrl":"https://doi.org/10.54941/ahfe1003561","url":null,"abstract":"In this article we discuss the role of Artificial Theory of Mind (AToM) in supporting human-agent teaming interactions. Humans are able to interpret, understand, and predict another’s behavior by leveraging core socio-cognitive processes, generally referred to as Theory of Mind (ToM). A human’s ToM is critical to their ability to successfully interact with others, especially in the context of teaming. Considering the increasing role of AI in team cognition, there is an emerging need for agents capable of such complex socio-cognitive processes. We report findings from a large multi-organization research program, DARPA’s Artificial Social Intelligence Supporting Teams (ASIST), designed to study teamwork with socially intelligent artificial agents serving as team advisors. We focus on agent-to-human communications, including content, intended purpose, and, particularly, the use of AToM attributions in both covert agent explanations as rationale for giving a certain intervention, as well as the use of agents making overt ToM attributions of players in the intervention itself. The findings suggest that agent teammates are able to demonstrate AToM and that that interventions based upon these can influence team outcomes. We discuss the impact of the various types of ASI interventions and their effect on teams, and provide recommendations for future research on human-AI teaming.","PeriodicalId":102446,"journal":{"name":"Human Factors and Simulation","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122498909","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Studying Control Room Operations on a Shoestring Budget - Reflections on the Rancor Microworld","authors":"T. Ulrich, R. Boring, Roger T. Lew","doi":"10.54941/ahfe1001488","DOIUrl":"https://doi.org/10.54941/ahfe1001488","url":null,"abstract":"As the U.S. continues to develop and mature advanced reactor designs, the nuclear industry is becoming increasingly aware of the need for good human factors are to ensure safe, reliable, effective, and economical concept of operations. Advanced reactor designs aim to reduce staffing, and significant operational costs, by adopting high levels of automation. The highly automated control system designs must be informed with human factors and human reliability data. The proposed concepts of operations are unlike the current, largely manual, concept of operations found in operating nuclear power plants. Human performance data collection has proven difficult to obtain for existing nuclear power plants. Human factors researchers working on advanced reactor designs will encounter these same fundamental challenges and more. The novel concept of operations and accompanying human-system interfaces are novel and require human performance data for validation and licensing. Methods to evaluate novel concepts of operations for diverse advanced reactor designs must be identified to aid vendors in their system design activities. The Rancor microworld is a simulation platform that is currently used to support advanced reactor vendors in developing their control room concepts. The rationale and historical use of the Rancor microworld demonstrates a unique and complimentary approach to traditional full-scope simulator data collection methods that rely on expert licensed operators. The Rancor microworld is a reduced-order model of a small modular reactor conceived and developed to support human performance research on nuclear operations topics. The microworld represents the core elements of a nuclear power plant sans the complexity associated with full-scope simulators that are typically used to support human factors and human reliability research. The impetus for the microworld as an alternative method to acquire human performance data stems from the challenges in performing full-scope simulator studies. Full-scope simulators are expensive to build and maintain. Furthermore, they require extensive expertise to develop scenarios to support specific hypothesis testing. Operations data is historically difficult to obtain since even large research organizations that can afford a full-scope simulator facility encounter sample size issues. Licensed operators are expensive and fully time committed to their employing nuclear power plant. As such, it is very difficult to perform research on nuclear control room operations with sufficient sample sizes to approach statistical significance and draw generalizable conclusions applicable to different designs. Therefore, an alternative population using a simplified simulator offers an approach to evaluate human factors issues. Through numerous studies, the Rancor microworld has demonstrated an effective means to leverage inexpensive and ubiquitous student participants to expand the data collection capability and build a corpus of h","PeriodicalId":102446,"journal":{"name":"Human Factors and Simulation","volume":"48 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122675902","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Human Factors for Machine Learning in Astronomy","authors":"John E. Wenskovitch, A. Jaodand","doi":"10.54941/ahfe1003580","DOIUrl":"https://doi.org/10.54941/ahfe1003580","url":null,"abstract":"In this work, we present a collection of human-centered pitfalls that can occur when using machine learning tools and techniques in modern astronomical research, and we recommend best practices in order to mitigate these pitfalls. Human concerns affect the adoption and evolution of machine learning (ML) techniques in both existing workflows and work cultures. We use current and future surveys such as ZTF and LSST, the data that they collect, and the techniques implemented to process that data as examples of these challenges and the potential application of these best practices, with the ultimate goal of maximizing the discovery potential of these surveys.","PeriodicalId":102446,"journal":{"name":"Human Factors and Simulation","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130101636","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Using Cognitive Models to Develop Digital Twin Synthetic Known User Persona","authors":"Audrey Reinert, S. Rebensky, Maria Chaparro Osman, Baptiste Prébot, Cleotilde González, Don Morrison, V. Yerdon, Daniel Nguyen","doi":"10.54941/ahfe1003572","DOIUrl":"https://doi.org/10.54941/ahfe1003572","url":null,"abstract":"A recurring challenge in user testing is the need to obtain a record of user interactions that is large enough to reflect the different biases of a single user persona while accounting for temporal and financial constraints. One way to address this need is to use digital twins of user personas to represent the range of decisions that could be made by a persona. This paper presents a potential use of cognitive models of user personas from a single complete record of a persona to test the web-based decision support system, ALFRED the BUTLER. ALFRED the BUTLER is a digital cognitive assistant developed to generate recommended articles for users to review and evaluate relative to a priority information request (PIR).Interaction data for three different user personas for the ALFRED the BUTLER system were created: the Early Terminator, the Disuser, and the Feature Abuser. These three personas were named after the type of interaction they would have with the data and were designed to represent different types of human-automation user interactions as outlined by Parasuraman & Riley (1997). The research team operationalized the definitions of use, misuse, disuse, and abuse to fit the current context. Specifically, the Early Terminator represented misuse by no longer meaningfully interacting with the system once a search criterion was met whereas the Disuser represented disuse by never using a certain feature. The Feature Abuser represented abuse by excessively using a single feature when they should be using other features. Each member of the research team was assigned a user persona, given a briefing related to their persona, and instructed to rate 250 articles as either relevant (thumbs up), irrelevant (thumbs down), or neutral (ignore). Subsequently, a cognitive model of the task was built. Cognitive models rely on mechanisms that capture human cognitive processes such as memory, learning, and biases to make predictions about decisions that humans would be likely to make (Gonzalez & Lebiere, 2005). To construct the cognitive model, we relied on the Instance-Based Learning (IBL) Theory (Gonzalez et al., 2003), a cognitive theory of experience-based decision making. The data for each user’s previous actions were added to the model’s memory to make predictions about the next action the user would be likely to make (thumbs up, thumbs down, or ignore an article). The model was run 100 times for each persona, with the 250 articles presented in the same order as they were judged by the persona. The results indicate an overall model prediction accuracy of the persona’s decisions above 60%. Future work will focus on refining and improving the model's predictive accuracy The authors discuss future applications, one of which is using this type of cognitive modeling to help create synthetic datasets of persona behaviors for evaluation and training of machine learning algorithms.ReferencesGonzalez, C., & Lebiere, C. (2005). Instance-based cognitive models of decisi","PeriodicalId":102446,"journal":{"name":"Human Factors and Simulation","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128237268","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Improving Trust in Power System Measurements","authors":"A. Riepnieks, H. Kirkham","doi":"10.54941/ahfe1003577","DOIUrl":"https://doi.org/10.54941/ahfe1003577","url":null,"abstract":"The power grid is a large and complex system. The system becomes larger and more complex daily. Distributed energy resources and a more active customer role are factors adding to the complexity. This complicated system is operated by a combination of human operators and automation. Effective control of the power grid requires an increasing amount of automation to support system operators. The need for more support from automation can only increase as operational complexity increases.Actions in controlling the system are entirely dependent on the results of measurements. Measurements inform decisions at all scales. How much trust can be placed in the measurements is essentially an unknown factor. North American Electric Reliability Corporation has generated reports showing that procedures and models have not always worked as expected. Part of the problem lies in the fact that system events can distort signal waveforms. Another part of the problem is that events taking place outside the control area of an operator can affect measured results. The companies involved, and their regulators, have had to change their requirements and guidelines.High “accuracy” measurements are available for most quantities of interest, but the problems are related to trustworthiness, rather than “accuracy.” Accuracy is established for a device within a controlled environment, where a “true value” can be estimated. Real-world conditions can be vastly different. The instrument may provide accurate output according to its specifications, but the measurement might not represent reality because what is happening in the real world is outside the bounds of these specifications. That is a problem that demands a solution. The crux of the matter is this: a real-world measurement’s usefulness as a decision-making aid is related to how believable the measurement is, and not to how accurate the owner’s manual says the instrument is. The concept of “uncertainty” that metrologists have refined over the last few decades is a statistical process that predicts the dispersion of future results. Such a measure is virtually meaningless for real-time power system use. The properties of the power system are not stationary for long periods. A low-quality result can lead to a bad decision, because power system measurements presently lack any kind of real-time “trustworthiness connection.”The signal model generally used in the electric power industry is that the voltages and currents are well-represented by mathematical sinusoids. Given that starting point, we describe two trust metrics that provide verifiable links to the real-time system being measured. The metrics capture any mismatch between the instrument measurement model and the actual signal. Our trust-awareness metrics can lead to ways to develop more robust operating models in the power system environment. Every measurement result is reported with an associated real-time trust (or no-trust) metric, allowing the user (whether human or n","PeriodicalId":102446,"journal":{"name":"Human Factors and Simulation","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127767478","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}