A Human Factors-Based Assessment Framework for Evaluating Performance in Control Room Interface Design

Abstract

OCCUPATIONAL APPLICATIONS Evaluating the effects of control room interface design on operator performance is crucial to maintaining productivity and safety. Current regulations in the petrochemical industry encourage the inclusion of human factors principles in designing displays and the environment but provide little guidance on evaluation. The framework presented in this article can be used to evaluate petrochemical control room interface designs under a variety of conditions, including workload and experience levels. Applications of this framework include the evaluation of novel interface designs to determine which alternatives result in the highest performance and reasonable perceived workload levels. This framework could also be used to evaluate the effects of other performance-shaping factors, such as fatigue. TECHNICAL ABSTRACT Background: The petrochemical industry has a need to improve the design of control room interfaces, but regulations on how to evaluate designs are sparse. Purpose: This study describes the development and initial evaluation of a human factors-based framework for evaluating performance using control room interfaces under varying user experience levels, display designs, and workload levels. The framework considers three steps of human behavior in systems: sensing/perception, information processing, and physical/verbal responses. Direct performance measures of speed and accuracy and measures of perceived workload (NASA task load index and subjective workload assessment technique) assess these three steps, and situation awareness (situation awareness global assessment technique) assesses sensing/perception and information processing. Methods: To provide initial validation of the framework, students and professional operators (two experience levels) used current standard interface designs and poor interface designs at three workload levels. The participants completed three 30-minute scenarios and responded to various alarms that signaled failures during each scenario, which were created using a commercially available refinery simulator. Results: In general, the framework was sensitive to differences in interface design, workload level, and experience. Future research can determine sensitivity to other control room parameters and may include additional metrics, such as secondary task workload measures and eye tracking, depending on the goals of the evaluation. Conclusion: This research demonstrates the feasibility of applying human factors tools to performance evaluation of commercially available petrochemical control room interface designs.