Enhancing Fatigue Detection Through Heterogeneous Multisource Data Integration and Cross-Domain Modality Imputation

Fatigue detection for human operators is important in safety-related applications such as aviation, mining, and long-haul transport. Reliable estimation of operator fatigue can support timely warnings, adaptive task scheduling, takeover reminders, and other safety-management decisions in human–machine systems. However, the effectiveness of these functions depends on whether fatigue-related signals can be reliably captured in the deployment environment. While many studies have shown the value of high-fidelity sensors in controlled laboratory environments, their performance often degrades when used in real-world settings because of noise, lighting conditions, and field-of-view constraints, thereby limiting their practical use. This article formalizes a deployment-oriented setting for real-world fatigue detection, where high-quality sensors are often unavailable in practical applications. To address this issue, we use knowledge from heterogeneous source domains, including high-fidelity sensors that are difficult to deploy in the field but commonly used in controlled environments, to assist fatigue detection in the real-world target domain. Based on this idea, we design a heterogeneous and multisource fatigue-detection framework that uses the available modalities in the target domain while leveraging diverse configurations in the source domains through cross-domain modality imputation based on shared modalities. As a proof-of-concept, we intentionally employ well-established algorithms for both knowledge-transfer and fatigue-detection components to test the framework’s core idea. Experiments using a field-deployed sensor setup and two publicly available human–fatigue datasets demonstrate the feasibility of the proposed framework in sensor-constrained scenarios and show that knowledge from heterogeneous source domains can support fatigue detection when high-fidelity modalities are unavailable in the target domain.