Optimal Fidelity Selection for Human-Supervised Search

Abstract

We study optimal fidelity selection in human-supervised underwater visual search, where operator performance is influenced by cognitive factors such as workload. In our experiments, participants perform two simultaneous tasks: detecting underwater mines in videos (primary) and responding to a visual cue to estimate workload (secondary). Videos arrive as a Poisson process and queue for review, with the operator choosing between normal fidelity (faster playback) and high fidelity. Rewards depend on detection accuracy, while penalties are tied to queue length. Workload is modeled as a hidden state using an Input-Output Hidden Markov Model, and fidelity selection is optimized via a Partially Observable Markov Decision Process. We evaluate two setups: fidelity-only selection and a version that also allows task delegation to automation for queue stability. Our approach improves performance by 26.5% without delegation and 50.3% with delegation, compared to a baseline where humans manually select fidelity.