On Robust and Adaptive Fidelity Selection for Human-in-the-loop Queues

We consider a human agent servicing a queue of homogeneous tasks. The agent can service a task with normal or high fidelity level, where fidelity refers to the degree of exactness and precision while servicing the task. We assume the parameters of the human’s service time distribution depend on the selected fidelity level and her cognitive state and are assumed to be unknown a priori. These parameters are learned online through Bayesian parameter estimation. We formulate a robust adaptive semi-Markov decision process (SMDP) to solve our optimal fidelity selection problem and extend the results on convergence of robust-adaptive Markov decision processes (MDP) to robust-adaptive SMDPs.