Online Non-Stationary Pricing Incentives for Budget-Limited Crowdsensing

Abstract

The promising applications of mobile crowdsensing (MCS) have attracted much research interest recently, especially for the posted-pricing scenes. However, existing works mainly focus on the stationary MCS, no matter whether in a stochastic or adversarial environment, where each price (or arm) remains identical over time. However, in many realistic MCS applications such as environment monitoring and recommendation systems, stationary bandits do not model the posted-pricing sequential decision problems where the reward distributions of each price (arm) and cost distribution vary over time due to the changes in light intensity and mobile devices’ remnant energy. While in this paper, we study a more general submodular crowdsensing scene to address the non-stationary sequential pricing problems, and construct a monotonic submodular function merging the marginal reward and temporal difference errors (TD-errors) of deep reinforcement learning (DRL). Moreover, we explore a weighted budget-limited non-stationary pricing mechanism by using the deep deterministic policy gradient (DDPG) method for submodular MCS from the perspectives of the hard-drop and soft-drop weights. Our mechanism can readily be extended to non-submodular MCS or other MCS scenes. Extensive simulations demonstrate that our mechanism outweighs existing benchmarks.