Salient object ranking with reinforcement learning

Objects in an image inherently draw attention due to their vivid colors and larger sizes, indicating their high saliency. The salient object ranking (SOR) task aims to prioritize multiple salient objects within a scene based on their saliency levels. Past research has predominantly treated the SOR task as a static process, typically formulating it as a regression or classification problem. However, these approaches overlook the dynamic nature of human attention, which shifts as context and inter-object correlations influence focus. To address this, we employ a reinforcement learning strategy, modeling SOR as a dynamic iterative sequence process. We train an actor to select salient objects from the environment. Additionally, we design a reward strategy that encourages the selection of the most prominent object among those not previously chosen. The selection order generated by the actor directly determines the ranking of object saliency in the scene. Furthermore, we identify limitations in existing SOR evaluation metrics, which may falter in certain scenarios. To address this, we introduce a simple and useful metric, referred to as the F1-Sor, into SOR tasks, improving the evaluation accuracy of the SOR tasks. Our model achieves state-of-the-art performance on publicly available SOR datasets.