Evolutionarily stable strategies in population games: An invader’s perspective

We revisit the various definitions of an Evolutionarily Stable Strategy ($\mathrm{ESS}$) in nonlinear population games from the standpoint of barrier functions. We show the equivalence between an $\mathrm{ESS}$ being uniformly uninvadable and the corresponding barrier function being lower semi-continuous (LSC). Moreover, it is sufficient to check this for strategies that are near an alternative best reply lying on an opposite face. We also provide some counterexamples that show that uniform stability cannot be taken for granted in nonlinear population games; we denote such $\mathrm{ESS}$s as singular $\mathrm{ESS}$s. Furthermore, we obtain conditions that are equivalent to the barrier function being LSC and are typically easier to verify. As a by-product, we identify a number of instances where being an $\mathrm{ESS}$ is equivalent to being uniformly uninvadable: 3-player games, payoffs inducing convex incentives, or differentiable payoffs with negative definite first derivative, when considered on alternative best replies.