From competition to cooperation: Co-evolution in a rewards continuum

In this study the hypothesis that zero-sum (i.e strictly competitive) games are more difficult targets for co-evolution than non-zero-sum (i.e. games that are not strictly competitive nor strictly cooperative) games is examined. Our method is to compare the co-evolutionary behavior of a three move zero-sum game (rock paper scissors) with that of a three move non-zero-sum game (coordination prisoner's dilemma) as well as with intermediate games obtained using weighted averages of the games's payoff matrices. The games are compared by examining the way use of moves evolves, by using transitivity measures on evolved agents, by estimating the complexity of the agents and by checking for non-local adaptation. Two different agent representations, finite state machines with 8 and 64 states, are used. Unexpectedly, these two representations are found to have large, qualitative differences. The results support the hypothesis that co-evolving good strategies for zero-sum games is more difficult than for non-zero-sum games. Many of the measurements used to compare different games are found to exhibit a nonlinear responses to the change in payoff matrix.