Play profiles: The effect of infinite-length games on evolution in the iterated Prisoner's Dilemma

It is well-known that the correct strategy in iterated Prisoner's Dilemma with a finite known number of rounds is to always defect. Evolution of Prisoner's Dilemma playing agents mirrors this: the more rounds the agents play against each other per encounter, the more likely the population will evolve to a cooperative state. Prior work has demonstrated that the result of evolution changes dramatically from very short games up to about 60-85 rounds, which yields substantially similar populations as those using 150 rounds. We extend this study using more powerful statistical tests and mathematical tools, including fingerprinting and play profiles, to consider the problem in the opposite direction: as the correct strategy in infinitely iterated Prisoner's Dilemma is to always cooperate, how many rounds are needed until evolution reflects this empirically? Within a very large plateau, from around 150 to a million rounds, evolution does not significantly change its behaviour. Surprisingly, behaviour does change again from millions to billions of rounds, but not further from billions to infinite-round games. This suggests that evolution operates on nontrivial categories of cooperativity depending on the number of rounds and the details of the representation.