PNAS paper explores the role of population structure in facilitating reciprocity

The “Early Edition” of the Proceedings of the National Academy of Sciences of the United States of America just posted online a paper entitled “Direct reciprocity in structured populations“. Authored by Matthijs van Veelen, Julián García, David G. Rand, and Martin A. Nowak, the paper combines two well-explored factors that influence how cooperation evolves: repeated interaction and population structure. This is one of those papers that make you say “wait, no one has done this model before?”. Given how well these authors investigate this system, I am thankful that they were the first to pull it off.

This paper has a lot of the hallmarks of a classic Martin Nowak paper: it focuses on the Prisoner’s Dilemma as the ultimate representation of all social dilemmas, produces a really comprehensive data set based on computer simulations, and then rounds off the whole package with separate analytical work that provides mathematical structure to the simulations. I have to say that this is a formula that works, and if I had the analytical chops of these authors this would be the approach that I would take to all of my research as well. You need the computer simulations in order to properly investigate a meaningful system: too many theorists allow a purism for analytical work to constrain the realism and value of their models. But once the simulations provide results, a retrospective analysis both confirms the validity of and provides structure for the simulation work.

Previous work in this area has shown that cooperation is not a stable result: while cooperation may be established and even persist for long periods of time, cooperation is always vulnerable to invasion. This paper does a good job of explaining exactly why this is the case: while vigilant cooperating strategies like tit-for-tat are quite robust to invasion by defectors, they are easily invaded by non-vigilant (i.e. naive) cooperators, who have the potential to displace strategies like tit-for-tat by a neutral process (i.e. drift). Once the vigilant cooperators are gone, the population is now very vulnerable to invasion by exploitative defector strategies. Van Veelen and collaborators call this “indirect invasion”, and point out that these invasions suggest that “unconditional cooperation is therefore cooperation’s worst enemy”. Interestingly, neutral invasion can also lead to the transition from defection to cooperation, although the paper reports that cooperation is more likely to be indirectly invaded than defection. In a final blow to the now-beleaguered concept of the Evolutionary Stable Strategy, van Veelen and collaborators report:

We can show that no strategy is ever robust against indirect invasions; there are always indirect paths out of equilibrium.

Previous studies have shown that the more likely interactions are to be repeated, the easier cooperation is to maintain. Similarly, adding spatial structure (such that cooperators are more likely to find themselves interacting with cooperators and defectors are more likely to find themselves interacting with defectors) increases the persistance of cooperation. By combining the two, this paper allows us to see how these two factors interact and when they produce near-persistant cooperation. Figure 2 is the ‘money figure’, showing both the results of simulations and their analytical breakdown (although someone forgot to label the areas of the analytic space, making it hard to follow the text discussion of the different zones of this parameter space). Perhaps the most interesting result is that while combining reciprocity and population structure does avert the “tragedy of the commons” manifested as a perpetual state of defection, once the threshold combination of reciprocity and structure are achieved, extremes of these two factors can actually make cooperation less likely. For this reason it is probably best to partition the results of this study: 1) population structure at a particular threshold (~0.5) will allow cooperation to persist, and is actually hurt by overly-persistant reciprocal interaction (which prevents evolution to more cooperative states); and 2) population structure enables rather short-term interactions to lead to stable cooperation where reciprocity alone would not have allowed for such persistance.

The way that this study incorporates population structure is interesting. Previous studies have been on grids, which suggest a spatial or geographical structuring. This view of structure works well for organisms that do not travel around too much, but does a poor job of describing the interactions between animals (most especially humans) that create population structure by means other than spatial proximity (for example by partner choice). This study avoids the limitations of spatial structuring by defining population structuring by a simple parameter (assortment, α), which represents the probability of a particular strategy being matched with itself in the next interaction.

Previous work by Nowak has also considered the effects of two types of error on these repeated Prisoner’s Dilemma games: 1) errors in interpreting the previous move of an opponent; and 2) errors in making the strategy-appropriate move in the next round. This study also goes the extra mile to show that while such errors do reduce the overall rate of cooperation slightly, they do not qualitatively change the results found without error.

The model investigated by van Veelen and colleagues is pretty complex. At its heart is the generation of algorithms for different strategies and pathways by which those algorithms can mutate into alternative algorithms. Depicting how these algorithms work is challenging, and the paper makes a valiant attempt at infographically representing these strategies in Figure 1. The infographics work well once you figure them out, but they are so poorly explained in the accompanying text that I would guess most people are going to be left scratching their heads trying to make sense out of Figure 1. This is definitely one of those papers for which you will have to get deeply into the Supporting Information if you wish to fully understand the model; this is not a task I have yet committed to.

There are some interesting political components to this paper, and I am talking about the politics of science rather than application to human political systems. Most prominently, the paper makes it clear that it is completely agnostic about whether the cooperation-promoting properties of population structure are examples of “kin selection, group selection, both, or neither”. In doing so it tries to avoid the controversy stoked by Nowak et al. (2010) over whether kin selection is a useless concept; perhaps the other authors here do not wish to join Nowak in trying to purge kin selection from the field.