Christopher X J. Jensen
Associate Professor, Pratt Institute

PNAS paper explores the role of population structure in facilitating reciprocity

Posted 12 Jun 2012 / 3

The “Early Edition” of the Proceedings of the National Academy of Sciences of the United States of America just posted online a paper entitled “Direct reciprocity in structured populations“. Authored by Matthijs van Veelen, Julián García, David G. Rand, and Martin A. Nowak, the paper combines two well-explored factors that influence how cooperation evolves: repeated interaction and population structure. This is one of those papers that make you say “wait, no one has done this model before?”. Given how well these authors investigate this system, I am thankful that they were the first to pull it off.

This paper has a lot of the hallmarks of a classic Martin Nowak paper: it focuses on the Prisoner’s Dilemma as the ultimate representation of all social dilemmas, produces a really comprehensive data set based on computer simulations, and then rounds off the whole package with separate analytical work that provides mathematical structure to the simulations. I have to say that this is a formula that works, and if I had the analytical chops of these authors this would be the approach that I would take to all of my research as well. You need the computer simulations in order to properly investigate a meaningful system: too many theorists allow a purism for analytical work to constrain the realism and value of their models. But once the simulations provide results, a retrospective analysis both confirms the validity of and provides structure for the simulation work.

Previous work in this area has shown that cooperation is not a stable result: while cooperation may be established and even persist for long periods of time, cooperation is always vulnerable to invasion. This paper does a good job of explaining exactly why this is the case: while vigilant cooperating strategies like tit-for-tat are quite robust to invasion by defectors, they are easily invaded by non-vigilant (i.e. naive) cooperators, who have the potential to displace strategies like tit-for-tat by a neutral process (i.e. drift). Once the vigilant cooperators are gone, the population is now very vulnerable to invasion by exploitative defector strategies. Van Veelen and collaborators call this “indirect invasion”, and point out that these invasions suggest that “unconditional cooperation is therefore cooperation’s worst enemy”. Interestingly, neutral invasion can also lead to the transition from defection to cooperation, although the paper reports that cooperation is more likely to be indirectly invaded than defection. In a final blow to the now-beleaguered concept of the Evolutionary Stable Strategy, van Veelen and collaborators report:

We can show that no strategy is ever robust against indirect invasions; there are always indirect paths out of equilibrium.

Previous studies have shown that the more likely interactions are to be repeated, the easier cooperation is to maintain. Similarly, adding spatial structure (such that cooperators are more likely to find themselves interacting with cooperators and defectors are more likely to find themselves interacting with defectors) increases the persistance of cooperation. By combining the two, this paper allows us to see how these two factors interact and when they produce near-persistant cooperation. Figure 2 is the ‘money figure’, showing both the results of simulations and their analytical breakdown (although someone forgot to label the areas of the analytic space, making it hard to follow the text discussion of the different zones of this parameter space). Perhaps the most interesting result is that while combining reciprocity and population structure does avert the “tragedy of the commons” manifested as a perpetual state of defection, once the threshold combination of reciprocity and structure are achieved, extremes of these two factors can actually make cooperation less likely. For this reason it is probably best to partition the results of this study: 1) population structure at a particular threshold (~0.5) will allow cooperation to persist, and is actually hurt by overly-persistant reciprocal interaction (which prevents evolution to more cooperative states); and 2) population structure enables rather short-term interactions to lead to stable cooperation where reciprocity alone would not have allowed for such persistance.

The way that this study incorporates population structure is interesting. Previous studies have been on grids, which suggest a spatial or geographical structuring. This view of structure works well for organisms that do not travel around too much, but does a poor job of describing the interactions between animals (most especially humans) that create population structure by means other than spatial proximity (for example by partner choice). This study avoids the limitations of spatial structuring by defining population structuring by a simple parameter (assortment, α), which represents the probability of a particular strategy being matched with itself in the next interaction.

Previous work by Nowak has also considered the effects of two types of error on these repeated Prisoner’s Dilemma games: 1) errors in interpreting the previous move of an opponent; and 2) errors in making the strategy-appropriate move in the next round. This study also goes the extra mile to show that while such errors do reduce the overall rate of cooperation slightly, they do not qualitatively change the results found without error.

The model investigated by van Veelen and colleagues is pretty complex. At its heart is the generation of algorithms for different strategies and pathways by which those algorithms can mutate into alternative algorithms. Depicting how these algorithms work is challenging, and the paper makes a valiant attempt at infographically representing these strategies in Figure 1. The infographics work well once you figure them out, but they are so poorly explained in the accompanying text that I would guess most people are going to be left scratching their heads trying to make sense out of Figure 1. This is definitely one of those papers for which you will have to get deeply into the Supporting Information if you wish to fully understand the model; this is not a task I have yet committed to.

There are some interesting political components to this paper, and I am talking about the politics of science rather than application to human political systems. Most prominently, the paper makes it clear that it is completely agnostic about whether the cooperation-promoting properties of population structure are examples of “kin selection, group selection, both, or neither”. In doing so it tries to avoid the controversy stoked by Nowak et al. (2010) over whether kin selection is a useless concept; perhaps the other authors here do not wish to join Nowak in trying to purge kin selection from the field.

Articles, Behavior, Cooperation, Evolutionary Modeling, Information Design, Reciprocity, Social Networks

3 Comments to "PNAS paper explores the role of population structure in facilitating reciprocity"

Yamir 9th July 2012 at 10:04 am

Nice post, but what about looking at how people behave when playing a dilemma? Check out

Chris Jensen 9th July 2012 at 5:50 pm

Thanks for sharing this Yamir. I totally agree that experimental work is needed to constraint model structure and parameterization: if there is one thing that the recent explosion in theoretical work exploring the evolution of cooperation has shown us, it is that the infinite varieties of model structures can produce nearly infinite outcomes. The theoretical world, whose explorations are indispensable, needs to be more influenced by empirical and experimental findings.

Looking forward to checking out your paper!

Julián García 20th July 2012 at 5:10 am

Yamir. I think these evolutionary models are not (should not?) aim at predicting what people do in the lab. For this problem, economists have a number of models that (to a certain extent work). In these models typically there is no population, and other psychological or decision making factors play a role; i.e., there is no mutation rate, because people are not mutating in the lab. I know they may be exploring, but they are also probably learning individually as well as socially and using the preferences they bring to the lab: it is much more complicated than just copying what others are doing. In my opinion, these evolutionary models aim at providing an ultimate explanation, i.e., how did we evolve (way in the past) in such a way that we acquired pro-social preferences. In this ultimate sense population structure may be an answer, but in the proximate sense of capturing what goes on in the lab other models like inequity aversion might turn out to be more appropriate. In my summary, we should not expect ultimate models to provide proximate answers. Your paper is important and timely because it shows just that, but it does not invalidate what ultimate models bring to the table.

Leave a Reply