Revisiting Tit-For-Tat

Tit-For-Tat as a strategy was hailed in the Iterated Prisoner's Dilemma (IPD) tournament of Axelrod (see Wikipedia background) as showing that altruism can be a product of evolution (Games People Play and How Nice Guys Finish First):

"Axelrod discovered that when these encounters were repeated over a long period of time with many players, each with different strategies, greedy strategies tended to do very poorly in the long run while more altruistic strategies did better, as judged purely by self-interest. He used this to show a possible mechanism for the evolution of altruistic behavior from mechanisms that are initially purely selfish, by natural selection."

I return to this topic for two reasons.

The first is when I found the software SciLab, the closest thing to MatLab and free as in free beer. SciLab is not a clone of MatLab, but translation from MatLab is relatively easy (there is function mfile2sci in SciLab to do it). I used MatLab before, and I was glad to use open source SciLab.

So the first thing I did was translate the MatLab codes which I got from Iterated prisoner’s dilemma in MATLAB into SciLab and run the tournament.

SciLab is now, together with paper & pen and Microsoft Excel, my favourite tools for doing Mathematics. Sage and Mathematica, although very powerful, are not as handy.

The second reason is the problem of reconciling Tit-For-Tat with Buddhist views.
Tit-For-Tat is said to be nice, forgiving, non-envious, but it also retaliates. It is the last attribute retaliation which is questionable from the Buddhist standpoint.
Tit-For-Tat will never defect first, but it will punish opponent's defection.
The idea of punishment is not acceptable in Buddhism.

Closer examination makes it clear that in the IPD setup, no communication between the players is possible. Hence we can say that IPD is not a realistic model of human interaction.
If communication were possible, it would be used first to reason with the opponent, but then it would not be IPD.
Nevertheless, although IPD is not a realistic model, it does at least show the possibility of the evolution of altruism.

A similar conclusion is reached when reading the New Scientist article Religion is a product of evolution, software suggests.
While the conclusion that religion is an emergent mental artifact of our evolution is quite plausible, the model is simply too crude and unrealistic. Incidentally the software is written in SciLab: SciLab program to simulate the evolution of religion

Link: Blog entries on game theory

Dalai Lama on Secular Ethics

In a recent interview "Ocean of wit and wisdoms", the Dalai Lama re-iterated his position about secular ethics, which are human values and positive emotions such as loving kindness and compassion, which "have got nothing to do with religious faith. I usually call these 'secular ethics.' Irrespective of whether you accept religion or not, this is according to our common sense, our common experience, and also scientific findings."

In an earlier interview, he said much the same thing:
"I call these secular ethics, secular beliefs. There’s no relationship with any particular religion. Even without religion, even as nonbelievers, we have the capacity to promote these things.......
No, these are not necessarily Buddhist teachings. These are old teachings based on human values. The way of presentation is different according to each religion."

Secular ethics are universal and independent from religions. They are probably related to the biological and social conditions of humans.

A New York Times article Is ‘Do Unto Others’ Written Into Our Genes?
discusses Jonathan Haidt book “The Happiness Hypothesis,” which is about the evolutionary view of morality.

Similarly, research on game-theoretic emergence of fairness and altruism, see e.g. "Games People Play and How Nice Guys Finish First" seems to support a non-religious foundation of morality.

Finally, research in neuroscience, see e.g. "Mirror Neurons Open New Vistas"
provides a biological basis for compassion and loving kindness through mirror neurons which are sometimes called Dalai Lama neurons.

I don't know if this is why the Dalai Lama is so interested in neuroscience, and in science in general. What is certain is that the scientific evidence of an evolutionary view of morality is growing

Other interesting topics in the interview is his statement, that the Dalai Lama institution need not necessarily be preserved, it could be replaced by a democratic process of election.

At the end, he expressed his views on theistic-religions:
"I'm Buddhist, I'm a Buddhist practitioner. So actually I think that according to nontheistic Buddhist belief, things are due to causes and conditions. No creator. So I have faith in our actions, not prayer. Action is important. Action is karma. Karma means action. That's an ancient Indian thought. In nontheistic religions, including Buddhism, the emphasis is on our actions rather than god or Buddha. So some people say that Buddhism is a kind of atheism. Some scholars say that Buddhism is not a religion — it's a science of the mind."

Most Terrifying Video on Global Warming

This is a video by a high school science teacher from Oregon, Greg Craven, called The Most Terrifying Video You'll Ever See.
The video received immense attention, and spread widely.
In the video, Greg Craven illustrated his argument using a game-theoretic setup. It is a game against nature, and our choice is whether to take precautionary actions against global warming, or do nothing, believing that nothing big will happen.
The conclusion, is that taking action is definitely a better choice.
Greg Craven challenged people to find flaws in his arguments, if any.

Games People Play and How Nice Guys Finish First

Game theory has become an important tool in socio-economic experimental studies ranging from explaining how cooperation arises to stock market simulations (see The Minority Game (MG) ).

Three of the most intensively studied games are:

• The Prisoner's Dilemma (PD) and the Iterated Prisoner's Dilemma (IPD), see Wikipedia Entry on Prisoner's Dilemma

• The Traveler's Dilemma (TD)

• The Ultimatum Game (UG)

The above games have forced us to reflect on human nature itself regarding our rationality, and selfishness. Are we rational, does rational mean optimizing self-interest, and is Nash Equilibrium the necessary outcome of rationality?

The results of the above games are often counter-intuitive, and can be seen as an attack on the concept of Homo economicus—a rational individual relentlessly bent on maximizing a purely selfish reward.

In all cases, assuming the players are rational but selfish, the solution given by the Nash Equilibrium is not the action real people choose in experiments.

In the PD selfishness wins, but in IPD, "Axelrod discovered that when these encounters were repeated over a long period of time with many players, each with different strategies, greedy strategies tended to do very poorly in the long run while more altruistic strategies did better, as judged purely by self-interest. He used this to show a possible mechanism for the evolution of altruistic behavior from mechanisms that are initially purely selfish, by natural selection."

TD was introduced by Kaushik Basu:

An airline loses the suitcases of two travelers. Both suitcases happen to be identical and contain identical pieces of antique. An airline manager tasked to settle the claims of both travelers explains that the airline is liable for a maximum of $100 per suitcase, and in order to avoid inflated claims he separates both travelers and asks them to write down a number no less than 2 and no larger than 100. He also tells them that if both write down the same number, he will treat this number as the true dollar value of both suitcases and reimburse both travelers that amount in dollars. However, if one writes down a smaller number than the other, this smaller number will be taken as the true dollar value, and both travelers will receive that amount plus a bonus/malus: a $2 extra amount for the traveler who wrote down the lower value and a $2 deduction for the person who wrote down the higher amount. The question is: what strategy should both travelers follow to decide which number to write down?

PD can be seen as a special case of TD with only 2 values.

The surprising result of analysis, is that the unique Nash Equilibrium is for both travelers to write down $2. If we only consider the choices $99 and $ 100, $99 is clearly better than $100, because if the other person chooses $100, we get $99+2=$101. The same argument can be applied to $98 and $99, and continuing by backwards induction, leads to the result that $2 is best. Of course this assumes that both players are selfish.

Bashu wrote about the economic implication: " The game and our intuitive prediction of its outcome also contradict economists' ideas. Early economics was firmly tethered to the libertarian presumption that individuals should be left to their own devices because their selfish choices will result in the economy running efficiently. The rise of game-theoretic methods has already done much to cut economics free from this assumption. Yet those methods have long been based on the axiom that people will make selfish rational choices that game theory can predict. TD undermines both the libertarian idea that unrestrained selfishness is good for the economy and the game-theoretic tenet that people will be selfish and rational."

In practice, experiments find that most people would choose a high number near to $100.

Does it mean that the rationality assumption is not valid?

Which leads Bashu to say: "Perhaps altruism is hardwired into our psyches alongside selfishness, and our behavior results from a tussle between the two."

Concluding the article, Bashu said: "If I were to play this game, I would say to myself: "Forget game-theoretic logic. I will play a large number (perhaps 95), and I know my opponent will play something similar and both of us will ignore the rational argument that the next smaller number would be better than whatever number we choose. What is interesting is that this rejection of formal rationality and logic has a kind of meta-rationality attached to it. If both players follow this meta-rational course, both will do well. The idea of behavior generated by rationally rejecting rational behavior is a hard one to formalize. But in it lies the step that will have to be taken in the future to solve the paradoxes of rationality that plague game theory and are codified in Traveler's Dilemma."

The key point in this statement is "I know my opponent will play something similar..."

More recent game theoretic analysis followed this line of reasoning by including knowledge or beliefs about our own rationality or irrationality, and that of the other person rationality or irrationality, into consideration. This field of study is known as interactive epistemics.

The third game, the Ultimatum Game (UG) deals with the concept of fairness.

The games is formulated as follows :

Imagine that somebody offers you $100. All you have to do is agree with some other anonymous person on how to share the sum. The rules are strict.
The two of you are in separate rooms and cannot exchange information. A coin toss decides which of you will propose how to share the money. Suppose that you are the proposer.
You can make a single offer of how to split the sum, and the other person—the responder—can say yes or no. The responder also knows the rules and the total amount of money at stake. If her answer is yes, the deal goes ahead. If her answer is no, neither of you gets anything. In both cases, the game is over and will not be repeated. What will you do?
Instinctively, many people feel they should offer 50 percent, because such a division is
“fair” and therefore likely to be accepted. More daring people, however, think they might
get away with offering somewhat less than half of the sum.

Experiments with people from different countries suggest that there is a concept of fairness, and some people would rather take home nothing than accepting a small amount, which they deem as not fair.

The authors said about the experiments: "Yet despite these cultural variations, the outcome was always far from what rational analysis would dictate for selfish players. In striking contrast to what selfish income maximizers ought to do, most people all over the world place a high value on fair outcomes."

It gets even more interesting when the game is iterated, and the evolution of strategies is studied (Evolutionary Game Theory )

Like in IPD, we can observe the evolution of altruism, cooperation, generosity, punishment of selfish people, sense of fairness, etc.

Nice guys finish first!

References:

• Kaushik Basu. The Traveler's Dilemma
• "Epistemic Conditions for Nash Equilibrium," by Robert Aumann and Adam Brandenburger, Econometrica, Vol. 63, pages 1161-1180 (1995). [Also available: Unpublished 1991 version (pdf).] http://pages.stern.nyu.edu/%7Eabranden/ecne-10-03-06.pdf

• "The Power of Paradox: Some Recent Developments in Interactive Epistemology," (pdf) by Adam Brandenburger, International Journal of Game Theory, Vol. 35, pages 465-492 (2007). http://www.springerlink.com/content/l3216230471x1874/fulltext.pdf

• Nowak MA, KM Page, K Sigmund (2000). Fairness versus reason in the ultimatum game. Science 289: 1773-1775 (PDF) http://www.fas.harvard.edu/%7Eped/people/faculty/publications_nowak/Science00.pdf

• Sigmund K, E Fehr, MA Nowak (2002). The economics of fair play. Sci Am 286: 82-87 (PDF) http://www.fas.harvard.edu/%7Eped/people/faculty/publications_nowak/SciAm02.pdf

• Christoph Uhlhaas . Is Greed Good? http://www.sciam.com/print_version.cfm?articleID=1D9DFCAC-E7F2-99DF-3C2DE7F657B943E5

• Stanford Encyclopedia of Philosophy. Evolutionary Game Theory http://plato.stanford.edu/entries/game-evolutionary/

• Econophysics And The Theory Of Games

• The True Test of The New Economies

The Myth of Homo economicus

The notion of Homo economicus (economic man) as a rational, selfish person who single-mindedly strives for maximum profit is often assumed in economic theories.

Yet many theories and experiments have shown that we are only bounded rational, and often have a sense of fairness instead of selfishness.

A 2002 Scientific American article titled "The Economics of Fair Play" , Sigmund. Fehr and Nowak showed that fair play can be the result of a Darwinian evolution.

In 2007, Scientifc American carries an article "Is Greed Good?":

"Economist Axel Ockenfels of the University of Cologne in Germany and his colleagues have spent the past several years figuring out ..... It turns out that humans do not always behave as if their sole concern is their personal financial advantage—and even when they do, they consider social motives in the profit-making equation. As Ockenfels has discovered, a sense of fairness often plays a big role in people’s decisions about what to do with their money and possessions, and it is also an essential part of what drives trust in markets full of strangers such as eBay.

Ockenfels’s Equity, Reciprocity and Competition (ERC) theory, which he developed with economist Gary Bolton of Pennsylvania State University, states that people not only try to maximize their gains but also watch to see that they get roughly the same share as others: they are happy to get one piece of cake as long as the next person does not get two pieces. This fairness gauge apparently even has a defined place in the brain. On eBay, however, fairness takes the system only halfway, researchers have now learned; eBay’s reputation system is critical for augmenting the level of trust enough for the market to work."

Research in this field utilizes Game Theory combined with evolution, see for example Carl Zimmer's "In Games, an Insight Into the Rules of Evolution" where he discussed Martin Nowak's work.

Such research has now gone beyond economics, and applied to social studies and to biology (cooperation as the cause of cancer).
Some popular articles about the social aspects appear in 2004 in the Neue Zuercher Zeitung "Warum einander helfen?" and Welt Online "Selbstlose Helfer setzen sich durch"

Is Go The Most Complex Game?

After working 18 years, Jonathan Schaeffer, who had earlier wrote a computer program to play Checkers which matched the world's best players, has now completed the proof that with best play the game is a draw.
The result is not surprising, but the proof is hailed as a milestone, since it is the most complex game ever solved by a machine. It was reported in the journal Scientific American

We know computers play Othello, Chess, Go, and other games. Playing Othello and Chess, computers have reached world class levels (see e.g.Man vs Machine: Deep Fritz beat Kramnik), sometimes better than world champions, but in Go computers have still a long way to go.

This is an indication that Go (or Weiqi in chinese) is indeed much more complex than Chess. Developing computer Go playing programs would probably need new insights and technologies, just as computer Chess has done before.
By contrast, computer Othello and Checkers do not seem to have generated many innovations.
I would venture that playing Go needs feeling of patterns and intuition. It would be interesting to see if computer programs can play Go at the level of champions, by relying on brute force only.

Is there a game more complex than Go? Artificially, one could invent games of very high complexity, but in asking the question, we should restrict ourselves to games which are widely played by people.
Poker is another genre, players having incomplete information, but it does not seem to have a great deal of complexity.

About the game Go

Examples of Prospect Theory Applications

To answer one of the comments made to the post Bounded Rationality and Prospect Theory, here are the mathematical formulas for the function ¶() and u(), adapted from the MIT Course on Behavioral Economics and Finance.

Of course the formulas are estimates only, and they contain parameters beta and lambda, which can be varied. Since Prospect Theory is descriptive rather than normative, it is an experimental task to find approximate formulas for both functions.

The function ¶(p) has one parameter beta:

¶(p) = (p^beta) / (p^beta + (1-p)^beta), with 0 ≤ beta ≤ 1

For example with beta =0.8, the graph looks like this:



The function u(x) has two parameters beta and lambda:

u(x) = x^beta for x >0 and u(x) = -lambda * |x|^beta for x ≤ 0

Graph of u()



To apply the formulas to concrete examples consider a situation in the 'Deal No-Deal game' where there are only 2 unopened cases with values $10,000 and $30,000 and the bank offers $18,000.

Using the above formulas,
PV(Deal) = 2536 and PV(No-Deal) = 2700, slightly favoring No-Deal.

A second example, is the problem of the unusual disease mentioned at the end of the post Bounded Rationality and Prospect Theory

The results:
PV(A) = 120.68
PV(B) = 125.51
PV(C) = -138.62
PV(D) = -82.81

is consistent, B preferred over A, and D over C

Bounded Rationality and Prospect Theory

According to Doyne Farmer, there are logical and economic rationalisms. The first has to do with truth, somebody is not logically rational if his or her statements or beliefs do not conform to logic, for instance being inconsistent. The second has to do with goodness of a decision we choose among alternatives.
Just as the first is fundamental in analyzing truth, the second is fundamental in decision making. In fact, Farmer discussed it in the context of artificial intelligence, how programs or robots are to make decisions.

In the following I am dealing with economic rationalism.
Economic rationalism is usually assumed in many economic theories. Economic agents are assumed to have perfect knowledge including knowledge of all past histories, and choose the best decision based on the knowledge. This kind of rationalism is actually hyper-rationalism. The efficient market hypothesis depends on it. Markets are efficient because the agents have hyper-rationality, and all information of past histories is therefore contained in the current prices. Markets can be inefficient for a while, then the arbitrage process will take them to become efficient again. Which is why, traditional economics is said to deal mostly with equilibriums.

People have been looking for other forms of rationality. In the early days of artificial intelligence, Herbert Simon introduced the concept of bounded rationality.

The boundedness is the result of three factors,

• because the utility is not clearly defined,

• because of limited information and resources to compute them,

• and because there may be multiple, conflicting goals.

In the context of constrained optimization, both the goals and the constraints are not clear, and the cost of computing, even with the help of computers, is not practical.
If optimization is not possible or practical, then it is replaced by sub-optimization and satisficing. Instead of optimal algorithms, we use heuristic rules to arrive at good solutions, though they may not be optimal. Analogical reasoning in the form of imitation of others' actions is a frequent form of how we make decisions. (This is why we have memes with imperfect copying mechanism).

When discussing the minority game (Econophysics and the Theory of Games), we touched on the rationality of the agents. It was assumed there that agents have bounded rationality, having a finite memory of past winning alternatives, and making decisions based on that memory. The Theory of Games itself can handle both assumptions of rationality or bounded rationality.

Sometimes the multiple goals are in simple relations, they can be combined using weights or arranged in a hierarchy of priorities, which reduces the problem to a single goal problem. General cases however cannot be reduced to a single goal.

In the following I will look at the first factor, that of utility.
Classical theory is based on expected utility, calculated as:

If i=1....N are the alternatives, and o(i) the outcome of alternative i, u(o(i)) the utility, and p(i) the probability of alternative I, then the expected utility value (EV) is:

EV = ∑ u(o(i)).p(i), where the summation is taken over all i.

For example, some banks tries to assess a customer's risk aversion profile by asking questions like:

"Would you prefer:

• (a) a certain income of $50K or
  
• (b) a 30% chance of $0K and 70$ chance of $100K?"

Here EV(a)= 50K, EV(b)=70K, assuming that u(i) is linear. According the expected theory, (b) is preferred. If however (a) is preferred, or there is no preference between the two, then we can try to deduce (from a set of such questions) the properties of the function u(i) for the particular customer. However, people are often not sure themselves which they prefer, and it leads to contradictions, and no function u(i) can be constructed.
So, the general conclusion is: either people are contradictory, or they are not, but we need a different utility theory.

It took two psychologists Kahneman and Tversky (KAHNEMAN, Daniel and Amos TVERSKY, 1979. Prospect Theory: An Analysis of Decision under Risk. Econometrica.) to suggest a different utility theory called the prospect theory. Kahneman got a Nobel prize in economics in 2002, but Tversky died before that.
The first difference of the two theories, is that here we take the relative gains or losses, not the outcome itself. Thus if W is the wealth, then the relative gain/loss is

g(i) = o(i) - W

The utility of alternative i is v(g(i)), where the same symbol v is the utility function here, analogue to u. In fact this function v is different from the function u used in the expected theory. u is defined for nonnegative values only, but v is for both losses (negative) and positive values.
The following graphs are the originals form the Kahneman Tversky paper, see Prospect Theory


We see from the graphic that v is not symmetric.

Kahneman and Tversky discovered the four-fold pattern of risk attitudes (the graph of v is convave for gains and convex for losses):

• Risk aversion in the domain of likely gains

• Risk seeking in the domain of unlikely gains

• Risk seeking in the domain of likely losses

• Risk aversion in the domain of unlikely losses

This might explain why some buy lotteries, but also buy insurance policies. They are speculative and conservative at the same time.

The third difference is the observance that we under and over estimate probabilities, i.e. overestimate small probabilities, and underestimate high probabilities. See the enclosed graph. Let us denote ¶(p(i)) as the estimate of probability p(i). If we take ¶() a linear function, then we are back in the classical case.
Note that the sum of all ¶(p(i)) is not 1, hence it is not a probability distribution.


Prospect theory uses the following equation to compute the prospect value PV:

PV = ∑ u(g(i)).¶(p(i))

One lighthearted example of an application of prospect theory is the TV game "Deal or No Deal", originally shown on Dutch TV, and now widely shown with variations in many countries.
To see the game online, visit JoyTube
The bank always offers amount below the expected value of all the unopened cases. Research by Thaler et.al. "Deal or No Deal? Decision making under risk in a large-payoff game show" indicates that the decision to continue the game or not is path dependent (how they arrive there) and that prospect theory, rather than expected theory, can better explain the preferences involved.

Among other things, Kahneman Tversky also found "framing effects" where decisions depend on how the question is posed.
The classic example is (Kahneman Tversky, The Framing of Decisions and the Psychology of Choice", Science 1981):

In an outbreak of an unusual disease, experts estimate that 600 people will die.

Choose A or B:
A: use tested drug, 400 people will be saved
B: use a new experimental drug, 80% chance all 600 will be saved, 20% chance all
will die.

Now choose C or D:
C: 200 people will die
D: 20% chance that 600 people will die, and 80% chance no one will die.

Note that A and C, and B and D are the same, but because they are posed differently, many make inconsistent choices.

Econophysics And The Theory Of Games

What has Econophysics to do with the Theory of Games?
Econophysics is the study of Economics by using analogies and methodologies from Physics, particularly that of Statistical Mechanics. The latter uses probability and statistics to derive aggregate properties (Thermodynamics) from a population of interacting particles/molecules. The macro properties of the population is then related to the microscopic properties of the particles.

-------------------------------------------------------------------------------------------------------------
Economics Nobel laureate Markowitz: "I believe that microscopic market simulations have an important role to play in economics and finance. If it takes people from outside economics and finance -- perhaps physicists -- to demonstrate this role, it won't be for the first time that outsiders have made substational contributions to these fields.''
--------------------------------------------------------------------