Participants were told to prepare for an outbreak expected to kill 600 people. Offered program A, which saves 200 people for certain, or program B, which has a one-in-three chance of saving all 600 and a two-in-three chance of saving no one, 72% chose A (N = 152).
A second group was offered the same two programs described by deaths: under C, 400 people die for certain; under D, there is a one-in-three chance nobody dies and a two-in-three chance all 600 die. 78% chose D (N = 155). The programs are identical. Only the description changed, from lives saved to lives lost, and the majority went from avoiding risk to seeking it.
Framed as lives saved (N = 152)
A: 200 saved for sure72%
B: ⅓ chance all 600 saved28%
Framed as deaths (N = 155)
C: 400 die for sure22%
D: ⅓ chance nobody dies78%
Figure 1. Share of participants choosing each program in the Asian disease problem. Source: Amos Tversky, Daniel Kahneman (1981), Science 211, 453–458, doi:10.1126/science.7455683.
Psychology · 1983
The conjunction fallacy
Linda is 31, single, outspoken and very bright. She majored in philosophy, and as a student she was deeply concerned with discrimination and social justice. Which is more probable: that Linda is a bank teller, or that Linda is a bank teller and active in the feminist movement?
85% of 142 undergraduates chose the second. A conjunction can never be more probable than either of its parts; the description resembles a feminist more than a bank teller, and resemblance stood in for probability.
Bank teller and feminist85%
Bank teller15%
Figure 2. Which is more probable? Share of 142 undergraduates choosing each answer. Source: Amos Tversky, Daniel Kahneman (1983), Psychological Review 90(4), 293–315, doi:10.1037/0033-295X.90.4.293.
Psychology · 1979
Prospect theory: the value function
People value outcomes as gains and losses from a reference point, not as final states of wealth. The value function is concave for gains, convex for losses, and steeper for losses than for gains.
Tversky and Kahneman's 1992 estimates put the curvature at 0.88 on both sides and the loss-aversion coefficient at 2.25: losing $100 is felt 2.25 times as strongly as gaining $100.
Value
v(x)
Figure 3. v(x) = x^0.88 for gains, −2.25(−x)^0.88 for losses, with the parameters from Tversky & Kahneman (1992), Journal of Risk and Uncertainty 5, 297–323. Source: Daniel Kahneman, Amos Tversky (1979), Econometrica 47(2), 263–291, doi:10.2307/1914185.
Psychology · 1995
Bayesian reasoning without instruction
One per cent of women at a routine screening have breast cancer. If a woman has it, the mammogram is positive 80% of the time; if she does not, it is positive 9.6% of the time. A woman tests positive. How likely is it that she has cancer? When David Eddy put the question to physicians in 1982, 95 of 100 answered about 75%.
The answer is 7.8%. Gigerenzer and Hoffrage restated the same numbers as natural frequencies: of 1,000 women, 10 have cancer and 8 of them test positive; of the 990 without cancer, 95 also test positive. The share of people reasoning to the correct answer rose from 16% to 46%.
8 have cancer and test positive
2 have cancer and test negative
95 test positive without cancer
895 test negative without cancer
Figure 4. 1,000 women, one dot each. Of the 103 who test positive, 8 have cancer. Source: Gerd Gigerenzer, Ulrich Hoffrage (1995), Psychological Review 102(4), 684–704, doi:10.1037/0033-295X.102.4.684.
Psychology · 1974
Anchoring and adjustment
A wheel of fortune, rigged to stop at either 10 or 65, was spun in front of each participant, who was then asked whether the percentage of African countries in the United Nations was higher or lower than that number, and then for their own estimate.
The median estimate was 25% from those who had seen 10, and 45% from those who had seen 65. A number everyone knew to be random moved the answer by twenty points.
Wheel stopped at 1025%
Wheel stopped at 6545%
Figure 5. Median estimate of the percentage of African countries in the United Nations. Source: Amos Tversky, Daniel Kahneman (1974), Science 185, 1124–1131, doi:10.1126/science.185.4157.1124.
Psychology · 1992
Cumulative prospect theory: probability weighting
People do not weigh outcomes by their probabilities. Small probabilities are overweighted and moderate and large ones underweighted, which is how the same person can buy both insurance and lottery tickets.
Tversky and Kahneman fitted w(p) = p^γ / (p^γ + (1 − p)^γ)^(1/γ), with γ = 0.61 for gains. A 1% chance is weighted as 5.5% and a 99% chance as 91.2%; the curve crosses the diagonal at p ≈ 0.34.
Decision weight
w(p)
Figure 6. The weighting function for gains, γ = 0.61. Source: Amos Tversky, Daniel Kahneman (1992), Journal of Risk and Uncertainty 5, 297–323, doi:10.1007/BF00122574.
Psychology · 2018
The hot hand, and the bias in finding it
Gilovich, Vallone and Tversky (1985) found that basketball players were no more likely to make a shot after a make than after a miss, and concluded that the 'hot hand' is a cognitive illusion. It became one of the best-known results in the field.
Miller and Sanjurjo showed that the method was biased. In a finite sequence of fair coin flips, the expected share of heads among flips that immediately follow a head is below 50%: 40.5% for four flips. Corrected for the bias, the original shooting data show a hot hand.
Expected share of heads after a head
Expected share
Figure 7. Computed exactly over every sequence of n fair flips that contains a flip after a head. Source: Joshua B. Miller, Adam Sanjurjo (2018), Econometrica 86(6), 2019–2047, doi:10.3982/ECTA14943.
Psychology · 1968
The selection task
Four cards each have a letter on one side and a number on the other. The rule: if a card has a vowel on one side, it has an even number on the other. Which cards must you turn over to test it?
Most people choose E, or E and 4. The answer is E and 7: only an odd number behind the E, or a vowel behind the 7, could break the rule, and nothing behind the 4 can. In the abstract version of the task, typically fewer than one person in ten gets it right. People look for cases that confirm a rule rather than cases that could refute it.
EK47
Figure 8. The cards to turn are outlined. Source: Peter Wason (1968), Reasoning about a rule, Quarterly Journal of Experimental Psychology 20(3), 273–281, doi:10.1080/14640746808400161.
Psychology · 1990
The endowment effect
Half of a class were given Cornell coffee mugs; the other half were not. Owners were asked the lowest price at which they would sell, and the others the highest at which they would buy. With the mugs handed out at random, standard theory predicts that about half should change hands.
The median owner asked $7.12; the median buyer offered $2.87. Far fewer mugs traded than predicted. Owning the mug for a few minutes was enough to more than double its price.
Median selling price (owners)$7.12
Median buying price$2.87
Figure 9. The same mug, priced by those given one and those not. Source: Daniel Kahneman, Jack Knetsch, Richard Thaler (1990), Journal of Political Economy 98(6), 1325–1348, doi:10.1086/261737.
Psychology · 2000
When choice is demotivating
A tasting booth at a California supermarket alternated between showing 6 jams and 24. The larger display drew more shoppers to stop, 60% against 40%, but 30% of those who stopped at the small display went on to buy a jar, against 3% at the large one.
The study became the standard example of choice overload. A 2010 meta-analysis by Scheibehenne, Greifeneder and Todd, of 50 experiments and 5,036 participants, found a mean effect of virtually zero.
Stopped at the booth
6 jams40%
24 jams60%
Of those who stopped, bought a jar
6 jams30%
24 jams3%
Figure 10. Iyengar & Lepper (2000), study 1. Meta-analysis: Journal of Consumer Research 37(3), 409–425. Source: Sheena Iyengar, Mark Lepper (2000), Journal of Personality and Social Psychology 79(6), 995–1006, doi:10.1037/0022-3514.79.6.995.
Psychology · 1999
Unskilled and unaware of it
Across four studies of humour, grammar and logical reasoning, participants who scored in the bottom quartile grossly overestimated how well they had done. Their scores put them in the 12th percentile; they estimated themselves in the 62nd.
Kruger and Dunning argued that the skills needed to do well are the same ones needed to recognise doing well, so the least competent are the least able to see it.
Bottom quartile
Estimated percentile62nd
Actual percentile12th
Figure 11. Averaged across the four studies, as reported in the abstract. Source: Justin Kruger, David Dunning (1999), Journal of Personality and Social Psychology 77(6), 1121–1134, doi:10.1037/0022-3514.77.6.1121.
Psychology · 1972
Subjective probability: a judgment of representativeness
A town has two hospitals. About 45 babies are born each day in the larger and about 15 in the smaller. For a year, each recorded the days on which more than 60% of the babies born were boys. Which recorded more such days?
Of 95 undergraduates, 21 said the larger hospital, 21 the smaller, and 53 about the same. The answer is the smaller, by a wide margin: small samples stray further from 50%. With a fair chance of a boy, more than 60% boys happens on 15.1% of days in the small hospital and 6.8% in the large one.
Answers (N = 95)
About the same53
The larger hospital21
The smaller hospital21
Days with more than 60% boys (binomial)
Small, 15 births15.1%
Large, 45 births6.8%
Figure 12. Answers from the paper; probabilities computed exactly. Source: Daniel Kahneman, Amos Tversky (1972), Cognitive Psychology 3(3), 430–454, doi:10.1016/0010-0285(72)90016-3.
Philosophy · 1969
Newcomb's problem
A predictor with a near-perfect record has already filled two boxes. The clear one holds $1,000. The opaque one holds $1,000,000 if the predictor foresaw that you would take the opaque box alone, and nothing if it foresaw you taking both.
Dominance reasoning says take both: whatever is in the opaque box is already there, and both boxes always hold $1,000 more than one. Expected-utility reasoning, conditioned on the prediction, says take one, since one-boxers almost always walk away rich. Nozick, who credited the problem to the physicist William Newcomb, found that people split roughly evenly and that each side thinks the answer is obvious.
You take
Predicted: one box
Predicted: both
One box
$1,000,000
$0
Both boxes
$1,001,000
$1,000
Figure 13. The payoffs. Across each column, taking both is $1,000 better; along the diagonal the predictor is right. Source: Robert Nozick (1969), Essays in Honor of Carl G. Hempel, 114–146.
Philosophy · 1926
The Dutch book argument
If your degrees of belief are your fair prices for bets, they had better obey the probability axioms. If they do not, a bookie can sell you a set of bets, each of which you consider fair, that loses money however the world turns out. Ramsey made the argument in 1926 and Bruno de Finetti independently in 1937.
Suppose you are 60% confident it will rain and 60% confident it will not. You will pay $0.60 for a ticket paying $1 if it rains and $0.60 for one paying $1 if it does not: $1.20 for a guaranteed $1.
If it
Paid
Rain ticket
No-rain ticket
Net
Rains
$1.20
$1.00
$0.00
−$0.20
Does not rain
$1.20
$0.00
$1.00
−$0.20
Figure 14. Credences of 0.6 and 0.6 in an event and its negation sum to 1.2. The loss is the excess. Source: Frank Ramsey (1926), 'Truth and Probability', in The Foundations of Mathematics (1931).
Philosophy · 2000
Sleeping Beauty
Beauty is put to sleep on Sunday and a fair coin is tossed. If heads, she is woken on Monday only. If tails, she is woken on Monday and again on Tuesday, with her memory of Monday erased. Each time she wakes, how confident should she be that the coin landed heads?
Elga argued for a third: of the three awakenings she could be in, only one follows heads. David Lewis replied, in Analysis the next year, that it is a half: she learns nothing on waking that she did not know on Sunday. The disagreement is about how to update on where you are, not on what the world is like, and it has not been settled.
Coin
Monday
Tuesday
Heads (½)
woken
asleep
Tails (½)
woken
woken
Figure 15. Three possible awakenings, one after heads. Thirders count awakenings; halfers count coin tosses. Source: Adam Elga (2000), Self-locating belief and the Sleeping Beauty problem, Analysis 60(2), 143–147, doi:10.1093/analys/60.2.143.
Philosophy · 1738
The St Petersburg paradox
A coin is tossed until it lands heads. If that happens on toss k, you win 2^k ducats. Each possible toss contributes one ducat to the expectation, so the expected value of the game is infinite, and yet almost no one would pay more than a few ducats to play.
Bernoulli proposed that the utility of money grows with its logarithm. To a player with log utility and no other wealth, the game is worth exactly 4. Real banks are finite: if the bank can pay at most 2^k, the expected value is only k + 1.
Value of the game (ducats)
Expected value
Certainty equivalent, log utility
Figure 16. The capped game, computed exactly. At k = 20 the bank can pay about a million ducats and the game is worth 21 in expectation, or under 4 to a log-utility player. Source: Daniel Bernoulli (1738), Exposition of a new theory on the measurement of risk, trans. in Econometrica 22(1), 23–36 (1954), doi:10.2307/1909829.
Philosophy · 1953
The Allais paradox
Choice one: $1 million for certain (1A), or a lottery with a 10% chance of $5 million, 89% of $1 million and 1% of nothing (1B). Choice two: an 11% chance of $1 million (2A), or a 10% chance of $5 million (2B). Most people choose 1A and 2B.
Laid out as a draw from 100 numbered tickets, each pair of lotteries differs only on tickets 1–11; tickets 12–100 pay the same within each pair ($1 million in the first, nothing in the second). By Savage's sure-thing principle, a common consequence should not matter, so anyone who prefers 1A should prefer 2A.
Ticket
1
2–11
12–100
1A
$1M
$1M
$1M
1B
$0
$5M
$1M
2A
$1M
$1M
$0
2B
$0
$5M
$0
Figure 17. Savage's layout of the four lotteries. The greyed column is the same within each pair. Source: Maurice Allais (1953), Econometrica 21(4), 503–546, doi:10.2307/1907921.
Philosophy · 1961
Risk, ambiguity, and the Savage axioms
An urn holds 30 red balls and 60 that are black or yellow in an unknown proportion. Most people would rather bet on red than on black. Most would also rather bet on black-or-yellow than on red-or-yellow.
The first preference implies they think black is less likely than red; the second implies they think it more likely. No single probability for black makes both consistent with expected utility. What people are avoiding is not risk but ambiguity: they prefer the odds they know.
30 red
60 black or yellow, proportion unknown
Figure 18. The urn, one dot per ball. Source: Daniel Ellsberg (1961), Quarterly Journal of Economics 75(4), 643–669, doi:10.2307/1884324.
Philosophy · 1950
A difficulty in the concept of social welfare
Three voters each rank three options, and each ranking is perfectly consistent. Put them to pairwise majority votes and A beats B, B beats C, and C beats A. Condorcet noticed the cycle in 1785: individually rational preferences can add up to a group preference that is not.
Arrow proved that the problem is not special to majority rule. No rule for turning individual rankings of three or more options into a social ranking satisfies all of unrestricted domain, Pareto efficiency, independence of irrelevant alternatives and non-dictatorship.
Voter
First
Second
Third
1
A
B
C
2
B
C
A
3
C
A
B
Figure 19. A beats B (voters 1 and 3), B beats C (1 and 2), C beats A (2 and 3). Source: Kenneth Arrow (1950), Journal of Political Economy 58(4), 328–346, doi:10.1086/256963.
Computer science · 2002
Exploration and exploitation
Each arm of a slot machine pays out with an unknown probability. Always pull the arm that looks best and you may never discover a better one; keep sampling the others and you pay for the information. The multi-armed bandit is the cleanest statement of the trade-off.
UCB1 adds to each arm's average an uncertainty bonus, √(2 ln t / n), which shrinks as the arm is tried, and Auer, Cesa-Bianchi and Fischer proved that its regret (the reward lost to not always playing the best arm) grows only logarithmically in time. A fixed ε-greedy rule explores at the same rate forever, so its regret grows linearly.
Cumulative regret
ε-greedy, ε = 0.1
UCB1
Figure 20. Simulated here: five arms paying 0.1–0.9, 10,000 pulls, averaged over 40 runs from a fixed seed. Source: Peter Auer, Nicolò Cesa-Bianchi, Paul Fischer (2002), Machine Learning 47, 235–256, doi:10.1023/A:1013689704352.
Computer science · 1975
The Monty Hall problem
A car is behind one of three doors and goats are behind the others. You pick a door. The host, who knows where the car is, opens one of the other two to show a goat and offers you the chance to switch. Switching wins two times in three: your first pick was right one time in three, and the host's choice moves the rest of the probability onto the one door he left closed.
When Marilyn vos Savant gave that answer in Parade in 1990, thousands of readers wrote in to tell her she was wrong, many of them with doctorates. The question is small enough to settle by simulation.
Share won
Switch
Stay
Figure 21. 1,000 simulated games from a fixed seed. Final: switching 68.0%, staying 32.0%. Source: Steve Selvin (1975), The American Statistician 29(1), 67.
Computer science · 1956
A new interpretation of information rate
Kelly, working on Shannon's information theory at Bell Labs, asked how much a gambler with an edge should stake. For an even-money bet won with probability p, the fraction of the bankroll that maximises the long-run growth rate of wealth is f = 2p − 1.
At p = 0.6, that is 20%, and wealth grows at 2.01% per bet. Staking twice that, 40%, gives a growth rate of −0.24%: every bet still has a positive expected value, and the bettor still goes broke in the long run.
Growth per bet
g(f)
Figure 22. g(f) = p ln(1 + f) + (1 − p) ln(1 − f), p = 0.6. Growth turns negative past f = 39%. Source: John L. Kelly Jr. (1956), Bell System Technical Journal 35(4), 917–926, doi:10.1002/j.1538-7305.1956.tb03809.x.
Computer science · 1957
Dynamic programming
Bellman's principle of optimality: whatever the first decision, the remaining decisions must be optimal for the state it leads to. So the value of a state is the best immediate reward plus the discounted value of where that action leads, V(s) = max over a of [R(s, a) + γ V(s′)].
Starting from zero everywhere and applying that one equation repeatedly converges on the optimal value of every state at once, and the best action in each state falls out of the values. It is the root of reinforcement learning.
→0.59
→0.66
→0.73
→0.81
→0.90
+1
↑0.53
↑0.59
↑0.66
↑0.73
↑0.81
−1
↑0.48
↑0.53
↑0.66
↑0.73
←0.66
↑0.43
↑0.48
→0.53
↑0.59
↑0.66
↑0.59
↑0.39
↑0.43
↑0.48
↑0.53
↑0.59
↑0.53
Figure 23. Value iteration on a small grid: +1 at the top right, −1 beneath it, one wall, γ = 0.9, computed to convergence. Arrows are the greedy policy. Source: Richard Bellman (1957), A Markovian decision process, Journal of Mathematics and Mechanics 6(5), 679–684.
Computer science · 1989
The secretary problem
n candidates arrive one at a time in random order. Each must be accepted or rejected on the spot, and only hiring the single best counts as success. The optimal rule is to reject the first n/e, about 37%, and then accept the first candidate better than everyone seen so far.
For n = 100, the best cut-off is to pass on the first 37, and the chance of ending with the best candidate is 37.1%. As n grows it approaches 1/e, about 36.8%, however large the pool.
Chance of hiring the best
P(success)
Figure 24. Exact success probability for n = 100, by the number of candidates rejected before the rule starts. Source: Thomas S. Ferguson (1989), Who solved the secretary problem?, Statistical Science 4(3), 282–296, doi:10.1214/ss/1177012493.
Computer science · 1981
The evolution of cooperation
In the prisoner's dilemma each player does better by defecting whatever the other does, yet two defectors both do worse than two cooperators. Played once, rational players defect.
Axelrod invited game theorists to submit programs to play it against each other many times over. The winner of both of his tournaments was one of the simplest programs entered, Anatol Rapoport's tit for tat: cooperate on the first move, then copy whatever the other player did last. It never beats its opponent in a single match; it won by eliciting cooperation.
You / Them
Cooperate
Defect
Cooperate
3, 3
0, 5
Defect
5, 0
1, 1
Figure 25. Axelrod's payoffs, yours first. Defecting is better in each column; mutual defection is worse for both than mutual cooperation. Source: Robert Axelrod, William D. Hamilton (1981), Science 211, 1390–1396, doi:10.1126/science.7466396.