On experts
Another fun wee column from John Kay in the Financial Times, summarising Philip Tetlock’s new book Expert Political Judgment -
Isaiah Berlin, historian of ideas, made a distinction between the intelligence of the hedgehog – which knows one big thing – and the intelligence of the fox – which knows many little things. Hedgehogs fit what they learn into a world view. Foxes improvise explanations case by case. The world needs both but today it needs fewer hedgehogs and more foxes. Berlin’s terms are used to describe styles of reasoning by the American psychologist Philip Tetlock, who has spent 20 years asking pundits to predict who will win elections, what countries will acquire nuclear weapons or enter the European Union and how the first Gulf war would end. He has tested 30,000 predictions from 300 experts against outcomes.
Mr Tetlock finds that his respondents are not very good. They do better than a chimp who answers at random, but not much, and worse than simple forecasting rules based on extrapolation. But some pundits are better than others. A little knowledge is helpful. Dilettantes – people with the information you will acquire from diligent reading of this newspaper – do much better than undergraduates who based their judgment on a one-page summary of the issues. But experts have little advantage over dilettantes. The reputation of the experts is a guide to which are worth following. But not in the way you might expect. Bad forecasters are consulted more frequently than good ones. The more famous the expert, the worse his prognostications.
Kay's own new book on business strategy, The Hare and the Tortoise, is also recently published. Must remember to seek out and buy.
Isaiah Berlin, historian of ideas, made a distinction between the intelligence of the hedgehog – which knows one big thing – and the intelligence of the fox – which knows many little things. Hedgehogs fit what they learn into a world view. Foxes improvise explanations case by case. The world needs both but today it needs fewer hedgehogs and more foxes. Berlin’s terms are used to describe styles of reasoning by the American psychologist Philip Tetlock, who has spent 20 years asking pundits to predict who will win elections, what countries will acquire nuclear weapons or enter the European Union and how the first Gulf war would end. He has tested 30,000 predictions from 300 experts against outcomes.
Mr Tetlock finds that his respondents are not very good. They do better than a chimp who answers at random, but not much, and worse than simple forecasting rules based on extrapolation. But some pundits are better than others. A little knowledge is helpful. Dilettantes – people with the information you will acquire from diligent reading of this newspaper – do much better than undergraduates who based their judgment on a one-page summary of the issues. But experts have little advantage over dilettantes. The reputation of the experts is a guide to which are worth following. But not in the way you might expect. Bad forecasters are consulted more frequently than good ones. The more famous the expert, the worse his prognostications.
Kay's own new book on business strategy, The Hare and the Tortoise, is also recently published. Must remember to seek out and buy.
posted by Tim Chapman at 10:35 am
0 Comments:
Post a Comment
<< Home