Thursday, January 31, 2013

Macro-forecasting rarely beats chance, and does worse using ideology

Wired.com - Do political experts know what they're talking about?
see also:
WSJ - Beware our blind seers
and:
Daily Beast - Why pundits get things wrong

This is an examination of the work of a social scientist who has made a career of studying pundits and their predictive ability. Philip Tetlock has given over 280 experts in the field of political or economic trends scores of questions, asking them to predict future events over 20 years. The average result was worse than chance. Even more interestingly, those who did better than their peers tended to have a messier, flexible, inductive style of prediction. The worst performers seemed be classifiable as experts with a particular lens or ideology, using one or a few "Big Ideas" about politics, economics, or human nature to make predictions about specific events. Those types of thinkers ("hedgehogs") also appeared more confident and more self-assured in their presentations of their predictions. The "foxes" couched their conclusions more tentatively and could be prone to seemingly contradictory opinions, but tended to perform better. Even better than both: mathematical models (see last bit of last article).

No comments: