Freakonomics » False Positive Science: Why We Can’t Predict the Future

What we have here is an illustration of what has more generally been called “false positive science” by Joseph Simmons and colleagues in a 2011 paper. They argue that “it is unacceptably easy to publish ‘statistically significant’ evidence consistent with any hypothesis.”  The fact that a statistically sophisticated model of Olympic medals leads to predictions that perform worse than a naïve prediction based on the results of the immediately previous Games should tell us that there is in fact a lot going on that is not accounted for in the statistical model developed by Johnson and Ali. Does such a poorly-performing statistical model provide much insight beyond “intuition”? I’m not so sure.

More generally, while anyone can offer a prediction of the future, providing a prediction that improves upon a naïve expectation is far more difficult. Whether it is your mutual fund manager who is seeking to outperform an index fund, or a weather forecaster trying to beat climatology, we should judge forecasts by their ability to improve upon simple expectations. If we can’t beat simple expectations in the controlled environment of forecasting the outcomes of a sporting event, we should have some considerable degree of skepticism when interpreting predictions related to the far more complex settings of the economy and human behavior more generally.

via Freakonomics » False Positive Science: Why We Can’t Predict the Future.

Advertisements

Tags:

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s