This year's Presidential election will be a referendum on the power of analytics.
The political pundits and conventional polling are all leaning one way in gauging the opinion of the electorate. And then there's this one guy, armed with advanced analytics tools, who's reading the landscape completely differently.
You've seen headlines like these in the past few days:
That represents the common wisdom about the election, from bloggers and pundits who've been covering politics for decades, armed with conventional polling tools. Most say the race is drumhead-tight. Some say Mitt Romney is winning.
And then there's Nate Silver, author of the blog FiveThirtyEight. He sees things very differently, giving President Barack Obama 71 percent odds of winning the election.
How does Silver reach such a widely different conclusion? He looks at all the polls, and runs them through an algorithm he developed in the 2008 election and fine-tuned in the spring. He has very little gut feeling and experience about politics -- indeed, until just a few years ago, he specialized in baseball stats.
It would be easy to dismiss Silver as a lone voice, if not for his track record. Using his analytics tools, he predicted the outcome of 49 of the 50 states in the 2008 election and projected the overall popular win to within a single percentage point. He also predicted primary winners, and the winners of all 35 Senate races that year. Impressive.
If you saw the movie Moneyball, or read the book by Michael Lewis, this sounds familiar. And the baseball tie-in is no coincidence. Before he got into political analysis, Silver analyzed baseball stats; he developed a system called PECOTA (Player Empirical Comparison and Optimization Test Algorithm), which predicts player performance, career development, and seasonal winners and losers. Just like in Moneyball (but a couple of years later).
Journalist James Fallows connects the dots. He says this is "a perfect test case of the Michael Lewis Moneyball hypothesis."
Apart from Silver's own background as a sports-stats analyst, we have an exceptionally clear instance of people judging from their experience, their "bones," their personal instinct, etc that things are going one way (like veteran scouts saying that a prospect "looks like a Big Leaguer"), while data (on-base efficiencies in one case, swing-state polls in another) point in the opposite direction.
I don't know who's right, but I note this as a moment of unusual clarity about two approaches to politics. Among the many things we'll learn two weeks from now (and in the assessment afterwards) is which of these approaches to political analysis has revealed a profound flaw.
The outcome of this election will have vast global significance, of course, but it'll have particular importance among business leaders who advocate analytics, many of whom face resistance from CEOs and line managers who believe their instincts and gut feelings, honed by years of experience, beat the results of analytics. If Silver's analyses prove as accurate this year as they were in 2008 -- if he can predict not only the winner of the election, but the state-by-state breakdowns and popular margins -- then he'll arm analytics advocates with a powerful example with which to convince the skeptics.
But Silver might be wrong. He's the first to admit that. That 71–29 lead for Obama doesn't mean that the incumbent is 40+ points ahead in the polls. Rather, it means that, of all the thousands of simulations Silver's algorithms have run, Obama won 71 percent of the time. There's still a better-than-one-in-four chance that Romney will win. I wouldn't bet my retirement savings on those kinds of odds.
For more from Silver, see a video interview with Silver at the IBM Information on Demand conference in Las Vegas. (IBM sponsors Internet Evolution.)
Related posts:
Reddit Crashes During Historic Obama Chat
Social Media Makes the President: The US Election on the Web
Social Beats Traditional Media in VP Debate
— Mitch Wagner 
, Editor in Chief, Internet Evolution
Email This
