This year's Presidential election will be a referendum on the power of analytics.
The political pundits and conventional polling are all leaning one way in gauging the opinion of the electorate. And then there's this one guy, armed with advanced analytics tools, who's reading the landscape completely differently.
You've seen headlines like these in the past few days:
That represents the common wisdom about the election, from bloggers and pundits who've been covering politics for decades, armed with conventional polling tools. Most say the race is drumhead-tight. Some say Mitt Romney is winning.
And then there's Nate Silver, author of the blog FiveThirtyEight. He sees things very differently, giving President Barack Obama 71 percent odds of winning the election.
How does Silver reach such a widely different conclusion? He looks at all the polls, and runs them through an algorithm he developed in the 2008 election and fine-tuned in the spring. He has very little gut feeling and experience about politics -- indeed, until just a few years ago, he specialized in baseball stats.
It would be easy to dismiss Silver as a lone voice, if not for his track record. Using his analytics tools, he predicted the outcome of 49 of the 50 states in the 2008 election and projected the overall popular win to within a single percentage point. He also predicted primary winners, and the winners of all 35 Senate races that year. Impressive.
If you saw the movie Moneyball, or read the book by Michael Lewis, this sounds familiar. And the baseball tie-in is no coincidence. Before he got into political analysis, Silver analyzed baseball stats; he developed a system called PECOTA (Player Empirical Comparison and Optimization Test Algorithm), which predicts player performance, career development, and seasonal winners and losers. Just like in Moneyball (but a couple of years later).
Journalist James Fallows connects the dots. He says this is "a perfect test case of the Michael Lewis Moneyball hypothesis."
Apart from Silver's own background as a sports-stats analyst, we have an exceptionally clear instance of people judging from their experience, their "bones," their personal instinct, etc that things are going one way (like veteran scouts saying that a prospect "looks like a Big Leaguer"), while data (on-base efficiencies in one case, swing-state polls in another) point in the opposite direction.
I don't know who's right, but I note this as a moment of unusual clarity about two approaches to politics. Among the many things we'll learn two weeks from now (and in the assessment afterwards) is which of these approaches to political analysis has revealed a profound flaw.
The outcome of this election will have vast global significance, of course, but it'll have particular importance among business leaders who advocate analytics, many of whom face resistance from CEOs and line managers who believe their instincts and gut feelings, honed by years of experience, beat the results of analytics. If Silver's analyses prove as accurate this year as they were in 2008 -- if he can predict not only the winner of the election, but the state-by-state breakdowns and popular margins -- then he'll arm analytics advocates with a powerful example with which to convince the skeptics.
But Silver might be wrong. He's the first to admit that. That 71–29 lead for Obama doesn't mean that the incumbent is 40+ points ahead in the polls. Rather, it means that, of all the thousands of simulations Silver's algorithms have run, Obama won 71 percent of the time. There's still a better-than-one-in-four chance that Romney will win. I wouldn't bet my retirement savings on those kinds of odds.
For more from Silver, see a video interview with Silver at the IBM Information on Demand conference in Las Vegas. (IBM sponsors Internet Evolution.)
Not only was Silver subjected to criticism from people who didn't like his forecast, he had to deal with some very nasty personal attacks too. He must be feeling pretty good this morning.
This is the big night for Silver, and for analytics in general.
Silver himself pointed out on Twitter this week that just because he's forecasting a 90% likelihood of victory for Obama doesn't mean he's forecasting a landslide. Quite the opposite: He's forecasting Obama most likely to win by a narrow margin.
This must be about the last forecast from Silver before the polls close. He's now showing Obama with a more than 90% choice of winning. Romney, thus 10-1 against, which are tough odds to overcome.
Stats don't come naturally to people. Imagine Romney winning, and Silver saying "There you are, I told you so." That would be a head-scratcher for most people, but consistent - as you say - with his data.
Joe - But people who don't understand math will need more clear-cut results. They'll fixate on the big prediction and not look at all the other forecasts Silver has made.
Another way of explaining what Silver does: If the weather forecaster says there's a 75% chance of snow tomorrow, he's not wrong if it doesn't snow. The key is -- how accurate are his predictions over time?
Silver will certainly need to explain what happened if Romney wins, but yes, Silver hasn't ruled it out. It's just appeared highly unlikely for a very long time.
Meanwhile, nice little piece on Silver at TPM. Interesting to see that he retains rights, essentially, to the 538 blog, and is really just "leasing" his brand to the Times.
The ThinkerNet does not reflect the views of TechWeb. The ThinkerNet is an informal means of communication to members and visitors of the Internet Evolution site. Individual authors are chosen by Internet Evolution to blog. Neither Internet Evolution nor TechWeb assume responsibility for comments, claims, or opinions made by authors and ThinkerNet bloggers. They are no substitute for your own research and should not be relied upon for trading or any other purpose.
Catch up on the week with one simple serving of Friday File. We've pieced together 10 interesting news bites you may have missed and put them together in bite-size morsels.
I've been excited by a few technology announcements, and bored by many, but Google's I/O announcement this week is the first where I found myself getting choked up and teary.
What's the prognosis for telehealth? Medical professionals have technologies they need; some state legislatures are enacting supportive laws, and both employers and employees tout the benefits. Yet other states have let proposals wither, insurers aren't sure how to charge for services, and physicians worry about liability and patient care.
New York's Metropolitan Transit Authority is conducting a pilot test of digital kiosks to guide subway users to where they want to go more efficiently and at lower cost.
Elizabeth Pizzinato, SVP of marketing and communications at Four Seasons Hotels and Resorts, calls content marketing "the new black" and explains how her brand engages its target audience.
Edmunds separates customers into segments based on the info it collects on its site and from partners, and uses that to push out custom content, said Brian Baron, director of business analytics for Edmunds.com, at Predictive Analytics Innovation Summit.
The automotive website uses propensity modeling to target ads and customer registration forms, said Brian Baron, director of business analytics for Edmunds.com, at Predictive Analytics Innovation Summit.
The automotive website uses propensity modeling of customer behavior to convert more site visitors into leads, says Brian Baron, director of business analytics, in an interview at the Predictive Analytics Innovation Summit.
Linda Descano, President and CEO of Women & Co., and managing director of partnerships and branded content of North America marketing at Citi, explains her firm's marketing opportunities and challenges.
Gil Elbaz, CEO of Factual, talks about the importance of data and analytics for marketers and how the technology is evolving to better assist automated, real-time decisions.
The whole Amazon.reader debate is a double-stupid. It's stupid to think that there's any e-book buyer who doesn't know Amazon's URL, and it was stupider to let ICANN launch the whole free-form TLD initiative to start with.
New York's Metropolitan Transit Authority is conducting a pilot test of digital kiosks to guide subway users to where they want to go more efficiently and at lower cost.
The whole Amazon.reader debate is a double-stupid. It's stupid to think that there's any e-book buyer who doesn't know Amazon's URL, and it was stupider to let ICANN launch the whole free-form TLD initiative to start with.
While NFC's original goal was to enhance mobile commerce applications, it is finding its way into a number of other uses, which is creating both opportunity as well as challenges for IT departments.
Enterprises would like to move to cloud computing but are hesitant because they are concerned about providers’ ability to secure company data. Here are some tips that help to ensure that if breaches occur, the business is not left holding the bag.
Edmunds separates customers into segments based on the info it collects on its site and from partners, and uses that to push out custom content, said Brian Baron, director of business analytics for Edmunds.com, at Predictive Analytics Innovation Summit.
The automotive website uses propensity modeling to target ads and customer registration forms, said Brian Baron, director of business analytics for Edmunds.com, at Predictive Analytics Innovation Summit.
Subsidized handsets, rather than locked handsets, should be the focus of regulators. We're not getting good deals, not fostering innovation, and weakening our power as buyers.
Expert Integrated Systems: Changing the Experience & Economics of IT In this e-book, we take an in-depth look at these expert integrated systems -- what they are, how they work, and how they have the potential to help CIOs achieve dramatic savings while restoring IT's role as business innovator. READ THIS eBOOK
your weekly update of news, analysis, and
opinion from Internet Evolution - FREE! REGISTER HERE
Wanted! Site Moderators Internet Evolution is looking for a handful of readers to help moderate the message boards on our site – as well as engaging in high-IQ conversation with the industry mavens on our thinkerNet blogosphere. The job comes with various perks, bags of kudos, and GIANT bragging rights. Interested?
To save this item to your list of favorite Internet Evolution content so you can find it later in your Profile page, click the "Save It" button next to the item.
Email This
