This year's Presidential election will be a referendum on the power of analytics.
The political pundits and conventional polling are all leaning one way in gauging the opinion of the electorate. And then there's this one guy, armed with advanced analytics tools, who's reading the landscape completely differently.
You've seen headlines like these in the past few days:
That represents the common wisdom about the election, from bloggers and pundits who've been covering politics for decades, armed with conventional polling tools. Most say the race is drumhead-tight. Some say Mitt Romney is winning.
And then there's Nate Silver, author of the blog FiveThirtyEight. He sees things very differently, giving President Barack Obama 71 percent odds of winning the election.
How does Silver reach such a widely different conclusion? He looks at all the polls, and runs them through an algorithm he developed in the 2008 election and fine-tuned in the spring. He has very little gut feeling and experience about politics -- indeed, until just a few years ago, he specialized in baseball stats.
It would be easy to dismiss Silver as a lone voice, if not for his track record. Using his analytics tools, he predicted the outcome of 49 of the 50 states in the 2008 election and projected the overall popular win to within a single percentage point. He also predicted primary winners, and the winners of all 35 Senate races that year. Impressive.
If you saw the movie Moneyball, or read the book by Michael Lewis, this sounds familiar. And the baseball tie-in is no coincidence. Before he got into political analysis, Silver analyzed baseball stats; he developed a system called PECOTA (Player Empirical Comparison and Optimization Test Algorithm), which predicts player performance, career development, and seasonal winners and losers. Just like in Moneyball (but a couple of years later).
Journalist James Fallows connects the dots. He says this is "a perfect test case of the Michael Lewis Moneyball hypothesis."
Apart from Silver's own background as a sports-stats analyst, we have an exceptionally clear instance of people judging from their experience, their "bones," their personal instinct, etc that things are going one way (like veteran scouts saying that a prospect "looks like a Big Leaguer"), while data (on-base efficiencies in one case, swing-state polls in another) point in the opposite direction.
I don't know who's right, but I note this as a moment of unusual clarity about two approaches to politics. Among the many things we'll learn two weeks from now (and in the assessment afterwards) is which of these approaches to political analysis has revealed a profound flaw.
The outcome of this election will have vast global significance, of course, but it'll have particular importance among business leaders who advocate analytics, many of whom face resistance from CEOs and line managers who believe their instincts and gut feelings, honed by years of experience, beat the results of analytics. If Silver's analyses prove as accurate this year as they were in 2008 -- if he can predict not only the winner of the election, but the state-by-state breakdowns and popular margins -- then he'll arm analytics advocates with a powerful example with which to convince the skeptics.
But Silver might be wrong. He's the first to admit that. That 71–29 lead for Obama doesn't mean that the incumbent is 40+ points ahead in the polls. Rather, it means that, of all the thousands of simulations Silver's algorithms have run, Obama won 71 percent of the time. There's still a better-than-one-in-four chance that Romney will win. I wouldn't bet my retirement savings on those kinds of odds.
For more from Silver, see a video interview with Silver at the IBM Information on Demand conference in Las Vegas. (IBM sponsors Internet Evolution.)
I'm not so sure I see it that way, Mitch. One way you know you've arrived is when you've got that many people criticizing your methods.
A Romney win will simply attract even more attention to Silver. His fans -- who understand how chance and probability work -- will continue to be his fans, his critics will continue to gleefully criticize, and all this means more press and attention.
Sure, get too many "wrong," and you might be done, but I daresay that a strong record of being "correct" followed by one big "error" (and then going back to being "correct" again) is far, far better for ensuring one's following and success than merely being boringly, astoundingly accurate every single time.
If Romney wins, it'll be a black eye for Silver and for analytics in general.
It won't mean Silver was wrong. If you say something has a 19% chance of happening (Silver's current prediction) and it happens, you're not wrong. But that's not the way most people will see it.
Today Nate Silver found a nice way to encapsulate my point that, because a race is close, doesn't mean it's a tossup:
(Obama's) chances of holding onto his Electoral College lead and converting it into another term are equivalent to the chances of an N.F.L. team winning when it leads by a field goal with three minutes left to play in the fourth quarter. There are plenty of things that could go wrong, and sometimes they will.
But it turns out that an N.F.L. team that leads by a field goal with three minutes left to go winds up winning the game 79 percent of the time. Those were Mr. Obama's chances in the FiveThirtyEight forecast as of Wednesday: 79 percent.
A recurrent theme in Silver's blog posts is the discrepancy between state polls (which have favored Mr. Obama) and national polls (which have favored Mr. Romney). Some interesting possibilities there. Silver discounts the likelihood of a popular vote/electoral college split, instead theorizing that it might have to do with overperformance and underperformance by the candidates in certain states.
In any case, the most important poll is the actual election. The results there remain to be fully seen.
It's not challenging to infer a prediction from the (interminably) consistent state of polling in battleground states, and the way the electoral college votes stack up. This doesn't mean the obvious prediction can't be wrong, but if it is, it's going to be interesting to try to figure out how the polls have so consistently misled us for so long.
Everyone and their dogs are coming out with predictions, that way, once Oct 7th comes in, they can say: "I ... TOLD YOU" and if you are trying to sell something, then you can say: "MY NEW PREDICTION TOOL ... TOLD YOU, BUY IT".
If it doesn't, people will forget and he can say it was too close and make it all fuzzy and technical.
Even though Silver desccribes what he does as forecasting, he says he's not trying oto predict the future. He's keeping score, like at a sporting event.
Looked at that way, Silver's current odds of 72% in favor of an Obama win doesn't look so much like a prediction as it looks like a score. At this stage, nearing the final bell, Team Blue is ahead by almost 3-1.
But teams have come back from worse odds.
On the other hand, a wise man said this: "People confuse a "close" election race with an "unpredictable" election race. The outcome has been fairly clear for weeks."
The ThinkerNet does not reflect the views of TechWeb. The ThinkerNet is an informal means of communication to members and visitors of the Internet Evolution site. Individual authors are chosen by Internet Evolution to blog. Neither Internet Evolution nor TechWeb assume responsibility for comments, claims, or opinions made by authors and ThinkerNet bloggers. They are no substitute for your own research and should not be relied upon for trading or any other purpose.
We never thought we'd write about carrier pigeons. Or the Pillsbury Dough Boy. Or corned beef hash. But these are some of the many ingredients you'll find in this week's mashup of top technology -- yes, really -- stories from around the world. Grab a fork and dig in.
Anyone still convinced that Amazon is essentially an online bookstore should at long last revise their opinions. Amazon may be gearing up to bring its full range of products and services right to your front door.
Google and Apple don't have a lock on wearable technologies. Neither do the Pebble watch or Sony. Plenty of other developers -- from well-established corporations to startups -- want a foothold in a space expected to be worth $6 billion by 2016, according to IMS Research. (See: Mary Meeker: The Future Will Be Wearable.)
IBM is advancing both its Mobile First and open-source strategies through a partnership with 10Gen, the company behind open-source NoSQL database MongoDB. Under the agreement, unveiled last week, both companies will work together on a new standard for mobile enterprise applications.
New York's Metropolitan Transit Authority is conducting a pilot test of digital kiosks to guide subway users to where they want to go more efficiently and at lower cost.
Elizabeth Pizzinato, SVP of marketing and communications at Four Seasons Hotels and Resorts, calls content marketing "the new black" and explains how her brand engages its target audience.
Big-data and analytics tools enable marketers to understand customers as individuals, identifying unmet needs and addressing each customer as a "segment of one," says John Kennedy, VP corporate marketing, IBM.
Edmunds separates customers into segments based on the info it collects on its site and from partners, and uses that to push out custom content, said Brian Baron, director of business analytics for Edmunds.com, at Predictive Analytics Innovation Summit.
The automotive website uses propensity modeling to target ads and customer registration forms, said Brian Baron, director of business analytics for Edmunds.com, at Predictive Analytics Innovation Summit.
The automotive website uses propensity modeling of customer behavior to convert more site visitors into leads, says Brian Baron, director of business analytics, in an interview at the Predictive Analytics Innovation Summit.
Linda Descano, President and CEO of Women & Co., and managing director of partnerships and branded content of North America marketing at Citi, explains her firm's marketing opportunities and challenges.
Big-data and analytics tools enable marketers to understand customers as individuals, identifying unmet needs and addressing each customer as a "segment of one," says John Kennedy, VP corporate marketing, IBM.
New York's Metropolitan Transit Authority is conducting a pilot test of digital kiosks to guide subway users to where they want to go more efficiently and at lower cost.
The whole Amazon.reader debate is a double-stupid. It's stupid to think that there's any e-book buyer who doesn't know Amazon's URL, and it was stupider to let ICANN launch the whole free-form TLD initiative to start with.
While NFC's original goal was to enhance mobile commerce applications, it is finding its way into a number of other uses, which is creating both opportunity as well as challenges for IT departments.
Enterprises would like to move to cloud computing but are hesitant because they are concerned about providers’ ability to secure company data. Here are some tips that help to ensure that if breaches occur, the business is not left holding the bag.
Edmunds separates customers into segments based on the info it collects on its site and from partners, and uses that to push out custom content, said Brian Baron, director of business analytics for Edmunds.com, at Predictive Analytics Innovation Summit.
The IBM Smarter Commerce Global Summit in Monaco kicked into high gear today, and we've already begun to see news emerging from that lovely city-state by the sea.
Expert Integrated Systems: Changing the Experience & Economics of IT In this e-book, we take an in-depth look at these expert integrated systems -- what they are, how they work, and how they have the potential to help CIOs achieve dramatic savings while restoring IT's role as business innovator. READ THIS eBOOK
your weekly update of news, analysis, and
opinion from Internet Evolution - FREE! REGISTER HERE
Wanted! Site Moderators Internet Evolution is looking for a handful of readers to help moderate the message boards on our site – as well as engaging in high-IQ conversation with the industry mavens on our thinkerNet blogosphere. The job comes with various perks, bags of kudos, and GIANT bragging rights. Interested?
To save this item to your list of favorite Internet Evolution content so you can find it later in your Profile page, click the "Save It" button next to the item.
Email This
