With social media use gaining strong ground in the midmarket, it's never too soon to check that metrics are in place and working well. That's easier said than done, of course. There's no shortage of vendors who offer social media metric solutions, even claiming to mine useable real-time data from the flood of feedback flowing along a range of social media channels.
One sharp challenge for social media metrics is sentiment analysis. Evidently, the channels through which customers react and respond to marketing initiatives throb with sentiment. Whether primarily concerned with B2B or B2C strategies, the midmarket enterprise needs to be sensitive to the ripples of sentiment about their offerings detectable in the social stream.
The problem is that sentiment analysis is notoriously unreliable, especially when automated -- and when channels are super-saturated with data, automated analytics are really the only game in town.
In a 2010 study, the social media agency FreshNetworks compared results of automated sentiment analysis of the Starbucks brand by several social media tools with the findings of a human analyst. It was all bad news for the automated tools. Although accuracy levels looked good, the results were skewed by the typically high level of neutral comments. The sentiment metrics tools could identify these correctly, giving an impression of high accuracy: The problem lay in correctly distinguishing positives from negatives in the smaller pool of non-neutral comments.
Compared with human results, automated sentiment analysis was right about positive or negative comments about 30 percent of the time, a result appropriately described as "disastrous." Have things gotten any better since 2010?
A recent discussion at Social Media Explorer doesn't provide much comfort, but does indicate some ways in which sentiment analysis tools might be improved, and in particular, what enterprises should look for when assessing the competing claims of analysis vendors.
Be aware, for example, of the challenges facing automation. Identifying the general sentiment of a message board post is one thing, identifying sentiment about a specific topic of interest within the post may be more difficult.
Fantastic day, very happy and pleased with everything (not so much the coffee).
It's easy to imagine sentiment metrics rating such a message as highly positive -- which is unhelpful if you're the coffee vendor.
Look out for proffers which allow you "train" the tool. There is no need to settle for "out of the box" accuracy. Generalized algorithms and taxonomies may not be well adapted to the needs of specific enterprises. Look for the option of customizing the analysis by importing your own word or phrase lists, and make sure the tool can be tuned to the channels of most interest.
Finally, be aware that accuracy is not necessarily everything. Capturing small numbers of highly positive leads using a sentimental analysis tool which misses large numbers of averagely positive leads might have sufficient business impact to be worthwhile. Also, depending on the business in question, it might be much more important to identify powerfully negative responses than powerfully positive (or vice versa).
When it comes to sentiment analysis, as with so many other smart business solutions, one-size-fits-all is not the way to go.
Maybe the hard part would be to create an analysis tool for any group. Maybe if you knew who was doing the talking, you could adapt. In a PC, LOL is as much of a word as The.
In my opinion, highly unlikely. That's why I don't see much of a future in sentiment analysis. I daresay influence measurement (a.k.a. Klout) has a better future than sentiment analysis, and I don't even think Klout is good at what it does. My Klout score went up 20 points last month because people greeted me on my birthday XD
Certainly going to be a tricky one in the beginning. Reason being even on social media there's going to be people from different cultures and contexts with different ways of using language.
I have a feeling that we'll see great improvements in sentiment analysis when we see the semantic web actually hitting the mainstream. If it doesn't, I doubt if we'd ever see sentiment analysis reaching 70% accuracy.
Good points! The terminology used by different audiences (groups) can vary dramatically which would make performing a dynamic analysis somewhat difficult.
I hadn't heard of sentiment analysis before. Thank you, very interesting post.
I can imagine how sentiment analysis, for now, suits certain types of websites better than others. For example, social networks for book lovers are easier (I would assume they have better grammar, writing, etc). Gaming sites are probably harder because of the slang, etc used by younger audiences.
Brand advocates (or haters) as you phrase it are a prime example of what I was referring to. If they are not tracking all aspects (good and bad) then they are likely to be making the results skewed in one direction or another.
Good points, but sample size is going to be relevant here. Hopefully it's a matter of following more than just one, small cohort across multiple channels. I think it's valuable to think in terms of training tools to look for phrases relevant to the product or service of interest rather than just general terms of approval - but yes, the results are going to be skewed if you just end up following brand advocates (or haters).
The analysis process can generally be skewed to one direction or another based upon how the data is interpreted. So how is the proecess of sentiment analysys supposed to be any different? It seems that the algorithms that are utilized can be tweaked as much as they want, but if a company is aggregating data from multiple channels to provide the overall analysis results then what is to say that following these same people across those channels could prove to be even more dramatically skewed.
The ThinkerNet does not reflect the views of TechWeb. The ThinkerNet is an informal means of communication to members and visitors of the Internet Evolution site. Individual authors are chosen by Internet Evolution to blog. Neither Internet Evolution nor TechWeb assume responsibility for comments, claims, or opinions made by authors and ThinkerNet bloggers. They are no substitute for your own research and should not be relied upon for trading or any other purpose.
In its mission to make car-buying simple, fair, and fun, TrueCar relies extensively on the Internet, social media, data, and -- increasingly -- mobile to reach its expanding audience of car dealers and consumers. The analysis firm is also relying on Mike Dunn, who joined TrueCar as chief technology officer on May 1, to help steer its technology investments and business priorities.
Marketing departments almost immediately latched onto Twitter as a great tool for spreading word about their brands, but Celina Insurance is using the microblog to help keep the entire company running in the case of emergency or disaster.
Everybody's talking about the rapidly growing importance of mobile channels, not only in social life, but for business too -- whether you're running a city, a hospital, or a school, selling B2B, or engaged in regular retail.
Rob Shoenfelt, CIO at Celina Insurance, is the first to admit that insurance firms aren't known as leading edge adopters of technology. But Celina Insurance isn't like most insurance firms.
When combined with training and management, today's affordable unified communications and collaboration solutions empower midsized organizations to be more efficient and productive. But only if you know how they work, and what they'll need to do their jobs even better.
A recent release of the popular TweetDeck app for Twitter power-users gives new life to software that had previously taken a wrong turn. Here's a quick walk-through of the new TweetDeck, to show you why it should be at the top of your Twitter toolkit.
Showing results is the best way to win over social business doubters, according to Mary Maida, Medtronic lead information solutions manager. Internet Evolution's Mitch Wagner interviewed Maida at the E2 Innovate conference.
A growing number of HR managers are suspicious of individuals who do not take part in social media and view them as anti-social in real life as well as online.
Enterprises are discovering that using social networking within the secure setting of a SaaS provider's network gives them an unusual opportunity to freely collaborate with partners, suppliers, and even competitors.
As social media make their way into company operations, IT'ers and engineers are using it to exchange ideas and collaborate on problem solving with others. But there is also a line to be drawn when it comes to proprietary information sharing.
The very low-tech "scrum" project technique introduces "crowd talking" to projects and also sets the entire crowd to problem solving. So far, these new social-media-style meetings appear to have supercharged project execution.
New tools like laptops, tablets, smartphone, and wireless connectivity let us work from San Diego to Katmandu, and anywhere in between. But time management remains a problem.
Facebook's Graph Search may face some profound challenges and risks, first, because Facebook users haven't been thinking of their posts as product reviews; and second, because Facebook will now have to contend with the social-network equivalent of SEO "gaming" of results.
Companies need to take advantage of new technologies to simplify interfaces, improve capabilities, and enhance back-office processes. But they can't upgrade their Websites too often.
New York's Metropolitan Transit Authority is conducting a pilot test of digital kiosks to guide subway users to where they want to go more efficiently and at lower cost.
The whole Amazon.reader debate is a double-stupid. It's stupid to think that there's any e-book buyer who doesn't know Amazon's URL, and it was stupider to let ICANN launch the whole free-form TLD initiative to start with.
While NFC's original goal was to enhance mobile commerce applications, it is finding its way into a number of other uses, which is creating both opportunity as well as challenges for IT departments.
Enterprises would like to move to cloud computing but are hesitant because they are concerned about providers’ ability to secure company data. Here are some tips that help to ensure that if breaches occur, the business is not left holding the bag.
Edmunds separates customers into segments based on the info it collects on its site and from partners, and uses that to push out custom content, said Brian Baron, director of business analytics for Edmunds.com, at Predictive Analytics Innovation Summit.
The automotive website uses propensity modeling to target ads and customer registration forms, said Brian Baron, director of business analytics for Edmunds.com, at Predictive Analytics Innovation Summit.
Expert Integrated Systems: Changing the Experience & Economics of IT In this e-book, we take an in-depth look at these expert integrated systems -- what they are, how they work, and how they have the potential to help CIOs achieve dramatic savings while restoring IT's role as business innovator. READ THIS eBOOK
your weekly update of news, analysis, and
opinion from Internet Evolution - FREE! REGISTER HERE
Wanted! Site Moderators Internet Evolution is looking for a handful of readers to help moderate the message boards on our site – as well as engaging in high-IQ conversation with the industry mavens on our thinkerNet blogosphere. The job comes with various perks, bags of kudos, and GIANT bragging rights. Interested?
To save this item to your list of favorite Internet Evolution content so you can find it later in your Profile page, click the "Save It" button next to the item.
M2M: Rise of the Machines? Not Yet David Weldon In the 1970 science fiction thriller Colossus: The Forbin Project, two giant supercomputers from the United States and Soviet Union secretly join forces to take control of the collective nuclear might of the two countries. In the film, the two machines discover each other's existence, communicate back-and-forth, share their collective data, and cut their human creators out of the process. It is the ultimate example of machine-to-machine communications, or M2M. CLICK FOR MORE
M2M: Rise of the Machines? Not Yet David Weldon In the 1970 science fiction thriller Colossus: The Forbin Project, two giant supercomputers from the United States and Soviet Union secretly join forces to take control of the collective nuclear might of the two countries. In the film, the two machines discover each other's existence, communicate back-and-forth, share their collective data, and cut their human creators out of the process. It is the ultimate example of machine-to-machine communications, or M2M. CLICK FOR MORE
M2M: Rise of the Machines? Not Yet David Weldon In the 1970 science fiction thriller Colossus: The Forbin Project, two giant supercomputers from the United States and Soviet Union secretly join forces to take control of the collective nuclear might of the two countries. In the film, the two machines discover each other's existence, communicate back-and-forth, share their collective data, and cut their human creators out of the process. It is the ultimate example of machine-to-machine communications, or M2M. CLICK FOR MORE
M2M: Rise of the Machines? Not Yet David Weldon In the 1970 science fiction thriller Colossus: The Forbin Project, two giant supercomputers from the United States and Soviet Union secretly join forces to take control of the collective nuclear might of the two countries. In the film, the two machines discover each other's existence, communicate back-and-forth, share their collective data, and cut their human creators out of the process. It is the ultimate example of machine-to-machine communications, or M2M. CLICK FOR MORE
Email This
