Customer experience and sentiment analysis
The notion that listening to your customer’s voice is important is well entrenched. Companies have long depended on data from customer surveys, call center transcripts and focus groups, captured in structured formats and handled through business intelligence applications, to help point the way to improved customer service, product enhancements and competitor vulnerabilities.
But the sheer volume of the customer choir in the Web 2.0 age often leaves companies scrambling to keep up. Publishing is now in the hands of the public, who have a vexing tendency to post with blunt honesty in unstructured formats via blogs, tweets, e-mails and forums about products and services that delight or disappoint them. And those opinions hold weight. A 2007 study by Jupiter Research (since acquired by Forrester), called “Social Networking Sites: Defining Advertising Opportunities in a Competitive Landscape,” found that 30 percent of frequent social networkers trust their peers’ opinions when making a major purchase decision, compared to the 10 percent who trust advertisements.
As Andreas Wiegend, former chief scientist of Amazon.com, predicted in a blog post for the Monitor Talent Group, “In 2009, more data will be generated by individuals than in the entire history of mankind through 2008.” Companies face a very real need not just to acknowledge the impact of unstructured social media on brand and product perception, but to understand and filter it sensibly, and to integrate it with structured customer data and get it into the hands of the right people to make it actionable.
For many companies, the burgeoning text analytics approach of sentiment analysis is becoming a critical component of their overall strategy, giving them a much-needed assist to stay responsive to customers, market opportunities and trends.
What is it?
In his white paper “Text Analytics 2009,” Seth Grimes, analytics strategist at Alta Plana, describes text analytics as “the software and the transformational steps that discover business value in ‘unstructured’ text.”
There’s special business value in discerning opinion, sentiment and subjectivity—the “voice of the customer”—in text as varied as blogs, forum postings, articles, e-mail and survey responses. That field of “customer experience analysis” applies sentiment analysis and other techniques to understand and help predict consumer behavior via text analysis coupled with analysis of customer transactions, profiles and demographics.
Vendors generally use a combination of statistical analysis of wordfrequency and co-occurrences with linguistics (involving lexicons, dictionaries and language rules) in an algorithmic approach to understanding exactly what the consumer is saying. Grimes, who will be presenting a talk in April on “Search for Sentiment” at the Search Engine Meeting 2010, which is co-sponsored by KMWorld, says, “The narrower you can frame the problem and the data you collect, the better, because you can then adjust your approach to match specific business requirements and information sources.”
The technological challenges are not for the faint-hearted or the linguistically timid. Suresh Vittal, analyst at Forrester, says, “For a long time, text analytics was a technology in search of a business need. Now, thanks to social media, the need is there; the question is whether the technology can ramp up fast enough to be commercial.” Early adoption by government agencies, which sought to apply text mining to mountains of classified documents, is giving way to more mainstream commercial demand from industries for whom customer perception is critical: hospitality, consumer brands and high-tech, among them.
Classifying the messy middle
Ours is a world in which online consumer reviews of hotels that might include the phrase “the lobby is baaaaad!” meant in a positive way, or a review of a holiday toy saying, “I would give this to all the children in my life, if I were Scrooge,” meant to disparage. Throw in slang, language evolution and socio-cultural gradations in word use, and you have a mammoth challenge for accurate computational treatment of opinion.
Larry Levy, co-founder and chief opinion gatherer at Jodange, an opinion utility that filters and aggregates thoughts, feelings and statements from traditional and social media, says accuracy remains a challenge in the industry. “The sentiment side is good at the two poles, positive and negative,” he says, estimating that Jodange’s Opinion Lens gets those sentiments right around 80 percent of the time. “But the neutrals are difficult. If you give four people in a room 100 neutral opinions and ask them to classify, even they will only agree 55 to 60 percent of the time.”
The level of granularity can also be important. If sentiment is assigned at a document level—that is, each tweet or blog post is assigned a positive, neutral or negative sentiment—how does the hypothetical tweet “I love Marriott’s bathrooms but the beds are lumpy” get classified? Marcel LeBrun, chief executive officer of Radian6, which offers clients a platform to listen, measure and engage with customers across the social web, cautions, “Ratings need to be assigned on a subject level at a minimum; a solution that assigns them at a document level is going to miss something.”
Whose opinion is it?
Even if a sentiment analysis tool were always accurate, the opinions don’t necessarily carry equal weight. LeBrun estimates that Radian6 customer Dell has 8,000 to 10,000 online conversations about its brand each day, which span the spectrum of positive to negative; the company needs to understand whose opinion actually has the power to move brand perception, and keep close tabs on those. “Sentiment analysis needs to be connected to social metrics and influence analysis to make sense,” says LeBrun.