Text Analytics: on the trail of business intelligence

"We see the pull coming from several types of companies," says Banerjee. "First, from those that have good quantitative data but want qualitative data on their customers’ experience." Typically, those companies are in the retail, consumer package goods or telecom markets.

"Focus groups and surveys are not getting these companies the information they need in order to act on a daily basis," Banerjee says. With Clarabridge, the companies can find out on an ongoing basis what the customers’ top complaints are, the most requested new features or the most frequently mentioned
competitors, he adds.

The unstructured content for text mining and analysis often comes from call center conversations that are captured by the customer service representatives as notes in a CRM system, from online chat exchanges and from e-mail. In addition, survey data and Web 2.0 content, such as forums and newsgroups in which customers are sharing information with each other, are very useful in terms of tracking customer responses. The insight from that text is used across an organization by market research, customer care, product development, quality assurance and risk management professionals.

The ability to tune in to customer sentiment attracts organizations across a broad range of the spectrum—from those that are tops in their industry and want to keep their lead, to those that are experiencing image problems and want to improve.

"One of our customers is a major hotel chain that changed some of its policies and wanted to monitor customer response," recalls Banerjee. "They were happy to find out that their guests were overwhelmingly positive about the change." Another customer is an insurance company that has a persistently low JD Power customer satisfaction ranking and wants to find out why.

In its analyses, Clarabridge uses multiple techniques specifically tailored toward quantifying and analyzing the "voice of the customer" in unstructured content, including natural language processing, which tags data into parts of speech and maps it grammatically; keyword and machine categorization; and clustering.

"The ability to get real-time feedback from any data source puts a completely new perspective on analytics," Banerjee says. "Companies can know right away if a campaign is having the desired effect or is backfiring."

"Companies realize they’ve got a handle on their transaction and financial systems, but also realize they are not fully understanding and acting on customer feedback, suggestions and perceptions," Banerjee says. "Text analytics offers the only real way to practically distill insights from the large and growing volume of unstructured data that companies need to understand."

Text mining in Web 2.0

Nstein Technologies provides a content management solution geared toward e-publishers, with an eye toward fostering migration from print to cross-media publishing. Leading news organizations such as Time and the BBC are among the users of Nstein’s Ntelligent Content Management (NCM) Suite.

Nstein’s semantic search capabilities and text mining modules are part of NCM and provide a robust search environment, including full text, Boolean and semantic search. The text mining modules carry out concept and entity extraction, clustering and classification. The also create an abstract on the fly and suggest documents that have related content.

"Information discovery is no longer just about search," says Laurent Proulx, senior VP and CTO of Nstein. "Users need to see what the content is about."

Nsentiment was introduced last year as a specific application area for text analysis, designed for "sentiment analysis." It taps into public opinion about consumer products, politicians or other topical areas. Nsentiment’s analysis first maps the grammatical structure, tagging words with metadata such as parts of speech and extracting concepts. Then, the software analyzes the text to filter out objective facts in order to focus on the subjective "opinion" content. In addition, it calculates whether the comments are positive or negative, and the strength of the comments, from mild to strong.

"Sentiment analysis and other text analytics capabilities, such as fact finding, are really powerful techniques when applied to social networks," says Proulx. "People are expressing themselves more and more in blogs and e-mails, but to be useful, this information must be extracted and analyzed."

Moreover, many companies are discovering that it pays to monitor such sources before a wave of dissatisfaction becomes a major news story. Nstein’s current research is related to the application of its text mining modules, including Nsentiment, to particular verticals.