WebFountain starts coming of age

You may remember some months ago when we discussed the WebFountain project underway at IBM's Almaden Research Center in San Jose. Then, Bob Carlson, IBM VP and WebFountain program director, described the project as a text analysis of the entire Web.

The initiative has started to take a much clearer form in these ensuing few months, and Carlson still sees the Web analytics market as growing from $400 million in 2003 to more than $10 billion in five years and views WebFountain's true value in terms of creating actionable business intelligence on unstructured information.

He says companies are spending $1 trillion a year worldwide trying to get us to buy "things," and the Internet has fundamentally changed the relationship between consumers and companies—we see an ad in a magazine and do our own research on the Web for a purchasing decision. So, the marketing domain has emerged as a fundamental application for WebFountain.

Few can argue that the Internet has created a different marketplace where corporations will need new generations of tools and new avenues of marketing, branding and communications to understand what consumers are actually saying about their products as they research them on the Internet. For example, Carlson says, blogs are becoming more of a source of personal consumer insight. They are tremendously rich sources of information, and if you can find a way to connect that information to customer segments, they can be remarkably valuable. Blogs are rapidly propagating in terms of numbers, and they do correlate to what customers are thinking and saying while giving near real-time views to consumers' wants and needs.

WebFountain continues to build its infrastructure. Carlson says the WebFountain supercomputer is easily one of the top 100 in the world and pushes about four gigabytes of data a second through the WebFountain environment. It continues to process 25 million to 50 million pages a day from the Internet and currently has more than one-and-a-half petabytes of storage online.

The Internet is growing at about 50 million pages a day, says Carlson. Much of the information an enterprise needs to develop this new marketing strategy and approach to consumer relations doesn't exist within the organization; it's "out there" on the Internet on consumer sites, the blogs, etc.

Organizations need to bring those external insights to the enterprise and match them to internal information. Carlson sees this technology as creating a new "sense-and-respond" marketing imperative--sense what's happening in the marketplace and respond with strategies to capture the opportunity or deal with a potential competitive disadvantage. The concept of annual planning cycles that are fixed for 12 months is dead in his mind. He believes the companies that stay fixed in that mode of thinking will have a tough time surviving the next five years.

But WebFountain isn't quite there yet. "I would not say today that we have transformative solutions yet," Carlson says, "but we're working with companies and academics to figure out how we can use this data in a way to make some of the fundamental decisions that drive product strategy.