The next-generation search market in 2002: Will Google reign in 2003?

Reviewing forecasts made in the first half of 2001 is a sobering experience. This time last year, Ovum’s report "Next-Generation Search: Building the Smart Portal" predicted that the enterprise market for next-generation search would be worth $1 billion by 2003. Not surprisingly, Ovum now think this target is going to take a bit longer to reach (more like 2006). However, it is not all doom and gloom. Take a look at the state of the market a year later, particularly at the impact of Google’s entry into the market.

By Eric Woods

Will Google (google.com) claim the same place in the enterprise as it has on the Web? We at Ovum think that is unlikely in its current guise--but that’s not to say it won’t give several other players a fright in the meantime, or that it cannot establish a foothold in the enterprise, which it may better exploit longer term.

The Google Appliance is not the vehicle for taking Google into the enterprise as a major player. However, Google will learn a lot about enterprise search in the process, and it learns quickly. It will find out how to adapt its distinct technology strengths to the enterprise, the gaps it needs to fill, and the channel and partner strategies that work best. "Google for the enterprise: the sequel" will be the offering that all competitors need to be concerned about.

Enterprise search today

In mid-2001, we estimated that the enterprise market for next-generation search would be worth $525 million. As with most forecasts made last year, it proved to be too optimistic. The market was worth $435 million in 2001—and that figure would have been lower if not for a reasonably strong first quarter from several vendors. Not surprisingly, it was the end of 2001 that saw the worst shocks, as the terrorist attacks of Sept. 11 added to problems of an already tough market. All parts of the market were affected—market leaders Verity and Autonomy both faced marked declines in revenues, while smaller vendors struggled (some unsuccessfully) to keep afloat. Grand visions of IPO (or post-IPO) fortunes faded quickly. So far, 2002 has been more about the slow grinding out of business rather than a further collapse. Vendors that have been able to bring costs in line with new expectations are feeling secure, if not brimming with optimism (both Autonomy and Verity are running profitably). However, for those with little pipeline and high running costs, it is becoming a month-by-month (or even week-by-week) survival operation.

As with many parts of the software market, we may have reached the bottom of the market, but it is a lot less clear how long it will be before we see a significant upturn. However, vendors and users of search technology still have reasons to be cheerful. The market leaders are in a strong position, but new challengers are coming along to maintain vigor and competition in the market; innovation continues to be a key characteristic and, above all, the enterprise requirement for better information handling continues to be a priority, and of growing importance.

With all those factors taken into account, our current estimate is that the search market will show some recovery in 2002 compared to 2001, with 8% overall growth in the market. Growth will be better in 2003, but real vigor won’t return until 2004 and 2005. By 2006, the market will be worth $1 billion. Realizing that opportunity, however, depends on the ability of vendors to take the innovative capabilities at the heart of next-generation search into the mainstream. If the market is to meet that forecast by 2006, advanced search will be a core infrastructure capability embedded into business processes and business networks--enabling access to information and people at new levels of sophistication.

While consolidation is inevitable among some of the smaller players, there will still be significant, new entrants. The industry "gorillas" will play a bigger part in the market. Related areas such as enterprise portals and content management will continue to develop in parallel, which will encourage the use of search technology.

Market segments and current players

The market for advanced enterprise search is splitting into three key segments:

• the infrastructure play—the market that Autonomy and Verity must make work if they are to return to the growth rates of previous years;

• the specialist knowledge management role—vendors such as Stratify and Inxight are offering technologies for the analysis and management of large-scale, unstructured data sets. They will have to compete with Autonomy and Verity in that, but they do not yet have their breadth of offering. Opportunities for such vendors will continue, however, and one or two may break into the next tier. For now, they need to establish a strong customer base and some key partnerships if they are to go further. For many, the main goal will be acquisition by a bigger player (from the database or content management market, for example);

• the enterprise search space—the pressures of commodification are pressing down on vendors focused on index-based search solutions. They need to establish themselves fast, and rapidly move upstream in their offering (possibly through the acquisition of specialist vendors). Inktomi is probably the vendor with the best opportunity there at the moment, but it is also the market that Google is targeting, and which it could use as a launch pad into a wider enterprise offering.;

Beefing up the infrastructure play

The market leaders in enterprise search—Autonomy, Verity and Convera—have been extending their operations well beyond the confines of the "search" tag for some time. Other significant providers such as Hummingbird (hummingbird.com) and Open Text (opentext.com) have rolled search into wider portal and collaboration offerings.

In 2001, Autonomy launched its IDOL platform, a key step in putting its technology at the center of the enterprise information architecture--integrating access to unstructured information of all forms (document, applications, images, voice and people). Verity has similar goals for its K2 platform, and an ambitious three-stage approach for taking information management to new levels (in simple terms, from indexing and categorization to "social networking"). Convera (formerly Excalibur) has bet its future on multimedia search, but has found that it was premature in its hopes for that market and now needs to recoup ground in its traditional knowledge management market.

Behind those companies, a host of smaller players are trying to hook into the advanced search and information analysis market. Although it’s a tough market for small players, companies such as Stratify and Mondosoft (mondosoft.com) are showing it is still possible to make an impact.

Gorillas in the mist

The industry giants are not oblivious to the importance of search—although they find it harder to know how to effectively productize impressive work done in their research labs. Microsoft has offered advanced search (and a big improvement on Index Server) as part of its Sharepoint Portal Server, Oracle is offering search as part of the latest version of its portal and SAP (sap.com) has been looking at what search technologies it needs to offer as part of its portal offering. IBM’s (ibm.com) labs have long been a major source of insight into search technology, and some of that has surfaced as part of Lotus’ (lotus.com) Discovery Server, as well as in its Clever project for Web searching.

The Web search vendors

It was already clear in early 2001 that Web search vendors would be forced to look to the enterprise market for future growth as Web advertising revenues dried up. Inktomi and AltaVista now have significant business in that market, FAST’sk enterprise revenues have also expanded quickly, and Ask Jeeves has its Jeeves Solutions arm for enterprise customers.

While growing enterprise revenues have helped those companies, their overall impact on the enterprise search market is still limited. The impact so far has been at the lower-to-middle section of the market, where the focus is on core indexing services.

Enter Google

According to recent research, Google is now a close second to Yahoo (yahoo.com) as the main source of referrals on the Web. Moreover, it has managed to achieve that position without losing its street credibility among serious Web users.

Google’s success is largely due to the fact that it set out to do one very important thing—to help people search the Web—and to do it well. In doing so, it avoided the siren call of the portal market (which led AltaVista off the righteous path of the pure-play search site). It has also successfully maneuvered through the potential pitfalls of search-related advertising, without sullying its valuable brand image.

Now Google is taking on the enterprise search market with Google Appliance. The offering was first launched in January 2002, and an updated version was released in June, along with some initial customer references.

Google’s approach is characteristically different. Rather than simply sell an enterprise software product, it is offering an integrated software and hardware solution starting at $28,000 and available on a try-before-you-buy basis.

Given that Yahoo for your intranet was an effective rallying cry for the portal market, Google for the enterprise would seem an equally attractive proposition. Google’s reputation remains high, and it will be seen by many potential customers as a refreshing new player in the enterprise market.

However, Google faces a number of challenges. First, the software/hardware combination may appear more as a gimmick than a serious, extensible solution to many enterprises. Second, it needs to show that its successful page-rank approach to Web searching works as well in the enterprise as it does on the Internet. Third, general requirements have to be met before entering the enterprise market at a profitable and sustainable level:

• sources and integration—enterprise search is not limited to HTML documents or even Microsoft Office documents—access to legacy sources, databases and business applications are all important. (Notably, Google has recently added access to IBM Lotus Domino repository to its solution.);

• security—enterprise search has much more complex security requirements, and those need to be met without hampering performance;

• application integration—enterprise search is often implemented as part of a wider e-business or knowledge management offering.;

In order to move up the food chain and away from the increasingly commoditized market, Google and other Web search vendors need to take their customers to the next level in their integration of search. They also must add higher value services, such as taxonomy generation and management, and user profiling.

Eric Woods is a research director at Ovum (ovum.com), e-mail exw@ovum.com.

Next-generation search: a definition

Ovum defines next-generation search as:

the technologies and products that are bringing new levels of intelligence, order and personalization to the search process.

Next-generation search technology is about providing more intelligent and more proactive tools to allow users to take control of the vast amounts of information that the networked world makes available. It overlaps with terms such as information retrieval, knowledge discovery and intelligent search, and it encompasses a wide variety of technologies and products.