Understanding Search: Seven Critical Questions (and answers)

3. To what degree has regulatory-compliance pressure affected your business? Do your customers consider "search" a critical part of their compliance?

David Bercovich, Google Enterprise

: Regulatory compliance has resulted in search projects that are mind-boggling in scope. The task of retaining e-mails and documents across tens of thousands of employees for years is difficult to imagine. The further task of retrieving a single important document from that archive sends chills down your spine...

Dave Schubmehl, Open Text: Customers are looking for a comprehensive solution that includes collection, monitoring, search, classification AND records retention. They also want technology that can deal with terabytes of unstructured information and deal with it quickly and effectively in a secure environment.

Andy Feit, Verity: Search and classification must be part of a greater business process; what you do with the information is more important than just finding it. This is the domain of business process automation and BPM. Task-based interfaces, tied into the business process, can help users along the way with the information they need for their roles in the process, all proactively.

Bjørn Olstad, Fast Search & Transfer: The time it takes to gather documents for legal discovery without rich full-text analysis can make you miss the deadline.

4. Which is the more effective approach in your opinion: A pre-arranged taxonomy/classification scheme, or a robust full-text search tool? And why?

Jim Baum, Endeca: I spend a lot of time talking to "information scientists" (a more glamorous name for librarians) who were thinking about this problem long before computers showed up on every desk. Guided Navigation is based on something librarians have called "faceted classification" since the 1930s. Essentially, old-style taxonomies put "books" on a single shelf. The faceted approach puts the same book on many shelves.

Alkis Papadopoullos, Convera: There is a common misconception that these two approaches compete, when in fact they must be viewed as complementary.

There are two familiar "use cases" that need to be supported. The first is when the user knows precisely what he is looking for and can express it well in a query. The second involves the need to explore information. "I need to answer the following specific question" versus "I need to obtain information about a topic." Both examples might be used by the same searcher, depending on his task. Directed search and browsing should therefore go hand in hand.

But in the marketplace, these two approaches are contrasted against each other with a quasi-religious fervor.

Andy Feit, Verity: There are times when you just want to see what information assets are out there, or perhaps you aren't sure how to describe what you're looking for in simple search terms. For example, in every company's HR organization there will be common information on, for example, "vacation policies." A search on that term would work just fine.

But, what about special benefits that your company has for health club membership, or discounts at local attractions, etc.? The user would never think to type those terms in. But, a directory—or taxonomy—of HR information, organized by topic, would have interesting categories that employees could browse, often discovering information that is of value.

Bjørn Olstad, Fast Search & Transfer: Clustering can reveal insight in the results that a user can instantly relate to but did not know how to enter the query to get to the results.

David Luzier, SER Solutions: Taxonomies are time-consuming to build and difficult to maintain and change. Ultimately, the goal is to provide users with immediate and relevant information. Since this can be achieved without the need to build and maintain taxonomies, robust, full-text search solutions that search for content, not just keywords, are a much better approach. Dave Schubmehl, Open Text: A good taxonomic scheme provides clues and context about the content that the user is working with and makes searching and analysis more effective by quickly letting users "drill down" to the information they're seeking...or, more importantly in compliance applications, find out if that information is not in the repository.

5. To what degree is "enterprise search" a front-of-mind item in the IT departments of the customers you deal with? How about among the line-of-business people you deal with?

Bjørn Olstad, Fast Search & Transfer: Line-of-business managers are moving quickly to specific solutions where they can define the ROI clearly for their business, whether it's gained from "selling more on the Web site" or from cost-savings.

Steve Kusmer, Atomz: The line-of-business people have the direct responsibility for the results of search, and are the ones who benefit. They need to be more in control of the search application.

David Luzier, SER Solutions: While finding information fast is important to line-of-business personnel, considering what alternatives are available to support this requirement is not front-of-mind for them. The IT organization is responsible for addressing the enterprise search needs of the firm and is therefore the primary driver.

Jim Baum, Endeca: The line-of-business people usually write the search RFPs. They're the visionaries who see how finding and reusing information can transform a business process.

IT sees enterprise search mostly as a plumbing problem. They only worry when they hit scalability or performance walls, or when hardware gets too expensive, or administration becomes too painful.

David Bercovich, Google Enterprise: Historically, enterprise search projects have been complex, resource-intensive and plagued by low adoption rates. We see a fairly radical shift underway as companies realize the incredible value of enterprise search and that—unlike many other IT priorities—the search problem can be solved quickly without breaking the bank or stretching the IT department.

Andy Feit, Verity: The more progressive IT departments are reaching out to their constituents, defining standards for search tools and corporate-wide taxonomies and ensuring solutions fit with the overall enterprise application architecture (e.g., factors such as security or Web services).

Dave Schubmehl, Open Text: IT departments are recognizing that there is a qualitative and quantitative difference between the search technologies needed to index and organize simple applications like corporate Web sites and the technologies needed to find, index, federate, organize and extract information from all of an organization's repositories, shared drives, corporate e-mail servers and databases.

6. What is the next challenge for the search-vendor community? Could be anything technologically you're looking at downstream (Messaging? Voice? Video?) or it could be market and economically related...

Bjørn Olstad, Fast Search & Transfer: The next challenge is the mobile user, and the need to apply "contextual insight." For example, if the query is "BBQ" from a mobile phone, the search needs to remove BBQ restaurants that are too far away, or are closed at this particular hour. It then needs to know the type of phone, so the results are presented in the best possible way.

Alkis Papadopoullos, Convera: There will be a very strong push for search to become a more integral part of the sharing and disseminating of information, and search applied to business intelligence to provide directed background data and search "reporting tools" to support and improve the decision-making process.

Also, ontologies, taxonomies and semantic networks will take on a greater role in helping to push relevant content to communities of interest, teams and business units ...in essence, push will start to take on much more importance than the current pull-based model.

Steve Kusmer, Atomz: The integration of Web analytics with site search and Web content management applications is fertile ground for innovation. It's a new era, where search and analytics combine to optimize the visitor experience.

Jim Baum, Endeca: These are exciting times—we're seeing the erosion of the long-standing walls between databases and other content repositories. This makes it possible for users to find content that was previously siloed in different enterprise systems.

Although that technological problem has been elegantly solved, there's a big lag in designing applications that unlock the full potential. Part of this lag is due to politics—the people who own the silos don't write RFPs for new systems to cross them.

But part of this lag is because people are still dreaming up new ways to use the technology. Literally every week, customers show me visionary new applications they have built on our platform—applications that are still unnamed that creatively solve business problems that were never addressed before.

7. Tell me what motivates your company the most about the enterprise search market and the opportunities it represents... in short, what's so hot about search?

Bjørn Olstad, Fast Search & Transfer: Search is evolving to deliver contextual results. A decade ago, people would say, "why doesn't the computer do what I mean versus what I say?" We are getting close to that now.

Andy Feit, Verity: This market continues to offer room for innovation. Other software segments are much more static. Databases, for example: How much has really changed in how we do this? Sure, it's become more scalable, or added a few new operators, but fundamentally it hasn't really changed much, and probably doesn't need to.

Alkis Papadopoullos, Convera: There are few things as satisfying as realizing that you have just shaved off 50% of effort to write a report because you located previous work done on the topic...or ensuring that a market analysis report includes detailed competitive information because you were able to access resources and content...or suddenly realizing that two seemingly unrelated concepts were in fact interacting in a very unexpected way that is critical to your understanding...

Dave Schubmehl, Open Text: Search, classification and related technologies are the glue that holds ECM systems together. The ability to effectively find and utilize that key piece of information is how ECM systems deliver their capabilities.

Jim Baum, Endeca: People STILL can't find what they're looking for! Every time I'm at a social gathering, when people ask what I do and I mention search, the floodgates just open and they pour out stories about information overload. What's so exciting about my job is that we have a genuinely new, elegant way for people to find things.

Search is in just the first few innings of the ballgame. I can already see part of how the next few innings will play out by looking in our own labs, but there's still so much opportunity, so much room for improvement, so much of the game left to play.

---

Andy Moore is a 25-year publishing professional, editor and writer who concentrates on business process improvement through document and content management. As a publication editor, Moore most recently was editor-in-chief and co-publisher of KMWorld Magazine. He is now publisher of KMWorld Magazine and its related online publications. As Editorial Director for the Specialty Publishing Group, Moore acts as chair for the "KMWorld Best Practices White Papers" and the "EContent Leadership" series, overseeing editorial content, conducting market research and writing the opening essays for each of the white papers in the series.

Moore has been fortunate enough to cover emerging areas of applied technology for much of his career, ranging from telecom and networking through to information management. In this role, he has been pleased to witness first-hand the decade's most significant business and organizational revolution: the drive to leverage organizational knowledge assets (documents, records, information and object repositories) to improve performance and improve lives.

Moore is based in Camden, Maine, and can be reached at andy_moore@verizon.net.