The role of information retrieval in knowledge management

The growing KM market should provide a welcome boost for the major players in the information retrieval space--text indexing and retrieval vendors like Verity (www.verity.com), Fulcrum (www.fulcrum.com), Dataware (www.dataware.com) and Excalibur (www.excalib.com). KM will also provide opportunity for Microsoft's (www.microsoft.com) new search technology bundled in Site Server. In fact, Microsoft's entrance into the searching and KM space helps legitimize the value (and market potential) of information retrieval.

Clearly, information retrieval vendors provide a key piece of the KM puzzle, and are uniquely positioned to capitalize on the KM trend. Consider:

• Information retrieval systems solve one of the biggest problems of KM: quickly finding useful information within massive data stores and ranking the results by relevance.

• Information retrieval can provide organizations with immediate value--while it's important to try to figure out ways to capture tacit knowledge, information retrieval provides a means to get at information that already exists in electronic formats.

• Information retrieval products are maturing beyond just searching, and now provide capabilities for KM functions such as information dissemination.

• Information retrieval systems are taking advantage of Web technologies and providing browser-based front ends.

• Information retrieval systems are already established in the marketplace--vendors have existing install bases and established revenue streams, which put them in a better position than small startups that are introducing completely new concepts and technologies.

Access information from any data store

The information most organizations use is located in a variety of different data stores. Those valuable repositories may include file servers, groupware systems, relational databases, legacy systems and even external sources such as the Web. Text indexing and retrieval systems can index information in those data stores and allow users to search against it.

Thus, retrieval systems give users online access to information that they might not know about, and they don't have to know or care where the information is located. With a single search, users can query all information that the administrator has seen fit to index.

In that way, companies can essentially "capture" information in any location and give users access to it. The value of that becomes clear as you begin to move text retrieval from a departmental deployment to an enterprise deployment. It's nice to be able to search information within your department, but the real value comes when the information can be shared throughout the organization.

The ability to index diverse data stores is hardly trivial. The free Web search engines can only search HTML. Products that include bundled searching capabilities can only search within their own systems. File systems such as Windows NT and Novell can search their own file stores, but they do not index the content, so searching is slow. Only advanced information retrieval systems provide the ability to simultaneously search indexes that have been built from different types of repositories.

While most information retrieval products can index a few different data stores, Fulcrum seems to support the widest range of repository types. Excalibur is leading the way in indexing and searching files types such as images, video and other specialized media formats--for example, users can search an image repository for images that resemble a particular sample.

Because so much information already exists in digital form, information retrieval can provide organizations with value right away. Granted, the KM goal includes sharing information that resides in people's heads by allowing users to actively contribute to the group memory--but that usually requires a change in business practices. In the meantime, there is value in giving a broad user base access to a broad information base. Of course, that technology can later be integrated into a more evolved KM practice.

Reducing the noise

We've all experienced the frustration that comes from searching the Web and receiving a results list of over 500,000 hits. You're not much closer to finding the information you wanted, and you may even give up on the process.

Information retrieval systems include technology designed to help reduce the noise and arrive at more precise results. Most systems provide advanced searching capabilities that allow users to create complex and sophisticated queries, and many systems provide behind-the-scenes functionality to improve precision. For example, Fulcrum and Verity can display document summaries in the result list, so users can quickly determine if the document meets their needs before downloading it. Verity offers the ability to cluster results into categories based on common themes, displaying documents in related groups.

Moving beyond search

As information retrieval systems mature, they are incorporating new features that address other areas of KM. No longer just limited to searching, they are addressing issues such as delivery, helping users find expertise instead of just data, and helping organizations learn from the types of queries that are being submitted by users.

For example, Verity and Fulcrum offer agent technology that allows users to create queries for information they're interested in, and to add the queries to their user profiles. The server executes the query on a schedule and automatically delivers the results to the user. Thus, users can receive information without having to actively search for it.

Dataware provides technology that helps users locate experts or specialists on particular subjects, instead of just locating information. The company's Knowledge Management Suite can store employee contact information or link to an existing LDAP directory, and can return contact information for particular employees as part of a query result list.

Microsoft's Site Server now includes the company's Index Server indexing and searching engine. In addition, Site Server can perform usage analysis on the queries that users have submitted. Administrators can understand how particular users employ the system by seeing the kinds of queries that users submit. Thus, indexes can be restructured or relationships created to help users get to the information they need more efficiently.

A key piece of a bigger puzzle

Information retrieval is a tool that can help you move toward your KM goals, but it's not full-blown, out-of-the-box KM. It still lacks key pieces of the KM puzzle, such as collaboration and the ability for users to contribute information to the corporate memory.

Still, information retrieval provides a key component that any KM system can leverage--broad-based access to diverse data stores, along with features that make it easier for your users to find and receive information that's relevant to them. For a lot of big corporations, that's something to get excited about.