Inverting the Corporate Information Pyramid
User Driven Information Discovery

The information chain is a pyramid

In past decades, information-intensive businesses profited from dropping storage costs. Computer systems offered simple and inexpensive data retention—in databases, document repositories and email archives. Within the scope of their design, these systems succeeded, creating massive amounts of digital information typically stored in application "silos."

The competitive landscape has changed. Return on intellectual capital drives success at least as much as transactional growth does, and infrastructures that served information storage well enough in the past fail miserably at the task of exploiting the enterprise's information. Companies also face new risks, such as compliance laws demanding higher levels of visibility. Information access has become strategic: In the knowledge-based economy, companies win by providing effective access to information.

Consider the traditional enterprise information infrastructure: an information value chain with production at the low end, consumption at the high end. The levels in the chain create a pyramid that widens from top to bottom, its width at any point corresponding to investment and use at that level. In most enterprises, expensive RDBMS storage and legacy architectures make this pyramid bottom-heavy. We believe there is an innovative way to turn the cost structure of this pyramid on its head.

Inverting the Pyramid
To quote the CIO of a large, multinational enterprise: "We are world champions at producing information. Finding what we need is an entirely different matter."

At the bottom of the pyramid are data sources, which in this enterprise include corporate, regional and local documents, in five languages and 220 formats. To manage this information and its associated applications, the enterprise has large teams of database architects, programmers and managers, plenty of software licenses and numerous high-end servers.

Further up are tools and people who organize and refine, people who provide access for knowledge workers. Librarians, application managers and user support spend man-year upon man-year building taxonomies, applying metadata, controlling access and developing new information services.

At the top of the chain are the information consumers: employees, analysts, management and customers looking for product information.

Information access in RDBMS and legacy systems is inflexible, costly and slow. Metadata maintenance is complex, expensive and imprecise; new information services are major undertakings; the system architecture is scattered and expensive to maintain; coordination of work is non-trivial. There is duplication of work; there is duplication of content; and still, important information gets lost. The information is often untimely.

If we introduce an information architecture based on a modern search platform, our bottom-heavy pyramid changes dramatically. A good search platform turns it on its head, and thereby improves information access, reduces total cost of ownership and increases overall enterprise performance.

Immediate Information Discovery
The traditional pyramid grew out of a focus on transactions. There was less data, and what there was of it was mostly structured. Times have changed: Information consumers understand the value of accurate, timely information and know how to use it. Better service affects their performance directly. Better information access is now a strategic requirement. This new "immediate information discovery" model, with intuitive search front ends, relevant results, sub-second performance and advanced tools to filter and refine the answers, drives many new competitive business models.

The immediate information discovery model addresses needs throughout the value chain. In our example enterprise, the management wants to grow the business, so new revenue streams and low TCO are important. IT managers like the innovation potential it affords their services; product managers love the increased user traffic. Existing users relish new ways to access and use information: Contextually sensitive search and navigation, user-specific relevancy ranking, spell checking, search for similar documents...all combine to create new ways to tap company information. New customers are attracted by new search-based services: vertically focused subject search, topic maps, update alerts and search for experts. These capabilities open new revenue channels, increase the number of visitors, enhance user satisfaction and improve business performance.