Intelligent Search—Making the Most of Metadata
Search is a conversation. If you ask me a question and I don't understand, I can ask you for more information. With time you learn more about my interests, and can give better answers. Well-designed, intelligent search systems can do the same. We can facilitate this dialog by addressing three critical requirements for effective search. These are:
1. Search needs to feel like navigation;
2. Search needs to be personal; and
3. Search needs to be adaptive, improving over time.
1. Search needs to feel like navigation. People find answers through haphazard and chaotic processes. Are you a searcher or a browser? It really depends. Most people shift perspectives between the two modes. We search when we know what we want and are trying to retrieve something. We browse when we don't know what we want and need to discover knowledge. Navigational structures can teach us about available content, but we tend to shift back and forth between retrieval and discovery.
Most consumers are now very familiar with faceted navigation. Search terms are part of the left navigation. When you click on a particular size, color or brand, the system executes a query. A relevant result set is returned. The user can also see how many documents that contain each term are in the collection. Search terms that make no sense or that would lead to zero results fall off the list, so users will not go down blind alleys.
Consumers are also employees. And as such, in the context of their work environments, they now expect that internal portals and knowledge management applications to use similar techniques. To meet this demand, progressive organizations are moving quickly to organize and tag their content to enable faceted navigation in high-value application areas.
Faceted navigation requires well-constructed taxonomies and clean metadata in order to function correctly. Even plain white box search is about metadata, even if documents do not contain explicit metadata. In that case, the search engine derives information about term occurrences in documents and places that in a search index. The index holds metadata associated with the documents.
2. Search needs to be personal. Search algorithms are getting better, but they cannot infer human context and intent. Though Google is beginning to do this with Web search based on your search history, this functionality is limited—especially with site search and intranet search. Search results may be relevant to some people and not relevant to others. The appropriateness of a set of results depends on the perspective, education, experience and intent of the user. For someone searching on "financial planning," results that would be fine for a recent college graduate beginning their investment planning would likely not be sophisticated enough for a seasoned investor. Again, the more we know about the user, the better we can tailor appropriate results. If users self-identify, I can use passive personalization techniques to target their result set. This means tagging the content with appropriate terms that match what we know about that user demographic or persona.
3. Search needs to be adaptive. If I know something about my users, I can anticipate what they need. By leveraging metadata and the structure of content, a search solution can present relevant choices to the user. As an example, consider the scenario of a salesperson following up on a marketing campaign. The salesperson can select "lead generation," using a step in the sales process to communicate a context for search. An intelligent search application can then execute a query to present all artifacts that support lead generation. These might include case studies, white papers, calling scripts, email templates, etc. The application is leveraging metadata and querying the content management system to surface this information. The user does not have to think about structuring a complex query or framing multiple search requests; the work is done by the search application.
By capturing search history and the value of search results to users in similar contexts, we can refine search to return more highly relevant results. Value can be measured by looking at the number of document opens and downloads, or by integrating social media techniques, such as user ratings. When search relevance improves over time, users get more value.
The Critical Role of Metadata
Enterprises that want to deliver a more efficient search experience need to think through content attributes (metadata) if they are to address the requirements for intelligent search. This requires starting with an information architecture exercise—understanding users, their tasks and the types of content they are interested in, and then tailoring the results returned by a search engine or presented in the UI to those user needs.
Search has evolved dramatically in the last few years. Web site search is more about information access and user experience than it is about a white box in the upper right corner of a page. Rather than thinking about search as a utility, something that is bolted onto an application, search needs to be approached as an application—an application that leverages, and in fact depends on, good content architecture and clean content repositories.
Most organizations have poor "content hygiene." They do not curate content; they have no content owners; content is not cleaned up; and tagging is haphazard. Content structure is not considered in the context of user needs and tasks and old out-of-date content clogs up the system. In fact, when organizations complain that search is broken, and install new search engines without solving the problem, they find, in the end, that the problem lies with their content!
Founded in 1994, Earley & Associates, www.earley.com, approaches information management solutions by taking into account how people, technology and content relate. We realize that focusing on specific business contexts and goals drives the success of an information management project. We work with companies in a broad range of industries including retail, high-tech, life sciences, manufacturing, non-profit and government. Our corporate headquarters is in Stow, Massachusetts, and our consultants work on projects around the globe.