Understanding knowledge and information architecture for GenAI at KMWorld 2024

Artificial intelligence (AI) is revolutionizing the enterprise, with the potential of generative AI (GenAI) at the forefront. However, many organizations struggle to harness GenAI’s full potential. The key lies in retrieving information from corporate sources, an application known as retrieval-augmented generation (RAG).

At KMWorld 2024, Seth Earley, CEO, Earley Information Science and Author, The AI Powered Enterprise; Sanjay Mehta, Earley Information Science; and Heather Eisenbraun, knowledge architect, Earley Information Science held a workshop discussing “Knowledge & Information Architecture for GenAI.” This interactive workshop covered essential principles of LLM applications, including ChatGPT, various LLM types, and the critical role of taxonomies, metadata, and ontologies.

The success of RAG hinges on the ability to search and retrieve accurate information. The success of search and retrieval depends on well-structured and organized data curated to provide a reliable ground truth for the algorithm.

The experts demonstrated innovative approaches to enhance data quality and completeness using Large Language Models (LLMs). There are a variety of LLMs to choose from including Hugging-Face, LLaMa, and more.

“When I first saw ChatGPT I had an existential crisis. I wondered if my work would become obsolete. But that’s not true,” Earley said. “There’s no AI without information architecture.”

GenAI is making it more difficult for organizations to focus on the foundation of core problems, he explained. It’s good at accessing your own information but you cannot retrieve things at a level of nuance without structure.

“You need some curation, you need some process,” Earley said.

AI cannot handle highly complex, open-ended tasks or replace humans in roles requiring emotional intelligence, creativity, and high-level reasoning.

“It’s always in support of human intelligence,” Early said.

GenAI is trained on content to learn the patterns and relationships characteristic of the data. It must have the information to solve the problem. It will take learned knowledge and generate new content with what you program into it. If you don’t have the answer from this data source, say you don’t know, Early said.

RAG allows the LLM to use corporate knowledge sources when answering questions. It can be constrained to eliminate/significantly reduce the likelihood of hallucinations—incorrect information.

“Every step to a process is a knowledge journey,” Earley said.

Use cases can act as a prompt to inform the knowledge architecture. Libraries of use cases allow for testing functionality, facet identification, and baseline metric comparison. This can be training material for how users need to ask questions. The knowledge architecture provides clues as to how to ask questions to get content in context.

The information architecture (IA) development process includes:

• Observe and gather data (pain) points
• Summarize into themes
• Translate themes into conceptual solutions
• Develop scenarios that comprise solutions
• Identify audiences who are impacted by scenarios
• Articulate tasks that audiences execute in scenarios
• Build detailed use cases around tasks and audiences
• Identify content needed by audiences in specific use cases
• Develop organizing principles for content (taxonomy development)

KMWorld returned to the J.W. Marriott in Washington D.C. on November 19-21, with pre-conference workshops held on November 18.

KMWorld 2024 is a part of a unique program of five co-located conferences, which also includes Enterprise Search & Discovery, Enterprise AI World, Taxonomy Boot Camp, and Text Analytics Forum.