Content intelligence 101

If there has been one constant in enterprise software over the past 4 decades, it is that good data is the basis of process automation, analytics, and optimization. 

While data is necessary to enterprise automation systems and foundational for computer-based decision making, there’s an assumption (or bias) that companies can only automate with data. With digital transformation has come the realization that many forms of automation rely on more than that—especially in applications that assist knowledge workers in applying their skill and experience to make decisions.

As digital initiatives focus more on experience, personalization, and conversant interactions, a new dimension of automation is emerging that goes beyond just data. In these applications, decision-making relies on applying specialized domain knowledge while contextualizing, interpreting, and acting on unstructured content—namely documents and messages.

While data extraction from content for workflow and task automation has been accomplished for decades by optical character recognition (OCR) technologies, the new challenge is automating decision-making from experienced knowledge workers that brings context and trained skills to content. This is where artificial intelligence (AI) has made its greatest impact in the past decade. Companies are turning to AI with machine learning (ML) to transform and process content, allowing humans and AI to work together as decision makers and evolve from simple data-driven task automation to AI-driven insight, decisions, and actions.

However, there is still the challenge of the "confirmation bias of data" when it comes to content. Leaders charged with digital transformation need to know how to overcome the data-only bias and leverage AI to get more out of their content in serving customers, preventing fraud, ensuring compliance, and gaining better financial outcomes. 

The confirmation bias in enterprise software

Data is necessary for systems of record, ranging from ERP, CRM, SCM, RPA, and OCR to analytics and BI, but it only yields one dimension of information. The assumption that data is the only thing needed for digital solutions is increasingly problematic considering the advances and adoption of AI with ML. This obsession with only focusing on data has become the confirmation bias of enterprise software and many digital transformation initiatives.

Confirmation bias is the tendency for people to search for, interpret, and favor information that confirms or supports an existing belief. If knowledge workers are only looking for data, that is all they will get, and software tools will be geared towards that end. But if companies want workers to make better use of information when serving customers, making decisions, or identifying potentially suspicious activity, they need a trained, experienced reading and understanding of their content that connects data with context as experienced knowledge workers do.

Alan Turing, considered to be the inventor of modern computing, recognized the dilemma from the outset: Can machines think and make computations faster than humans? Yes. But can machines think like humans? Humans, he suggested, have uniquely complex, experience-based forms of reasoning and intelligence that machines cannot replicate. Yet technologists continue to operate as if better data architectures can somehow replicate human intelligence.  

One only needs to look at processes that include variable content and human interaction to see the limitations of data-only approaches. Take invoice and payment processing, for example. Most content can be digitized, yet according to Levvel Research, 65% of invoice processing is still manual, arriving on paper or email and printed out for processing. The data included in tables within invoices is highly variable and can be very specific, meaning they include several layouts that simply cannot be templated for data-only approaches. Typically, accounting staff, even with the help of OCR tools and RPA, spend a significant amount of time collecting invoices and entering field data from them into systems. This approach relies too heavily on human intervention to handle the complex variability and exceptions among invoices, making straight-through processing and strategic initiatives, such as e-invoicing and Procure-to-Pay, continually elusive to most enterprises. 

Automation omits intelligence

So often, digitization surrounds the decision process with data collection without assisting intelligent decision making; a couple of common use cases include using bots to collect invoices from an inbox and routing them or verifying data with the ERP system. Similarly, OCR has been used in a scan-to-archive role, while invoices are processed by accounting staff. Then data is entered from on-screen or printed images into systems that consume them. In many cases, the process contributes to the problem because it’s tailored to data entry instead of assisting the skilled accounting staff in getting the job done. Many organizations call this ‘swivel-chair automation’ because data entry requires toggling between multiple applications and screens.

Advances in AI and ML are finally expanding the scope of enterprise computing. It’s bringing a new focus to experience, engagement, and bot-assisted problem-solving as layers on top of data. While digital transformation has made significant advances in customer engagement, content-heavy processes supporting knowledge work often remain locked in simple data extraction and workflow-driven processes. The opportunity with AI and ML is to transform those processes, such as accounts payable, to work directly with the skilled professionals who depend on them. This involves re-thinking processes from merely being data entry to becoming digital assistants with AI. This can help extend human intelligence into processing content, thereby expanding their effectiveness to handle decision-making while spotting suspicious activity and maintaining compliance assurance.

AI and ML can “see” things the “human API” cannot. This opportunity has prompted everyone—from RPA vendors and startup OCR vendors to established tech giants—to jump into the fray with AI-enabled solutions. But most have targeted AI/ML at data field extraction rather than content intelligence. It needs to go further to assist knowledge workers and connect the human dimension with the content for better processing and more by being both process-aware and entity-aware.

Data versus content

There is a clear distinction between data and content. Data involves abstracting information for the purpose of computing; content is a representation of human thought and is an analog communication form. Historically, the interface between humans, content, and computing has been data entry. OCR has been established for decades as a tool for entering data from content into computer systems and their application workflows. Companies can now capture unthinkable amounts of data—both structured and unstructured—that inundates businesses on a day-to-day basis. But the volume of data entered is nearly insignificant compared to the volume of content used to conduct business. Gartner research has stated that 80% of enterprise information is unstructured, often in the form of documents and other forms of content. Customer service is a prime example, where decisions on opening a checking account, underwriting a loan, settling an insurance claim, or paying an invoice are all made by experienced, trained workers who apply their skill to make judgments about the request or problem.