Rebooting the information refinery

For a decade now, data has been the headline. The bigness of it. The growth of it. The velocity of it. The SQL-ness or the NoSQL-ness of it. The (usually polluted) lake of it. The cloud as a home for it. The ownership of it. The threat from it. Surprisingly, there has been comparatively little attention devoted to the refining of it.

It seems that the mere handling of data—the networks, servers, processing, and storage architectures necessary to keep pace—is where the money is flowing. But at the same time, some smart money has been looking ahead to the challenges of mining value from this data. Innovators are looking beyond compliance requirements for the data to opportunities to transform businesses altogether.

Refined data

In the field of knowledge management, of course, the idea of turning data into information into knowledge has been a foundation concept for knowledge managers. But frankly, the ability to achieve this alchemy of data to knowledge has not been broadly demonstrated in practice. A next generation information refinery is required to make something meaningful and valuable out of the raw data flying around the firm and throughout the internet economy.

The first consideration to appreciate is that cognitive computing applications depend not on raw data but on refined data. The raw data itself is simply too opaque and silent to be able to deliver the value users and organizations are looking for. Alchemy is called for.

Dino Eliopulos, managing director at Earley Information Science, put it this way at KMWorld 2018: If you want a gold ring, you could simply whack rocks together for a long time and hope that a ring will magically emerge from the chips of stone. Or, as our first metallurgist ancestors discovered, you can dream up a process (which we now call smelting) through which the raw stone material is refined to release the kind of ore that you need. The ring makers needed to design a multi-stage, controlled process through which the pure gold could emerge, raw stone having been transformed into a material that now could be made directly into the object desired—the gold ring. In the information economy, we need similar kinds of processes to render information that can be made directly into intelligent business applications.

The idea of the information refinery has been around for decades, but today’s cognitive computing projects are reviving the idea with a new urgency. Let’s look at three examples that show why the refining operations needed for today’s applications require bigger, smarter refineries than those we have attempted to build in the past.