Lost knowledge: file not found

Last week, I spent several hours in my father's library, which consists of thousands of books as well as print material relating to the history of photography. The material goes back to the early 1800s. As I was reading through documents, journals and newspaper articles, most from the mid-1800s, I thought back to the challenge I faced a week earlier when I tried to locate a document from circa 1988. I easily found the documents I was looking for in my father's library. However, my own document, from a mere 18 years ago, was not accessible.

I did, however, come close. I eventually found two places where my desired document existed: One was on a network backup tape for a file server long gone; the other was on a 5.25-in. floppy diskette. (Think about it. In order to read the backup tape, we need the backup tape drive, software for the drive, hardware and an operating system compatible with the drive and its software. Forget about finding a working computer with a 5.25-in. drive. Come to think of it, does anyone even have a 3.5-in. floppy drive nowadays?)

Although I was able to read documents from 150 years ago with ease, the same could not be said for documents one-tenth that age. Despite having the media that held the files, the hardware (and software) required to access the files was nowhere to be found.

Consider now how much digital content we are creating every day; some estimate it takes the world only 15 minutes to create a quantity of information equivalent to that currently in the Library of Congress. Less and less information is in analog form, such as that created with pen and paper. The proliferation of digital data is a sure sign that we are getting deeper and deeper into the knowledge economy. But have we made any effort to ensure that this information will be available 200 years from now, or better yet in 20 years? As more and more enterprise content exists solely in the digital domain, enterprise knowledge managers are starting to face this paradox in their daily lives.

Certainly someone, somewhere must be doing something about this, you, dear reader, must be wondering. Slowly, but surely, someone is.

The Library of Congress launched its Digital Preservation program in 1998 to assess the current state of digital archiving and preservation. In 2000, the United States Congress appropriated about $100 million for the National Digital Information Infrastructure and Preservation Program (NDIIPP), a collaborative effort led by the Library of Congress, whose mission is to "make its resources available and useful … and preserve a universal collection of knowledge and creativity for future generations." The Library will make its final report to Congress under the program in 2010.

But enterprises shouldn't--and can't--wait that long. Every industry faces the compatibility conundrum. One way of addressing the problem, while NDIIPP is figuring out the long-term solution, is to ensure that information is preserved in a format as generic as possible and that knowledge retention audits are routinely performed to ensure that information is not lost. People move frequently from job to job, both within an organization and to new companies. Products can span decades and have life cycles that outlast several generations of employees. The NASA space shuttle, perhaps one of the most complex engineering projects in modern times, started in 1981 and is scheduled to continue at least through 2009. Engineering drawings of earlier space boosters are gone. Any attempt to bring back such equipment for alternate space exploration purposes means literally reinventing the wheel.

Retaining knowledge isn't always at the forefront of a manager's list of priorities; it has little short-term payback and can be expensive and labor-intensive, and there is no guarantee that a company won't experience a loss of information over time.

For now, there is only one sure answer for critical knowledge, and it's almost antithetical to how we work: Maintain this information in analog form. While we sort all this out, at least we'll know that knowledge workers of the future will not have to depend on digital system upgrades (or retrogrades) and transitions that took place in the intervening years since the information was created, because the ability to view and retrieve such information will always be available.