Records management rides again
Way back when, in the earliest days of electronic document technology, there was no workflow or business process re-engineering, no document management, no World Wide Web, certainly no knowledge management. Few people even had PCs on their desk. It was amazing that companies managed to get anything done at all, but somehow they muddled through.
Out of that primordial fog came imaging, an incredible new technology developed to revolutionize the world of records management. The world of what? You know, records! Those huge rooms of color-coded paper files, sometimes taking up whole warehouses, filled with frazzled file clerks searching for folders that were always somehow hopelessly lost. Fantastic tales abounded of buildings that had to be structurally reinforced just to support the weight of all those file cabinets. It was a fearful time.
I was a hardware guy at Wang then. We had discovered a way to robotically select microfilm cassettes using a contraption originally designed to fill jelly doughnuts, and index each frame of microfilm in a database. With a single keystroke, the doughnut machine would whirl, a cassette would spin and the selected frame would be scanned and digitized. Any record could be called up on the screen in a flash, and they never got lost! Sure it was expensive, but anything that could make life easier for records managers was worth it. We weren't sure exactly what records managers did, but we knew their work was important, since they spent the big bucks.
Over time, of course, it turned out we were mistaken, at least about the big bucks part. Once the few customers with floor-busting file rooms were satisfied, it seemed that most companies' technology acquisition budget for records management ranked somewhere below that for the bowling team. The real customer demand for imaging, and later for workflow and document management technology, came from the lines of business that needed to process customer requests faster and more efficiently, control and revise internal documents, collaborate across the enterprise and publish information easily. Those applications promised line VPs a financial return on investment, while records management was ... well, just a legal obligation of the company as a whole.
Besides, rapid advances in technology made the records management problem look to vendors like a solved problem. Records managers had evolved elaborate and tightly controlled indexing schemes, which they weren't particularly looking to automate. With imaging, they usually seemed to know which document they were looking for and exactly how to find it, without recourse to content retrieval, relevancy ranking, spiders and bots, and the other knowledge management exotica now in fashion. All they needed was a controlled-access archive, a standalone application. In local government and education markets where paper records ruled, companies like Laserfiche quietly prospered with image-oriented records management. But among the Fortune 1000 and the leading document systems vendors that serve them, records management soon drifted off the radar screen.
But now something weird is happening. Records management is coming back as a strategic issue in business, and as a technological challenge. What's driving it is no longer efficient storage of the paper mountain, but managing internal electronic documents as corporate records. It's a question most companies don't even think about, and they're putting themselves at risk as a result. At Cohasset Associates' recent Managing Electronic Records conference, that point was made quite dramatically by executives from Prudential Insurance, who related the tale of how the company was socked with a $1 million fine by an irate federal judge when it couldn't produce internal documents in a class action suit. In fact, it appeared that some documents had been willfully destroyed by a Prudential employee, but that did not cause the fine. What made the judge mad was that the company had no policy--not to mention any system--for managing internal electronic documents as corporate records, the principal records on the conduct of its business.
Well, of course not! Who does? And what is a record anyway?
As I learned at the conference, the term means recorded information in any form that is created, received or maintained in the transaction of business, and kept as evidence. That means that many of the Word files, E-mails and Web postings spread around your company, if they are created, received or maintained in the normal transaction of your company's business, are records, and you could be obliged to produce them in litigation, an audit, regulatory hearing, etc. You need a company policy that defines which documents--now on users' hard drives and E-mail servers--are corporate records owned by the company and how they need to be maintained.
Well, you say, I have an enterprise document management system. Doesn't that do the trick? Think again, Jack. In fact, records managers roll their eyes at the mere suggestion that EDM software gives them what they need. In practice, the three key features needed for records management are not adequately implemented in standard EDM software: record selection, classification and retention. Record selection means not all documents are records. In one click a user must be able to say yes, that is a record. Classification means assigning an official category to the record, selecting from a list created and controlled by the company's records management staff. While EDM offers elaborate indexing and advanced query, simple classification isn't there.
Retention means a date, controlled both by policy and intervening events (such as litigation), until which a record must be maintained and after which the record is to be destroyed. Records managers are manic about the destruction part. EDM software just erases the index entry to the document at the end of the retention period, or perhaps migrates it to secondary storage, but that's not enough.
Records managers want special features that hook into the file system and physically overwrite the disk sectors where the record was formerly stored. (Ironically, that is more easily done with magnetic media than with WORM, once believed the only safe medium for records because it was non-erasable.)
There are a few products, like ForeMost from Provenance Systems, that provide this functionality as an application layer on top of EDM software like Documentum, Open Text, or FileNet, but as businesses are increasingly driven by electronic documents, electronically communicated and stored, we can expect to see records management functionality built into the core of document systems.