Records management: an expanding role

Keeping good records can also aid a company’s competitive position. “Some companies have used their records to justify their patents,” Sigal explains. “For example, if they have design documents, meeting minutes and other evidence, they can prove that they were the first to market. But they need to be able to produce the information quickly to the lawyers and courts.”

Big data and records management

Big data is associated with consuming and analyzing rapidly incoming data such as clickstream or customer information, but the issue of managing big data, especially internal data, has not received as much attention. “Users have learned the hard way that big data management is very difficult, most obviously because of the volume,” says Kon Leong, CEO of ZL Technologies. “This is an incoming tsunami and is only going to get worse.”

To give an idea of the order of magnitude, Leong cites the U.S. Library of Congress, which has 100 million documents, as a baseline. “In one of our large deployments, the document count is already at 12 billion, with document ingestion rates peaking at 50 million per day,” Leong says.

This explosion is partly because the fundamental definition of “record” has been changing, according to Leong. “The old guideline of ‘not relevant,’ is being revamped,” he explains. “Now, even if the relevance of the content itself is not apparent, the existence of the document may be. For example, enough personal e-mails sent by someone at work can be an indication of unproductive work practices, or even questionable activities.”

ZL Technologies’ Unified Archive product offers the Records Manager component as well as Storage, Compliance and E-Discovery Manager modules. Through a unified architecture, ZL Technologies provides the ability to store each document just once and have it accessible for multiple purposes. “E-mail is one of the worst offenders,” Leong says, “because with one click, you can send out thousands of copies. Shared file servers are another problem area. However, with the Unified Archive, one and only one copy can be stored, and the metadata provides all relevant information such as who received it and when.”

The Unified Archive uses a grid-based architecture that virtualizes computing and storage resources and supports the management of large volumes of unstructured data. It allows multiple functions such as archiving, search, compliance, e-discovery and records management to be handled within one system environment.

Modernizing RM

The amount of internal data often dominates any that is coming in from sources more typically considered big data. “The e-mails, word documents, spreadsheets and presentations also constitute big data, and need to be accounted for,” Leong says. “Not only do they represent valuable content, but they reflect materials created by one human for another. If you can use data analytics to harness the information associated with these interactions, you can use it to strategic advantage. It changes the old paradigm of narrow usage for legal, compliance and records.”

Leong expects pushback from the potential intrusiveness of analytics based on internal records. “It could be Big Brother times 10,” he says. “But like any powerful invention or discovery, such as fire, this technology can create or destroy, but walking away from it does not make sense. We simply have to learn to use it with judgment.”

The next step in modernizing records management will be to use metadata to drive retention more automatically. “The vast amount of office information is left to the user to file and declare, and quality assurance often shows that the accuracy is low,” Forrester’s McKinnon says. “Some form of automated categorization would improve records capture and thereby support better information governance.”

Archive vendors have moved forward with categorization more quickly than straight ECM and records management vendors, McKinnon adds. “They are keeping up with the different channels that can produce records, regardless of whether it is e-mail, chat, social media or other content,” she says. “Connectors to new content sources help separate the relevant content from the container.”