Optimizing mass storage to improve business processes

As organizations adopt business processes that focus on data-intensive applications such as data warehousing and knowledge management, strategic planning for mass storage systems becomes essential. Although the obvious requirement for a storage system is the ability to store data effectively, when the data in the system is mission-critical, new storage requirements arise.

Typically, organizations with various functional departments implement numerous storage systems according to the respective application needs of each department. While that approach successfully meets the needs of each unit, it does little to streamline the overall business objectives of the organization. When an organization implements data warehousing or knowledge management applications, it must address new requirements surrounding the storage system.

Beyond storage, there are two key requirements for systems that are designed for data-intensive applications. First, the various types of data in a system must be stored on media that best suits the priorities of the myriad users that will access the system. Second, the system must have the ability to manage data based on the various phases of its ever changing life cycle. As data moves through its life cycle, it becomes pertinent to different types of users. The system must ensure that the data is always stored on the media type that suits the needs of the user.

To fulfill the key storage requirements of data-intensive business processes, organizations must implement an adaptive storage system strategy. With such a design, all the storage media types within an organization are part of a single system. That adaptive system recognizes the application needs of each department and the respective media types on which to store data to best meet those application needs. Furthermore, as the data moves through the various phases in its life cycle, the system recognizes that it may no longer be pertinent to a particular application. The system then migrates the data to a more appropriate media type and the data is used for a new application.

Of course, cost must also be taken into account. The cost is different for the various media types and should be weighed against the benefits that the media type provides. Typically, high performance means high cost, while stable archival with low performance means low cost.

Application needs determine storage type

The various types of storage media on the market today are highly differentiated in their functions and costs. Some types boast round-the-clock availability, while others offer high performance. And, of course, some systems are attractive simply because they offer a low-cost solution for storing massive amounts of data. Within a single organization, there is typically a need to address more than one of those storage functions and thus also a need for numerous media types.

Consider, for example, users from two departments in an investment firm: customer service representatives and financial analysts. The two types of users have distinct application needs. The stored data must meet those needs for them to perform well at their individual jobs, as well as for the firm, as a whole, to be successful.

The priority for customer service representatives is performance. Such users need a system that can access and retrieve customer-specific information within seconds of the customer's request. In that application scenario, performance takes a higher priority than cost per unit of storage. With such an application type, a RAID system at the server and intelligent caching at the client hard drive is the optimal storage media type and configuration.

The financial analysts, on the other hand, have different priorities. Their main applications involve performing trends analysis and sales analysis that affect investment decisions. Although to a certain degree such applications have performance requirements, the true datatype priority of analysts is availability. Financial analysts depend on the availability of data; without it, they cannot perform their job. Within a storage system, the key to availability is redundancy. Redundancy can be achieved with various media types. For example, analysts might prefer RAID as the primary storage system because it meets performance as well as availability requirements. However, because performance is costly and not the top priority for analysts, they could use a less expensive system such as optical (or even the latest high-speed tape technologies) as the backup system. Optical maintains the required availability of the data at a lower cost than RAID while also offering respectable performance.

When data is no longer applicable to the investment firm's daily applications, it can be archived. Archiving also represents an application requirement. With archival, performance and availability are no longer a priority, and cost-effectiveness comes into play. At that point in the data life cycle, optical--including MO, WORM, CD and tape--offer the best low-cost storage options for archiving data in digital format. Microfiche is also a viable option.

Adaptive storage strategies

One consideration that is often overlooked is that applications change with time. For example, the daily stock performance data that was once of utmost importance to the customer service representative quickly becomes outdated for the customer service application. However, the daily stock performance data is highly useful to the financial analyst for trends analysis long after the customer service representative is finished with it. As the applications for data change, the data must be migrated to the media type that meets the requirements of the current user.

Essentially, the priorities surrounding data continually change, and organizations need storage systems that are designed to adapt to change. A well-designed storage system moves data between various media types as the priorities surrounding that data change. The storage system is not an isolated jukebox or RAID system. Rather, it consists of all of the media types within the organization. By continually moving the data based on the applications for which it is best suited, the data (not to mention the whole storage system) is always optimized for overall performance.

Newer storage technologies, such as fiber channels and various DVD formats, provide previously unheard of storage volumes. To take advantage of those latest technologies, however, organizations must effectively design and implement adaptive storage strategies based on application needs and the changing requirements surrounding the data.

With an integrated storage strategy, each piece of the whole system is optimized for its function. That means that each business unit receives maximum benefit from the storage system. As a whole, the storage system helps the organization streamline and effectively meet its business objectives.