Beyond RAID: Wide Area Storage for Large-Scale Disk Archives

- Janet Lafleur, Product Marketing Manager at Quantum, says:

RAID was built to protect data from disk failures, but was it built to scale?  When you consider what happens when disk capacities grow to the multi-terabyte level and how many disks are needed to meet the growing demand for big data and media files, the answer is clearly “No.” 

When a 3TB or larger drive fails, it can require over 24 hours to rebuild the data on a replacement drive.  Until the failed drive is detected, replaced and the rebuild is completed, the storage array is left vulnerable to data loss from another drive failure, experiences degraded performance or both.  Add to that the regular supervision needed to replicate data across RAID arrays to deliver large files to remote users and for disaster recovery, and the operational challenges soar along with the data volumes.

For those who manage large repositories of large files shared by a large number of users, scale-out NAS storage built on RAID arrays has been the go-to technology for years. But when the storage demand scales toward and beyond the petabyte level, a new architecture is needed for disk-based archives, one with extreme scalability and durability that reduces both capital and operating expenses.  That new architecture is wide area storage.

Wide area storage combines next generation dispersed object storage with file system technologies in a new approach to archiving that overcomes these limitations and inefficiencies. Data repositories built on wide area storage are extremely scalable, durable and easy to maintain--allowing data to be stored forever on disk without business-halting service interruptions or painful data migrations.

Unlike RAID, wide area storage uses fountain coding, a type of forward error correction algorithm developed for communication over unreliable networks. The fountain code algorithm encodes the data into a set of equations in such a way that fewer than the full set is needed to reconstruct the original data. The equations are then dispersed across the storage which can be in multiple geographic sites.

The higher redundancy gives wide area storage 15 nines of durability—far greater than RAID—which means there’s no rush to replace failed drives and initiate rebuilds. Maintenance can be performed on a scheduled, not rushed basis.

Unlike RAID, wide area storage can mix and match drives from different technologies within the storage system. This makes the upgrade process as simple as removing a set of drives from active service and replacing with new ones. The wide area storage will re-calculate and disperse the data on the new drives and continue to run throughout the upgrade process.

The result: disk storage that scales indefinitely, preserves data indefinitely, and reduces maintenance for long-term, large-scale archives.