RAID

What RAID is and why

RAID (redundant array of independent disks) combines several physical drives into one logical drive. The operating system sees a single volume; a RAID controller or software layer decides where each block really goes. (The I originally stood for "inexpensive"; both expansions are in use.) RAID can serve three goals, and each level trades them off differently:

• Performance: several drives working in parallel read and write faster than one.
• Redundancy: the data survives when a drive fails, and the system keeps running.
• Capacity: several drives form one large volume.

Fault tolerance is the ability of a system to keep working when a component fails. For RAID, it is counted in drives: how many can fail at the same time before data is lost. Hard drives and SSDs are among the components that fail most often (see Storage), which is why servers and network-attached storage (NAS) devices almost always use RAID.

Striping, mirroring and parity

All RAID levels are built from three techniques.

Striping

Striping splits data into chunks (stripe units, often 64 KiB to 256 KiB) and writes them across the drives in turn: chunk 1 to drive 1, chunk 2 to drive 2, and so on. A large file is then read from all drives at once, so throughput grows with the number of drives. Striping alone adds no redundancy: every file has pieces on every drive, so losing one drive destroys all of them.

Mirroring

Mirroring writes identical copies of every block to two (or more) drives. If one drive fails, the other holds everything and the system continues. Reads can be served by either copy, so they can get faster; writes are not faster, because every block is written to every drive. The price is capacity: two mirrored 8 TB drives give 8 TB.

Parity

Parity is extra data calculated from the data blocks, from which any one missing block can be rebuilt. It uses the exclusive or (XOR) operation: bit by bit, the parity is 1 when an odd number of data bits is 1. XOR has a useful property: if P = A ^ B ^ C, then B = A ^ C ^ P. XOR the survivors with the parity and the lost block reappears:

disk1, disk2, disk3 = b"SALES", b"Q3-EU", b"12500"   # one block per data disk

parity = bytes(a ^ b ^ c for a, b, c in zip(disk1, disk2, disk3))
print(f"First byte:  {disk1[0]:08b} ^ {disk2[0]:08b} ^ {disk3[0]:08b} = {parity[0]:08b}")
print("Parity block:", parity.hex(" "))

# Disk 2 fails. XOR the surviving blocks with the parity to get it back.
rebuilt = bytes(a ^ c ^ p for a, c, p in zip(disk1, disk3, parity))
print("Rebuilt disk 2:", rebuilt, rebuilt == disk2)

First byte:  01010011 ^ 01010001 ^ 00110001 = 00110011
Parity block: 33 40 54 30 36
Rebuilt disk 2: b'Q3-EU' True

One parity block protects any number of data blocks against the loss of one of them, which is why parity costs much less capacity than mirroring. The cost is work: every write must also update the parity, and a rebuild must read every surviving drive completely. For the XOR operation itself, see Boolean Logic.

RAID levels

RAID 0: striping

Data is striped across two or more drives, with no redundancy. You get the full capacity and the best read and write speed, and one failed drive loses the whole array. The chance of losing data is higher than with a single drive, because any of several drives can fail. RAID 0 suits scratch space and data that can be recreated, such as video editing caches, never the only copy of anything.

RAID 1: mirroring

Two drives hold identical copies. Usable capacity is one drive; the array survives the loss of either drive. Simple and robust, RAID 1 is a common choice for the operating system drive of a server and for small two-bay NAS devices.

RAID 5: striping with distributed parity

Data and parity are striped across three or more drives. Each stripe has one parity block, and the parity rotates from drive to drive so no single drive becomes a bottleneck. Usable capacity is all drives but one; the array survives one failed drive. Reads are fast. Small writes are slower, because the controller has to read the old data and parity, calculate, and write both back (the write penalty).

RAID 6: striping with double parity

Like RAID 5, but with two independent parity blocks per stripe (the second one is not a simple XOR). It needs at least four drives, gives the capacity of all drives but two, and survives any two drive failures at the same time. Writes are slower still than RAID 5. With today's large drives, RAID 6 is often preferred over RAID 5 because the array stays protected during a long rebuild.

RAID 10: a stripe of mirrors

RAID 10 (also written RAID 1+0) first mirrors drives in pairs and then stripes data across the pairs. It needs at least four drives, in an even number, and gives half the total capacity. It always survives one failed drive, and can survive more as long as no pair loses both of its drives. It combines good read and write speed with fast rebuilds (a rebuild only copies the partner drive), which makes it popular for databases and virtual machine storage.

The levels compared

In the table, n is the number of drives and s the size of one drive.

The formulas assume identical drives. With mixed sizes, most RAID implementations treat every drive as if it were the size of the smallest one, and the extra space is wasted. A small calculator makes the trade-offs concrete:

def raid(level, drives, size_tb):
    """Return (usable TB, failures always survived) for identical drives."""
    if level == 0 and drives >= 2:
        return drives * size_tb, 0
    if level == 1 and drives >= 2:
        return size_tb, drives - 1
    if level == 5 and drives >= 3:
        return (drives - 1) * size_tb, 1
    if level == 6 and drives >= 4:
        return (drives - 2) * size_tb, 2
    if level == 10 and drives >= 4 and drives % 2 == 0:
        return drives // 2 * size_tb, 1
    raise ValueError(f"RAID {level} cannot be built from {drives} drives")

for level, drives in [(0, 4), (1, 2), (5, 4), (6, 4), (6, 8), (10, 4), (5, 2)]:
    try:
        usable, survives = raid(level, drives, size_tb=8)
    except ValueError as error:
        print(error)
        continue
    share = usable / (drives * 8)
    print(f"RAID {level:<2} {drives} x 8 TB: {usable:>2} TB usable ({share:4.0%}), survives {survives}")

RAID 0  4 x 8 TB: 32 TB usable (100%), survives 0
RAID 1  2 x 8 TB:  8 TB usable ( 50%), survives 1
RAID 5  4 x 8 TB: 24 TB usable ( 75%), survives 1
RAID 6  4 x 8 TB: 16 TB usable ( 50%), survives 2
RAID 6  8 x 8 TB: 48 TB usable ( 75%), survives 2
RAID 10 4 x 8 TB: 16 TB usable ( 50%), survives 1
RAID 5 cannot be built from 2 drives

Two things stand out. RAID 6 and RAID 10 on four drives give the same capacity, but RAID 6 survives any two failures while RAID 10 survives two only if they are in different pairs; RAID 10 is faster, especially for writes. And the more drives an array has, the smaller the share that parity costs.

JBOD

JBOD ("just a bunch of disks") is not a RAID level. It means the drives are used without RAID: either each drive appears on its own, or they are concatenated (spanned) into one volume that fills one drive after the other. Capacity is the sum of all drives and there is no redundancy; with spanning, a failed drive usually makes the whole volume unusable.

Hardware, software and firmware RAID

• Hardware RAID: a RAID controller card (or a controller built into a server motherboard) with its own processor does the work. The operating system sees one drive. Good controllers have a write cache protected by a battery or flash module, so cached writes survive a power cut. You configure the array in the controller's setup utility during boot or in the vendor's management tool.
• Software RAID: the operating system builds the array from ordinary drives, for example with Storage Spaces in Windows or mdadm and ZFS on Linux. No special hardware is needed, the array can move to another computer with the drives, and today's processors handle parity easily.
• Firmware RAID (sometimes called "fake RAID"): the motherboard's chipset offers RAID in the setup of its firmware, the Basic Input/Output System (BIOS) or Unified Extensible Firmware Interface (UEFI), but a driver in the operating system does the actual work. It is cheap and convenient, but ties the array to that platform and driver.

Failures, hot spares and rebuilds

When a drive in a redundant array fails, the array becomes degraded: it keeps working and serving data, but with less or no protection left. RAID 5 in degraded mode has no redundancy at all until the failed drive is replaced and its contents are rebuilt. Controllers and NAS devices signal this with an alert, a warning in the management tool, a beep or an amber light on the drive bay; a technician's job is to notice and act.

A hot spare is an extra drive that sits in the system unused until a drive fails; the controller then starts the rebuild onto it automatically, without waiting for someone to swap hardware. That shortens the time the array is unprotected. Afterward, the failed drive is replaced and the new one becomes the next spare.

A rebuild has to write the whole replacement drive, and for parity levels read every surviving drive completely. That takes a long time with large drives, here at an assumed average of 200 MB/s; smaller, older drives are slower:

speed_mb_s = 200    # assumed average write speed of the replacement drive

for size_tb in (4, 8, 16, 24):
    seconds = size_tb * 10**12 / (speed_mb_s * 10**6)
    print(f"{size_tb:>2} TB drive: about {seconds / 3600:4.1f} hours at {speed_mb_s} MB/s")

 4 TB drive: about  5.6 hours at 200 MB/s
 8 TB drive: about 11.1 hours at 200 MB/s
16 TB drive: about 22.2 hours at 200 MB/s
24 TB drive: about 33.3 hours at 200 MB/s

In practice rebuilds are often slower, because the array keeps serving users at the same time; days are not unusual for large parity arrays. During that window the remaining drives work harder than ever, and drives bought together tend to age together. An unreadable sector on a surviving drive can then make part of the data unrecoverable. This is why RAID 6 or RAID 10 is generally recommended over RAID 5 for arrays of large drives, and why failed drives should be replaced promptly.

RAID is not a backup

RAID protects against one thing: a drive failing. It does nothing against the far more common causes of data loss, because the array faithfully copies every change to all its drives:

• A user deletes or overwrites a file: it is gone from every drive at once.
• Ransomware encrypts the files: the array stores the encrypted versions redundantly.
• The controller fails, the file system is corrupted, or two drives fail during a rebuild.
• Fire, flood or theft takes the whole device, all drives included.

A backup is a separate copy, ideally with older versions, kept where the same event cannot reach it. A common guideline is the 3-2-1 rule: three copies of the data, on two different kinds of media, one of them offsite. Backup methods and strategies are covered in Backup and Recovery.