Storage Arrays in Practice

What is inside an array

A storage array is a dedicated system that pools many drives, protects them with RAID or a similar scheme, and presents slices of that pool to servers as volumes over a SAN or as file shares (see DAS, NAS and SAN). It is built so that no single component failure — drive, controller, power supply, fan, cable — interrupts access, and so that most maintenance, including firmware upgrades, can be done while servers keep running.

Controllers and cache

A storage controller is the computer inside the array that does the work: it receives commands on the front-end ports (FC, iSCSI, NVMe-oF or NFS/SMB), manages RAID, snapshots and data reduction, and reads and writes the drives through back-end SAS or NVMe connections. Enterprise arrays have two (some have more), each able to take over the other's work. Arrays differ in how they share it: in an active/active design both controllers serve every volume equally; in an ALUA design each volume is owned by one controller and the other serves it more slowly (see ALUA and path policies).

The controllers' memory is the array's write cache. A write is acknowledged to the server as soon as it sits in the cache of both controllers, long before it reaches the drives — which is why arrays answer writes in a fraction of a millisecond. The mirror means one controller can fail without losing acknowledged data, and batteries or capacitors give the controllers time to save the cache to flash if power is lost. If cache protection fails (a worn battery, a partner controller down), many arrays switch to write-through mode: safe, but much slower.

Drives and shelves

Behind the controllers sit drive shelves (enclosures, often called JBODs — see JBOD enclosures), cabled to both controllers so either can reach every drive. Modern arrays rarely use fixed RAID groups with dedicated hot spares: they spread data, parity and spare space over all drives ("distributed RAID"), so a rebuild after a drive failure writes to many drives at once and finishes in hours rather than days. Raw versus usable capacity still applies; see Storage Fundamentals.

LUNs, volumes and host mappings

A LUN (logical unit number) is, strictly, the number under which a target presents a logical unit to a host; in daily use it means the block volume itself. Arrays call them volumes, virtual disks or LUNs; it is all the same thing: a chunk of the pool with a size, a unique identifier, and a list of hosts allowed to see it.

Presenting a LUN takes three steps on the array:

1. Create the volume in a pool, with a size and (on most arrays) thin provisioning.
2. Define the host: a host object with its WWPNs or IQN, and the operating system type so the array answers the way that host's multipathing expects. Cluster members go into a host group.
3. Map the volume to the host or host group. This is LUN masking: other hosts, even if zoned to the same ports, do not see it.

Map shared datastores to the host group, not host by host, so every cluster member sees the same volume with the same LUN ID. A volume mapped to all hosts but one is a classic cause of "the VM cannot migrate to host 4". The reverse error is worse: a volume mapped to a host that should not see it may be formatted by that host.

Thin provisioning

With thin provisioning, a volume takes physical space from the pool only when data is actually written to it. A host sees a 40 TiB volume; the pool holds only the 22 TiB written so far. A thick volume, by contrast, reserves its full size up front. Thin provisioning lets you give every application the size it asks for without buying all of it on day one — which means the total provisioned capacity can exceed the physical pool. That ratio is the overcommit.

Overcommit and days until full

The two numbers to watch are the overcommit ratio (how much you have promised) and the time until the pool is full at the current growth rate (how long you have):

# A thin-provisioned pool: how overcommitted is it, and when does it fill up?
POOL_TIB = 120.0              # usable capacity of the pool
GROWTH_TIB_PER_DAY = 0.35     # average of the last 30 days
ALERT_AT = 0.80               # warn at 80% full

volumes = [
    # name, provisioned TiB (what the host sees), written TiB (what the pool holds)
    ("vmfs-prod-01", 40, 22.5),
    ("vmfs-prod-02", 40, 19.0),
    ("billing-db", 30, 18.2),
    ("file-shares", 60, 26.8),
    ("test-clones", 50, 6.5),
]

provisioned = sum(size for _, size, _ in volumes)
written = sum(used for _, _, used in volumes)
print(f"Provisioned: {provisioned:6.1f} TiB  (overcommit {provisioned / POOL_TIB:.2f} : 1)")
print(f"Written:     {written:6.1f} TiB  ({written / POOL_TIB:.1%} of the pool)")

days_to_alert = (POOL_TIB * ALERT_AT - written) / GROWTH_TIB_PER_DAY
days_to_full = (POOL_TIB - written) / GROWTH_TIB_PER_DAY
print(f"Days until {ALERT_AT:.0%}: {days_to_alert:5.1f}")
print(f"Days until full: {days_to_full:5.1f}")
print(f"If every volume filled up, the pool would be "
      f"{provisioned - POOL_TIB:.1f} TiB short")

Provisioned:  220.0 TiB  (overcommit 1.83 : 1)
Written:       93.0 TiB  (77.5% of the pool)
Days until 80%:   8.6
Days until full:  77.1
If every volume filled up, the pool would be 100.0 TiB short

Nothing is broken yet, but at 77.5% the pool is already past the 75% early warning recommended below, the 80% alert fires next week, and buying and installing a drive shelf can take longer than the 77 days left. The overcommit of 1.83 : 1 is normal for virtualization; the risk is the growth rate, not the ratio.

Getting space back

Deleting a file frees space in the host's file system, but the array does not know that: the blocks stay allocated in the pool. Hosts tell the array with UNMAP (SCSI) or deallocate (NVMe) commands, the SAN equivalent of TRIM on an SSD (see TRIM). Modern Windows, Linux and VMware versions can send them automatically or on a schedule, but it has to be enabled and supported end to end. Also beware of thin-on-thin: a thin virtual disk on a thin LUN overcommits twice, and each layer has to be watched separately.

Snapshots and clones

An array snapshot freezes a volume's contents at one moment, in seconds, regardless of its size, because it copies no data at the moment it is taken. It works the same way as the VM snapshots in Virtualization, but one layer lower: the array snapshots a whole LUN, with every VM or file on it.

Copy-on-write and redirect-on-write

A new snapshot is just a list of pointers to the volume's current blocks. The work happens when the host later changes a block that the snapshot still needs. With copy-on-write (COW), the array first copies the old block to a snapshot area, then overwrites the original in place. Every first write to a block after a snapshot therefore costs an extra read and write.

Many modern arrays use redirect-on-write instead: the new data is written to a fresh location and the volume's pointer moves, while the snapshot keeps pointing at the old block. That avoids the extra copy, which is why such arrays can keep hundreds of snapshots per volume with little performance cost. Either way, a snapshot grows with every block changed after it was taken: a snapshot of a busy database kept for a month can grow to a large share of the volume's size.

Snapshots are not backups

A snapshot lives on the same array, and shares most of its blocks with the live volume. It is excellent for undoing a mistake — a bad patch, a deleted folder, an application upgrade gone wrong — within minutes. It does nothing when the array itself fails, burns or is wiped by an attacker with administrator access. Real protection needs copies elsewhere: replication to a second array and backups on separate, preferably immutable, storage (see Backup and Recovery and the OS-level 3-2-1 rule).

That said, snapshots have become part of ransomware defense: many arrays can make snapshots immutable (they cannot be deleted before they expire, not even by an administrator) or require a second person to approve deleting them. Ransomware that encrypts the data inside VMs cannot touch such snapshots, which can make recovery very fast — as long as the array itself survives.

Clones

A clone is a writable volume created from a snapshot. Because it starts out sharing all its blocks with the original, a 20 TiB database clone is ready in seconds and uses almost no space until someone writes to it. Teams use clones to refresh test and acceptance environments from production every night, to give developers their own copy, or to mount yesterday's data next to today's to compare. Treat a clone of production as production data: it holds the same personal data and needs the same access control.

Data reduction: deduplication and compression

Most all-flash arrays reduce data before it reaches the drives, in two complementary ways.

Deduplication

Deduplication stores identical chunks of data only once. The array splits incoming data into chunks, computes a cryptographic hash of each, and when a hash is already known it records a reference instead of writing the chunk again. Twenty VMs built from the same Windows Server template share gigabytes of identical system files — a perfect case. Inline deduplication does this before writing; post-process deduplication writes first and removes duplicates later.

# Fixed-size deduplication: split each VM disk into 4 KiB chunks and store
# every distinct chunk once, keyed by its SHA-256 hash.
import hashlib

CHUNK = 4096


def blocks(label, count):
    """Build `count` distinct 4 KiB blocks whose content starts with `label`."""
    return b"".join(f"{label} {i}".encode().ljust(CHUNK, b"\0") for i in range(count))


os_image = blocks("windows-server-2025 system block", 600)   # shared by all VMs
vms = {
    "web01": os_image + blocks("web01 data", 40),
    "web02": os_image + blocks("web02 data", 40),
    "app01": os_image + blocks("app01 data", 120),
    "db01": os_image + blocks("db01 data", 400),
}


def store(disks):
    chunks = {}                      # hash -> chunk length
    logical = 0
    for disk in disks:
        for offset in range(0, len(disk), CHUNK):
            chunk = disk[offset:offset + CHUNK]
            chunks.setdefault(hashlib.sha256(chunk).hexdigest(), len(chunk))
            logical += len(chunk)
    return logical, sum(chunks.values())


logical, physical = store(vms.values())
print(f"Written by hosts: {logical / 2**20:5.1f} MiB")
print(f"Stored on disk:   {physical / 2**20:5.1f} MiB")
print(f"Dedup ratio:      {logical / physical:.2f} : 1")

# Insert one byte at the start of a copy of web01: every chunk boundary shifts.
shifted = b"\n" + vms["web01"]
_, before = store([vms["web01"]])
_, after = store([vms["web01"], shifted])
print(f"Shifted copy of web01 adds {(after - before) / 2**20:.1f} MiB of new chunks")

Written by hosts:  11.7 MiB
Stored on disk:     4.7 MiB
Dedup ratio:      2.50 : 1
Shifted copy of web01 adds 2.5 MiB of new chunks

Four VMs sharing one operating system image deduplicate 2.5 : 1. The last line shows the weakness of fixed-size chunking: insert a single byte and every chunk boundary after it shifts, so none of the chunks match any more. Backup appliances therefore often use variable-size chunking, which finds boundaries from the content itself; arrays rely on the fact that file systems and VMs align their data to fixed blocks.

Compression

Compression shrinks each chunk by encoding repeated patterns more compactly, like a ZIP file (see compressors). It works on unique data too: databases and text often compress 2 : 1 or better. Already compressed or encrypted data — video, images, ZIP archives, and anything encrypted by the host — barely compresses or deduplicates at all, because good encryption makes every block look random.

The data reduction ratio

The data reduction ratio is the logical data written by hosts divided by the physical space it occupies after deduplication and compression; 3 : 1 means 30 TiB of data fits in 10 TiB. The two effects multiply: 1.5 : 1 deduplication and 2 : 1 compression give 3 : 1.

Read vendor figures critically:

• Some dashboards report a larger "total efficiency" that also counts thin provisioning (space never written) and snapshots (blocks shared). Those are real savings compared with thick copies, but not data reduction, and they do not buy you more room for new data.
• Ratios are workload-specific: VDI might reach 5 : 1 or more, databases around 2 : 1 to 3 : 1, encrypted or media data about 1 : 1.
• A ratio can drop. If the security team enables encryption inside all VMs, the ratio falls toward 1 : 1 and the pool fills up accordingly. Encrypt at the array (encryption at rest) instead, which happens after data reduction.

Automated tiering and caching

Automated tiering moves data between storage tiers inside an array without the host noticing: the array tracks how often each region of a volume is read and written, promotes busy regions to flash and demotes quiet ones to large, cheaper drives, typically in a scheduled run or continuously in small steps. In a hybrid array, a few percent of flash can then serve most of the I/O, because in most workloads a small share of the data receives most of the access.

Tiering reacts to the past, which is its weakness: the month-end billing run reads data that has been cold for four weeks, finds it on slow drives, and runs slowly the first time. Caching is the faster-reacting cousin: a flash cache keeps copies of hot blocks instead of moving them. With all-flash arrays now the norm for primary storage, tiering inside one array matters less; it lives on between arrays and the cloud, where cold snapshots and volumes are moved to object storage.

Operating an array day to day

Arrays are reliable, which makes it easy to stop looking at them. A short routine catches almost every slow-building problem: