Cloud Storage

Three kinds of storage

Cloud providers offer storage in three shapes, which match three very different ways software uses data:

• Object storage: an application stores and fetches whole files by name over the web, such as photos, backups and log archives.
• Block storage: an operating system sees a raw disk, puts a file system on it and reads and writes small blocks, as with the boot disk of a virtual machine (VM) or a database's data files.
• File storage: several servers mount the same folder tree over the network and work on shared files.

Choosing the wrong one is a common source of slow systems and high bills, so it pays to understand the differences before looking at products.

Object storage

Object storage stores data as objects: each one is a complete file plus metadata, stored under a unique key, in a flat namespace without real folders. You read and write it over HTTPS (Hypertext Transfer Protocol Secure) with a simple application programming interface (API), not by mounting a disk. Capacity is virtually unlimited and billed per gigabyte per month; you never create or resize a volume.

Buckets, keys and metadata

A bucket is the container for objects: it has a name, lives in one region (or a chosen set of regions) and carries the settings that apply to its contents — access rules, encryption, versioning, lifecycle. In Amazon Simple Storage Service (S3) and Google Cloud Storage, bucket names are globally unique, because they appear in web addresses (Uniform Resource Locators, URLs). Azure Blob Storage uses a storage account (globally unique name) that holds containers, which hold the blobs.

Every object has three parts:

• the key, its full name, such as reports/2026/09/sales-2026-09.csv. The slashes are part of the name; consoles show them as folders for convenience, but there is no directory to create or delete;
• the data, from a few bytes to terabytes;
• metadata: system values such as size, content type and last-modified time, plus key-value pairs you add yourself.

Objects are written as a whole. In most object stores you cannot change byte 500 of an object as you would in a file on disk: you upload a new version of the entire object. That makes object storage ideal for files that are written once and read many times, and unsuitable for a database that updates small pieces of a file constantly.

An HTTP API

Every operation is an HTTP request: PUT uploads an object, GET downloads it, DELETE removes it, and a list request returns keys with a given prefix — the same Representational State Transfer (REST) style described in REST APIs. The Amazon S3 API has become a de facto standard: many other products and providers offer an S3-compatible API, so tools written for S3 often work with them too. Command-line tools and software development kits wrap these requests:

$ aws s3 cp sales-2026-09.csv s3://example-reports-2026/reports/2026/09/

upload: ./sales-2026-09.csv to s3://example-reports-2026/reports/2026/09/sales-2026-09.csv

$ aws s3 ls s3://example-reports-2026/reports/2026/09/

2026-10-01 08:15:42      48213 sales-2026-09.csv

The Azure command-line interface (CLI) equivalent is az storage blob upload, and Google Cloud's is gcloud storage cp.

Block storage

Block storage provides virtual disks that attach to an instance and appear to its operating system as an ordinary drive. The operating system partitions and formats it with its own file system, exactly as with a physical disk (see File Systems). AWS calls these Elastic Block Store (EBS) volumes, Azure calls them managed disks, and Google Cloud calls them Persistent Disk or Hyperdisk.

• A volume is normally attached to one instance at a time, and it normally lives in one availability zone: an instance in another zone cannot use it.
• It survives when the instance stops, and can be detached and attached elsewhere — unlike local instance storage (see Compute).
• Snapshots copy the volume to cheaper, more durable storage. They are incremental (each one stores only the blocks changed since the previous one) and can be used to create new volumes, also in other zones.
• You pay for the provisioned size, used or not, and sometimes for extra performance.

IOPS and throughput

Volume types differ in speed, described by two numbers. IOPS (input/output operations per second) counts reads and writes per second and matters for databases doing many small random accesses. Throughput, in mebibytes per second (MiB/s), measures the volume of data moved and matters for large sequential reads such as backups or video. A volume has a limit for each, and whichever is reached first applies. This script shows a volume with a baseline of 3,000 IOPS and 125 MiB/s, similar to a general-purpose solid-state drive (SSD) volume at the time of writing:

# A volume with a baseline of 3,000 IOPS and 125 MiB/s throughput.
MAX_IOPS = 3000
MAX_MIB_PER_S = 125

for io_kib in [4, 16, 64, 256]:
    by_iops = MAX_IOPS * io_kib / 1024          # MiB/s if IOPS is the limit
    actual = min(by_iops, MAX_MIB_PER_S)
    limit = "IOPS" if by_iops < MAX_MIB_PER_S else "throughput"
    iops = actual * 1024 / io_kib
    print(f"{io_kib:3} KiB I/O: {actual:6.1f} MiB/s at {iops:5.0f} IOPS, limited by {limit}")

  4 KiB I/O:   11.7 MiB/s at  3000 IOPS, limited by IOPS
 16 KiB I/O:   46.9 MiB/s at  3000 IOPS, limited by IOPS
 64 KiB I/O:  125.0 MiB/s at  2000 IOPS, limited by throughput
256 KiB I/O:  125.0 MiB/s at   500 IOPS, limited by throughput

A database doing 4 KiB random reads hits the IOPS limit while moving less than 12 MiB/s; a backup reading 256 KiB blocks hits the throughput limit at only 500 IOPS. Instances also have their own limits for disk traffic, so a fast volume on a small instance type can still be slow.

File storage

File storage is a managed network file system that many instances — and on-premises computers over a private connection — can mount at the same time. It offers real folders, file locking and permissions, through the standard protocols: Network File System (NFS), common on Linux, and Server Message Block (SMB), common on Windows.

• AWS: Amazon Elastic File System (EFS) for NFS; the Amazon FSx family for Windows File Server (SMB), NetApp ONTAP and others.
• Azure: Azure Files (SMB and NFS shares).
• Google Cloud: Filestore (NFS).

Typical uses are content shared by a group of web servers, home directories, and older applications that expect a shared drive and cannot be changed to use object storage. File storage costs more per gigabyte than object storage, and network latency makes it slower than a local block volume for small, frequent writes. On Linux, an EFS file system is mounted like any NFS share:

$ sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport \
    fs-0123456789abcdef0.efs.eu-central-1.amazonaws.com:/ /mnt/shared

Comparing the three

Storage classes

Most data is read often when it is new and rarely afterward. A storage class (Azure: access tier) sets how an object is stored and billed: the colder the class, the lower the price per gigabyte per month, and the higher the cost and delay of reading it back. Colder classes also have a minimum storage duration: deleting or moving an object earlier is billed as if it had stayed the full period.

Minimum durations and retrieval times are those published at the time of writing. Note the differences in the archive row: an Azure archive blob must be rehydrated to an online tier before it can be read, which can take hours, and S3 Glacier Flexible Retrieval and Deep Archive need a restore request first. Google Cloud's Archive class answers in milliseconds but charges heavily for reading. If you cannot predict access patterns, automatic tiering (S3 Intelligent-Tiering, Google Cloud Autoclass) moves objects for you, for a small monitoring fee.

Lifecycle policies

Moving objects between classes by hand does not scale. A lifecycle policy is a set of rules on a bucket that moves objects to colder classes, or deletes them, automatically after a number of days — for example "move reports to infrequent access after 30 days, archive them after 90". Rules can apply to a key prefix or to objects with a tag, and can also clean up old versions and unfinished uploads.

This S3 lifecycle configuration in JSON (JavaScript Object Notation) does exactly that for everything under reports/ (GLACIER is the API name of S3 Glacier Flexible Retrieval):

{
  "Rules": [
    {
      "ID": "reports-to-archive",
      "Filter": { "Prefix": "reports/" },
      "Status": "Enabled",
      "Transitions": [
        { "Days": 30, "StorageClass": "STANDARD_IA" },
        { "Days": 90, "StorageClass": "GLACIER" }
      ]
    }
  ]
}

$ aws s3api put-bucket-lifecycle-configuration \
    --bucket example-reports-2026 \
    --lifecycle-configuration file://lifecycle.json

What a lifecycle saves

The script below reads that same rule and compares a year of storing 10 TB in the hot class with the lifecycle, including the cost of reading data back from the archive. The prices are fictitious round numbers chosen to show the pattern — real prices differ per provider and region, and per-request and transition fees are left out. It also simplifies by writing all data on day 0 and using 30-day months.

import json

LIFECYCLE = """
{
  "Rules": [
    {
      "ID": "reports-to-archive",
      "Filter": { "Prefix": "reports/" },
      "Status": "Enabled",
      "Transitions": [
        { "Days": 30, "StorageClass": "STANDARD_IA" },
        { "Days": 90, "StorageClass": "GLACIER" }
      ]
    }
  ]
}
"""

# Fictitious round prices, for the comparison only.
PRICE_PER_GB_MONTH = {"STANDARD": 0.020, "STANDARD_IA": 0.010, "GLACIER": 0.002}
RETRIEVAL_PER_GB = {"STANDARD": 0.0, "STANDARD_IA": 0.01, "GLACIER": 0.05}

DATA_GB = 10_000      # 10 TB, all written on day 0
MONTHS = 12           # one year of 30-day months

def storage_class_in_month(month, transitions):
    current = "STANDARD"
    for step in transitions:            # sorted by Days
        if month * 30 >= step["Days"]:
            current = step["StorageClass"]
    return current

rule = json.loads(LIFECYCLE)["Rules"][0]
transitions = sorted(rule["Transitions"], key=lambda t: t["Days"])

hot_only = DATA_GB * PRICE_PER_GB_MONTH["STANDARD"] * MONTHS

tiered = 0.0
months_per_class = {}
for month in range(MONTHS):
    cls = storage_class_in_month(month, transitions)
    tiered += DATA_GB * PRICE_PER_GB_MONTH[cls]
    months_per_class[cls] = months_per_class.get(cls, 0) + 1

restore_gb = 500      # an audit asks for 5% of the data back in month 10
restore_cost = restore_gb * RETRIEVAL_PER_GB["GLACIER"]

print("Months per class:", months_per_class)
print(f"Hot all year:          {hot_only:8.2f}")
print(f"With lifecycle:        {tiered:8.2f}")
print(f"  + restoring 500 GB:  {tiered + restore_cost:8.2f}")
print(f"  + restoring all:     {tiered + DATA_GB * RETRIEVAL_PER_GB['GLACIER']:8.2f}")

Months per class: {'STANDARD': 1, 'STANDARD_IA': 2, 'GLACIER': 9}
Hot all year:           2400.00
With lifecycle:          580.00
  + restoring 500 GB:    605.00
  + restoring all:      1080.00

With these prices the lifecycle costs about a quarter of keeping everything hot, and even restoring all data once leaves it well ahead. The real risk is not the fee but the wait: data in an offline archive class may take hours to come back, which matters if it is needed during an incident. Match the class to how fast you might need the data, not only to how often.

Durability and availability

Two promises are easily confused:

• Durability is the probability that stored data is not lost or corrupted over a year. It is achieved by keeping several copies on different devices, and often in different availability zones, and by constantly checking and repairing them.
• Availability is the share of time you can actually read and write the data. A storage service can be unreachable for an hour without losing a single byte.

Amazon S3 Standard and Google Cloud Storage are designed for 99.999999999% durability — "eleven nines" — while their designed availability is around 99.9 to 99.99%. Azure states at least eleven nines for locally redundant storage and more for zone- and geo-redundant options.

What eleven nines means

Eleven nines means an annual chance of 0.000000001% (one in 100 billion) that a given object is lost. The number only becomes meaningful when you multiply it by how many objects you store:

OBJECTS = 10_000_000

print(f"Expected objects lost per year, out of {OBJECTS:,}:")
for nines in [4, 9, 11]:
    durability = "99." + "9" * (nines - 2) + "%"
    annual_loss_rate = 10 ** -nines      # chance that one object is lost in a year
    lost = OBJECTS * annual_loss_rate
    print(f"  {durability:>14}  {lost:12,.4f}")

years = 1 / (OBJECTS * 10 ** -11)
print(f"At eleven nines: on average one object every {years:,.0f} years")

Expected objects lost per year, out of 10,000,000:
          99.99%    1,000.0000
     99.9999999%        0.0100
   99.999999999%        0.0001
At eleven nines: on average one object every 10,000 years

Ten million objects at eleven nines: on average, one lost object every 10,000 years. Hardware failure is no longer the realistic risk. The realistic risks are a person or script deleting data, ransomware encrypting it, a wrong lifecycle rule expiring it, or an account being closed or compromised — and durability protects against none of these, because the storage faithfully keeps doing what it was told.

Replication

Providers copy your data automatically within the limits of the option you choose. Azure makes it an explicit choice per storage account: locally redundant storage (LRS, three copies in one data center), zone-redundant storage (ZRS, across three zones), geo-redundant storage (GRS, also copied to the paired region) and geo-zone-redundant storage (GZRS, both). S3 Standard spreads data over at least three availability zones; Google Cloud lets you choose a region, dual-region or multi-region bucket location. To keep a copy in another region, you can also configure replication yourself, such as S3 Cross-Region Replication.

Replication protects against losing a data center or region. It does not protect against mistakes: a deleted or overwritten object is deleted or overwritten in the replica too. Like a Redundant Array of Independent Disks (RAID), it is not a backup (see RAID is not a backup).

Versioning and immutability

With versioning turned on, a bucket keeps every previous version of an object. Overwriting creates a new version; deleting only adds a delete marker, and the old versions stay retrievable. Old versions are billed like any other data, so pair versioning with a lifecycle rule that removes noncurrent versions after, say, 90 days.

For backups and records that must not be changed, buckets can also be made immutable for a retention period (S3 Object Lock, Azure immutable storage, Cloud Storage retention policies): nobody, not even an administrator in some modes, can delete the data before the period ends. That is a strong defense against ransomware, and part of the 3-2-1 approach described in Backup and Recovery.

Securing storage

Storage holds the data attackers want, and object storage is reachable over the internet by design. The defaults have become safer over the years, and the work is mostly to keep them that way.

Blocking public access

New buckets are private: only identities you grant access to can read them. On top of that, providers offer a switch that overrides any attempt to make data public — S3 Block Public Access (on by default for new buckets since 2023), the Azure storage account setting that disallows anonymous blob access, and Google Cloud's public access prevention. Leave it on at the account level, and make exceptions only for buckets that are deliberately public, such as the assets of a public website — and preferably serve those through a content delivery network (CDN) instead.

$ aws s3api get-public-access-block --bucket example-reports-2026

{
    "PublicAccessBlockConfiguration": {
        "BlockPublicAcls": true,
        "IgnorePublicAcls": true,
        "BlockPublicPolicy": true,
        "RestrictPublicBuckets": true
    }
}

To share one file with someone outside your organization, create a time-limited signed link instead of opening the bucket: a presigned URL in S3 and Google Cloud, a shared access signature (SAS) in Azure. A signed link works for anyone who holds it until it expires and cannot be withdrawn on its own, so keep the expiry short. In Azure, prefer a user delegation SAS, which is signed with Entra ID credentials instead of the storage account key.

$ aws s3 presign s3://example-reports-2026/reports/2026/09/sales-2026-09.csv --expires-in 3600

Encryption

The large providers encrypt stored data by default at the time of writing, with keys they manage (one exception: Amazon EBS volumes are encrypted only if you turn on "encryption by default" per region or choose it per volume). You can choose keys you control in a key management service instead, which adds an audit trail of key use and the ability to revoke access by disabling the key. Data in transit should always use TLS (Transport Layer Security); a bucket policy can refuse any request that does not:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyInsecureTransport",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::example-reports-2026",
        "arn:aws:s3:::example-reports-2026/*"
      ],
      "Condition": { "Bool": { "aws:SecureTransport": "false" } }
    }
  ]
}

Keys, encryption options and compliance are covered in Cloud Security and Compliance; who may access which bucket is a matter of Identity and Access Management.