Cloud Security and Compliance

Shared responsibility in practice

In the cloud, security is split between provider and customer. The provider is responsible for the security of the cloud: data center buildings, hardware, the hypervisor and the global network. You are responsible for security in the cloud: your data, your identities and permissions, and how you configure the services you use. Where exactly the line falls depends on the service model (see Service Models: IaaS, PaaS, SaaS).

The bottom line is the same everywhere: identities, access and data always remain yours. A provider can offer encryption, logging and multi-factor authentication; whether they are switched on is usually your decision.

The most common risks

Attacks on the provider's infrastructure itself are rare. Most cloud incidents follow a handful of patterns on the customer's side.

• Misconfiguration: a storage bucket readable by anyone on the internet, a security group that allows Secure Shell (SSH) or Remote Desktop Protocol (RDP) from 0.0.0.0/0, an identity with administrator rights "to make it work", logging switched off. The typical lesson: a backup or export left in a public bucket, found by automated scanners that search for exactly that, exposing millions of customer records.
• Stolen credentials: access keys committed to a public code repository, phished passwords without multi-factor authentication (MFA), tokens copied from a developer's laptop. Leaked keys in public repositories are often found and abused within minutes, typically to start expensive machines for cryptocurrency mining.
• Unpatched workloads: virtual machines and container images with known vulnerabilities. On infrastructure as a service (IaaS), patching the operating system is your job.
• Insecure application programming interfaces (APIs) and applications: an application flaw lets an attacker make the server send requests on their behalf, for example to the instance metadata service, which hands out the server's temporary credentials. If the server's role is allowed to read every bucket, so is the attacker. Requiring session tokens for the metadata service (IMDSv2 on Amazon Web Services, or AWS) and least-privilege roles limit the damage.
• Data leakage: snapshots or database copies shared with "everyone", sensitive data in logs, production data copied into a less protected test environment.

Note what these have in common: in almost every case, the security feature that would have prevented the incident existed and was not used. The rest of this page is about using them by default.

Identity first

In a data center, the network perimeter was the main defense. In the cloud, every action is an API call that can come from anywhere, so identity is the new perimeter: whoever has valid credentials with enough permissions can act, regardless of firewalls. The basics are covered in Identity and Access Management: MFA on every human account and especially on the root or global administrator account; no access keys for the root account; least privilege; roles with temporary credentials for workloads (including continuous integration (CI) pipelines, through OpenID Connect (OIDC) federation) instead of long-lived keys; and identity federation so leavers lose access everywhere at once.

Encryption and key management

Encryption at rest

Encryption at rest protects stored data — disks, object storage, databases, backups — so that someone who obtains the raw storage media or a copy of the files cannot read it without the key. In the large clouds it is increasingly the default: at the time of writing, Amazon Simple Storage Service (S3) encrypts new objects automatically, Azure Storage encrypts all data, and Google Cloud encrypts customer data at rest by default. Some services still leave it to you: on AWS, encryption by default for Elastic Block Store (EBS) disks is an account setting per region.

Encryption at rest does not protect against an attacker who uses valid credentials: the service decrypts the data for anyone who is allowed to read it. That is why access control matters more than encryption, and why who may use a key is itself a permission.

Encryption in transit

Encryption in transit protects data while it moves over a network, so it cannot be read or changed on the way. In practice this means Transport Layer Security (TLS): Hypertext Transfer Protocol Secure (HTTPS) for websites and APIs, TLS for database connections, and SSH or a virtual private network (VPN) for administration. Provider APIs only accept TLS. For your own services, redirect HTTP to HTTPS, use TLS 1.2 or newer, and use the provider's certificate services (AWS Certificate Manager, Azure Key Vault certificates, Google Certificate Manager) so certificates renew automatically. Encrypt traffic inside your virtual network as well: "it never leaves our network" is a weaker guarantee in a shared cloud than in your own building.

Key management services

Encryption is only as safe as its keys. A key management service (KMS) creates, stores and controls the use of encryption keys, so that keys never appear in code or configuration files, every use is checked against permissions and recorded in the audit log, and rotation can be automated. Examples are AWS Key Management Service, Azure Key Vault and Google Cloud Key Management Service.

• Provider-managed keys: the provider creates and rotates the keys for you. No work, but you have little control over them.
• Customer-managed keys: you create the key in the KMS and decide who may use it, when it rotates (for example yearly) and when it is disabled. Disabling the key makes all data encrypted with it unreadable — a strong control, and a real risk if done by mistake. Some services also let you import or supply your own key material ("bring your own key").
• A hardware security module (HSM) is tamper-resistant hardware that generates and uses keys without ever exposing them. KMS services use HSMs internally; dedicated HSMs (AWS CloudHSM, Azure Managed HSM, Google Cloud HSM's single-tenant option) are for organizations whose regulations demand keys in a single-tenant, certified device.

Large amounts of data are not sent to the KMS to be encrypted. Instead, services use envelope encryption: the KMS key encrypts a small data key, and the data key encrypts the data.

Secrets: passwords, keys and tokens

Applications need secrets: database passwords, API keys for payment providers, signing keys. A secrets manager stores such secrets encrypted, hands them to authorized applications at run time, records every access, and can rotate them automatically. Examples are AWS Secrets Manager, Azure Key Vault and Google Cloud Secret Manager.

The application asks for the secret when it needs it, using its own role or managed identity — so no secret is needed to fetch the secret:

import json

import boto3


def database_password():
    """Fetch the password at run time; never store it in the code."""
    client = boto3.client("secretsmanager")
    response = client.get_secret_value(SecretId="prod/shop/db")
    return json.loads(response["SecretString"])["password"]

Better still is a secret that does not exist. Where a cloud service supports identity-based access (a virtual machine reading from storage with its role, a function connecting to a database with a managed identity), use that instead of a password.

Network controls

Network controls still matter as a second layer: place databases and internal services in private subnets, allow only the ports a service needs in its security groups, reach managed services through private endpoints instead of over the internet, and use a bastion service or VPN instead of opening SSH and RDP to the world. The building blocks are explained in Cloud Networking and the principles behind them in Network Security Basics.

For object storage, the provider-level switch against accidental exposure is worth knowing by name. On AWS, S3 Block Public Access overrides any bucket policy or access control list that would make data public. It is on by default for new buckets at the time of writing; for older buckets, check and set it:

$ aws s3api put-public-access-block --bucket shop-invoices \
    --public-access-block-configuration \
    BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true
$ aws s3api get-public-access-block --bucket shop-invoices

{
    "PublicAccessBlockConfiguration": {
        "BlockPublicAcls": true,
        "IgnorePublicAcls": true,
        "BlockPublicPolicy": true,
        "RestrictPublicBuckets": true
    }
}

Azure Storage accounts have a similar setting that disallows anonymous blob access, and Google Cloud has public access prevention for buckets.

Logging and detection

Prevention is never perfect, so you also need to notice when something goes wrong. The foundation is the audit log of every action on the account (see Monitoring, Logging and Observability), stored where attackers cannot delete it. On top of that, providers offer security services in three broad categories:

Cloud security posture management (CSPM) deserves a closer look, because misconfiguration is the largest risk. A CSPM tool reads the configuration of every resource through the provider's APIs, compares it with rules (often based on benchmarks from the Center for Internet Security (CIS)), and reports findings with a severity — continuously, so a bucket made public at 15:02 is flagged minutes later, not at next year's audit. On AWS, the newer AWS Security Hub builds on Security Hub CSPM and correlates its findings with those of other security services, such as GuardDuty and Inspector.

Scanning for misconfigurations in Python

The core of such a tool is simple: describe each resource as data, and run checks against it. This sketch checks four classic problems.

RESOURCES = [
    {"type": "bucket", "name": "shop-invoices", "public_access": True, "encrypted": True},
    {"type": "bucket", "name": "shop-assets-private", "public_access": False, "encrypted": True},
    {"type": "security_group", "name": "sg-web",
     "inbound": [{"port": 443, "source": "0.0.0.0/0"}]},
    {"type": "security_group", "name": "sg-admin",
     "inbound": [{"port": 22, "source": "0.0.0.0/0"},
                 {"port": 3389, "source": "198.51.100.0/24"}]},
    {"type": "disk", "name": "db-data-01", "encrypted": False},
    {"type": "disk", "name": "web-boot-01", "encrypted": True},
    {"type": "account", "name": "root", "access_keys": 1, "mfa": True},
]

ADMIN_PORTS = {22: "SSH", 3389: "RDP"}


def check(resource):
    """Yield (severity, message) for every problem in one resource."""
    kind = resource["type"]
    if kind == "bucket" and resource["public_access"]:
        yield "HIGH", "bucket is publicly accessible"
    if kind in ("bucket", "disk") and not resource["encrypted"]:
        yield "MEDIUM", "not encrypted at rest"
    if kind == "security_group":
        for rule in resource["inbound"]:
            if rule["port"] in ADMIN_PORTS and rule["source"] == "0.0.0.0/0":
                name = ADMIN_PORTS[rule["port"]]
                yield "HIGH", f"{name} (port {rule['port']}) open to the whole internet"
    if kind == "account" and resource["name"] == "root":
        if resource["access_keys"] > 0:
            yield "HIGH", "root account has an access key"
        if not resource["mfa"]:
            yield "HIGH", "root account has no MFA"


findings = []
for resource in RESOURCES:
    for severity, message in check(resource):
        findings.append((severity, resource["name"], message))

findings.sort(key=lambda f: (f[0] != "HIGH", f[1]))  # HIGH first, then by name
for severity, name, message in findings:
    print(f"{severity:<7}{name:<16}{message}")
print(f"{len(findings)} finding(s) in {len(RESOURCES)} resources")

HIGH   root            root account has an access key
HIGH   sg-admin        SSH (port 22) open to the whole internet
HIGH   shop-invoices   bucket is publicly accessible
MEDIUM db-data-01      not encrypted at rest
4 finding(s) in 7 resources

Port 443 open to the internet on sg-web is not flagged: a public web server is supposed to accept HTTPS from everyone. Real tools work the same way, with hundreds of rules, IPv6 (::/0) as well as IPv4, and exceptions for resources that are public on purpose. The hard part is not finding issues; it is getting each one fixed by its owner — another reason for good tagging.

Secure baselines and guardrails

Detecting mistakes is good; making them impossible is better. A secure baseline is the standard configuration every new account or subscription starts with: audit logging on and sent to a central, protected account, threat detection enabled, budget alerts, MFA enforced, default encryption on. Guardrails are organization-wide policies that block dangerous actions, whatever permissions a user has inside an account:

• AWS Organizations service control policies (SCPs) set the maximum permissions for every account in the organization. Even an administrator in a member account cannot do what an SCP denies.
• Azure Policy evaluates resources against rules and can deny non-compliant deployments (for example storage accounts that allow public access), audit them, or fix them automatically.
• Google Cloud Organization Policy Service applies constraints across an organization or folder, for example enforcing public access prevention on buckets or disabling the creation of service account keys.

An SCP that protects the audit trail: nobody in any member account can stop or delete CloudTrail logging. Real policies add a Condition that exempts a dedicated security-admin role (aws:PrincipalArn); SCPs never restrict the management account.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ProtectAuditTrail",
      "Effect": "Deny",
      "Action": [
        "cloudtrail:StopLogging",
        "cloudtrail:DeleteTrail",
        "cloudtrail:UpdateTrail",
        "cloudtrail:PutEventSelectors"
      ],
      "Resource": "*"
    }
  ]
}

Compliance

Compliance means being able to show that you meet the requirements of laws, regulations, contracts and standards. A compliance framework is a published set of such requirements, often with an independent audit or certification, that organizations use to demonstrate that their security and privacy controls are in place. Which frameworks apply depends on your sector and country; some common ones:

The GDPR and data residency

The General Data Protection Regulation (GDPR, in Dutch the AVG) is the European Union law on personal data. In cloud terms, the organization that decides why personal data is processed is the controller and stays responsible; the cloud provider is usually a processor, and the two need a data processing agreement. The GDPR also restricts transfers of personal data outside the European Economic Area, which is why choosing a region inside the EU (data residency, see Regions and Availability Zones) is often the first step. Whether that is enough — given support staff, subprocessors and foreign laws that may reach the provider — is a question for your privacy officer and lawyers, and the legal picture changes over time. This is also why some organizations look at sovereign cloud offerings (see Deployment Models).

Provider reports versus your own responsibility

Large providers are audited against dozens of frameworks and publish the reports: AWS Artifact, the Microsoft Service Trust Portal and the Google Cloud Compliance Reports Manager. You can use them as evidence for the controls the provider runs — physical security, hardware disposal, the hypervisor. They say nothing about your own configuration. A provider being PCI DSS compliant does not make your shop PCI DSS compliant; it means the infrastructure you build on can be part of a compliant setup, if you configure and operate your part correctly. Shared responsibility applies to compliance just as it does to security.

Cloud security builds on general security knowledge: see also Linux Security Basics for hardening a server, and the Security topic of this encyclopedia (coming later) for the wider field.