SLIs, SLOs and SLAs

The terms service level indicator, objective and agreement are introduced on the cloud page about high availability, and the ITSM page on service level management covers how SLAs are negotiated and reviewed. This page is about the engineering side: which numbers to measure, how to measure them, and how to pick a target that means something.

Start from the user

Users do not care whether db01's CPU is at 40%. They care whether they can log in, see their energy usage and download last month's invoice. Each of those is a critical user journey: a sequence of steps a user takes to get something done that matters to them and to the business. SRE starts from these journeys, not from the components.

For Kestrel Energy's customer portal the critical journeys are:

1. Log in.
2. View energy usage (the smart-meter readings).
3. Download an invoice as PDF.

Behind them sit web01 and web02, the meter-api that receives a reading from every smart meter every 15 minutes, PostgreSQL on db01 and the nightly billing export at 02:30. Every one of those can fail, but the users only notice when a journey fails. So the measurements that matter are the ones that describe journeys.

Choosing SLIs: good events and total events

A service level indicator (SLI) is a carefully defined measurement of some aspect of the service. The SRE Workbook recommends one shape for almost all of them: "the ratio of two numbers: the number of good events divided by the total number of events." A good event is one that met the user's expectation: a request that returned a correct answer, fast enough, or a reading that showed up in the portal in time. The ratio form has two advantages. It always lies between 0% and 100%, so every SLI reads the same way, and it maps straight onto an error budget: the bad events are what you spend.

Most services need two or three SLIs, not ten. For request-driven services, availability and latency cover most of what users feel. Data pipelines such as meter-api and the billing export need freshness or completion instead: nobody waits for a page to load, but a reading that arrives a day late is still a failure.

Request-based and time-based availability

There are two ways to turn "the service was available" into a number, and they give different answers for the same day. Time-based availability is the share of time slices (often minutes) in which the service was considered up: good minutes divided by all minutes. It is the classic uptime figure, and what most SLAs use. Request-based availability is the share of requests that succeeded: good requests divided by all requests. The SRE book calls it aggregate availability and explains why Google prefers it: a large service is rarely completely down, but some of its requests often fail.

The example below takes one day of portal traffic with two incidents that each fail exactly 600 requests: a 30-minute outage at 03:00, when only 20 people a minute use the portal, and four minutes at 14:00 when one of two web servers fails a quarter of the requests at 600 requests a minute.

# One day of portal traffic, minute by minute: (requests, failed)
minutes = []
for minute in range(24 * 60):
    hour = minute // 60
    requests = 20 if hour < 6 else 600      # quiet at night, busy by day
    failed = 0
    if 180 <= minute < 210:                  # 03:00-03:30 total outage
        failed = requests
    if 840 <= minute < 844:                  # 14:00-14:04 one node fails
        failed = requests // 4
    minutes.append((requests, failed))

total = sum(r for r, _ in minutes)
good = sum(r - f for r, f in minutes)
print(f"Requests: {total:,}, good: {good:,}")
print(f"Request-based availability: {good / total:.3%}")

# Time-based: a minute counts as good if fewer than 1% of its requests failed
good_minutes = sum(1 for r, f in minutes if f / r < 0.01)
print(f"Good minutes: {good_minutes} of {len(minutes)}")
print(f"Time-based availability: {good_minutes / len(minutes):.3%}")

Requests: 655,200, good: 654,000
Request-based availability: 99.817%
Good minutes: 1406 of 1440
Time-based availability: 97.639%

Request-based availability weighs both incidents equally, because both hurt the same number of requests. Time-based availability counts the night outage as 30 bad minutes and the daytime one as 4, although far more people noticed the second. Neither is wrong; they answer different questions. Request-based numbers follow user impact and work well for busy services. Time-based numbers are easier to explain in a contract and do not swing wildly when there is little traffic, which matters at night (see low-traffic services). Whichever you choose, write down how a "bad minute" is defined.

SLIs in Prometheus

With Prometheus (see Prometheus and PromQL), an availability SLI is a ratio of two counter rates. The query below divides the rate of meter-api requests without a 5xx status by the rate of all meter-api requests over the last five minutes. It was recorded during a test incident in which meter-api failed about one request in ten:

sum(rate(http_requests_total{job="meter-api",code!~"5.."}[5m]))
/
sum(rate(http_requests_total{job="meter-api"}[5m]))

promtool query instant http://localhost:9090 'sum(rate(http_requests_total{job="meter-api",code!~"5.."}[5m])) / sum(rate(http_requests_total{job="meter-api"}[5m]))'

{} => 0.9180620585667805 @[1791306938.499]

91.8% of requests succeeded in those five minutes. The number after @ is the evaluation time as a Unix timestamp.

A latency SLI uses the histogram's buckets: the requests in the bucket for "at most 0.5 seconds" divided by all requests. The customer portal slowed down in the same test, and the same query shifted ten minutes back shows the normal situation:

sum(rate(http_request_duration_seconds_bucket{job="portal",le="0.5"}[5m]))
/
sum(rate(http_request_duration_seconds_count{job="portal"}[5m]))

{} => 0.5338541666666666 @[1791306938.627]

During the slowdown only 53.4% of portal requests were answered within half a second; ten minutes earlier (the same query with offset 10m) it was 97.6%. Note that the threshold must be one of the histogram's bucket boundaries. The portal uses the common default buckets (…, 0.25, 0.5, 1, …), so the 300 ms target in the table above cannot be measured exactly until someone adds a 0.3 bucket. Choose SLO thresholds and buckets together; the metrics page explains histograms.

Recording the SLI per window

Alerts and dashboards need the same ratio over several windows, so teams store it with recording rules. The naming follows the SRE Workbook's convention, job:slo_errors_per_request:ratio_rate5m: the error ratio (1 minus the SLI) per job, over five minutes. This is the first group of Kestrel's rule file for meter-api; the error budget page adds the alerts that use it.

rules/slo-meter-api.yml (first part)

# SLO: 99.9% of meter-api requests succeed (no 5xx), measured over 30 days.
# Error budget: 0.1% of requests.
groups:
  - name: slo-meter-api-sli
    rules:
      - record: job:slo_errors_per_request:ratio_rate5m
        expr: |
          sum by (job) (rate(http_requests_total{job="meter-api",code=~"5.."}[5m]))
          /
          sum by (job) (rate(http_requests_total{job="meter-api"}[5m]))
      - record: job:slo_errors_per_request:ratio_rate30m
        expr: |
          sum by (job) (rate(http_requests_total{job="meter-api",code=~"5.."}[30m]))
          /
          sum by (job) (rate(http_requests_total{job="meter-api"}[30m]))
      - record: job:slo_errors_per_request:ratio_rate1h
        expr: |
          sum by (job) (rate(http_requests_total{job="meter-api",code=~"5.."}[1h]))
          /
          sum by (job) (rate(http_requests_total{job="meter-api"}[1h]))
      - record: job:slo_errors_per_request:ratio_rate6h
        expr: |
          sum by (job) (rate(http_requests_total{job="meter-api",code=~"5.."}[6h]))
          /
          sum by (job) (rate(http_requests_total{job="meter-api"}[6h]))

The four recorded series, queried together seven minutes into the test incident:

promtool query instant http://localhost:9090 '{__name__=~"job:slo_errors_per_request:ratio_rate.*"}'

job:slo_errors_per_request:ratio_rate5m{job="meter-api"} => 0.08178246894964486 @[1791306938.37]
job:slo_errors_per_request:ratio_rate30m{job="meter-api"} => 0.04266863068151602 @[1791306938.37]
job:slo_errors_per_request:ratio_rate1h{job="meter-api"} => 0.04266863068151602 @[1791306938.37]
job:slo_errors_per_request:ratio_rate6h{job="meter-api"} => 0.04266863068151602 @[1791306938.37]

Setting objectives

A service level objective (SLO) is a target for an SLI over a period: "99.9% of portal requests succeed over a rolling 30 days". Choosing the number is a business decision with engineering input:

• Start from what you achieve now. If the portal reached 99.95% in each of the last three months, 99.9% is a target you can keep while still releasing. Tighten it later if users complain at that level; loosen it if it blocks all change.
• Ask what users need. A dashboard used once a day can be less reliable than login. Different journeys may get different SLOs.
• Never 100%. It leaves no room for any change, and your users' own networks are less reliable than that anyway (see embracing risk).
• Use percentiles or thresholds, not averages. An average latency of 120 ms can hide that one request in twenty on the invoice-PDF route takes almost a second (see tail latency). "99% of requests within 300 ms" says something users can feel.

Kestrel's portal team settles on these SLOs:

Rolling and calendar windows

Every SLO needs a period. A rolling window always covers the most recent stretch of time, such as the last 30 days: an incident counts against the SLO for exactly 30 days and then drops out. A calendar window is fixed to the calendar, such as October or the third quarter, and starts again from zero on the first day of the next one.

Rolling windows match how users feel: a bad outage on 30 September is not forgotten on 1 October. They are the usual choice for engineering decisions and alerts. Calendar windows match how businesses report and bill, which is why SLAs almost always use them. The billing export uses a calendar quarter on purpose, because it runs only once a night. Over 30 nights, a single failed run is already 3.3% of the runs, more than a 98% target allows; over a quarter of about 91 nights, a 98% target allows one bad night and still catches a pattern.

SLOs versus SLAs

The SRE book gives a simple test for telling them apart: ask "what happens if the SLOs aren't met?" If nothing explicit happens, it is an SLO. A service level agreement (SLA) is a contract that includes consequences, usually service credits or penalties. The book also notes that SRE "doesn't typically get involved in constructing SLAs": that is the job of the business and legal people, with service level management in between.

The relation matters for the numbers. Margriet de Wit, Kestrel's CFO, signed an SLA with a large business customer that promises 99.5% monthly portal availability, with a service credit if it is missed. The internal SLO is 99.9%. The gap is deliberate: when the team starts slowing down releases because the SLO is at risk, the contract is still far away.

Kestrel's internal SLA with Customer Operations (see service level management) also says 99.9%, but per calendar month, measured by a synthetic login, and without service credits; by the SRE book's test it works as an objective, and the agreement with consequences is the 99.5% contract.

The nines, and which month

Availability targets are often spoken of in nines: the number of nines in the percentage, so "three nines" is 99.9% and "three and a half nines" is 99.95% (the SRE book's target for Google Compute Engine). Each extra nine divides the allowed failure by ten.

Converting a percentage into minutes needs a period, and here the site has two conventions on purpose. The cloud page's nines table uses a month of one twelfth of a 365-day year (43,800 minutes); the ITSM page on availability per month uses 30 days (43,200 minutes), as most SLO tooling and the SRE Workbook do. This topic always uses 30 days = 43,200 minutes for SLO windows. The table below uses that basis and a 365-day year, and adds the request budget for a service with 1.2 million requests in 30 days:

MONTH = 30 * 24 * 60        # minutes in a 30-day window: 43,200
YEAR = 365 * 24 * 60        # minutes in a 365-day year: 525,600


def duration(minutes):
    """Format minutes as hours and minutes, or minutes and seconds."""
    seconds = round(minutes * 60)
    hours, rest = divmod(seconds, 3600)
    mins, secs = divmod(rest, 60)
    if hours:
        return f"{hours} h {mins:02d} min"
    if mins:
        return f"{mins} min {secs:02d} s"
    return f"{secs} s"


print(f"{'Target':<9}{'per 30 days':>14}{'per year':>14}{'failed per 1.2M req':>22}")
for target in ("99", "99.5", "99.9", "99.95", "99.99", "99.999"):
    budget = 1 - float(target) / 100
    failed = round(1_200_000 * budget)
    print(f"{target + '%':<9}{duration(budget * MONTH):>14}"
          f"{duration(budget * YEAR):>14}{failed:>22,}")

Target      per 30 days      per year   failed per 1.2M req
99%          7 h 12 min   87 h 36 min                12,000
99.5%        3 h 36 min   43 h 48 min                 6,000
99.9%       43 min 12 s    8 h 45 min                 1,200
99.95%      21 min 36 s    4 h 22 min                   600
99.99%       4 min 19 s   52 min 34 s                   120
99.999%            26 s    5 min 15 s                    12

Hours drop their seconds in this format: 99.9% per year is exactly 8 h 45 min 36 s. For 99.9%, the three month bases give 43 min 12 s (30 days), 43 min 48 s (a year/12) and 44 min 38 s (31 days). The difference looks small until a contract dispute turns on it, so every SLO and SLA document should name its period.

For request-based SLOs the budget is a count, not a time: at 99.9%, a service with 1.2 million requests in 30 days may fail 1,200 of them, whether that happens in one burst or spread out.

Writing SLOs down: OpenSLO

An SLO that only exists in a slide deck is forgotten. At minimum, write each one down with its SLI definition (what counts as good, measured where), the target, the window, the owner and the date of the next review. OpenSLO is an open specification (Apache 2.0) for exactly that, in YAML, so SLOs can live in version control and be read by tools. As of October 2026 the stable schema is openslo/v1. The portal's availability SLO looks like this:

slo/portal-availability.yaml

apiVersion: openslo/v1
kind: SLO
metadata:
  name: portal-availability
  displayName: Customer portal availability
spec:
  description: 99.9% of portal requests succeed over a rolling 30 days
  service: customer-portal
  indicator:
    metadata:
      name: portal-non-5xx
    spec:
      ratioMetric:
        counter: true
        good:
          metricSource:
            type: Prometheus
            spec:
              query: sum(http_requests_total{job="portal", code!~"5.."})
        total:
          metricSource:
            type: Prometheus
            spec:
              query: sum(http_requests_total{job="portal"})
  timeWindow:
    - duration: 30d
      isRolling: true
  budgetingMethod: Occurrences
  objectives:
    - displayName: Good
      target: 0.999

The fields follow this page: ratioMetric holds the good and total events, timeWindow the window (rolling here; a calendar window has a calendar: block instead), and budgetingMethod chooses between counting events (Occurrences, request-based) and counting good time slices (Timeslices, time-based). Some SLO tools read OpenSLO directly or generate Prometheus rules from it; even without a tool, the file is a precise, reviewable definition.