Quality Attributes and Trade-offs

What quality attributes are

Two systems can have exactly the same features and still be very different: one answers in 100 ms and survives a server failure, the other takes five seconds and is down every Monday. A quality attribute is a measurable property of a system that says how well it does its job rather than what it does: how fast, how available, how secure, how easy to change. Features rarely force an architecture; quality attributes do. "Show the invoice" fits any structure. "Show the invoice within 300 ms while the billing export is running" does not.

The older name is non-functional requirement (NFR): any requirement about how a system should behave rather than which function it offers. The term is everywhere in tenders and requirement documents, but many architects avoid it because "non-functional" sounds optional. A portal that shows the right invoice to the wrong customer is fully functional and completely unacceptable. "Quality requirement" says the same without that suggestion.

Constraints are different

An architectural constraint is a decision that has already been made for you and is not up for trade-offs: "customer data stays in the EU", "billing runs on our own servers in the colocation", "the database is PostgreSQL because that is what the DBA team supports". A quality attribute has a target you can do better or worse on; a constraint is a fixed boundary. Keep them in separate lists. Mixing them hides which things can still be discussed, and a constraint that nobody wrote down tends to be discovered after the design is finished.

The ISO/IEC 25010 quality model

Lists of qualities are easy to start and hard to complete. A standard checklist helps you notice the ones you forgot. ISO/IEC 25010 is the international standard that defines a product quality model: a list of quality characteristics, each split into sub-characteristics, for describing and evaluating software. Its second edition, ISO/IEC 25010:2023, is current as of October 2026 and has nine characteristics:

The standard itself uses British spelling for names such as time behavior and analyzability; this page uses US spelling. Quality in use (how effective and satisfying the system is for its users in practice) moved out of 25010 into a separate standard, ISO/IEC 25019:2023.

Maintainability

Maintainability is how easily and safely a system can be changed: to fix a bug, add a feature or adapt to a new rule. ISO 25010 splits it into modularity (a change stays in one part), reusability, analyzability (you can find out what is wrong and what a change would affect), modifiability and testability. It is the quality nobody asks for in a tender and everybody pays for later: most of a business system's cost comes after its first release. The design principles and layers and boundaries in this topic are mostly about maintainability.

Scalability

Scalability is a system's ability to handle more work, more users, more data or more requests, by adding resources, without a redesign and without performance falling off a cliff. ISO 25010:2023 files it under Flexibility. It is not the same as performance: a system can be fast for 1,000 users and unusable at 10,000, or slow but able to grow in a straight line. Scaling is done by adding machines (horizontal scaling) or making one machine bigger (vertical scaling); the patterns that make the first possible are on Scalability and Caching Patterns.

Quality attribute scenarios

"The system must be fast and scalable" cannot be tested, so it cannot be met or failed, so it will not influence any decision. A quality attribute scenario is a short, testable description of how the system must respond to one specific event, with a number attached. The form comes from Bass, Clements and Kazman's Software Architecture in Practice, which gives it six parts:

• Source: who or what causes the event (smart meters, a customer, an attacker, a developer);
• Stimulus: the event itself (a burst of readings, a login attempt, a change request);
• Environment: the conditions at that moment (normal operation, peak load, one server down);
• Artifact: the part of the system that is hit (meter-api, KBS, the Tariffs module);
• Response: what the system should do;
• Response measure: the number that says whether it did.

Ravi Kumar and Joost Bakker wrote four scenarios for meter-api and KBS. The numbers for freshness and the export come straight from the service level objectives Kestrel already measures, which is the point: a scenario and an SLO are two views of the same target.

Because each scenario ends in a number, it can be checked, in a load test, from monitoring, or from the change history:

from dataclasses import dataclass


@dataclass
class Scenario:
    name: str
    stimulus: str
    response: str
    measure: str        # the response measure in words ...
    limit: float        # ... and as a number
    higher_is_better: bool


scenarios = [
    Scenario("reading burst", "400,000 readings arrive within 60 s after :00",
             "all readings stored, none lost", "seconds until the last one is stored",
             300, False),
    Scenario("freshness", "a meter sends a reading",
             "reading visible in the portal", "% visible within 30 minutes",
             99.5, True),
    Scenario("export after failover", "db01 fails over during the 02:30 export",
             "export resumes from the last committed batch", "minutes after 02:30 until done",
             210, False),
    Scenario("new tariff", "billing asks for a dynamic tariff",
             "change only in the Tariffs module", "working days until in production",
             5, False),
]

# What load tests, monitoring and the last change actually showed
measured = {"reading burst": 212, "freshness": 99.71,
            "export after failover": 241, "new tariff": 8}

for s in scenarios:
    value = measured[s.name]
    ok = value >= s.limit if s.higher_is_better else value <= s.limit
    sign = ">=" if s.higher_is_better else "<="
    print(f"{'PASS' if ok else 'FAIL'}  {s.name:22} {value:>6} (target {sign} {s.limit}: {s.measure})")

PASS  reading burst             212 (target <= 300: seconds until the last one is stored)
PASS  freshness               99.71 (target >= 99.5: % visible within 30 minutes)
FAIL  export after failover     241 (target <= 210: minutes after 02:30 until done)
FAIL  new tariff                  8 (target <= 5: working days until in production)

The two failures are the useful output. The export failure says something about recovery (restarting from the beginning instead of from the last batch); the tariff failure says something about modularity (the last tariff change touched invoicing and the database as well). Both point at architecture, not at a bug.

From numbers to decisions

A measurable scenario lets you calculate before you build. Meters send one reading per 15 minutes, which averages out to a modest load, but most of them send in the first minute of each quarter hour. How much capacity does meter-api need to meet the burst scenario?

meters = 400_000
average = meters / (15 * 60)              # one reading per meter per 15 minutes
burst = meters / 60                       # ... but most arrive in the first minute
print(f"average load: {average:,.0f} readings/s, burst: {burst:,.0f} readings/s")

for capacity in (1_000, 2_000, 7_000):    # readings meter-api can store per second
    done_after = max(60, meters / capacity)
    busy = average / capacity
    verdict = "meets" if done_after <= 300 else "misses"
    print(f"{capacity:>5}/s: last reading stored after {done_after:3.0f} s "
          f"({verdict} 300 s), average utilization {busy:.0%}")

average load: 444 readings/s, burst: 6,667 readings/s
 1000/s: last reading stored after 400 s (misses 300 s), average utilization 44%
 2000/s: last reading stored after 200 s (meets 300 s), average utilization 22%
 7000/s: last reading stored after  60 s (meets 300 s), average utilization 6%

Without the scenario, the obvious answer is "size for the peak": 7,000 readings per second, about 16 times the average, mostly idle. With the scenario, 2,000 per second is enough, provided readings can wait in a queue for up to a few minutes without being lost. That turns a hardware question into an architectural one (accept the readings quickly, store them a little later), which Service Communication picks up. Queueing has its own costs, explained in Capacity Planning and Performance.

Trade-offs between qualities

Qualities compete. Almost every technique that improves one costs something in another, so "maximize all of them" is not a strategy. Some pairs that come up again and again:

Evaluation methods have names for this. A sensitivity point is a decision that strongly affects one quality (the size of meter-api's queue decides how long readings may be delayed). A trade-off point is a decision that affects several qualities in opposite directions (moving the export to a read replica of db01 protects the portal's performance, but the export may then read data that is a few seconds old). Finding the trade-off points is the most valuable part of reviewing an architecture, because that is where a change for one stakeholder hurts another.

Ranking qualities: the utility tree

The Architecture Tradeoff Analysis Method (ATAM), developed at the Software Engineering Institute (SEI) of Carnegie Mellon University, is a structured way to evaluate an architecture against its quality goals. Its outputs are risks, non-risks, sensitivity points, trade-off points and recurring risk themes. A full ATAM takes days and many stakeholders, but one of its tools is useful on its own.

A utility tree is a hierarchy that starts at "utility" (overall goodness), branches into quality attributes, and ends in concrete scenarios. Each scenario gets two ratings, high, medium or low: how important it is to the business, and how hard it is to achieve. The scenarios rated high on both are where the architecture needs the most attention.

# Utility tree for KBS: quality -> scenario -> (importance, difficulty)
tree = {
    "Performance": {
        "Reading burst at :00 stored within 5 minutes": ("H", "M"),
    },
    "Availability": {
        "Billing export finishes by 06:00 after a db01 failover": ("H", "H"),
        "Portal invoices readable during a KBS deployment": ("M", "M"),
    },
    "Maintainability": {
        "New dynamic tariff in production within 5 working days": ("H", "H"),
        "Invoice layout change without a database change": ("M", "L"),
    },
    "Security": {
        "Invoice of another customer never shown in the portal": ("H", "L"),
    },
}

for quality, scenarios in tree.items():
    print(quality)
    for scenario, (importance, difficulty) in scenarios.items():
        print(f"  ({importance},{difficulty}) {scenario}")

rank = {"H": 3, "M": 2, "L": 1}
leaves = [(rank[i] + rank[d], rank[i], s) for q in tree.values() for s, (i, d) in q.items()]
print("\nDiscuss first:")
for _, _, scenario in sorted(leaves, reverse=True)[:3]:
    print(" ", scenario)

Performance
  (H,M) Reading burst at :00 stored within 5 minutes
Availability
  (H,H) Billing export finishes by 06:00 after a db01 failover
  (M,M) Portal invoices readable during a KBS deployment
Maintainability
  (H,H) New dynamic tariff in production within 5 working days
  (M,L) Invoice layout change without a database change
Security
  (H,L) Invoice of another customer never shown in the portal

Discuss first:
  New dynamic tariff in production within 5 working days
  Billing export finishes by 06:00 after a db01 failover
  Reading burst at :00 stored within 5 minutes

"Invoice of another customer never shown" is rated high importance but low difficulty: it matters enormously, but the team knows how to do it (access checks and tests), so it needs discipline rather than architectural debate. The numbers are judgment, and the sort order is only a way to start the conversation; the real output is that Femke Jansen (billing), Hamid Rahimi (database) and the developers agreed on what "good" means before arguing about solutions.