Quality in Agile Teams

The whole-team approach

In a traditional project, developers build and a separate test department tests afterward. Agile frameworks remove that hand-off. The Scrum Guide has no tester role at all: the Developers are everyone who creates the Increment, and a cross-functional team has all the skills it needs, testing included.

The whole-team approach, as the ISTQB (International Software Testing Qualifications Board) Foundation syllabus calls it, makes this explicit: everyone in the team is responsible for quality. Developers write automated tests, the Product Owner clarifies what "right" means, people with testing skills bring the testing mindset (curiosity, skepticism, attention to what could go wrong) and share it, and the team decides together how much testing an item needs.

Kestrel Energy's Portal team has no dedicated tester. Its five Developers (Ravi Kumar, Sofia Lindqvist, Kwame Asante, Priya Shah and Bram Peeters) all test, together with Product Owner Isabel Santos and Scrum Master Lars de Boer. That works only because they treat testing as a skill they practice, not as something that happens to code by itself.

The three amigos

Many bugs that reach users are not coding slips but misunderstandings: the team built what it thought was asked. The cheapest place to find them is a conversation before any code exists. The three amigos is a short meeting, a name usually credited to George Dinwiddie (around 2009), in which three perspectives look at one story together:

• Business: what problem does this solve, and for whom? Usually the Product Owner or a stakeholder.
• Development: how could we build it, and what is hard or risky about that?
• Testing: what could go wrong? What about the edge cases? How will we know it works?

The "amigos" are perspectives, not job titles; there may be more than three people. In the Portal team, Isabel Santos brings the business view on PBI-450 ("Show this month's costs split by day and night"), Ravi Kumar the development view, and Sofia Lindqvist takes the testing view this time. The conversation takes twenty minutes in a refinement session:

Questions raised (testing view)
- Is 07:00 day or night? And 23:00?            -> 07:00 day, 23:00 night
- What does the page show on 25 October (100 quarter-hours)?
                                               -> every quarter-hour, the
                                                  repeated hour labeled
- Dynamic contracts have no day/night split.   -> show "not applicable",
                                                  link to hourly prices
- Must the page total match the invoice to the cent?
                                               -> open: Isabel asks Joost
- Readings still missing for today?            -> show "incomplete" badge

Out of scope (new PBI)
- Comparing with the same month last year

Every answer becomes an example, and every open question is a risk that someone has to resolve before the item is ready. That one question about rounding would have prevented the difference Sofia later found by exploration (see Exploratory and Manual Testing).

Acceptance criteria and examples

Acceptance criteria are the conditions one item must meet; they are the "confirmation" in the card, conversation, confirmation of a user story. The ISTQB syllabus names two styles: scenario-oriented (the Given/When/Then format) and rule-oriented (a list of rules or a checklist). Rules alone are easy to misread. "Night is before 07:00 and from 23:00" is clear until somebody asks about 06:59:59.

Specification by example, a name popularized by Gojko Adzic's book of that title, answers with concrete examples: the team writes down key examples of input and expected outcome, agrees on them, and uses them both as the requirement and as the tests. For PBI-450, with the day price €0.3120 and the night price €0.2240 per kWh:

Written as scenarios, the same examples read like this. Tools such as Cucumber can run scenarios like these as automated tests; that practice, behavior-driven development, is covered on Test-Driven Development.

Scenario: a reading at the start of the day rate
  Given a customer with a day_night contract
  When a reading of 0.8 kWh for 07:00 arrives
  Then the cost page shows it at the day rate
  And its cost is EUR 0.25

Scenario: a reading at the start of the night rate
  Given a customer with a day_night contract
  When a reading of 0.8 kWh for 23:00 arrives
  Then the cost page shows it at the night rate
  And its cost is EUR 0.18

Scenario: the day the clocks go back
  Given a customer with a day_night contract
  When the customer opens the costs for 25 October 2026
  Then the page lists 100 quarter-hours
  And the two quarter-hours starting at 02:15 are labeled
      "summer time" and "winter time"

The Definition of Done

The Definition of Done is the quality bar that applies to every item. Acceptance criteria are specific to one item; an item is finished only when it meets both. The Portal team's Definition of Done is shown on Artifacts and Commitments. After an escaped bug in the night rule (BUG-1041, found by a customer, not by a test), the team will propose three additions at the Sprint 23 Retrospective on 16 October:

- Every pricing rule has automated boundary tests
  (e.g. 06:59 / 07:00 and 22:59 / 23:00)
- Changed screens pass an automated accessibility check without
  serious findings, plus a keyboard-only check by a person
- No open bugs with severity critical or major on the item

Each line is checkable, so nobody has to argue later about whether it was met. Avoid vague items such as "well tested" or "good quality", and avoid lines nobody can afford to meet: a Definition of Done that the team routinely ignores is worse than a shorter one it keeps. See Test Design Techniques for boundary tests and Accessibility Testing for the accessibility check.

Testing inside the Sprint

In an agile team, an item is not done until it is tested, so testing happens while the item is built, not after the Sprint. In practice that means:

• Tests start with the conversation. The examples from the three amigos become the first automated tests, often written before the code (see Test-Driven Development).
• Small items, finished one by one. Five stories that are each 80% done at the end of the Sprint deliver nothing. Limiting work in progress makes the team finish, and test, before starting the next item.
• Developers test each other's work. A second pair of eyes in a pull request, a pairing session, or a short exploratory session on someone else's story.
• Automated checks run on every change in the pipeline, so regressions show up in minutes (see Testing and Quality in Pipelines).

Two patterns signal that testing has quietly become a separate phase again. The first is a "testing" or "QA" column on the board where items pile up, waiting for the one person who tests: a small waterfall inside the Sprint. The second is a "hardening Sprint" every few Sprints to test and fix what was built before. Both mean the Increments were not really done, which is what the Water-Scrum-Fall anti-pattern looks like from the inside.

Test debt

Test debt is the testing work a team has postponed: tests not written, flaky tests not fixed, manual checks never automated, test data nobody understands any more. Like technical debt, it is sometimes a reasonable short-term choice, and it charges interest: every later change is slower and riskier because the safety net has holes. Make it visible as Product Backlog items (for example "Add boundary tests for the variable tariff price change") instead of a vague feeling, and pay it back steadily rather than in a hardening Sprint.

The agile testing quadrants

Which kinds of tests does a team need? The agile testing quadrants give a checklist. Brian Marick described two axes in a blog series in 2003; Lisa Crispin and Janet Gregory made the four quadrants widely known in their book Agile Testing. One axis asks whether a test is business-facing (described in terms a user or Product Owner understands) or technology-facing (described in terms of the code or the system). The other asks whether it supports the team while building, or critiques the product once something exists.

For the cost page, the quadrants map to concrete work:

Use the quadrants as a gap check in refinement or in the Sprint Retrospective: "which quadrant did we skip for this story?" A team with a strong Q1 and nothing in Q3 ships code that works exactly as written and still confuses customers.

Quality metrics

A team that wants to improve needs some numbers, and the wrong numbers do damage. An escaped defect is a defect that was not found before release and was found afterward, by users, by monitoring or by the service desk. BUG-1041 (the night rate at 23:00) is one: Mei Chen's service desk heard about it from a customer. Defect density is the number of defects divided by the size of what was examined, for example defects per thousand changed lines of code (KLOC). The Portal team compares four Sprints:

# Portal team, last four Sprints: defects found before release, defects
# found after release (escaped), and thousands of lines of code changed
sprints = [
    ("Sprint 19", 14, 1, 3.2),
    ("Sprint 20", 9, 2, 2.1),
    ("Sprint 21", 11, 3, 4.0),
    ("Sprint 22", 6, 4, 1.8),
]

print(f"{'Sprint':<10} {'found':>5} {'escaped':>7} {'escape rate':>11} {'per KLOC':>8}")
for name, found, escaped, kloc in sprints:
    total = found + escaped
    escape_rate = escaped / total
    density = total / kloc
    print(f"{name:<10} {found:>5} {escaped:>7} {escape_rate:>11.0%} {density:>8.1f}")

Sprint     found escaped escape rate per KLOC
Sprint 19     14       1          7%      4.7
Sprint 20      9       2         18%      5.2
Sprint 21     11       3         21%      3.5
Sprint 22      6       4         40%      5.6

Read the columns together. "Found" fell from 14 to 6, which on its own looks like better code. But the defect density stayed around five per thousand changed lines, so the code did not get better; the team found less of what was there, and the escape rate went from 7% to 40%. That is a signal about testing, not about any one person, and it is exactly what the Definition of Done additions above respond to. Two cautions: escaped defects keep arriving for weeks after a release, so Sprint 22's number is still incomplete; and four Sprints of small numbers are a hint to discuss, not a statistic.

The reason is Goodhart's law, usually quoted as "when a measure becomes a target, it ceases to be a good measure". Reward testers for bugs found and you get many small duplicate reports; punish developers for bugs and the bugs stop being logged. The agile topic covers this under metrics that hurt. Bring the trends to the Sprint Retrospective and ask the three questions the ISTQB syllabus suggests: what went well and should be kept, what did not and could be improved, and how to make the improvement happen.

The role of testers

If everyone tests, are testers still needed? Many agile teams do have one or more people who specialize in testing, and the whole-team approach changes what they do rather than removing them. A tester in an agile team typically:

• asks the testing questions in refinement and three amigos sessions, before code exists;
• helps developers design better automated tests (boundaries, partitions, test data) instead of writing all tests alone;
• explores new features and risky areas, and runs usability sessions;
• keeps an eye on the whole test suite: gaps across the quadrants, flaky tests, test debt;
• coaches the team in testing skills.

A tester in this sense is a member of the team, not a gatekeeper who approves releases. The ISTQB syllabus also notes the value of some independence: a person who did not write the code sees different things. Teams without testers, like Kestrel's Portal team, get some of that independence by having Developers test each other's stories and by rotating who takes the testing view.

AI in testing

AI assistants can draft test cases, suggest edge cases and write test code, and browser tools such as Playwright now include AI-driven test agents (see End-to-End and UI Testing). Treat that output like a colleague's first draft: review every test for what it actually asserts. A generated test that checks the current output of the code will happily confirm a bug as the expected behavior. Writing the tests first and letting the AI write the code is often safer; see test-first prompting. Testing AI systems themselves is a separate field, with its own ISTQB syllabus (CT-AI, version 2.0 since April 2026).