Test Design Techniques

Why use techniques?

Asked to test the night rule, most people try a time in the night and a time in the day, see two green results and stop. That is how the 23:00 defect on What Is Software Testing? survived. Exhaustive testing is impossible, so the question is which few cases to pick. A test design technique is a systematic procedure for deriving test cases (and for knowing when you have enough), so that the choice does not depend on luck or mood. Two people applying the same technique to the same rule should end up with largely the same tests.

The techniques on this page come mostly from the ISTQB (International Software Testing Qualifications Board) Foundation Level syllabus, which groups them into black-box, white-box and experience-based techniques, plus collaboration-based approaches such as writing acceptance criteria together (see test-driven development). The result of applying a technique is a list of test conditions and values; turning them into documented test cases or automated tests is the next step.

Black-box and white-box testing

Black-box testing derives tests from what the software should do (requirements, user stories, business rules, the documentation of an API) without looking at the code inside. It finds missing and misunderstood behavior, and the tests stay valid when the code is rewritten.

White-box testing (also called structure-based or glass-box testing) derives tests from the internal structure of the code: its statements, branches and paths. It finds code that no test runs, but it cannot find behavior that is missing from the code altogether. The two complement each other; most of this page is black-box, and the white-box techniques section below introduces the other side.

Equivalence partitioning

Equivalence partitioning divides the possible values of an input (or output) into groups, called partitions, that the software is expected to handle in the same way. If 02:30 is priced at the night rate, then 03:10 surely is too: they are in the same partition, so testing both adds little. One representative value per partition covers it. Mathematically, the partitions are the classes of an equivalence relation: every value belongs to exactly one.

The night rule has three partitions, not two. The output has two values (night, day), but the input splits into three ranges, and the code handles the two night ranges with different conditions (hour < 7 and hour >= 23). A defect in one condition leaves the other range working, which is exactly what happened with BUG-1041.

Partitions apply to any input. The energy of one 15-minute reading, as the reading validator sees it:

Do not skip the invalid partitions. Real meters send negative values after a replacement, impossible spikes after a firmware update, and exports from spreadsheets with decimal commas. Test one invalid value at a time: a test that combines a negative value with a missing meter ID cannot tell you which check rejected it.

Boundary value analysis

Defects gather at the edges of partitions, because that is where a programmer chooses between < and <=, starts a loop at 0 or 1, or decides whether the end time is included. Boundary value analysis (BVA) tests those edges: the minimum and maximum value of each ordered partition and the values right next to them. It only applies to partitions with an order (numbers, times, dates, lengths), and it is one of the most productive techniques there is.

For the night rule, with a precision of one minute, the boundary values are 06:59 and 07:00, and 22:59 and 23:00. These four tests in pytest, run against Priya's code with hour > 23:

from datetime import time

import pytest


def is_night(at):
    return at.hour < 7 or at.hour > 23  # the defect from BUG-1041


BOUNDARIES = [
    pytest.param(time(6, 59), True, id="06:59"),
    pytest.param(time(7, 0), False, id="07:00"),
    pytest.param(time(22, 59), False, id="22:59"),
    pytest.param(time(23, 0), True, id="23:00"),
]


@pytest.mark.parametrize("at, expected", BOUNDARIES)
def test_night_rate_boundaries(at, expected):
    assert is_night(at) == expected

...F                                                                     [100%]
=================================== FAILURES ===================================
______________________ test_night_rate_boundaries[23:00] _______________________

at = datetime.time(23, 0), expected = True

    @pytest.mark.parametrize("at, expected", BOUNDARIES)
    def test_night_rate_boundaries(at, expected):
>       assert is_night(at) == expected
E       assert False == True
E        +  where False = is_night(datetime.time(23, 0))

test_example.py:20: AssertionError
=========================== short test summary info ============================
FAILED test_example.py::test_night_rate_boundaries[23:00] - assert False == True
1 failed, 3 passed in 0.18s

The boundary test at 23:00 catches the defect at once, where the "obvious" tests at 02:30 and 14:00 never could. The table of boundary values feeds one parameterized test, and the readable ids ([23:00]) make the failing case obvious in the report; the page on unit testing with pytest explains parametrize in detail. With >= 23 restored, all four pass.

Two-value and three-value boundaries

There are two common variants. Two-value BVA tests each boundary value and its closest neighbor in the adjacent partition: 22:59 and 23:00. Three-value BVA tests the boundary value and both its neighbors: 22:59, 23:00 and 23:01. The third value catches a rarer kind of defect. The ISTQB syllabus gives the classic example: a rule "x ≤ 10" coded by mistake as "x = 10". Two-value BVA tests 10 and 11, and both give the right answer; only the three-value test at 9 reveals the defect.

The neighbor depends on the precision of the input. Times in minutes have neighbors one minute apart; meter readings arrive per quarter-hour, so in the reading pipeline the neighbors of 23:00 are 22:45 and 23:15. For energy recorded with three decimals, the boundaries of the validator are −0.001 and 0, and 25 and 25.001. Choosing the wrong precision (testing 24 and 26 instead of 25 and 25.001) leaves a gap where defects hide; the page on mutation testing shows a tool finding exactly such gaps.

The boundaries of a day

Boundaries are not only in numbers you can see. "A day has 96 quarter-hours" is true on most days, and Kestrel's code assumed it. But a day in the Netherlands is a local calendar day, and twice a year the clocks change:

from datetime import date, datetime, timedelta
from zoneinfo import ZoneInfo

AMSTERDAM = ZoneInfo("Europe/Amsterdam")


def quarter_hours_in_day(day):
    """Number of 15-minute readings a meter sends on a local calendar day."""
    start = datetime(day.year, day.month, day.day, tzinfo=AMSTERDAM)
    end = start + timedelta(days=1)  # same wall-clock time, next day
    seconds = end.timestamp() - start.timestamp()
    return int(seconds // 900)


for day in [date(2026, 3, 29), date(2026, 10, 7), date(2026, 10, 25)]:
    print(day, quarter_hours_in_day(day))

2026-03-29 92
2026-10-07 96
2026-10-25 100

On 29 March 2026 the clocks went forward and the day had 23 hours; on 25 October 2026 they go back and the day has 25. The partitions for "quarter-hours in a day" are therefore 92, 96 and 100, and any test of daily totals needs all three. The cost page still assumed 96, so the team recorded the 100-quarter-hour problem as BUG-1042 (see test cases and bug reports). Other boundaries of time worth a test: midnight, the first and last day of a month, 29 February (next in 2028) and the turn of the year.

Decision table testing

Partitions and boundaries look at one input at a time. When the result depends on a combination of conditions, use a decision table. Decision table testing lists the conditions as rows, every relevant combination as a column (a rule), and the expected action for each rule; every rule becomes at least one test. The table also exposes gaps: a combination nobody specified, or two rules that contradict each other. It is the testing cousin of a truth table.

Which price per kWh applies depends on the tariff type, on whether the reading falls in the night and, for the variable tariff, on the date (its price rose from €0.2500 to €0.2600 on 1 August 2026). Written out in full, that is 4 × 2 × 2 = 16 combinations, but many conditions do not matter for some tariffs. A dash ("—") marks a condition that does not influence the outcome, which collapses the table to six rules:

Each column becomes one parametrized case. For a "don't care" condition the test still has to pass some value; choose one deliberately, and vary it between rules, so that the table's claim "this does not matter" is tested too. The date for R4 and R5 sits on the boundary, 31 July and 1 August. The dynamic tariff uses a fixed market price, a stub standing in for the real price feed (see test doubles):

from datetime import date
from decimal import Decimal

import pytest

VARIABLE_PRICE_CHANGE = date(2026, 8, 1)


def price_per_kwh(tariff, night, day, market_price=None):
    """Price in euros per kWh for one reading."""
    if tariff == "fixed":
        return Decimal("0.2700")
    if tariff == "day_night":
        return Decimal("0.2240") if night else Decimal("0.3120")
    if tariff == "variable":
        if day >= VARIABLE_PRICE_CHANGE:
            return Decimal("0.2600")
        return Decimal("0.2500")
    if tariff == "dynamic":
        return market_price
    raise ValueError(f"unknown tariff: {tariff}")


# One test per rule (column) of the decision table.
RULES = [
    pytest.param("fixed", True, date(2026, 9, 1), "0.2700", id="R1-fixed"),
    pytest.param("day_night", True, date(2026, 9, 1), "0.2240", id="R2-dn-night"),
    pytest.param("day_night", False, date(2026, 9, 1), "0.3120", id="R3-dn-day"),
    pytest.param("variable", False, date(2026, 7, 31), "0.2500", id="R4-var-before"),
    pytest.param("variable", True, date(2026, 8, 1), "0.2600", id="R5-var-from"),
    pytest.param("dynamic", False, date(2026, 9, 1), "0.1874", id="R6-dynamic"),
]


@pytest.mark.parametrize("tariff, night, day, expected", RULES)
def test_price_rule(tariff, night, day, expected):
    market = Decimal("0.1874")  # stub value for the hourly market price
    assert price_per_kwh(tariff, night, day, market) == Decimal(expected)


def test_unknown_tariff_is_rejected():
    with pytest.raises(ValueError, match="unknown tariff"):
        price_per_kwh("weekend", False, date(2026, 9, 1))

============================= test session starts ==============================
collecting ... collected 7 items

test_example.py::test_price_rule[R1-fixed] PASSED                        [ 14%]
test_example.py::test_price_rule[R2-dn-night] PASSED                     [ 28%]
test_example.py::test_price_rule[R3-dn-day] PASSED                       [ 42%]
test_example.py::test_price_rule[R4-var-before] PASSED                   [ 57%]
test_example.py::test_price_rule[R5-var-from] PASSED                     [ 71%]
test_example.py::test_price_rule[R6-dynamic] PASSED                      [ 85%]
test_example.py::test_unknown_tariff_is_rejected PASSED                  [100%]

============================== 7 passed in 0.19s ===============================

The last test covers the combination the table leaves out: a tariff that does not exist. Decision tables grow quickly (with n yes/no conditions there are 2ⁿ combinations, see the product rule), so collapse "don't care" conditions and split large tables by topic. When the combinations are settings rather than business rules, pairwise testing (below) is usually the better tool.

State transition testing

Some behavior depends on history: what the system does next depends on what happened before. A meter reading at Kestrel is first received, then either validated or rejected; a validated reading is billed, and a rejected one can be corrected and resubmitted. State transition testing models such behavior as states and the events that move between them, and derives tests from the model. The Portal team drew its model of a reading like this:

From the model come three kinds of tests, in increasing strength:

• All states: every state is reached at least once.
• All valid transitions: every arrow is taken at least once. This is the usual minimum, and it implies all states.
• All transitions, including invalid ones: every combination of state and event that is not an arrow is tried, and the system must refuse it. Billing a rejected reading, or resubmitting a billed one, would be a serious defect.

With four states and four events there are 16 combinations: four valid and twelve invalid. Writing the model as a dictionary lets the tests generate both lists from it:

import pytest

# The Portal team's model of a reading: (state, event) -> next state.
TRANSITIONS = {
    ("received", "pass_checks"): "validated",
    ("received", "fail_checks"): "rejected",
    ("rejected", "resubmit"): "received",
    ("validated", "bill"): "billed",
}
STATES = ["received", "validated", "rejected", "billed"]
EVENTS = ["pass_checks", "fail_checks", "resubmit", "bill"]


def next_state(state, event):
    try:
        return TRANSITIONS[(state, event)]
    except KeyError:
        raise ValueError(f"{event} is not allowed in state {state}") from None


@pytest.mark.parametrize("state, event", list(TRANSITIONS))
def test_valid_transition(state, event):
    assert next_state(state, event) == TRANSITIONS[(state, event)]


INVALID = [
    (s, e) for s in STATES for e in EVENTS if (s, e) not in TRANSITIONS
]


@pytest.mark.parametrize("state, event", INVALID)
def test_invalid_transition_is_refused(state, event):
    with pytest.raises(ValueError, match="not allowed"):
        next_state(state, event)


def test_rejected_reading_can_be_billed_after_correction():
    state = "received"
    for event in ["fail_checks", "resubmit", "pass_checks", "bill"]:
        state = next_state(state, event)
    assert state == "billed"

.................                                                        [100%]
17 passed in 0.17s

Four valid transitions, twelve invalid ones and one sequence: 17 tests. The last test checks a whole path through the model, which single transitions cannot: that a reading that was once rejected can still end up on a bill. Here the code under test is the table, which keeps the example short; in the portal the same tests call the real reading service, and the model in the test is the specification it is checked against.

Pairwise testing

The cost page must work in four browsers (Chrome, Edge, Firefox, Safari), for four tariffs, on desktop and phone, in Dutch and English. All combinations: 4 × 4 × 2 × 2 = 64 test runs, and every new browser or language multiplies the number again. Research by the US National Institute of Standards and Technology (NIST) found that most software failures are caused by one or two parameters together, with progressively fewer by three or more. Pairwise testing uses that: instead of every combination, it picks a set of tests in which every pair of values (every browser with every tariff, every tariff with every language, and so on) appears at least once.

Tools such as Microsoft's PICT or NIST's ACTS generate such sets. A simple greedy search shows the idea: keep picking the configuration that covers the most pairs not yet covered.

from itertools import combinations, product

FACTORS = {
    "browser": ["Chrome", "Edge", "Firefox", "Safari"],
    "tariff": ["day_night", "dynamic", "fixed", "variable"],
    "device": ["desktop", "phone"],
    "language": ["en", "nl"],
}
names = list(FACTORS)


def pairs_of(row):
    """All (factor, value) pairs that one test configuration covers."""
    return {
        ((names[i], row[i]), (names[j], row[j]))
        for i, j in combinations(range(len(row)), 2)
    }


all_rows = list(product(*FACTORS.values()))
uncovered = set().union(*(pairs_of(row) for row in all_rows))
print("All combinations:", len(all_rows))
print("Pairs to cover:", len(uncovered))

# Greedy: keep picking the configuration that covers the most new pairs.
chosen = []
while uncovered:
    best = max(all_rows, key=lambda row: len(pairs_of(row) & uncovered))
    chosen.append(best)
    uncovered -= pairs_of(best)

print("Pairwise set:", len(chosen))
for row in chosen:
    print("  " + "  ".join(f"{value:9}" for value in row).rstrip())

All combinations: 64
Pairs to cover: 52
Pairwise set: 16
  Chrome     day_night  desktop    en
  Chrome     dynamic    phone      nl
  Edge       fixed      desktop    nl
  Edge       variable   phone      en
  Firefox    day_night  phone      nl
  Firefox    dynamic    desktop    en
  Safari     fixed      phone      en
  Safari     variable   desktop    nl
  Chrome     fixed      desktop    en
  Chrome     variable   desktop    en
  Edge       day_night  desktop    en
  Edge       dynamic    desktop    en
  Firefox    fixed      desktop    en
  Firefox    variable   desktop    en
  Safari     day_night  desktop    en
  Safari     dynamic    desktop    en

Sixteen configurations instead of 64, and 16 is the minimum possible: the 16 browser-and-tariff pairs alone need 16 tests. The saving grows with more factors; ten factors with four values each have over a million combinations, while a pairwise set needs only a few dozen. The price is that a failure caused by three settings together can slip through, so add specific tests for combinations you know are risky (Safari on a phone with a dynamic contract, if that is where the complaints come from).

White-box techniques

White-box techniques measure tests against the code's structure. Statement testing aims to run every executable statement at least once; branch testing aims to take every branch, both the true and the false outcome of every if, at least once. Branch testing is stronger: an if without an else can have all its statements run while the "condition is false" path is never tried.

In practice you rarely design white-box tests from scratch. You write black-box tests, measure which statements and branches they run with a coverage tool, and look at what is missing: each uncovered branch is either a missing test or dead code. Code coverage and its limits are covered on code coverage and mutation testing (and briefly in pipelines). One warning belongs here: the night rule return at.hour < 7 or at.hour >= 23 is a single line without an if, so a test at 02:30 already gives it 100% statement coverage, and the branch coverage of coverage.py, which measures jumps between lines, sees no branch at all. Coverage would never have pointed at the 23:00 defect; boundary value analysis did.

Experience-based techniques

Systematic techniques find the defects that a specification predicts. Experience finds the others. Three experience-based techniques appear in the ISTQB syllabus:

• Error guessing: anticipating which mistakes developers are likely to make and which defects and failures follow from them, based on past bugs, knowledge of the domain and of how similar systems fail, and designing tests aimed at exactly those. A more aggressive form, a fault attack, deliberately tries to make the software fail, for example with extreme or malformed input.
• Checklist-based testing: working through a list of things to check, built from experience (the team's own list of "things that went wrong before" is a good start).
• Exploratory testing: designing and running tests at the same time while learning about the product, covered on exploratory and manual testing.

After a few years with meter data, the Portal team's error-guessing list for anything that handles readings looks like this:

• Midnight, the clock changes (92 and 100 quarter-hours), month ends and 29 February.
• An empty export file, a file with only a header, and a file whose last line has no line break.
• Decimal commas (0,25) in a semicolon-separated KBS export, and thousands separators in amounts.
• The same quarter-hour sent twice, and a reading that arrives a day late.
• A meter replaced in the middle of the month, so the meter total suddenly goes down.
• A customer who changes tariff on the 15th: which price applies to which half of the month?

Every bug the team fixes adds a line to the list, which is how experience turns into a technique others can use. Error guessing does not replace the systematic techniques; it adds the cases that no specification thought to mention.