Property-Based Testing

From examples to properties

Almost every test on the other pages of this topic is an example-based test: you pick an input, work out the expected output by hand, and assert that the code produces it. "1.5 kWh at 23:15 on a day/night contract costs €0.34" is an example. Examples are clear and cheap, but they only test what you thought of. The night-rate bug at 23:00 survived because nobody thought of 23:00.

Property-based testing turns this around. You describe the inputs (any amount between 0 and 25 kWh, any time of day) and a rule the output must obey for all of them, and a library generates many inputs, runs the test with each, and reports any input that breaks the rule. Such a rule is a property: a statement about the code's behavior that is true for every valid input, such as:

• the cost of a reading is never negative;
• using more energy never costs less;
• the day part and the night part of a day's readings add up to the day's total;
• writing readings to a CSV (comma-separated values) file and reading them back gives the same readings.

None of these says what the cost is; each says something that must always be true. That is why properties find surprises: the library has no idea which inputs are "normal" and happily tries 0, 25, 00:00, 23:00 and very small amounts that a person would skip. The idea comes from QuickCheck, a Haskell library by Koen Claessen and John Hughes (ICFP 2000); similar libraries exist for most languages today.

Your first property with Hypothesis

Hypothesis is Python's property-based testing library (version 6.168.5 as of October 2026). It is not a test runner: you write ordinary pytest test functions and add the @given decorator, which says where each argument comes from. Hypothesis then calls the function about a hundred times with generated values. (Not to be confused with a hypothesis test in statistics, which is something else entirely.)

The file below has one example-based test and two properties for a day/night cost function with exact decimal prices:

from datetime import time
from decimal import ROUND_HALF_UP, Decimal

from hypothesis import given, strategies as st

DAY = Decimal("0.3120")
NIGHT = Decimal("0.2240")
CENT = Decimal("0.01")


def energy_cost(kwh, moment):
    """Cost of one reading on a day/night contract, rounded to cents."""
    if kwh < 0:
        raise ValueError("kWh cannot be negative")
    price = NIGHT if moment.hour < 7 or moment.hour >= 23 else DAY
    return (kwh * price).quantize(CENT, ROUND_HALF_UP)


kwh_values = st.decimals(min_value=0, max_value=25, places=3)


def test_example_quarter_past_eleven():
    assert energy_cost(Decimal("1.5"), time(23, 15)) == Decimal("0.34")


@given(kwh_values, st.times())
def test_cost_is_never_negative(kwh, moment):
    assert energy_cost(kwh, moment) >= 0


@given(kwh_values, kwh_values, st.times())
def test_more_energy_never_costs_less(a, b, moment):
    low, high = sorted([a, b])
    assert energy_cost(low, moment) <= energy_cost(high, moment)

============================= test session starts ==============================
collecting ... collected 3 items

test_example.py::test_example_quarter_past_eleven PASSED                 [ 33%]
test_example.py::test_cost_is_never_negative PASSED                      [ 66%]
test_example.py::test_more_energy_never_costs_less PASSED                [100%]

============================== 3 passed in 0.42s ===============================

Three things to notice. Each property ran with a hundred generated inputs (the default max_examples), yet pytest reports it as one test. The properties never compute an expected price, so they cannot copy a mistake from the code. And the run is slower than the example test alone: properties trade a little time for many more cases.

Strategies: describing the inputs

The arguments of @given are strategies. A strategy is a description of a set of possible values that Hypothesis knows how to generate and, later, how to simplify. You build them from the functions in hypothesis.strategies, usually imported as st:

The example below generates a whole day of 96 quarter-hour readings and checks that the day/night split loses nothing. Because the readings are Decimal values, the sums are exact and the property can use ==:

from decimal import Decimal

from hypothesis import given, strategies as st

day_of_readings = st.lists(
    st.decimals(min_value=0, max_value=25, places=3), min_size=96, max_size=96
)


def split_day_night(readings):
    """Return (day kWh, night kWh) for 96 readings starting at 00:00."""
    day = night = Decimal(0)
    for index, kwh in enumerate(readings):
        hour = index // 4
        if hour < 7 or hour >= 23:
            night += kwh
        else:
            day += kwh
    return day, night


@given(day_of_readings)
def test_day_and_night_add_up_to_the_total(readings):
    day, night = split_day_night(readings)
    assert day + night == sum(readings)


@given(day_of_readings)
def test_night_covers_32_quarter_hours(readings):
    only_night = [kwh if i < 28 or i >= 92 else Decimal(0)
                  for i, kwh in enumerate(readings)]
    assert split_day_night(only_night)[0] == 0

..                                                                       [100%]
2 passed in 2.62s

The second property checks the boundaries from the other side: readings only in the 32 night quarter-hours (00:00–06:45 and 23:00–23:45) must leave the day part at zero. If someone wrote hour > 23, the 23:00 readings would land in the day part and Hypothesis would find a list with a non-zero reading there on the first run.

Shrinking: the smallest failing case

The reading validator sometimes has to spread one reading over several quarter-hours, for example after a gap in the data. The first version divides and rounds to three decimals (whole watt-hours). The property says the parts must add up to the original amount:

from hypothesis import given, strategies as st


def split_kwh(total: float, parts: int) -> list[float]:
    """Split a reading over equal 15-minute parts, rounded to 3 decimals."""
    share = round(total / parts, 3)
    return [share] * parts


@given(st.integers(min_value=0, max_value=100_000), st.integers(min_value=1, max_value=96))
def test_parts_add_up(total_wh, parts):
    total = total_wh / 1000
    assert round(sum(split_kwh(total, parts)), 3) == total

F                                                                        [100%]
=================================== FAILURES ===================================
______________________________ test_parts_add_up _______________________________

    @given(st.integers(min_value=0, max_value=100_000), st.integers(min_value=1, max_value=96))
>   def test_parts_add_up(total_wh, parts):
                   ^^^

test_example.py:11:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

total_wh = 1, parts = 2

    @given(st.integers(min_value=0, max_value=100_000), st.integers(min_value=1, max_value=96))
    def test_parts_add_up(total_wh, parts):
        total = total_wh / 1000
>       assert round(sum(split_kwh(total, parts)), 3) == total
E       assert 0.002 == 0.001
E        +  where 0.002 = round(0.002, 3)
E        +    where 0.002 = sum([0.001, 0.001])
E        +      where [0.001, 0.001] = split_kwh(0.001, 2)
E       Failing test case: test_parts_add_up(
E           total_wh=1,
E           parts=2,
E       )

test_example.py:13: AssertionError
=========================== short test summary info ============================
FAILED test_example.py::test_parts_add_up - assert 0.002 == 0.001
1 failed in 0.30s

The first input that broke the property was probably something like 73,519 Wh over 41 parts: correct but hard to debug. Hypothesis did not stop there. It tried simpler and simpler variations of that input, kept each one that still failed, and stopped when nothing simpler failed. That process is called shrinking, and its result is the report above: 1 Wh split over 2 parts. Half a watt-hour rounds up to 0.001 kWh per part, so the parts add up to 2 Wh. Reading the minimal case, the bug is obvious: rounding each part separately can create or lose energy.

The fix is to work in whole watt-hours and hand out the remainder one by one, so the parts differ by at most 1 Wh and always add up. The same property, plus a second one, now passes:

from hypothesis import given, strategies as st


def split_wh(total_wh: int, parts: int) -> list[int]:
    """Split whole watt-hours over parts; the first parts get the remainder."""
    share, rest = divmod(total_wh, parts)
    return [share + 1 if i < rest else share for i in range(parts)]


totals = st.integers(min_value=0, max_value=100_000)
part_counts = st.integers(min_value=1, max_value=96)


@given(totals, part_counts)
def test_parts_add_up(total_wh, parts):
    assert sum(split_wh(total_wh, parts)) == total_wh


@given(totals, part_counts)
def test_parts_differ_by_at_most_one_wh(total_wh, parts):
    shares = split_wh(total_wh, parts)
    assert len(shares) == parts
    assert max(shares) - min(shares) <= 1

..                                                                       [100%]
2 passed in 0.35s

Reproducing failures

Hypothesis keeps every failing input in a small example database, a folder called .hypothesis/ in the directory you run the tests from. The next run replays those inputs first, so a failure does not disappear just because the random generator went another way. Add the folder to .gitignore; for a failure that must never come back, copy the shrunk case into the test as @example(total_wh=1, parts=2) so it is in version control. A few settings are worth knowing:

• @settings(max_examples=500) tries more inputs (the default is 100); use it for important properties, or a smaller number for slow ones.
• deadline (default 200 milliseconds per input) fails a test whose single call is too slow; set deadline=None for tests that are slow by nature.
• pytest --hypothesis-seed=0 fixes the random generator for one run, which helps when you need to reproduce exactly what a pipeline did.

Properties worth testing

The hard part of property-based testing is not the library but finding properties. A useful property is simpler than the code it tests; if you have to re-implement the function inside the test, you will copy its bugs. These patterns cover most cases:

Round trips

A round-trip property says that converting data to another form and back returns the original. It is the easiest property to write and catches real problems in parsers, serializers, exports and imports: lost decimals, wrong quoting, a separator inside a value. Kestrel's KBS export uses semicolons, as many Dutch exports do:

import csv
import io
from decimal import Decimal

from hypothesis import given, strategies as st

readings = st.lists(
    st.tuples(
        st.from_regex(r"M-[0-9]{5}", fullmatch=True),
        st.decimals(min_value=0, max_value=25, places=3),
    )
)


def write_readings(rows):
    out = io.StringIO()
    writer = csv.writer(out, delimiter=";")
    writer.writerow(["meter", "kwh"])
    for meter, kwh in rows:
        writer.writerow([meter, kwh])
    return out.getvalue()


def read_readings(text):
    reader = csv.reader(io.StringIO(text), delimiter=";")
    next(reader)  # header
    return [(meter, Decimal(kwh)) for meter, kwh in reader]


@given(readings)
def test_csv_round_trip(rows):
    assert read_readings(write_readings(rows)) == rows

.                                                                        [100%]
1 passed in 2.82s

It passes, and it is worth having: if someone later "simplifies" the export to write float(kwh) or two decimals, the property fails on the first value that does not survive the trip.

Metamorphic relations

Sometimes you cannot say what the right output is for one input, but you can say how the output must change when the input changes. Testing such a relation between two runs is metamorphic testing. For the cost function: doubling the energy must double the cost. Written exactly like that, with energy_cost(2 * kwh) == 2 * energy_cost(kwh), the property fails within a second, on a reading of a few watt-hours: each cost is rounded to whole cents, and twice a rounded amount is not always the rounded double. That is not a bug in the cost function; it is a rule the team has to state precisely. The relation that does hold is "within one cent":

from decimal import ROUND_HALF_UP, Decimal

from hypothesis import given, strategies as st

DAY = Decimal("0.3120")
NIGHT = Decimal("0.2240")
CENT = Decimal("0.01")


def energy_cost(kwh, night):
    return (kwh * (NIGHT if night else DAY)).quantize(CENT, ROUND_HALF_UP)


@given(st.decimals(min_value=0, max_value=25, places=3), st.booleans())
def test_double_energy_costs_double_within_a_cent(kwh, night):
    difference = energy_cost(2 * kwh, night) - 2 * energy_cost(kwh, night)
    assert abs(difference) <= CENT

.                                                                        [100%]
1 passed in 0.25s

Questions like "do we round per reading or per invoice?" are exactly what properties bring to the surface. If KBS rounds once per invoice line and the portal rounds every reading, the cost page and the invoice will disagree by a few cents; better that Joost Bakker and the Portal team settle the rule now than that a customer finds the difference.

Dates, time zones and the 100-quarter-hour day

Dates are where properties shine, because the dangerous inputs are rare: a day in Amsterdam has 96 quarter-hours, except the last Sunday of March (92) and the last Sunday of October (100). The next one is 25 October 2026. Instead of listing those days, state what must hold for every date and every year:

from datetime import date, datetime, time, timedelta, timezone
from zoneinfo import ZoneInfo

from hypothesis import given, strategies as st

AMSTERDAM = ZoneInfo("Europe/Amsterdam")


def quarter_hours_in_day(day):
    start = datetime.combine(day, time(0), tzinfo=AMSTERDAM)
    end = datetime.combine(day + timedelta(days=1), time(0), tzinfo=AMSTERDAM)
    seconds = (end.astimezone(timezone.utc) - start.astimezone(timezone.utc)).total_seconds()
    return int(seconds // 900)


@given(st.dates(min_value=date(2020, 1, 1), max_value=date(2035, 12, 31)))
def test_a_day_has_92_96_or_100_quarter_hours(day):
    assert quarter_hours_in_day(day) in {92, 96, 100}


@given(st.integers(min_value=2020, max_value=2035))
def test_a_year_has_96_quarter_hours_per_day_on_average(year):
    first = date(year, 1, 1)
    days = (date(year + 1, 1, 1) - first).days
    total = sum(quarter_hours_in_day(first + timedelta(n)) for n in range(days))
    assert total == days * 96

..                                                                       [100%]
2 passed in 0.32s

The conversion to UTC (Coordinated Universal Time) before subtracting is essential: Python subtracts two times in the same time zone object as wall-clock times, so without it every day would have 96 quarter-hours and the bug the team is guarding against (BUG-1042, "daily total shows 96 quarter-hours on 25 Oct 2026") would be built into the test helper. The second property is a good sanity check: the 92 in spring and the 100 in autumn cancel out over a year.

When to use properties, and when not

Property-based tests pay off most for:

• Calculations with rules: prices, splits, rounding, units, time zones.
• Parsers and formats: CSV and JSON (JavaScript Object Notation) import and export, meter ID formats, anything with a round trip.
• Code with a simple reference: an optimized version of something you can also write slowly and obviously.
• Input validation: "no input, however strange, makes this function crash" is a property. Hypothesis is not a coverage-guided fuzzer, but it finds many of the same crashes; Security Testing covers fuzzing proper.

They are a poor fit, or need care, when:

• the only property you can think of is "the output equals what the code computes", which means re-implementing it;
• each call is slow (a database, a network call): a hundred inputs become a hundred slow calls;
• the business wants specific, readable cases in the test suite, such as the boundaries of a tariff rule. Keep those as examples and add properties next to them.

Property-based tests also kill surviving mutants well, because generated inputs reach the boundaries that hand-picked ones miss. Similar libraries exist outside Python, such as fast-check for JavaScript and TypeScript, jqwik for Java and FsCheck for .NET; the ideas on this page carry over unchanged.