Test Cases and Bug Reports

Kestrel Energy's Portal team is building "your costs this month": the cost per day, split into day and night for customers on a day_night contract (night is before 07:00 and from 23:00). The team has no dedicated tester; everyone tests, and Mei Chen's service desk reports what customers run into. This page follows one defect from a customer's phone call to a closed bug report, and shows the test case that now guards against it.

What a test case is

A test case is a written set of preconditions, inputs or actions, expected results and postconditions that checks one specific thing about the software. That "one specific thing" is often called the test condition: "a reading at 23:15 on a day_night contract is charged at the night price". The test case says exactly how you check it: with which data, in which order, and what must happen.

Test cases come from somewhere. For the cost page they come from the acceptance criteria of the backlog item PBI-450 (see acceptance criteria), from the tariff sheet on kestrel.example.com, and from the test design techniques on the previous page: the boundary 22:59 / 23:00 is there because boundary value analysis put it there. Whatever describes how the software should behave is called the test basis.

TC-0231  Night price applies to a reading at 23:15

Reference      PBI-450, acceptance criterion 3 (night rate from 23:00)
Environment    Portal test environment
Preconditions
  - Test customer C-10006 with contract CT-20007 (day_night), meter M-30007
  - Prices loaded: day 0.3120, night 0.2240 euro per kWh
  - One reading for M-30007: 2026-10-05 23:15, 1.5 kWh
Steps
  1. Log in to the portal as test customer C-10006.
  2. Open "Costs this month" and choose 5 October 2026.
  3. Find the row for 23:15.
Expected result
  - The row shows tariff "night", 1.5 kWh and EUR 0.34
    (1.5 x 0.2240 = 0.336, rounded to cents).
  - The night subtotal for 5 October includes this reading.
Postconditions
  - None: the test only reads data.

Expected results

The expected result is the behavior or output the software should show under the stated conditions, worked out from the test basis before the test is run. In TC-0231 the €0.34 comes from the tariff sheet and a multiplication, not from the screen. That order matters: if you first look at what the software shows and then decide whether it seems right, you will accept a plausible wrong number. €0.47 for 1.5 kWh does not look strange at all.

"The page works as expected" or "the cost is correct" are not expected results; they leave the decision to whoever runs the test. Write the values, the messages and the state. Where you cannot know the exact answer in advance, say what you can check: "the day and night subtotals add up to the day total". The source that tells you the right answer is called a test oracle; for many outputs there is no easy one, and exploratory testing deals with that problem.

How detailed a test case should be

Some teams write test cases at two levels. A high-level (logical) test case says what to check without concrete values: "a reading in the late-night window uses the night price". A low-level (concrete) test case, like TC-0231, fixes the customer, the time and the amount. High-level cases are quick to write and leave room for the person running them; concrete cases are repeatable and can be automated, but they take more upkeep when the screens or the data change.

Write as much detail as the person who will run the test needs. A step-by-step script suits a test that someone new runs before every release, or a check an auditor wants to see. For a team that knows the product, a short charter often finds more, which is the idea behind exploratory testing.

From test case to automated test

Most of what TC-0231 checks does not need a browser. The price rule can be tested directly in code, in a fraction of a second, on every change. Here is the same condition as an automated test with pytest (explained on Unit Testing with pytest), together with its neighbor on the other side of the boundary. The test name and docstring carry the test case ID, so a failure points straight back to the written case and the bug:

from datetime import datetime
from decimal import ROUND_HALF_UP, Decimal

DAY_PRICE = Decimal("0.3120")    # euro per kWh, day_night tariff
NIGHT_PRICE = Decimal("0.2240")


def is_night(at):
    return at.hour < 7 or at.hour >= 23    # fixed: was "> 23" (BUG-1041)


def energy_cost(kwh, at):
    price = NIGHT_PRICE if is_night(at) else DAY_PRICE
    return (Decimal(str(kwh)) * price).quantize(Decimal("0.01"), ROUND_HALF_UP)


def test_tc_0231_reading_at_23_15_uses_night_price():
    """TC-0231, regression test for BUG-1041."""
    assert energy_cost(1.5, datetime(2026, 10, 5, 23, 15)) == Decimal("0.34")


def test_reading_at_22_45_still_uses_day_price():
    assert energy_cost(1.5, datetime(2026, 10, 5, 22, 45)) == Decimal("0.47")

============================= test session starts ==============================
collecting ... collected 2 items

test_example.py::test_tc_0231_reading_at_23_15_uses_night_price PASSED   [ 50%]
test_example.py::test_reading_at_22_45_still_uses_day_price PASSED       [100%]

============================== 2 passed in 0.16s ===============================

The written test case does not disappear. At system level, in the portal test environment, TC-0231 still checks what the unit test cannot: that the page reads the right contract, loads the right prices from the database and shows the rounded amount. One such check is enough there; the boundaries themselves (06:59, 07:00, 22:59, 23:00) are cheaper to cover in code. Which check belongs at which level is the subject of Test Levels and Test Types.

Test suites

A test suite is a set of test cases or test scripts grouped to be run together, because they cover the same feature, belong to the same level or serve the same purpose. A suite is how you answer "what do we run, and when?". The Portal team's suites for the cost page look like this:

Keep a trace from requirement to test to result. If every acceptance criterion of PBI-450 points to at least one test case, and every test result is recorded, then "is the night rule tested?" has an answer you can show, and a criterion without a test stands out. Teams keep this in a test management tool (such as a Jira app or Azure Test Plans), in a spreadsheet, or simply in the repository, with test IDs in the test names as above. The tool matters less than the habit.

Writing a bug report

A bug report is a written record of a failure, or of a defect found by reading code or documents, with enough information for someone else to reproduce it, understand it, decide what to do with it and follow its progress. The ISTQB syllabus calls it a defect report; the documentation standard ISO/IEC/IEEE 29119-3 calls it an incident report, which is not the same as an incident in IT service management. In daily work most people say bug, ticket or issue.

A useful report contains, according to the ISTQB syllabus:

• a unique ID and a short title that names the symptom;
• the date, the author and the test object with its version and environment;
• the context: which test case or activity found it, and what you were trying to do;
• a description of the failure: steps to reproduce, logs, screen captures;
• the expected result and the actual result;
• severity and priority (see below);
• status and references, such as the test case or the story.

Here is BUG-1041, written after Mei Chen's service desk took the call. Notice what is not in it: no guess about the cause, no name of whoever wrote the code, and no personal details of the customer who called (those stay in the incident record).

BUG-1041  Cost page charges 23:00-23:59 at day rate for day_night contracts

Reported     2026-10-07, Mei Chen (service desk), from INC-22061
Status       New
Severity     Major
Priority     High
Component    Portal, "Costs this month" page
Environment  Portal test environment, build of 2026-10-07; Firefox and Chrome
References   PBI-450; tariff sheet day_night (night 23:00-07:00); TC-0231

Steps to reproduce
  1. In the portal test environment, log in as test customer C-10006
     (contract CT-20007, day_night, meter M-30007).
  2. Make sure M-30007 has a reading for 2026-10-05 23:15 of 1.5 kWh.
  3. Open "Costs this month" and choose 5 October 2026.
  4. Look at the row for 23:15.

Expected result
  Tariff "night", cost EUR 0.34 (1.5 kWh x EUR 0.2240 = EUR 0.336).

Actual result
  Tariff "day", cost EUR 0.47 (1.5 kWh x EUR 0.3120 = EUR 0.468).

Notes
  - Happens every time (5 of 5 attempts, both browsers).
  - Readings at 06:45 (night) and 22:45 (day) are priced correctly.
  - All four quarter-hours 23:00, 23:15, 23:30 and 23:45 get the day rate.
  - Attached: screenshot cost-page-2315.png, portal log of the request.

Steps to reproduce

Steps to reproduce are the numbered, minimal sequence of actions, data and conditions that makes the failure happen again, starting from a known state. They are the most valuable part of a report: a developer who can see the failure for themselves usually finds the cause quickly, and one who cannot will spend the time guessing. The same skill is described for troubleshooting in general on reproducing the problem.

• Start from a known state: which account, contract and data, in which environment and build.
• One action per step, with the exact values you typed or chose.
• Make it small. Remove every step that is not needed to trigger the failure; a minimal reproducible example is the goal.
• Say how often it happens: always, or 2 out of 10 attempts. An intermittent failure is still worth reporting, with everything you noticed about when it happens.
• Say what you ruled out. "22:45 and 06:45 are correct" in BUG-1041 already points at the 23:00 boundary.

Titles, facts and tone

The title is what people read in a list of fifty bugs. It should name the symptom and where it happens, so that someone can find it again and spot duplicates:

Weak:    Night prices wrong!!! Customers complaining, fix ASAP
Weak:    Cost page broken
Better:  Cost page charges 23:00-23:59 at day rate for day_night contracts
Better:  Daily total shows 96 quarter-hours on 25 Oct 2026 (should be 100)

Report one problem per bug: if the 23:15 row is also hard to read on a phone, that is a second report, because the two will be fixed, tested and closed at different times. Search for an existing report before you file a new one. Stick to observations ("shows €0.47") rather than conclusions ("the price table is corrupt"), and leave blame out entirely: a defect report describes the product, not a person. A report that reads like an accusation gets defended instead of fixed.

Service desk tickets and bug reports

On the service desk the same event shows up first as an incident: a customer, called here only by ticket number, phoned on Wednesday 7 October about a 23:15 reading of Monday 5 October on her cost page that was priced at the day rate. Mei Chen logged INC-22061. The incident and the bug are two records with two goals:

Link the two both ways, so that the service desk can tell other callers that a fix is coming, and the developers can see how many customers are affected. When many incidents point to the same defect, problem management records it as a known error with a workaround. The service desk is often the first to see a failure in production, and the details it collects (the exact time, the contract type, the browser, what the customer clicked) are exactly what a good reproduction needs.

Severity and priority

Defect severity is how much impact a defect has on the use or operation of the software: does it lose data, block a task, give wrong results, or just look odd? Defect priority is how urgently the defect should be fixed compared with other work, which is a business decision. The person who finds a defect usually proposes the severity; at Kestrel the Product Owner, Isabel Santos, sets the priority, because she orders the work of the Portal team.

The scales differ per team: critical, major, minor and trivial is common for severity, and high, medium and low or P1 to P4 for priority. Write down what each level means for your product, with an example, or everyone will use "major" for everything that annoys them.

The life of a defect

The defect life cycle is the set of states a bug report goes through from the moment it is reported until it is closed, together with the allowed moves between them. A typical cycle:

The ISTQB syllabus lists statuses such as open, deferred, duplicate, waiting to be fixed, awaiting confirmation testing, reopened, closed and rejected, and it deliberately does not prescribe one workflow. Jira, Azure Boards and GitHub Issues all let a team define its own; GitHub Issues has only the states open and closed (with a reason for closing) and uses labels or a project's status field for the rest. What matters is that everyone knows what each state means and who moves a bug out of it.

BUG-1041 went through the cycle like this:

1. New (7 October): Mei Chen files the report from INC-22061.
2. Assigned: in triage, Isabel Santos and Ravi Kumar confirm it is a defect, keep severity major and set priority high; Priya Shah picks it up.
3. Fixed: hour > 23 becomes hour >= 23, with the regression test shown above. A failing test was written first, so the fix is proven to make a difference.
4. Retest: after the build is deployed to the test environment, Sofia Lindqvist runs TC-0231. This is a confirmation test (see Test Levels and Test Types); she also checks 22:45 and 06:45, so that the fix did not move the problem.
5. Closed: TC-0231 passes. The automated test stays in the suite and guards against a regression.

Common mistakes

• "It doesn't work." A report without steps, environment or expected result costs a round trip of questions before anyone can start.
• Several bugs in one report. Half of it gets fixed and the report can neither be closed nor kept open honestly.
• No expected result in a test case, or one written after looking at the screen.
• Inflated severity to get attention. It works once; after that, nobody trusts the field.
• Closing without a retest, because the developer said it was fixed. The fix may not be deployed, or may only cover the exact case in the report.
• Test cases nobody updates. A suite of outdated scripts gives false confidence. Retire or rewrite cases when the feature changes, just like code.