What Is Software Testing?

The Testing & QA pages follow the Portal team of Kestrel Energy, a fictional Dutch energy supplier, while they build a page that shows customers what their electricity cost this month, split into day and night. The team has no separate tester: Product Owner Isabel Santos, Scrum Master Lars de Boer and the developers Ravi Kumar, Sofia Lindqvist, Kwame Asante, Priya Shah and Bram Peeters test as a whole team. Two dates keep them busy: on 25 October 2026 the clocks go back and a day has 100 quarter-hours instead of 96, and in December and January the winter evening peak brings the most traffic of the year.

Why we test

Software testing is the set of activities that evaluate the quality of a piece of software and of the work products around it, such as requirements, designs and user stories. Its purpose is to give people the information they need to decide: is this good enough to release, and if not, what is wrong with it? Testing finds problems, but it also builds confidence, prevents problems (writing a test forces someone to think about what "correct" means), and reduces the risk that a failure reaches a customer.

Testing comes in two kinds. Dynamic testing runs the software: a unit test calls a function, a person clicks through the portal. Static testing examines the software without running it: a colleague reviews the code or the user story, or a tool performs static analysis. Both count as testing, and they find different things, as the section on errors, defects and failures shows.

Testing and debugging are separate activities. Testing shows that something is wrong; debugging finds the cause and removes it, usually with a systematic method. After the fix, testing returns: a confirmation test checks that the problem is gone, and regression tests check that the fix broke nothing else (see testing after a change).

The word "bug" is older than computers; the programming page tells the story of the moth in the Harvard Mark II. Testing professionals use more precise words for the different stages of a bug, explained below.

What testing can and cannot prove

In October 1969, at a NATO software engineering conference in Rome, Edsger Dijkstra put the central limit of testing into one sentence, recorded in the conference report:

Testing shows the presence, not the absence of bugs.

A passing test proves that the software works for the cases that were tried, in the environment where they were tried. It says nothing about the cases nobody thought of. Priya Shah copied Kestrel's night rule (night rate before 07:00 and from 23:00) into the portal and wrote three tests. All three pass:

from datetime import time
from decimal import Decimal

DAY_PRICE = Decimal("0.3120")
NIGHT_PRICE = Decimal("0.2240")


def is_night(at):
    """Night rate applies before 07:00 and from 23:00."""
    return at.hour < 7 or at.hour > 23


def price_per_kwh(at):
    return NIGHT_PRICE if is_night(at) else DAY_PRICE


def test_early_morning_is_night():
    assert price_per_kwh(time(2, 30)) == NIGHT_PRICE


def test_afternoon_is_day():
    assert price_per_kwh(time(14, 0)) == DAY_PRICE


def test_seven_oclock_is_day():
    assert price_per_kwh(time(7, 0)) == DAY_PRICE

...                                                                      [100%]
3 passed in 0.16s

Three green dots, and the code is still wrong: hour > 23 is never true, because the hour of a time runs from 0 to 23. Every reading between 23:00 and midnight gets the day price. The tests simply never asked about that hour. One more test, for a reading at 23:15, makes the problem visible:

from datetime import time
from decimal import Decimal

DAY_PRICE = Decimal("0.3120")
NIGHT_PRICE = Decimal("0.2240")


def is_night(at):
    """Night rate applies before 07:00 and from 23:00."""
    return at.hour < 7 or at.hour > 23


def price_per_kwh(at):
    return NIGHT_PRICE if is_night(at) else DAY_PRICE


def test_early_morning_is_night():
    assert price_per_kwh(time(2, 30)) == NIGHT_PRICE


def test_quarter_past_eleven_is_night():
    assert price_per_kwh(time(23, 15)) == NIGHT_PRICE

.F                                                                       [100%]
=================================== FAILURES ===================================
______________________ test_quarter_past_eleven_is_night _______________________

    def test_quarter_past_eleven_is_night():
>       assert price_per_kwh(time(23, 15)) == NIGHT_PRICE
E       AssertionError: assert Decimal('0.3120') == Decimal('0.2240')
E        +  where Decimal('0.3120') = price_per_kwh(datetime.time(23, 15))
E        +    where datetime.time(23, 15) = time(23, 15)

test_example.py:22: AssertionError
=========================== short test summary info ============================
FAILED test_example.py::test_quarter_past_eleven_is_night - AssertionError: a...
1 failed, 1 passed in 0.17s

The failing test does not tell you why the price is wrong, only that it is; finding the > is debugging. The page on unit testing with pytest explains how to read this report, and boundary value analysis shows how to find the 23:00 case systematically instead of by luck.

Quality assurance, quality control and testing

Three terms are often used as if they meant the same thing. They do not, and the difference explains who is responsible for what.

Quality assurance (QA) is about the process: the activities that give confidence that quality requirements will be met, by improving the way software is made. Code review rules, a Definition of Done, training, coding standards and retrospectives that fix the causes of past bugs are all QA. It is preventive: it aims to make fewer defects in the first place.

Quality control (QC) is about the product: the activities that evaluate the quality of a specific piece of software, find what is wrong with it and get it corrected. Testing is the main form of quality control. It is corrective: it finds defects that already exist.

The ISTQB (International Software Testing Qualifications Board) Foundation Level syllabus, version 4.0, a widely used testing syllabus, draws the line between testing and QA in exactly this way: testing is product-oriented and corrective, QA is process-oriented and preventive. Both feed each other: test results show where the process needs improving.

Errors, defects and failures

"Bug" is used for everything from a typo in the code to a crash on a customer's screen. Testing separates three links in a chain, because each one is found in a different way:

• An error (also called a mistake) is a human action that produces a defect: misreading a requirement, a slip of the keyboard, a wrong assumption.
• A defect (also fault or bug) is the imperfection that the error leaves in a work product: a wrong line of code, but also a wrong sentence in a requirement or a missing case in a design.
• A failure is an event in which the running system does not do what it should: a wrong result, a crash, a page that takes a minute to load.

At Kestrel the chain ran its full length. Priya's error was a single character. The defect, hour > 23, sat in the code where a reviewer could have spotted it. The failure happened every night between 23:00 and midnight, for every customer on a day_night contract. On Wednesday 7 October 2026 a customer called Mei Chen at the service desk: her reading of Monday 23:15, 1.5 kWh, was priced at €0.47 instead of €0.34. Mei logged it as incident INC-22061, and the team recorded the defect as BUG-1041 (its report is on the page on test cases and bug reports).

Not every defect causes a failure: a defect in code that never runs stays silent, and so would this one for a customer on a fixed tariff. And not every failure comes from a defect: a full disk, a network outage or a wrong configuration can make correct code fail. The root cause is the source of the defect: remove it, and defects of this kind become rarer. Why could Priya's mistake happen, and why did it go unnoticed? Here the answers were that the night rule was copied by hand into the portal, and that nobody tests the boundaries of pricing rules: process problems, and so a job for quality assurance. Finding root causes is covered under root cause analysis.

Verification and validation

Two questions sum up what testing is for. A classic formulation, often attributed to the software engineer Barry Boehm, puts them side by side:

• Verification: are we building the product right? Does the software meet its specification? The test for 23:15 verifies the night rule as it is written down.
• Validation: are we building the right product? Does it meet the real needs of the people who will use it, in the way they will use it? Whether customers understand a cost page split into day and night is a validation question, and no amount of verification answers it.

Most automated tests verify. Validation needs people: the Product Owner, real users, acceptance testing and usability tests (see acceptance testing and exploratory and manual testing). A product can pass every verification test and still fail validation, which is the last of the seven principles below.

The seven testing principles

The seven testing principles are general truths about testing that the ISTQB Foundation Level syllabus (version 4.0, 2023, with corrections in 4.0.1 of September 2024, current as of October 2026) lists as the basis of the profession. Each one explains a mistake that teams keep making.

1. Testing shows the presence, not the absence of defects. Dijkstra's point, above. Testing lowers the chance that defects remain, but cannot show that there are none.
2. Exhaustive testing is impossible. The night rule alone has 1,440 possible minutes in a day; combine it with four tariffs, every date, every browser and every amount of energy and the number of cases is effectively infinite. Instead of testing everything, choose cases with test design techniques and spend effort where the risk is (see risk-based testing).
3. Early testing saves time and money. A defect found in a review of the user story costs a conversation; the same defect found after release costs an incident, a correction of bills and an apology. Testing activities should start as early as possible (see shift-left).
4. Defects cluster together. A small number of components usually contain most of the defects, an illustration of the Pareto principle. At Kestrel, anything that deals with time (night hours, daylight saving time, month ends) produces far more bugs than the rest. Where you found defects before, look again.
5. Tests wear out. Running the same tests again and again finds fewer and fewer new defects: the code they cover has been fixed, and the code they do not cover stays untested. Tests need to be reviewed and new ones added. (Older versions of the syllabus called this the "pesticide paradox".) Repeated tests are still valuable as regression tests; they just stop finding new problems.
6. Testing is context dependent. A billing system, a game and the software in a medical device need very different testing. The right amount and kind of testing depends on the risks, the technology, the team and any rules that apply.
7. Absence-of-defects fallacy. Finding and fixing many defects does not make a system successful. Software that meets its specification perfectly can still be useless if the specification was wrong: that is why validation matters as much as verification.

The tester's mindset

Writing code and testing it ask for different attitudes. A developer's job is to make something work; testing asks "how could this fail?" Good testing combines a few habits:

• Curiosity. What happens at 23:00? On the day the clocks change? With a reading of 0 kWh, or −4 kWh? With a meter that sent nothing at all?
• Healthy skepticism. "It works on my machine" and "that can never happen" are claims to check, not facts.
• Attention to detail, together with a sense of what matters to the business: a wrong price matters more than a misaligned icon.
• Clear, blameless communication. A test result is news about the product, not a verdict on a person. "The cost page prices 23:15 at the day rate" helps; "Priya broke billing" does not.

People find it hard to see the mistakes in their own work, which is why some independence helps: a second pair of eyes on a pull request, a colleague who did not write the code exploring it, or a Product Owner who checks it against the business need. Independence does not require a separate test department. In the Portal team, the developer who did not write a story reviews and explores it, and everyone writes automated tests for their own code. The page on quality in agile teams describes this whole-team approach.