What you are learning — 5 topics
Choosing what to assert
A test is a sentence about what must stay true, written so a machine can check it while you are asleep.
Reach for it whenWrite a test when a rule must survive future edits; reach for a quick print statement instead when you are still exploring and the answer will not matter tomorrow.
Think of it likeA smoke alarm. Nobody installs one expecting a fire tonight — it earns its cost on the night you would otherwise have slept through it.
The valuable assertion is about behaviour, not implementation. "Returns the two indices that sum to the target" survives a rewrite; "calls the helper twice" breaks the moment you tidy the code, having caught nothing.
Assert on the edges, because that is where code fails: empty input, one item, duplicates, the largest value, the value that sits exactly on the boundary. The middle of the range is where code is already correct.
Syntax you can useassert total([]) == 0- States the expected behaviour for an empty input.
assert find([7], 7) == 0- Covers the one-item boundary where loops often go wrong.
assert sorted(values) == [1, 2, 3]- Checks a visible result rather than how the function reached it.
with pytest.raises(ValueError):
parse("bad")- Asserts that invalid input fails in the promised way.
Several ways to see itWhy it existsIt exists because your memory is shortYou know today why that condition uses <= and not <. In six weeks you will not, and neither will anyone else. A test is that reasoning written down in a form that speaks up when someone breaks it.
How it breaksPassing on the example proves nothingThe buggy Contains Duplicate in the practice lab passes [1, 2, 3, 1] — and returns True for every input ever given to it. A single passing example is perfectly compatible with a completely broken function, which is why the edge cases carry all the weight.
Against the alternativeAgainst printing and lookingA print statement checks the code once, using your eyes, and then the evidence is gone. An assertion checks it on every run forever, using nobody's attention. The two cost about the same to write; the difference is entirely in what happens next month.
In the wild- SQLite — ships with several hundred times more test code than library code, which is why it runs unattended on billions of devices.
- Python's standard library — keeps a regression test for each behaviour that has ever broken, so old bugs cannot quietly return.
Commonly gets misread as- "Full coverage means the code is correct." Correction: coverage records which lines ran, not whether the assertions meant anything — a test with no assertion at all still covers every line it touches.
- "Tests are for large projects." Correction: the value scales with how long you keep the code, not how big it is; a script you rerun monthly repays a test immediately.
What it costsTime up front, repaid the first time a test fails instead of a user.
Go and read
Examples versus properties
An example checks one answer; a property states a rule and lets the test runner hunt for an input that breaks it.
Reach for it whenUse examples for named cases with one clear expected answer. Use properties when many inputs share the same rule and you cannot afford to list them all.
Think of it likeInspecting one bridge versus stating the load rule every bridge must survive. The first proves that bridge passed once; the second tells you what to test across the whole design.
A good property describes an invariant without repeating the implementation. Reversing a list twice returns the original; encoding then decoding preserves the value; adding an item and removing it restores the collection. The test runner supplies awkward inputs you would not think to write by hand, then shrinks a failure to the smallest example that still breaks the rule.
Properties need examples beside them. A function that always returns an empty list satisfies "the result is sorted," so add the second rule that no items disappear. One sharp example documents a recognizable case; several independent properties fence in the whole behavior.
Syntax you can useassert reverse([1, 2, 3]) == [3, 2, 1]- A named example with one known answer.
assert reverse(reverse(items)) == items- A property: reversing twice restores any input.
assert sorted(result) == sorted(items)- Checks that a sort kept every original item.
assert all(result[i] <= result[i + 1] for i in range(len(result) - 1))- Checks the ordering property of every adjacent pair.
Several ways to see itIts shapeA property draws a fence, not a dotAn example marks one coordinate in the input space. A property surrounds a region and says every point inside must obey the same rule. The useful shift is from "what answer do I expect here?" to "what can never be allowed to change?"
What the machine doesShrinking turns noise into a diagnosisA generated failure may begin as a list of two hundred strange values. The runner repeatedly removes and simplifies pieces until perhaps [0, 0] is all that remains. That tiny counterexample usually exposes the missing condition immediately.
How it breaksOne weak rule can bless nonsenseChecking only that a sorted result is ordered approves an implementation that deletes every item. Properties become convincing in combination: ordered, same length, and same item counts. Each closes a loophole left by the others.
In the wild- Hypothesis — generates Python inputs from declared strategies, records failures, and shrinks a large failing case to a small reproducible one.
- QuickCheck — popularized executable properties in Haskell by generating test data and minimizing counterexamples automatically.
- Ethereum execution-spec-tests — generates state-transition fixtures that multiple clients must execute to the same post-state, checking a protocol rule across implementations.
Commonly gets misread as- "Property tests replace example tests." Correction: properties search a wide space while examples explain important named cases; strong suites use both.
- "Random tests cannot reproduce a failure." Correction: property-testing tools report and save the minimized failing input, so the next run can replay it exactly.
What it costsMore thought per assertion and more runtime than a few examples, traded for much wider input coverage.
Go and read
Keeping fixed bugs fixed
Every escaped bug is a test case the real world wrote for you; keep it so the same failure cannot return unnoticed.
Reach for it whenAdd a regression test whenever you fix reproducible wrong behavior. Skip only when the failure lives outside your control and cannot be represented reliably in the test environment.
Think of it likeA repaired roof with the leak marked on the inspection plan. Future workers can change the tiles, but that exact weak spot is checked after every job.
The order matters: reproduce, write the failing test, then change the implementation. Seeing the test fail proves it reaches the bug; seeing it pass after the fix proves the change addresses that bug rather than merely moving symptoms around.
Name the behavior, not the ticket number, and use the smallest input that triggers it. Preserve only what users could observe unless an internal invariant is the failure itself. A regression test should survive the next refactor while making the old defect cheap to recognize.
Syntax you can usedef test_empty_input_regression():- Names a test after the bug it prevents from returning.
assert parse("") == []- Locks down the smallest input that previously broke.
pytest -k regression- Runs only tests whose names include regression.
pytest -x- Stops at the first failure while you are fixing a bug.
Several ways to see itWhy it existsProduction discovered an input your plan missedA bug report is evidence that your previous model of the input space was incomplete. Keeping its input expands that model permanently. Deleting it throws away information you paid for with a real failure.
What the machine doesRed before green proves the test has teethRun the new test against the unfixed code and watch the expected assertion fail. If it passes already, it is not reproducing the defect, however convincing the test body looks. That red run is part of the evidence, not ceremony.
What it costsThe second fix should cost almost nothingWithout a test, a returning bug needs another user report, investigation and patch. With one, CI identifies the exact broken behavior on the commit that revived it. A few lines now buy back an entire future incident.
In the wild- CPython — bug fixes add focused cases under Lib/test so every supported platform reruns the formerly failing behavior in continuous integration.
- SQLite — keeps historical malfunction cases in its TCL test suites and reruns them across configurations before a release.
- Chromium — turns minimized ClusterFuzz crash inputs into regression tests so later browser builds cannot silently reintroduce the crash.
Commonly gets misread as- "The fix is obvious, so a test is unnecessary." Correction: obvious fixes are still removed by later cleanups when nobody remembers the case that made them necessary.
- "A regression test should copy the entire production incident." Correction: reduce it to the smallest input and observable failure; incidental logs, timing and data make the test brittle.
What it costsA steadily growing suite to maintain and run; keep cases focused so the protection grows faster than the friction.
Go and read
Judging generated code
Treat fluent code as an unverified claim: identify its contract, attack the boundaries, and trust only evidence you can rerun.
Reach for it whenUse this review whenever code arrived faster than you could understand it: from a model, a copied answer, a code generator or an unfamiliar contributor.
Think of it likeInspecting a beautifully forged key. Smooth metal and a convincing shape mean nothing; you try it in the lock, turn it under pressure, and check that it opens only the door it should.
Check the contract before the style. Write down accepted inputs, returned shape, mutations, errors, ordering and external effects, then compare those promises with every exit path. Generated code often has excellent names and familiar structure while inventing a library call, swallowing an error, mutating the caller's data or solving a neighboring problem. Read dependencies and security boundaries next: file paths, SQL, shell commands, permissions, secrets and network retries can turn a nearly correct helper into an incident.
Then search for the boundary it fails on. Partition the input space—empty versus non-empty, one versus many, missing versus present, minimum versus maximum, trusted versus hostile—and choose one test from each side of a branch. Run the existing suite, add focused examples or properties, and independently check unfamiliar APIs in primary documentation. Approval means you can explain why the code is correct and show the checks that would fail if it were not; fluency, comments and a green happy-path demo are not evidence.
Syntax you can useassert function_name(input_value) == expected- Checks one generated claim against a concrete expected result.
assert function_name([]) == expected_empty- Tests the empty boundary a plausible first draft often misses.
assert function_name(data) == function_name(list(data))- Checks that equal inputs give equal answers without unexpected mutation.
git diff --check- Finds whitespace errors in a generated patch before reviewing its logic.
Several ways to see itHow it breaksFluency is a camouflage for missing branchesConsistent names, tidy helpers and confident comments make code feel finished before it has earned that status. Reviewers then skim the exact lines that deserve friction: defaults, exception paths and boundary comparisons. The more effortless the code feels, the more deliberately you should enumerate what it forgot.
Its shapePartition the input space before choosing testsDraw simple boxes: empty, single, many; below, at, above; valid, malformed, hostile. Each condition in the code creates a border between boxes. Put at least one test directly on both sides of the borders that matter, because generated code commonly gets the broad shape right and one border wrong.
What the machine doesTrace side effects all the way outA helper may return the right value while writing twice, leaking a file handle or retrying a payment after a timeout. Follow calls that cross the process boundary and ask what happens when each succeeds, fails or partly completes. The machine executes effects, not intentions written in comments.
Against the alternativeGenerated and handwritten code owe the same proofAuthorship changes your review strategy, not the definition of correctness. With your own code, memory supplies hidden context and can make you overconfident; with generated code, polished presentation does the same. In both cases the contract, tests, documentation and failure behavior are the evidence.
In the wild- GitHub CodeQL — traces untrusted data through code to dangerous sinks such as SQL execution, catching security paths that a smooth line-by-line read can miss.
- Chromium ClusterFuzz — feeds malformed and extreme inputs into browser code, minimizes crashes, and files reproducible cases at the boundary where plausible implementations fail.
- Linux kernel — routes patches to subsystem maintainers and automated build-test bots; acceptance depends on review and reproducible checks, not on who or what produced the patch.
Commonly gets misread as- "If it compiles and the tests pass, it is safe." Correction: compilation checks shape, and existing tests check only cases somebody already imagined; neither proves the new contract or its hostile boundaries.
- "I can judge it by reading carefully." Correction: readable code lowers your suspicion and makes missing cases harder to notice; turn each claim you believe into an executable check or a verified API fact.
What it costsSlower than accepting a convincing first draft, but far cheaper than debugging code whose reviewer never established what it promised.
Go and read
Debugging as a procedure
Debugging is a loop: reproduce the failure, narrow where it begins, state one explanation, and run the smallest test that could disprove it.
Reach for it whenUse the loop for every repeatable failure, especially when your first urge is to edit the suspicious line immediately. Intuition is useful for choosing a hypothesis, not for skipping evidence.
Think of it likeFinding which bulb broke a string of lights. You confirm the string is dark, test the midpoint, choose the bad half, and repeat instead of replacing bulbs at random.
Start with a stable reproduction: exact input, environment and expected versus actual result. If it fails only sometimes, record timing, seeds and external responses until the difference is visible. Then bisect across commits, pipeline stages or input pieces—wherever you can divide the search space—and stop at the first point the state becomes wrong.
Write the hypothesis as a prediction: "the cache key omits the user ID, so two users with the same query will share a result." Choose an observation that would prove it wrong, run it, and update your model before editing. After the fix, rerun the reproduction, the surrounding suite and a new regression test; otherwise you know only that the symptom moved.
Syntax you can useassert actual == expected, (actual, expected)- Stops at the first broken assumption and shows both values.
print(f"{index=}, {value=}")- Prints labelled state while tracing a small failing case.
breakpoint()- Pauses at the suspicious line so you can inspect live state.
pytest -k failing_test -x- Runs one failing test and stops immediately, keeping the feedback loop short.
Several ways to see itWhat the machine doesFind the first wrong stateThe crash is often several calls after the cause. Trace backward until a value changes from expected to wrong, then ask what wrote it. Fixing the first bad transition is more reliable than guarding the final explosion.
What it costsHalving beats wanderingChecking files or commits one by one grows with the whole search space. A useful midpoint observation discards half at once: 128 candidates become one in seven checks. The discipline matters most when the codebase is too large to hold in your head.
The Python you typeWrite predictions in the debug logBefore each experiment, write "If X is true, I expect Y." Record the actual result beside it. This tiny habit stops repeated guesses and leaves a map another person can inspect when your model is wrong.
In the wild- git bisect — binary-searches commit history by repeatedly asking whether a chosen revision is good or bad until it identifies the introducing commit.
- rr — records a Linux program once and replays the same execution backward, making an intermittent failure inspectable instead of fleeting.
- Delta Debugging — systematically removes pieces of a failing input or configuration while preserving the failure, leaving a minimal cause-inducing case.
Commonly gets misread as- "Changing a likely line is faster than reproducing." Correction: without a reproduction you cannot tell whether the edit fixed the bug, hid it or changed nothing.
- "A hypothesis is an explanation that sounds right." Correction: it must predict an observation that could prove it wrong; otherwise it is only a story.
What it costsA few deliberate minutes before editing, repaid by fewer speculative changes and a fix you can actually verify.
Go and read
You are done with this stage when
Exit checkGiven a plausible-looking function you did not write, you can name an input that would break it and write the test that proves it.
If you cannot do this, repeat the stage rather than moving on. Nothing later gets easier by skipping it.