Weeks 7–10 · About 30 hours

Reading crashes, working with files, and finding bugs

How to read an error message instead of panicking, how to save and load data, and how to work out why a program is doing the wrong thing.

Why this stage exists

This is the difference between someone who can write code and someone who can fix code. Once you are into , most of your time will be spent working out why your returns the wrong answer. Being able to investigate calmly is what makes that survivable rather than demoralising. It is the least glamorous stage here and probably the highest-value one.

What you are learning — 8 topics

Reading a stack trace

A is the breadcrumb trail of function calls that led to a failure.

Reach for it whenRead the stack trace before adding print statements; start at the final exception and your nearest application frame, not the first library frame.

Read it from the bottom up. The last line names the kind of problem and describes it — for example NameError: name 'totl' is not defined, which means you misspelt a variable. That bottom line is usually all you need.

Above it is the traceback: the chain of function calls that led to the failure, oldest at the top, most recent at the bottom. When your own code called a library that failed, this chain shows which of your lines started it.

Syntax you can use

raise ValueError("n must be positive"): Signals a failure yourself, with a message the traceback will show.
import traceback: The module for handling tracebacks as data.
traceback.print_exc(): Prints the current exception and its traceback from inside an except block.
raise ValueError("bad row") from err: Chains a new error onto the original, so the traceback shows both causes.

Notice that the deepest frame detects the error, while the frames below explain how execution reached it.

Several ways to see it

Its shapeThe traceback is a route through nested calls

Each frame names a file, line, and function on the path to failure. Read from the final exception upward to find the immediate cause, then use earlier frames to learn who supplied the bad state.

What the machine doesThe last frame is where execution became impossible

Python unwinds active calls after an exception is not handled, recording each frame as it goes. The bottom line names the exception class and message; it is the compact summary, not incidental noise.

How it breaksThe failing line may only reveal an older mistake

A conversion can fail because malformed text entered five functions earlier. Treat the traceback as the location where the invariant was detected, then trace the suspect value backward to where it was created.

In the wild

Sentry — groups captured exceptions and displays their with source context.
pytest — shows the failing assertion and a filtered Python traceback for a failed test.

Commonly gets misread as

"The top line is where the bug is." Correction: the final exception and last relevant frame are usually the best start.
"Every frame is broken." Correction: frames show the call path; often only one assumption along it is wrong.

Go and read

Python tutorial: errors and exceptionsDocsOfficial explanation of what the different errors mean and how to handle them.docs.python.org/3/tutorial/errors.html Python docs: list of built-in exceptionsDocsLook up any error name you do not recognise here.docs.python.org/3/library/exceptions.html

Handling exceptions

An exception is a failure signal that travels up through callers until code deliberately handles it.

Reach for it whenCatch a specific exception instead of predicting every failure when the operation itself is the reliable test; do not catch Exception merely to hide a bug.

The shape is try: followed by the risky steps, then except SomeError: followed by what to do about it. If nothing goes wrong, the except block is skipped entirely.

Catch the specific error you expect, not everything. A bare except: swallows every problem including the ones you have not thought of, which turns a loud crash into silent wrong behaviour. That is strictly worse — a crash tells you something is broken, silence does not.

Syntax you can use

try:: Starts a protected block. The risky work goes here.
except ValueError:: Handles one specific failure. Prefer this to a bare except.
except (KeyError, IndexError) as err:: Catches several kinds and binds the object to a name.
else:: Runs only if nothing was raised, keeping its own errors out of the handler.
finally:: Runs either way. The place to release things.
raise: Re-raises the exception you are handling, unchanged, after logging it.
class ParseError(Exception):: Your own exception type, so callers can catch exactly this failure.
with open(path) as f:: A context manager: cleanup happens even if the block raises.

Several ways to see it

Why it existsErrors needed a route separate from results

Returning a special value forces every caller to remember to inspect it and reserves part of the result space for failure. Exceptions carry failure up the call chain until code with enough context can respond.

Against the alternativeCatch the failure you can actually repair

A broad `except Exception` can keep a process alive by hiding programming defects and corrupted state. Catch `ValueError`, `FileNotFoundError`, or another expected case at the boundary where you can retry, substitute, or explain.

The Python you typeElse and finally state two different promises

An `else` block runs only when the protected work succeeded, keeping its own errors out of the catch clause. `finally` runs either way, which makes it the place for releasing resources rather than reporting success.

In the wild

FastAPI — turns a raised HTTPException into an HTTP response with a chosen status and detail.
SQLAlchemy — raises database exceptions that application code can catch before rolling back a transaction.

Commonly gets misread as

"Exceptions are only crashes." Correction: they are signals that can be caught at a boundary able to recover.
"A broad except makes code robust." Correction: swallowing unexpected errors hides defects and bad state.

Go and read

Python tutorial: handling exceptionsDocstry, except, else and finally, with examples.docs.python.org/3/tutorial/errors.html

Reading and writing files

A file is data kept under a name so it survives after the program stops.

Reach for it whenUse a file instead of memory for durable local data or interchange; use a database when concurrent updates, queries, and integrity rules matter.

Everything held in a variable disappears when the program ends. A file is how you keep it. You open a file, read from it or write to it, and close it.

The safe pattern is with open("notes.txt") as f: — the with part guarantees the file is properly closed even if something goes wrong in the middle. Get into the habit of always using it.

Syntax you can use

with open("notes.txt") as f:: Opens for reading, and closes automatically at the end of the block.
text = f.read(): Reads the whole file into one string.
for line in f:: Streams one line at a time, so the file's size stops mattering.
open(path, "w"): Opens for writing.w TRUNCATES the file immediately. Use a to append.
f.write("text\n"): Writes a string. Unlike print, it adds no newline for you.
open(path, encoding="utf-8"): States the encoding explicitly, so the file reads the same on every machine.
import json: The standard reader and writer for structured data files.
json.load(f): Reads JSON from a file into dictionaries and lists. json.dump(obj, f) writes it back.
from pathlib import Path: Paths as objects: Path("data") / "x.txt", .exists(), .read_text().

Several ways to see it

What the machine doesA file handle owns an operating-system resource

`open` asks the operating system for a descriptor and tracks a current byte position. The `with` statement closes that descriptor even if parsing raises an exception midway through the block.

How it breaksText is bytes plus an agreed encoding

A file does not store characters directly; decoding turns byte sequences into characters. Specify UTF-8 when you control the format, or a file that works on one machine may fail or change symbols on another.

What it costsLine-by-line reading keeps memory flat

`read()` allocates space for the entire file, which is convenient for small inputs. Iterating over the handle streams one line at a time, so a gigabyte log does not require a gigabyte string.

In the wild

Git — stores repository objects and references as files inside the .git directory.
SQLite — keeps an entire transactional database in a single cross-platform file.

Commonly gets misread as

"Opening with w adds to a file." Correction: it truncates first; use a for append.
"close is optional." Correction: a with block closes promptly even when an exception occurs.

Go and read

Python tutorial: reading and writing filesDocsOfficial coverage, including the with pattern.docs.python.org/3/tutorial/inputoutput.html Python docs: the csv moduleDocsFor anything spreadsheet-shaped. Read the reader and DictReader sections.docs.python.org/3/library/csv.html

Debugging

Debugging is narrowing the gap between what you expected and what the program actually did.

Reach for it whenUse a debugger instead of scattered print calls when control flow or changing state is unclear; use a focused test or log when failure only occurs remotely.

Start with printing. Put print statements at the points where you have an assumption — print the value going into a function and the value coming out. Most bugs turn out to be a value that was not what you assumed, and this finds them.

Then graduate to the debugger, which is much better. You click beside a line number to set a breakpoint, run the program, and it pauses there. You can then look at every variable, and step forward one line at a time watching what changes. In VS Code this is the Run and Debug panel.

Syntax you can use

print(f"{value=}"): Prints the expression AND its value — value=3. Made for exactly this.
breakpoint(): Stops execution here and drops you into the debugger. Delete it before committing.
n: In the debugger: run the next line. s steps into a call, c continues, q quits.
p variable: In the debugger: print a value. pp pretty-prints, l lists the source around you.
assert total >= 0, total: Crashes immediately if an assumption is false, showing the offending value.
import sys; sys.exit(1): Stops the program with a failure code, for when continuing would do damage.

Several ways to see it

Why it existsDebugging begins when observation disagrees with belief

The useful question is not 'why is Python wrong?' but 'which assumption first became false?' A debugger, assertion, or focused print narrows the distance between the last known truth and the bad result.

What the machine doesA breakpoint freezes a complete program state

Pausing lets you inspect local names, the , and the next instruction without changing the source to print them. Stepping over runs a call as one unit; stepping into follows execution into that call's frame.

The Python you typeA tiny failing case is a debugging tool

Remove unrelated inputs and operations until the failure barely remains. The reduced example makes state easier to inspect and often exposes which condition is actually necessary for the bug.

In the wild

VS Code Python debugger — uses debugpy to pause at breakpoints and inspect frames, variables, and expressions.
pdb — steps through Python code and inspects the current call stack from a terminal.

Commonly gets misread as

"Debugging means guessing fixes." Correction: first reproduce, observe, and isolate the first wrong state.
"The line that crashes caused the bug." Correction: an earlier state change may only become visible there.

Go and read

VS Code: Python debuggingDocsHow to set breakpoints and step through code in the editor you are already using.code.visualstudio.com/docs/python/debugging Python docs: pdb, the built-in debuggerDocsThe terminal-based debugger, for when you are not in an editor. Worth knowing it exists.docs.python.org/3/library/pdb.html

Reproducing a bug before fixing it

Reproducing a bug means finding a specific input or sequence that makes it happen every time you ask.

Reach for it whenReproduce before you change anything; go straight to a fix only when the traceback names the mistake outright.

Start from the report and cut it down. Remove inputs, shorten the data, delete steps — each time checking that the failure still happens. What remains is the smallest thing that breaks, and it is usually small enough to reason about directly.

Then write it down as a test. It should fail before your change and pass after it. That single habit is the difference between fixing a bug once and fixing it again next year.

Syntax you can use

def test_empty_input_returns_zero():: A test is a function whose name starts with test_. pytest finds it for you.
assert total([]) == 0: The whole assertion vocabulary you need to start. pytest shows both sides on failure.
pytest: Runs every test it can find under the current folder.
pytest -k empty: Runs only tests whose name contains “empty” — the fast loop while fixing one bug.
pytest -x: Stops at the first failure instead of reporting all of them.
with pytest.raises(ValueError):: Asserts that the block DOES raise. A test that expects a crash.
@pytest.mark.parametrize("n,want", [(0, 1), (5, 120)]): Runs the same test body over several cases, reporting each separately.

Several ways to see it

Why it existsA fix you cannot test is a guess you cannot retire

Without a reliable way to trigger the failure, there is no way to tell a real repair from a coincidence. The reproduction is what converts 'it seems better' into 'it was broken, and now it is not'.

The Python you typeThe reproduction belongs in the test suite

Write the smallest input that fails as a test before touching the code. It fails now, passes after the fix, and keeps failing forever if someone reintroduces the bug.

How it breaksIntermittent failures usually have a hidden input

A bug that appears only sometimes still has a cause: the clock, the ordering of a set, a leftover file, a second thread. Hunt for the input you are not controlling rather than concluding the machine is unreliable.

In the wild

Bug trackers — ask for exact steps to reproduce, because a report nobody can trigger cannot be verified as fixed.
pytest — turns the reproduction into a permanent regression test that runs on every future change.

Commonly gets misread as

"It happens randomly." Correction: it happens under a condition you have not identified yet.
"The bug is gone." Correction: without a reproduction you only know it did not happen this time.

Go and read

pytest: getting startedDocsHow to write the small failing test that pins a bug down.docs.pytest.org/en/stable/getting-started.html

Logging you will thank yourself for at 3am

A log is a record your program writes about what it was doing, aimed at whoever reads it after the fact.

Reach for it whenLog at the boundaries — what came in, what was decided, what went out — and at every failure you catch; do not log inside a tight loop.

Use the logging module rather than print. logging.info("loaded %s rows from %s", count, path) gives you a timestamp, a level, and the ability to turn that whole category off later. A print gives you none of that.

Write each line for a stranger. Include the value that identifies the case — the filename, the user id, the record number — because the reader will be searching for one specific run among thousands. And log the failure you caught, not merely the fact that you caught one.

Syntax you can use

import logging: The standard library's logging system. Prefer it to print for anything long-lived.
logging.basicConfig(level=logging.INFO): Turns logging on and sets the threshold. Call it once, at program start.
log = logging.getLogger(__name__): A logger named after the module, so the output says where each line came from.
log.info("loaded %s rows from %s", count, path): Logs with the values filled in.Pass values as arguments rather than building an f-string; then the formatting is skipped entirely when the level is off.
log.warning("retrying %s", url): Something odd but survivable. debug < info < warning < error < critical.
log.exception("could not parse %s", path): Logs at error level AND attaches the traceback. Use it inside except.
logging.basicConfig(filename="app.log"): Sends records to a file instead of the terminal.

Notice that the level is a threshold rather than a label: raising it silences everything below without touching the calls that produce those records.

Several ways to see it

Against the alternativeA print is for now; a log is for later

Prints are written to answer today's question and deleted afterwards. A log line is written for a future reader who cannot attach a debugger and has only the file to work from.

What the machine doesLevels are a dial the reader controls, not decoration

Every record carries a level, and the configured threshold decides what is emitted. That lets one program stay quiet in production and turn verbose for a single investigation with no code change.

How it breaksA log without context cannot be searched

'Something went wrong' identifies nothing when it appears ten thousand times a day. Include the identifiers that let a reader select one request, one user, or one file out of the noise.

In the wild

Python's logging module — routes records through levels and handlers, so the same code can print to a terminal or write JSON to a file.
Cloud log viewers — search across millions of lines, which only works when each line carries the identifiers to search by.

Commonly gets misread as

"Logging is print with extra steps." Correction: levels, timestamps and routing are what make it readable months later.
"Log everything." Correction: volume is what buries the one line that mattered.

Go and read

Python docs: logging HOWTODocsThe practical introduction — levels, formatting, and writing to a file.docs.python.org/3/howto/logging.html

Bisecting a failure

Bisecting is halving the space of possible causes with each test until only one remains.

Reach for it whenBisect when something used to work and now does not and the code does not show why; skip it when the traceback already names the cause.

With git this is automatic. git bisect start, mark a commit you know was fine as good and the current one as bad, and git checks out the midpoint for you. You test, say good or bad, and it repeats — about ten steps for a thousand commits.

The same idea rescues you without git. If a data file breaks your parser, delete half of it and try again; if a config file breaks startup, comment out half. Every test you run should rule out half of what is left, which is what keeps the search short.

Syntax you can use

git bisect start: Begins a bisect session.
git bisect bad: Marks the current commit as broken.
git bisect good a1b2c3d: Marks a commit you know was fine. Git then checks out the midpoint for you.
git bisect run pytest -k parse: Runs the whole search automatically, using the command's exit code as the answer.
git bisect reset: Ends the session and puts you back where you started.
git log --oneline -20: Finds a commit hash to use as your known-good point.

Notice that one test at the midpoint eliminates half the candidates, whichever way the answer comes out.

Several ways to see it

Its shapeIt is binary search over history

A known-good point and a known-bad point bracket the change that broke things. Testing the midpoint discards half the remaining commits, so a thousand of them take about ten tests.

Against the alternativeThe same halving works on inputs and configuration

When history is not the variable, bisect something else: delete half the data, disable half the settings, comment out half the file. The method is the halving, not the tool.

How it breaksThe answer is only as good as the test

A flaky or vague check sends the search down the wrong half and names an innocent commit. Decide precisely what counts as broken before starting, and prefer a command that can answer without you.

In the wild

git bisect — walks the commit history for you, asking only whether each checked-out point is good or bad.
Incident response — narrows a production failure by halving the suspect surface — one service, one region, one config change — rather than reading everything.

Commonly gets misread as

"Bisecting needs version control." Correction: halving works on inputs, settings, or code you comment out.
"The commit it finds is the bug." Correction: it is the change that exposed the bug, which is where to start reading.

Go and read

Git docs: git bisectDocsThe commands, including running the search automatically with a test script.git-scm.com/docs/git-bisect

Modules — splitting code across files

A module is a Python file used as a named bundle of code by other files.

Reach for it whenSplit code into modules instead of one long file when parts have distinct jobs or reuse; do not create a new file for every tiny function.

Any .py file is a module. If you have helpers.py containing a function called clean(), then in another file you write import helpers and call helpers.clean(), or from helpers import clean to use the name directly.

Python also ships with a large — code that comes with the language, already tested, that you can import without installing anything. csv, math, random, json and collections are ones you will use constantly.

Syntax you can use

import math: Makes the module available as math.sqrt(9).
from math import sqrt: Pulls one name in directly, so you write sqrt(9).
import numpy as np: Imports under a shorter name.
from .helpers import clean: A relative import, from a module beside this one in the same package.
if __name__ == "__main__":: Runs the block only when the file is executed directly, not when it is imported.Without it, importing your file runs its script code as a side effect.
python3 -m mypackage: Runs a package as a program.

Several ways to see it

What the machine doesImport executes a module once

On first import, Python runs the file top to bottom and stores the resulting module object in `sys.modules`. Later imports in the same process reuse that object, so top-level side effects do not run afresh.

Its shapeA package mirrors a namespace tree

Dots in `shop.payments.cards` walk nested package names, not arbitrary text paths. That hierarchy lets separate modules use the same short internal names without colliding at the application level.

How it breaksCircular imports expose a missing boundary

If module A needs B while B is still importing A, one side sees a partially initialized module. Move shared definitions downward into a third module or move coordination upward so dependencies point in one direction.

In the wild

Django — imports settings, URL configuration, models, and app modules to assemble a project.
pytest — imports test modules during collection to discover test functions and classes.

Commonly gets misread as

"Import pastes the file's text at every use." Correction: Python executes a module once and caches the module object.
"from x import * improves organization." Correction: it hides where names came from and invites collisions.

Go and read

Python tutorial: modulesDocsimport, from-import, and how Python finds your files.docs.python.org/3/tutorial/modules.html The Python standard library indexDocsEverything that comes free with Python. Skim the contents page once so you know what is there.docs.python.org/3/library/index.html

Build these yourself

Reading about this stage is not the same as finishing it. Type every one of these from a blank file rather than copying — the writing is the part that teaches.

Text file statistics

Read a text file and report how many lines, words and characters it contains. Handle the file not existing, rather than crashing.

To-do list that survives restarting

Add tasks, mark them done, list them — and save to a file so everything is still there next time you run it. Your first program with persistence, which is a genuine step up.

CSV column averages

Read a CSV file and compute the average of a numeric column. Use the csv module. Deal with rows where the value is missing or is not a number, instead of letting it crash.

Break something on purpose, then fix it

Take one of your earlier programs and deliberately introduce three bugs: a typo in a variable name, an off-by-one error in a loop, and a wrong comparison. Then find each one using the debugger and a breakpoint, rather than by remembering what you changed. This is practice at the actual skill.

You are done with this stage when

Exit checkGiven a program that crashes, you can find and fix the bug using the traceback and a breakpoint, without asking anyone for help.

If you cannot do this, repeat the stage rather than moving on. Nothing later gets easier by skipping it.

This stage corresponds to Phase 0 — Errors, Files, and Debugging in the source roadmap.