Does This Code
Actually Work?

Who This Is For

This guide is for people who write code — or work with code that AI generates — and want to verify it actually does what it's supposed to do. You might be a developer who knows tests are important but isn't sure what makes a test good. You might be someone who asks AI to write tests and suspects the results are hollow but can't articulate why.

You don't need testing experience. You need to understand what tests are actually for, what makes them trustworthy versus theatrical, and how to evaluate whether the tests in front of you are doing real work.

Plan for 35 minutes. You'll leave with a mental model that makes bad tests immediately recognizable, working vocabulary for the test categories you'll encounter, concrete patterns for what's worth testing and what isn't, and a specific framework for using AI to generate tests that actually find bugs rather than just look like tests.

Why This Matters Now

Testing has always been important. It matters more now for a specific reason: AI writes code that looks correct much more often than it is correct.

The code is fluent, well-formatted, handles the obvious case, and fails silently on the cases nobody described. Without tests that probe the edge cases, you have no way to know where the silent failures are.

The workflow "ask AI to write the code, then ask AI to write tests for it" is now extremely common, and it has a serious structural problem. AI generates tests the same way it generates code: optimized for appearance, covering the cases that were described, missing the cases that weren't. If you ask AI to write tests for code AI wrote, you get tests that confirm the code does what was described — which is the least interesting question.

The Mental Model: A Test Is a Falsifiable Claim

Here is the one insight that separates tests that find bugs from tests that don't: a test is a falsifiable claim about behavior, and most tests people write aren't actually falsifiable.

A falsifiable claim is one that can be proven wrong. "This function returns the correct discount" is not falsifiable — what does "correct" mean? "Given a price of 100 and a discount rate of 0.20, this function returns 80.00" is falsifiable — it will fail if the function returns 79.99, 80.01, or "eighty."

The Test That Isn't Falsifiable

def test_calculate_discount():
    result = calculate_discount(100, 0.20)
    assert result is not None

This test passes if the function returns 80. It also passes if the function returns -500, or "hello", or 0. The only thing it falsifies is "the function crashes or returns None." That's almost never the interesting question.

The Test That Is Falsifiable

def test_calculate_discount():
    assert calculate_discount(100, 0.20) == 80.00

This fails if the function returns anything other than exactly 80.00. It makes a specific claim that can be proven wrong.

But this is still incomplete. It only tests one case — the happy path with clean inputs. A genuinely useful test suite makes falsifiable claims across the full space of inputs the function will encounter: normal cases, edge cases, boundary conditions, and invalid inputs.

The Testing Pyramid: What Each Layer Is For

Tests are usually organized into three layers, each with a different purpose, speed, and scope of what they verify.

Unit Tests

Test a single function or class in isolation. No database, no network, no filesystem — just the function and its inputs and outputs. Fast (milliseconds), specific (when they fail, you know exactly what broke), and abundant (you should have many of them).

Integration Tests

Test multiple components working together. A function that queries a real database. Two services communicating. A module that reads a file and parses it. Slower than unit tests, but they catch the bugs that happen at the seams between components — which is where most bugs actually live.

End-to-End (E2E) Tests

Test the full system from the outside, the way a user would. A browser navigating to a page and clicking through a flow. An API call that touches every layer from HTTP handling to database and back. Slowest, most fragile, but the only tests that verify the system actually works as a whole.

What Makes a Test Trustworthy

Before the mechanics of how to write tests, the properties that make a test worth having. These are the criteria by which you evaluate AI-generated tests or any tests you inherit.

Specificity. The assertion makes a precise claim. Not "the result is truthy" but "the result is exactly 80.00." Not "an error is raised" but "a ValueError is raised with a message containing 'must be positive.'" Vague assertions pass too many wrong implementations.

Isolation. The test doesn't depend on other tests running first, on external state from a previous test run, or on global configuration that might change. A test that fails when run in a different order has a hidden dependency.

Determinism. The test produces the same result every time, regardless of when it runs or what order tests execute in. Tests that fail intermittently — "flaky tests" — are worse than no tests because they train developers to ignore test failures.

Coverage of behavior, not implementation. Tests coupled to implementation details break when you refactor, even if the behavior hasn't changed. A test that verifies a function calls a specific private method is testing implementation. A test that verifies the function returns the correct output is testing behavior. Prefer the latter.

A single reason to fail. Each test should test one thing. A test that makes twenty different assertions about a complex object can fail for twenty different reasons. One assertion per test (or per logical concept) produces failures that are immediately diagnostic.

Readability. A test that reads clearly is documentation as well as verification. If you can't understand a test without reading the implementation, it's not providing documentation value.

What to Test: The Territory Map

The most common question from people new to testing is "what should I write tests for?" The answer: the behaviors that matter, across the input space where your code will actually operate.

The Happy Path — Necessary but Never Sufficient

The happy path is the case where everything goes right. Every test suite starts here. But happy path coverage alone is almost worthless as a bug-catching mechanism: the code was written to handle the happy path (of course it passes), bugs live in the cases the developer didn't think about, and AI-generated code almost always handles the happy path correctly.

Edge Cases and Boundary Conditions

Edge cases are the inputs at the boundary of what the function accepts — the first and last elements of a list, the minimum and maximum valid values, the empty collection, the single-element collection, the exactly-at-the-limit value.

Boundary conditions are where most off-by-one errors live. If a function accepts ages 18 to 65, the cases to test are: 17 (should reject), 18 (should accept), 65 (should accept), 66 (should reject).

def test_age_validation():
    assert is_valid_age(18) == True    # minimum boundary
    assert is_valid_age(65) == True    # maximum boundary
    assert is_valid_age(17) == False   # just below minimum
    assert is_valid_age(66) == False   # just above maximum
    assert is_valid_age(40) == True    # interior case

Error Cases and Invalid Inputs

What happens when the function receives something it shouldn't? Wrong type, null value, negative number where only positive makes sense, empty string where content is required, malformed data?

def test_calculate_discount_invalid_inputs():
    with pytest.raises(ValueError, match="discount_rate must be between 0 and 1"):
        calculate_discount(100, -0.10)

    with pytest.raises(ValueError):
        calculate_discount(100, 1.50)

    with pytest.raises(TypeError):
        calculate_discount("not a number", 0.20)

Concurrent and Stateful Behavior

If your code maintains state between calls, test that the state transitions are correct. If your code runs concurrently, test that concurrent access doesn't produce incorrect results. These are the hardest tests to write and the most likely to be omitted from AI-generated test suites — and where some of the most dangerous bugs live.

The Critical Paths

Not all behavior is equally important. Identify the paths where a failure would be most consequential — authorization checks, payment calculations, data modification operations, security boundaries — and test those more thoroughly than everything else. A bug in your recommendation algorithm is annoying. A bug in your payment calculation or authorization check is catastrophic.

The Coverage Trap

Code coverage measures what percentage of your code is executed by your test suite. It sounds like a quality metric. It's not — it's a proxy for quality that breaks badly when optimized directly.

Coverage measures whether a line of code was executed during tests. It doesn't measure whether the assertion verified anything meaningful. It doesn't measure whether edge cases were tested. It doesn't measure whether the test would catch a bug.

The Trap in Practice

def apply_discount(price, is_premium_user):
    if is_premium_user:
        return price * 0.80
    return price

# "100% branch coverage" but catches nothing:
def test_apply_discount():
    result = apply_discount(100, True)
    assert result is not None    # tests almost nothing

    result = apply_discount(100, False)
    assert result is not None    # tests almost nothing

Compare to tests with lower nominal coverage but actually falsifiable claims:

def test_apply_discount():
    assert apply_discount(100, True) == 80.0     # specific claim
    assert apply_discount(100, False) == 100.0   # specific claim
    assert apply_discount(0, True) == 0.0        # edge case

Where coverage is genuinely useful: finding code that has zero tests at all. A module with 0% coverage is completely unverified. Use coverage to find the completely untested areas, then use judgment to decide what those areas need.

The Annotated Tests: Bad, Better, Best

Walking through a single function with three generations of tests shows the progression more clearly than abstract description.

The Function

def transfer_funds(from_account, to_account, amount):
    """Transfer funds between accounts. Raises ValueError for invalid inputs."""
    if amount <= 0:
        raise ValueError(f"Amount must be positive, got {amount}")
    if from_account.balance < amount:
        raise ValueError(f"Insufficient funds: balance {from_account.balance}, requested {amount}")

    from_account.balance -= amount
    to_account.balance += amount
    return True

The Bad Tests (AI First Draft)

def test_transfer_funds():
    account_a = Account(balance=1000)
    account_b = Account(balance=500)

    result = transfer_funds(account_a, account_b, 200)
    assert result == True

One test, one case, only checks the return value. Doesn't verify balances changed. Doesn't test any error case. Would pass even if the function debited from account_a but forgot to credit account_b.

The Better Tests

def test_transfer_funds_happy_path():
    account_a = Account(balance=1000)
    account_b = Account(balance=500)

    transfer_funds(account_a, account_b, 200)

    assert account_a.balance == 800   # verifies debit happened
    assert account_b.balance == 700   # verifies credit happened

def test_transfer_funds_raises_on_negative_amount():
    account_a = Account(balance=1000)
    account_b = Account(balance=500)

    with pytest.raises(ValueError):
        transfer_funds(account_a, account_b, -50)

def test_transfer_funds_raises_on_insufficient_funds():
    account_a = Account(balance=100)
    account_b = Account(balance=500)

    with pytest.raises(ValueError):
        transfer_funds(account_a, account_b, 200)

Better: verifies state change, tests error cases. Still missing things.

The Best Tests

def test_transfer_funds_debits_source_and_credits_destination():
    account_a = Account(balance=1000)
    account_b = Account(balance=500)

    transfer_funds(account_a, account_b, 200)

    assert account_a.balance == 800
    assert account_b.balance == 700

def test_transfer_funds_exact_balance_succeeds():
    """Boundary: transferring exactly the available balance should succeed."""
    account_a = Account(balance=200)
    account_b = Account(balance=0)

    transfer_funds(account_a, account_b, 200)

    assert account_a.balance == 0
    assert account_b.balance == 200

def test_transfer_funds_raises_on_zero_amount():
    account_a = Account(balance=1000)
    account_b = Account(balance=500)

    with pytest.raises(ValueError, match="Amount must be positive"):
        transfer_funds(account_a, account_b, 0)

def test_transfer_funds_raises_on_negative_amount():
    account_a = Account(balance=1000)
    account_b = Account(balance=500)

    with pytest.raises(ValueError, match="Amount must be positive"):
        transfer_funds(account_a, account_b, -1)

def test_transfer_funds_raises_with_helpful_message_on_insufficient_funds():
    account_a = Account(balance=100)
    account_b = Account(balance=500)

    with pytest.raises(ValueError, match="Insufficient funds"):
        transfer_funds(account_a, account_b, 200)

def test_transfer_funds_does_not_modify_balances_on_failure():
    """On failure, neither account should be modified."""
    account_a = Account(balance=100)
    account_b = Account(balance=500)

    with pytest.raises(ValueError):
        transfer_funds(account_a, account_b, 200)

    assert account_a.balance == 100   # unchanged
    assert account_b.balance == 500   # unchanged

Mocks, Stubs, and Fakes: Isolating What You're Testing

Unit tests test functions in isolation. But functions often call other things — databases, APIs, the filesystem, external services. Test doubles are stand-ins for real dependencies.

The Three Types

A stub returns a predefined response. You're not verifying how it was called, just ensuring the code under test gets the response it needs to continue.

A mock is like a stub but also records how it was called. You can then assert that it was called with specific arguments, the right number of times, in the right order.

A fake is a lightweight working implementation — an in-memory database instead of a real one, a local filesystem instead of S3. Fakes are the most realistic of the three.

from unittest.mock import Mock, patch

def test_sends_confirmation_email_after_order():
    # Create a mock email service
    mock_email = Mock()

    # Inject it instead of the real service
    order_service = OrderService(email_service=mock_email)
    order_service.place_order(customer_id=1, items=["SKU-001"])

    # Verify the mock was called correctly
    mock_email.send.assert_called_once_with(
        to="[email protected]",
        template="order_confirmation",
        order_id=mock_email.send.call_args[1]["order_id"]
    )

The Over-Mocking Trap

When you mock everything and test only that the right mocks were called in the right order, you're not testing behavior — you're testing your mental model of the implementation. These tests break when you refactor, even when the behavior stays correct.

Patching for Determinism

from unittest.mock import patch
import datetime

def test_is_business_hours():
    # Control "now" so the test is deterministic
    with patch("mymodule.datetime") as mock_dt:
        mock_dt.now.return_value = datetime.datetime(2024, 1, 15, 10, 0)  # 10am Monday
        assert is_business_hours() == True

    with patch("mymodule.datetime") as mock_dt:
        mock_dt.now.return_value = datetime.datetime(2024, 1, 15, 22, 0)  # 10pm
        assert is_business_hours() == False

Code that calls datetime.now() directly is hard to test because "now" changes. Patching it makes the test deterministic. This is also why injecting dependencies (passing them as arguments) is easier to test than hard-coding them.

Test-Driven Development: The Idea, the Value, and the Limits

TDD is a practice where you write the test before the code. The cycle: write a failing test, write the minimum code to make it pass, refactor. Repeat.

The appeal: it forces you to think about behavior before implementation. You can't write a test for code you haven't designed — you have to decide what the function's interface is, what it accepts, what it returns, what errors it raises. This design pressure often produces better interfaces than writing code first.

The evidence: TDD's value as a design tool is better established than its value as a bug-prevention tool. The process tends to produce more testable code and more coherent interfaces.

The practical reality: strict TDD — always write the test first — is rarely practiced as written. What most teams actually do is "test-alongside," writing tests as they write code, or "test-after" with discipline to write real tests.

Integration Tests: Testing the Seams

Most bugs don't live inside individual functions. They live in how functions interact — in the assumptions function A makes about what function B returns, in the data that moves between layers, in the connection between the application and its database.

Database Integration Tests

Testing that your queries actually work against a real database schema — particularly valuable and often underrepresented.

import pytest

@pytest.fixture
def test_db():
    """Provide a real test database, clean it up after each test."""
    db = create_test_database()
    yield db
    db.drop_all_tables()

def test_user_can_be_created_and_retrieved(test_db):
    user_id = test_db.create_user(email="[email protected]", name="Test User")

    retrieved = test_db.get_user(user_id)

    assert retrieved.email == "[email protected]"
    assert retrieved.name == "Test User"
    assert retrieved.id == user_id

def test_duplicate_email_raises_constraint_error(test_db):
    test_db.create_user(email="[email protected]", name="First User")

    with pytest.raises(IntegrityError):
        test_db.create_user(email="[email protected]", name="Second User")

Fixtures

The @pytest.fixture decorator is the testing pattern for setup and teardown. A fixture creates the state a test needs, provides it, and cleans up afterward. Using fixtures instead of setup/teardown methods keeps tests independent and readable.

Test Database Isolation

Each test run should start from a known clean state. Tests that share database state can interact in unexpected ways — one test creates records that another test doesn't expect, causing intermittent failures. Common patterns: use transactions that roll back after each test, use a fresh database per test run, or use fixtures that explicitly reset state.

End-to-End Tests: When and How

E2e tests simulate real user behavior against the full running system. They're the most realistic tests and the most expensive to write and maintain.

When They're Worth the Cost

Critical user flows where a failure would be immediately visible and consequential: user registration and login, checkout and payment, document export, data import. These are worth e2e coverage because the cost of failure is high and the full-stack nature of the flow means unit and integration tests can miss coordinating failures.

When They're Not

Functionality already covered by reliable unit and integration tests. Simple UI interactions with no complex backend logic. Any flow that changes frequently — e2e tests for fast-changing UI are constantly broken and become maintenance burden rather than safety net.

Tools and Example

Playwright and Cypress are the current standards for browser-based e2e testing.

# Playwright example
from playwright.sync_api import Page

def test_user_can_complete_checkout(page: Page):
    page.goto("https://localhost:3000")
    page.click("text=Add to Cart")
    page.click("text=Checkout")
    page.fill("[name=email]", "[email protected]")
    page.fill("[name=card_number]", "4242424242424242")
    page.click("text=Place Order")

    page.wait_for_selector("text=Order confirmed")
    assert page.is_visible("text=Order confirmed")

Using AI to Write Tests: The Good, the Hollow, and the Gap

AI is genuinely useful for generating tests. It's fast, it handles the boilerplate, and it can enumerate test cases you might not have thought of. It also has specific, predictable failure modes.

What AI Does Well

Generating the mechanical structure of tests — the setup, the fixture, the assertion syntax. Covering the obvious happy path. Generating tests for simple, pure functions. Suggesting edge cases when you ask directly. Explaining what a test is doing and why.

What AI Does Badly

Generating assertions that falsify anything interesting. The most common AI test pattern is asserting that the result is not null, that the function returned successfully, or that some attribute exists — all of which pass for most wrong implementations.

Testing side effects. If a function modifies state — writes to a database, sends an email, logs something — AI often tests only the return value and ignores the state change.

Generating the adversarial test — the one that catches the specific bug the implementation is likely to have. That requires reasoning about failure modes, not about what the function is supposed to do.

The Diagnostic Question

For each AI-generated test, ask: what implementation would make this test pass while the function was still broken?

# AI-generated test
def test_process_payment():
    result = process_payment(amount=50.00, card="4242424242424242")
    assert result["status"] == "success"

What makes this test pass while still broken? A function that returns {"status": "success"} without actually processing the payment. A function that charges the wrong amount. A function that doesn't record the transaction. Almost nothing is falsified.

How to Prompt for Better Tests

Even better: write the first test yourself — the most important one. Then ask AI to write tests for the remaining cases, given the style and rigor of your example. AI follows examples well; showing it what a real test looks like produces better results than describing it.

The Review Checklist

□ Does each assertion make a specific claim (exact value, not "not None")?
□ Are state changes verified, not just return values?
□ Are error cases tested with specific error types and messages?
□ Is the boundary condition tested (exactly at limit, one above, one below)?
□ Is there a test that verifies failure doesn't cause partial state modification?
□ Would these tests pass for a plausible broken implementation?

Testing in Practice: The Habits That Scale

The conceptual model and the mechanics matter less than the habits you develop for when and how you test.

Test as you write, not after. Tests written after code tend to be shaped by the implementation — you know what the code does, so you test what it does, rather than what it should do. Tests written alongside code are more likely to be specification-driven.

Fix the bug, write the test, then fix the bug. When you find a bug, the first move is writing a test that reproduces it — a test that fails because of the bug. Then fix the bug. Then verify the test passes. This gives you a regression test that guarantees the bug doesn't come back.

Treat test failures as signal, not noise. A failing test is telling you something. The reflex should be "what does this tell me about what I just changed?" not "how do I get this test to pass?" If the test is correct and your change broke it, the test did its job.

Keep tests close to the code they test. Tests in the same directory as (or adjacent to) the code they cover are easier to find, easier to update, and more likely to be maintained.

Delete tests that no longer serve. A test for behavior that no longer exists is noise. Accumulated obsolete tests make the suite harder to read and failures harder to interpret.

Where This Breaks Down

The testing model in this guide covers the vast majority of situations you'll encounter. But there are real limits.

Async and concurrent code is genuinely hard to test. Race conditions, timing-dependent behavior, and subtle interaction bugs often aren't caught by standard techniques. Dedicated tools and patterns exist, but they require significantly more effort.

Tests don't verify correctness — they verify conformance to the test. If your tests are wrong about what the code should do, passing tests don't mean working code. The test suite is only as good as the specification it's encoding.

High coverage of the wrong things. Excellent coverage of validation logic and no coverage of business logic is a real antipattern. Reading the tests and asking "if this suite passed, what am I confident about?" is a better assessment than any metric.

Integration tests against external services — third-party APIs, payment processors — are hard to run reliably. The usual solution is mocking at the boundary, which trades realism for reliability. Real bugs at the external service boundary require contract tests, dedicated integration environments, or careful manual testing with monitoring.

E2e tests are expensive to maintain. Budget for it. A test suite that isn't maintained — tests skipped because they're flaky, never updated as the UI changes — is worse than no test suite because it creates false confidence.

Quick Reference

Everything you need in one place.

Test Quality Checklist

□ Assertion makes a specific claim (not "not None", not "is truthy")
□ State changes verified, not just return values
□ One logical concept per test
□ Test name describes the scenario and expected behavior
□ Passes only for correct implementations
□ Error cases test specific error type and message
□ Boundaries tested at limit, one below, one above

The Boundary Test Pattern

# For any function with a numeric boundary condition:
assert func(LIMIT)     == expected_at_limit
assert func(LIMIT - 1) == expected_below
assert func(LIMIT + 1) == expected_above

The Error Case Pattern

with pytest.raises(SpecificErrorType, match="expected message fragment"):
    function_that_should_fail(bad_input)

The State Change Pattern

# Record state before
before_a = account_a.balance
before_b = account_b.balance

# Perform operation
transfer_funds(account_a, account_b, 100)

# Verify state after
assert account_a.balance == before_a - 100
assert account_b.balance == before_b + 100

The Non-Modification on Failure Pattern

original_state = get_state()
with pytest.raises(SomeError):
    operation_that_should_fail()
assert get_state() == original_state  # nothing changed

AI Test Prompting

Instead of: "Write tests for this function"

Use: "Write tests for this function that would fail if:
  - [specific bug 1]
  - [specific bug 2]
  Include boundary tests at [specific boundary].
  Verify failed operations don't partially modify state."

Coverage Interpretation

0-40%    → significant untested areas, use coverage to find them
40-70%   → reasonable baseline, use judgment for what to add
70-90%   → good coverage, diminishing returns on the metric
90%+     → focus on assertion quality, not coverage number
>95%     → verify you're not writing hollow coverage-filling tests

Pyramid Ratios (Rough Target)

Unit tests:        many (hundreds in a medium project)
Integration tests: some (dozens)
E2e tests:         few (single digits to low tens for critical paths)

What You Can Do Now

Find a function — one you wrote, one AI wrote, one you inherited — that has at least one test. Apply the falsifiability question: for each assertion, what would have to be true for this test to fail?

If the answer is "almost nothing" — the function crashes, returns None, raises any exception at all — the test is not doing real work. Write one replacement test that makes a specific, falsifiable claim: the exact return value, the exact state change, the exact error type and message. One real test.

Then ask: does this function have a test for what happens when inputs are at the boundary? When inputs are invalid? When a dependency fails partway through? If not, pick one of those cases and write it.

You're not building a complete test suite. You're building the judgment to distinguish tests that verify behavior from tests that verify the code runs. The first time you write a test that actually catches a bug — one that would have made it to production without the test — that judgment becomes permanent.

Continue Learning

Related field guides that build on what you've learned here:

Coding Basics — variables, functions, and the foundations this guide builds on.
Debugging — a systematic method for finding the bugs your tests reveal.
Reading Code — how to approach unfamiliar code and evaluate what AI generates.
SQL — queries, databases, and where AI-generated SQL goes wrong.
APIs — what APIs are and how to build things that talk to external services.