Scripts you can trust

Where your checker is

At the end of Lesson 6, your checker visits real websites, reads real status codes, and keeps going when a site doesn't answer. It works, as long as you're sitting there reading its output.

Lesson 8 is going to run it on a schedule, when nobody's watching. Before that, the checker has to earn some trust. It needs a way to say "something's wrong" that another program can understand, without reading its output. And you need a way to know that a change you make tomorrow hasn't quietly broken a rule that works today. This lesson adds both, in three steps.

One job for the bottom of the file: main()

Right now, the bottom of checker.py is a run of loose lines: load the sites, loop, save, print. Python runs them the moment the file is loaded. That's about to cause a problem, because in a few minutes another file is going to import checker.py to borrow its functions, and importing a file runs its loose lines too. Borrowing is_up() shouldn't mean checking every website.

A recipe book lets you borrow one recipe without cooking the whole book. The fix is to give the checker the same split: put the main work in a function called main(), and only call it when the file is run directly.

Step 1. In checker.py, find the line sites = load_sites("sites.csv"). Put this line just above it:

def main():

Then indent everything from sites = load_sites("sites.csv") to the end of the file by 4 spaces, so it's all inside main(). (Most editors will indent a whole selection at once: select the lines and press Tab.) Then, at the very end of the file, back at the left edge and after two blank lines, add:

if __name__ == "__main__":
    main()

That line reads strangely, but it says something simple. When you run a file directly, with python3 checker.py, Python sets a special variable called __name__ to "__main__". When another file imports it, __name__ is set to the file's name instead, "checker". So this means "only run main() if this file was run directly, not if it was imported". You'll see it at the bottom of almost every Python script.

Run the checker. It does exactly what it did before. The difference shows up later in this lesson, when a test file imports it.

Exit codes: one signal at the end

A dishwasher doesn't make you read anything when it finishes: one light tells you whether it worked. Programs do the same. Every program, when it ends, hands back a number called its exit code. By convention, 0 means "everything was fine", and anything else means "something went wrong".

You can see the exit code of the last command you ran. Run the checker, then straight afterwards:

echo $?

$? holds the exit code of the last command, and echo prints it. Right now you'll see 0, even though a site was down and another unreachable, because the checker finished without crashing, and that's all Python reports by default. Anything that runs your checker without reading its output, like the alert you'll build in Lesson 9, would think everything was fine.

Step 2. Make the exit code tell the truth. At the end of main(), after the last print() and still indented inside it, add:

    if down > 0 or errors > 0:
        return 1
    return 0

Then change the last line of the file, the one under if __name__, so it passes that number on:

    sys.exit(main())

sys.exit() ends the script with the exit code you give it. It comes from the standard library's sys module, so add import sys to the imports at the top, between import json and import time. Now run the checker, and echo $? again:

python3 checker.py
echo $?

This time it prints 1, because one site was down and one unreachable. Remove the two failing sites from sites.csv, run both commands again, and you'll see 0. Put the sites back afterwards.

Tests: checking the rules before you need them

You press the test button on a smoke alarm now and then, so you find out it's broken before there's a fire, not during one. A test does that for your code: a small piece of code that checks another piece of code still gives the right answer.

The most important rule in the checker is is_up(): get it wrong, and every report is wrong. So test it. The tool almost everyone uses for testing Python is pytest. Install it the way you installed requests, with your virtual environment active:

pip install pytest

Step 3. Create a new file called test_checker.py in your site-checker folder, next to checker.py:

from checker import is_up, to_ms


def test_200_is_up():
    assert is_up(200)


def test_301_is_up():
    assert is_up(301)


def test_400_is_down():
    assert not is_up(400)


def test_503_is_down():
    assert not is_up(503)


def test_to_ms_rounds():
    assert to_ms(0.2504) == 250

Here's what's new:

• from checker import is_up, to_ms is another way to import. Instead of the whole module, it brings in just those two functions from your own checker.py, so you can call them by name. This is the import that would have checked every website, before step 1.
• Each test is a function whose name starts with test_. That's how pytest finds them.
• assert means "this must be true". If it is, nothing happens. If it isn't, the test fails, and pytest tells you exactly what the values were.
• Look at which codes are tested. 200 and 503 are the obvious cases. 301 and 400 sit right at the edges of the rule, one just inside and one just outside. Mistakes almost always happen at the edges, so that's where tests earn their keep.

Run them, from the site-checker folder:

pytest

============================= test session starts ==============================
platform linux -- Python 3.10.12, pytest-9.1.1, pluggy-1.7.0
rootdir: /home/you/site-checker
collected 5 items

test_checker.py .....                                                    [100%]

============================== 5 passed in 0.02s ===============================

Your platform and rootdir lines will show your own computer and folder. pytest found the test file, found 5 tests, and printed a dot for each one that passed. Notice how fast that was: tests check the rules without visiting a single website.

Checkpoint: your checker so far

Your checker.py should now look like this. The functions at the top haven't changed; the bottom is now inside main():

# A website checker, built one piece at a time.
import csv
import json
import sys
import time

import requests


def is_up(status_code):
    return status_code >= 200 and status_code < 400


def to_ms(seconds):
    return round(seconds * 1000)


def load_sites(path):
    with open(path, newline="") as file:
        return list(csv.DictReader(file))


def fetch(url):
    attempts = 0
    while attempts < 3:
        attempts = attempts + 1
        try:
            return requests.get(url, timeout=5)
        except requests.RequestException:
            print(f"      attempt {attempts} failed")
            time.sleep(1)
    return None


def check_site(site):
    name = site["name"]
    response = fetch(site["url"])
    if response is None:
        print(f"ERROR {name}: no answer after 3 tries")
        return "error"

    status_code = response.status_code
    response_ms = to_ms(response.elapsed.total_seconds())
    if is_up(status_code):
        print(f"UP    {name}: {status_code} in {response_ms} ms")
        return "up"
    print(f"DOWN  {name}: {status_code} in {response_ms} ms")
    return "down"


def save_results(path, results):
    with open(path, "w") as file:
        json.dump(results, file, indent=2)


def main():
    sites = load_sites("sites.csv")
    print("Checking sites...")
    print(f"{len(sites)} sites to check")

    down = 0
    errors = 0
    results = []
    for site in sites:
        result = check_site(site)
        results.append({"name": site["name"], "result": result})
        if result == "down":
            down = down + 1
        elif result == "error":
            errors = errors + 1

    save_results("results.json", results)
    print(f"{down} of {len(sites)} sites down, {errors} unreachable")
    print("Results saved to results.json")

    if down > 0 or errors > 0:
        return 1
    return 0


if __name__ == "__main__":
    sys.exit(main())

Run the checker and check its exit code:

python3 checker.py
echo $?

The output is the same as in Lesson 6, with your own times, followed by 1. Then run pytest, and you should see 5 passed.

Break it on purpose

Make each mistake, then put it back before the next.

A rule that's off by one. On line 11, in is_up(), change status_code < 400 to status_code <= 400, and run pytest:

test_checker.py ..F..                                                    [100%]

=================================== FAILURES ===================================
_______________________________ test_400_is_down _______________________________

    def test_400_is_down():
>       assert not is_up(400)
E       assert not True
E        +  where True = is_up(400)

test_checker.py:13: AssertionError
=========================== short test summary info ============================
FAILED test_checker.py::test_400_is_down - assert not True
========================= 1 failed, 4 passed in 0.03s ==========================

The F is the failed test. pytest shows which one, the line that failed, and the actual values: is_up(400) gave back True, when the test expected it to be down. A one-character slip, and every site answering 400 would have been reported as up. Your real checker would never have shown it, because none of your sites happen to answer 400 today. The test caught it in a fraction of a second. Change <= back to <.

A main() that forgets to answer. Delete the three lines at the end of main() that return 1 or 0, then run the checker and echo $?:

1 of 4 sites down, 1 unreachable
Results saved to results.json

followed by 0. A function with no return hands back None, as you saw in Lesson 4, and sys.exit(None) means "everything was fine". The output says a site is down, but the exit code says all is well, and in Lesson 9, it's the exit code that decides whether you get an alert. No error, just a lie. Put the three lines back.

Lose the name check. Replace the last two lines of checker.py with just sys.exit(main()), at the left edge, and run pytest. It takes several seconds this time, and ends with a wall of INTERNALERROR lines, finishing like this:

INTERNALERROR>     sys.exit(main())
INTERNALERROR> SystemExit: 1

============================ no tests ran in 3.55s =============================

When test_checker.py imported checker.py, nothing stopped main() from running, so the import checked every website, then sys.exit() ended the whole test run before a single test ran. That's exactly what the name check prevents. Put the two lines back:

if __name__ == "__main__":
    sys.exit(main())

Where you are now

Your checker can now tell another program whether everything was fine, with one number, and you have tests that check its most important rule in a fraction of a second. Lesson 8 runs the checker on a schedule and keeps a log of every run, and Lesson 9 alerts you when the exit code says something's wrong.

Check yourself

Key terms

• main(): the function that holds a script's main work, called only when the script runs directly.
• __name__: a special variable Python sets to "__main__" when a file is run directly, and to the module's name when it's imported.
• Exit code: the number a program hands back when it ends. 0 means success; anything else means something went wrong. echo $? shows the last one.
• sys.exit(): ends a script with the exit code you give it.
• Test: a small piece of code that checks another piece still gives the right answer.
• pytest: the most widely used tool for writing and running Python tests. It runs every test_ function in every test_ file.
• assert: "this must be true". If it isn't, the test fails.
• Edge case: a value right at the boundary of a rule, like status code 400, where mistakes are most likely.

Go deeper

• Get started with pytest (opens in a new tab) covers installing it, writing tests, and reading failures.
• Top-level code environment (opens in a new tab) in the Python documentation explains __name__ and "__main__" in more depth.