Running it on a schedule

Where your checker is

At the end of Lesson 7, your checker tells the truth with its exit code, and tests guard its most important rule. But it only runs when you type python3 checker.py, from inside its own folder, and the only record of a run is what scrolled past in your terminal.

This lesson makes it run by itself, every 15 minutes, and leave evidence that it did. That takes three things: a way to tell it which files to use, so it works from any folder; a log file, so there's a record when nobody's watching; and a scheduler to start it. You'll change checker.py in three steps, then hand it to the scheduler.

Keyword arguments and default values

When you order "a coffee", you get a medium with regular milk, unless you say otherwise. Those are defaults: sensible choices that apply when you don't specify. Functions can have them too.

You've been using them since Lesson 5 without the name. In requests.get(url, timeout=5) and json.dump(results, file, indent=2), timeout=5 and indent=2 are keyword arguments: values passed by name, instead of by position. And requests.get() works without a timeout, because timeout has a default value.

Your own fetch() has two numbers buried inside it: 3 tries, and 5 seconds. Give them names and defaults, so they're easy to see and easy to change.

Step 1. In checker.py, change the first two lines of fetch(), and the while and requests.get() lines, so it reads:

def fetch(url, max_attempts=3, timeout=5):
    attempts = 0
    while attempts < max_attempts:
        attempts = attempts + 1
        try:
            return requests.get(url, timeout=timeout)

The rest of fetch() stays the same. max_attempts=3 means "if the caller doesn't say, use 3". So fetch(site["url"]) works exactly as before, but fetch(site["url"], timeout=10) waits longer for a slow site, without touching anything else. In timeout=timeout, the left side is the name requests.get() expects, and the right side is your parameter's value. Run pytest and the checker: nothing has changed, which is the point.

Command-line arguments

When you print something, the Print button is the same every time, but the dialog lets you choose which pages, and how many copies. Command-line arguments do that for a script: the words you type after its name, which change what it does without changing its code.

Python collects them in a list called sys.argv. If you ran python3 checker.py --sites work.csv, then sys.argv would be:

['checker.py', '--sites', 'work.csv']

You could pick through that list yourself, but the standard library's argparse module does it properly. You describe the arguments you accept, and it reads them, fills in defaults, writes a help message, and rejects anything it doesn't recognise.

Step 2. Add import argparse at the top of the imports (they're in alphabetical order, so it goes first). Then add this function just above main():

def parse_args():
    parser = argparse.ArgumentParser(description="Check that websites are up.")
    parser.add_argument("--sites", default="sites.csv", help="CSV file of sites to check")
    parser.add_argument("--results", default="results.json", help="where to save the results")
    return parser.parse_args()

Each add_argument() describes one option: its name, its default, and a line of help. parse_args() reads what was typed and hands back an object with one value per option, as args.sites and args.results. Then, in main():

1. Add args = parse_args() as the first line.
2. Change load_sites("sites.csv") to load_sites(args.sites).
3. Change save_results("results.json", results) to save_results(args.results, results).

Now try the help that argparse wrote for you:

python3 checker.py --help

usage: checker.py [-h] [--sites SITES] [--results RESULTS]

Check that websites are up.

options:
  -h, --help         show this help message and exit
  --sites SITES      CSV file of sites to check
  --results RESULTS  where to save the results

Running python3 checker.py on its own still works, using the defaults. But now you can also run it from somewhere else. Remember the FileNotFoundError from Lesson 5, when you ran the checker from the folder above? Try that again, this time telling it where the files are:

cd ..
python3 site-checker/checker.py --sites site-checker/sites.csv --results site-checker/results.json
cd site-checker

It works, because the checker no longer guesses where its files are. You told it.

Logging: a record of every run

A plane's flight recorder notes everything that happens, with the time, so that when something goes wrong, there's a record to look at. When the checker runs by itself, nobody sees its output, so it needs one too: a log.

The standard library's logging module writes messages like print() does, but each one gets a timestamp and a level that says how serious it is: INFO for normal news, WARNING for something worth a look, and ERROR for something that failed. And it can write to a file and the screen at the same time.

Step 3. This step touches several functions, so here they are in full.

1. Add import logging to the imports, between json and sys.
2. Add a third argument to parse_args(), under --results:

    parser.add_argument("--log", default="checker.log", help="where to write the log")

3. Replace fetch() and check_site() with these versions, where each print() becomes a logging call at the right level:

def fetch(url, max_attempts=3, timeout=5):
    attempts = 0
    while attempts < max_attempts:
        attempts = attempts + 1
        try:
            return requests.get(url, timeout=timeout)
        except requests.RequestException:
            logging.warning(f"attempt {attempts} failed for {url}")
            time.sleep(1)
    return None


def check_site(site):
    name = site["name"]
    response = fetch(site["url"])
    if response is None:
        logging.error(f"{name}: no answer")
        return "error"

    status_code = response.status_code
    response_ms = to_ms(response.elapsed.total_seconds())
    if is_up(status_code):
        logging.info(f"{name}: up, {status_code} in {response_ms} ms")
        return "up"
    logging.warning(f"{name}: DOWN, {status_code} in {response_ms} ms")
    return "down"

4. Replace main() with this version. It sets up logging first, then does the same work as before, logging instead of printing:

def main():
    args = parse_args()
    logging.basicConfig(
        level=logging.INFO,
        format="%(asctime)s %(levelname)s %(message)s",
        handlers=[logging.FileHandler(args.log), logging.StreamHandler()],
    )

    sites = load_sites(args.sites)
    logging.info(f"Checking {len(sites)} sites from {args.sites}")

    down = 0
    errors = 0
    results = []
    for site in sites:
        result = check_site(site)
        results.append({"name": site["name"], "result": result})
        if result == "down":
            down = down + 1
        elif result == "error":
            errors = errors + 1

    save_results(args.results, results)
    logging.info(f"{down} of {len(sites)} sites down, {errors} unreachable")
    logging.info(f"Results saved to {args.results}")

    if down > 0 or errors > 0:
        return 1
    return 0

logging.basicConfig() sets logging up once, at the start. Its keyword arguments say: record INFO and anything more serious; start each line with the time and the level; and send every message to two places, a handler for the log file and one for the screen. The %(asctime)s parts are placeholders that logging fills in, a bit like an f-string.

Checkpoint: your checker so far

Your checker.py should now look like this:

# A website checker, built one piece at a time.
import argparse
import csv
import json
import logging
import sys
import time

import requests


def is_up(status_code):
    return status_code >= 200 and status_code < 400


def to_ms(seconds):
    return round(seconds * 1000)


def load_sites(path):
    with open(path, newline="") as file:
        return list(csv.DictReader(file))


def fetch(url, max_attempts=3, timeout=5):
    attempts = 0
    while attempts < max_attempts:
        attempts = attempts + 1
        try:
            return requests.get(url, timeout=timeout)
        except requests.RequestException:
            logging.warning(f"attempt {attempts} failed for {url}")
            time.sleep(1)
    return None


def check_site(site):
    name = site["name"]
    response = fetch(site["url"])
    if response is None:
        logging.error(f"{name}: no answer")
        return "error"

    status_code = response.status_code
    response_ms = to_ms(response.elapsed.total_seconds())
    if is_up(status_code):
        logging.info(f"{name}: up, {status_code} in {response_ms} ms")
        return "up"
    logging.warning(f"{name}: DOWN, {status_code} in {response_ms} ms")
    return "down"


def save_results(path, results):
    with open(path, "w") as file:
        json.dump(results, file, indent=2)


def parse_args():
    parser = argparse.ArgumentParser(description="Check that websites are up.")
    parser.add_argument("--sites", default="sites.csv", help="CSV file of sites to check")
    parser.add_argument("--results", default="results.json", help="where to save the results")
    parser.add_argument("--log", default="checker.log", help="where to write the log")
    return parser.parse_args()


def main():
    args = parse_args()
    logging.basicConfig(
        level=logging.INFO,
        format="%(asctime)s %(levelname)s %(message)s",
        handlers=[logging.FileHandler(args.log), logging.StreamHandler()],
    )

    sites = load_sites(args.sites)
    logging.info(f"Checking {len(sites)} sites from {args.sites}")

    down = 0
    errors = 0
    results = []
    for site in sites:
        result = check_site(site)
        results.append({"name": site["name"], "result": result})
        if result == "down":
            down = down + 1
        elif result == "error":
            errors = errors + 1

    save_results(args.results, results)
    logging.info(f"{down} of {len(sites)} sites down, {errors} unreachable")
    logging.info(f"Results saved to {args.results}")

    if down > 0 or errors > 0:
        return 1
    return 0


if __name__ == "__main__":
    sys.exit(main())

Run it, with your virtual environment active:

python3 checker.py

2026-10-24 23:24:55,908 INFO Checking 4 sites from sites.csv
2026-10-24 23:24:56,113 INFO github.com: up, 200 in 111 ms
2026-10-24 23:24:56,290 INFO pypi.org: up, 200 in 131 ms
2026-10-24 23:24:56,519 WARNING missing page: DOWN, 404 in 223 ms
2026-10-24 23:24:56,525 WARNING attempt 1 failed for https://example.invalid
2026-10-24 23:24:57,539 WARNING attempt 2 failed for https://example.invalid
2026-10-24 23:24:58,553 WARNING attempt 3 failed for https://example.invalid
2026-10-24 23:24:59,559 ERROR no such site: no answer
2026-10-24 23:24:59,561 INFO 1 of 4 sites down, 1 unreachable
2026-10-24 23:24:59,562 INFO Results saved to results.json

Your dates and times will be your own. The same lines are now in a new file, checker.log, next to checker.py. Run the checker again and open checker.log: the second run's lines are added below the first. Unlike results.json, which "w" replaces every time, a log file grows, so it remembers every run. Run pytest too: still 5 passed.

Scheduling with cron

An alarm clock set to "weekdays at 7" goes off by itself every morning; you set it once. cron is that for scripts, on macOS and Linux: a scheduler that's always running in the background, starting jobs at the times you give it. (Windows has its own, called Task Scheduler, which you set up in a window rather than a file; the full paths and output file below apply there too.)

Your list of cron jobs is called your crontab. Each job is one line: five fields that say when, then the command to run. The five fields are minute, hour, day of the month, month, and day of the week, and * means "every".

cron doesn't know anything you haven't told it. It doesn't start in your site-checker folder, so file names like sites.csv won't be found. It doesn't activate your virtual environment, so python3 won't have requests. And it doesn't show you the output, because there's no terminal. So the command has to spell everything out.

First, find your folder's full path. In your site-checker folder, run:

pwd

pwd (print working directory) shows where you are, like /Users/you/site-checker. Use your own path wherever you see that below.

Step 4. Open your crontab for editing. This sets the editor to nano, which is simpler than the default:

EDITOR=nano crontab -e

Add this as one single line, with your own path in each of the six places:

*/15 * * * * /Users/you/site-checker/.venv/bin/python /Users/you/site-checker/checker.py --sites /Users/you/site-checker/sites.csv --results /Users/you/site-checker/results.json --log /Users/you/site-checker/checker.log >> /Users/you/site-checker/cron-output.txt 2>&1

It's long, but every part has a job:

• */15 * * * * runs it every 15 minutes.
• .venv/bin/python is the Python inside your virtual environment. Running it directly uses the environment's packages, with no need to activate anything.
• --sites, --results and --log give every file its full path, so it doesn't matter where cron starts.
• >> cron-output.txt adds anything the command prints to a file, as >> did with echo in the Git track. 2>&1 sends error messages there too. That's your safety net: if the checker crashes before logging even starts, the reason lands here.

Save and exit nano with Ctrl+O, Enter, then Ctrl+X. To check it's there, list your crontab:

crontab -l

Then wait. Within 15 minutes, checker.log gets a new run at a time you weren't there. That's it: the checker now runs by itself, and leaves a record every time.

On a Mac, keep your site-checker folder somewhere like your home folder, not inside Desktop, Documents or Downloads. macOS protects those folders, and cron isn't allowed into them unless you give it Full Disk Access in System Settings. If your job seems to run but nothing happens, check cron-output.txt for "Operation not permitted".

When you no longer want the job, run EDITOR=nano crontab -e again and delete the line. Avoid crontab -r, which deletes your whole crontab without asking.

Break it on purpose

Try each one, then put it back.

Mistype an argument. Run the checker with --stes instead of --sites:

python3 checker.py --stes sites.csv

usage: checker.py [-h] [--sites SITES] [--results RESULTS] [--log LOG]
checker.py: error: unrecognized arguments: --stes sites.csv

argparse refuses to run with an option it doesn't know, and reminds you which ones it does. That's far better than silently ignoring the typo and checking the wrong file. Run echo $? straight afterwards and you'll see 2: argparse's exit code for "you used me wrong", which is still "not 0", so anything watching knows the run didn't happen.

Forget the virtual environment. cron doesn't activate anything, so the path to Python in your crontab line matters. To see what happens if you get it wrong, run the cron command by hand, but with plain python3 instead of the .venv path. First run deactivate, so your terminal isn't using the environment either, then:

python3 /Users/you/site-checker/checker.py --sites /Users/you/site-checker/sites.csv >> /Users/you/site-checker/cron-output.txt 2>&1

Nothing appears on screen, and checker.log gets no new lines at all. Open cron-output.txt:

Traceback (most recent call last):
  File "checker.py", line 9, in <module>
    import requests
ModuleNotFoundError: No module named 'requests'

(Your File line shows the full path.) The checker crashed on line 9, before logging had even started, so the log can't help you. Only cron-output.txt caught it. That's why the >> cron-output.txt 2>&1 part is in the crontab line: when a scheduled job seems to do nothing, it's the first place to look. Activate the environment again with source .venv/bin/activate.

Turn the volume down. In main(), change level=logging.INFO to level=logging.WARNING, and run the checker with a separate log file, so your real one stays tidy:

python3 checker.py --log quiet.log

Open quiet.log:

2026-10-24 23:25:23,284 WARNING missing page: DOWN, 404 in 102 ms
2026-10-24 23:25:23,287 WARNING attempt 1 failed for https://example.invalid
2026-10-24 23:25:24,302 WARNING attempt 2 failed for https://example.invalid
2026-10-24 23:25:25,317 WARNING attempt 3 failed for https://example.invalid
2026-10-24 23:25:26,322 ERROR no such site: no answer

The INFO lines are gone: no "up" sites, no summary. The level is a volume knob: it records that level and anything more serious, and drops the rest. A quiet log is easier to scan for problems, but it can no longer prove that a run happened when everything was fine. For a checker that runs while you're away, that proof matters, so set it back to logging.INFO, and delete quiet.log.

Where you are now

Your checker runs by itself every 15 minutes, works from any folder, and keeps a timestamped log of every run. But to find out something's wrong, you still have to go and read the log. Lesson 9 makes it tell you: an alert on your phone whenever the exit code says there's a problem, with its settings kept out of the code.

Check yourself

Key terms

• Keyword argument: a value passed by name, like timeout=5.
• Default value: the value a parameter uses when the caller doesn't pass one, like max_attempts=3.
• Command-line arguments: the words typed after a script's name, collected in sys.argv.
• argparse: the standard library module for reading command-line arguments, with defaults and --help built in.
• Log: a timestamped record of what a program did, kept in a file that grows with every run.
• Log level: how serious a message is: INFO, WARNING or ERROR. The configured level drops anything less serious.
• cron: the scheduler on macOS and Linux that runs commands at set times.
• crontab: your list of cron jobs, one per line: five time fields, then the command. Edit it with crontab -e.
• Full path: a file's complete location, like /Users/you/site-checker/sites.csv, which works from any folder.

Go deeper

• The argparse tutorial (opens in a new tab) in the Python documentation, from simple options to more advanced ones.
• The logging how-to (opens in a new tab) covers levels, formats, and logging to files.
• crontab.guru (opens in a new tab) explains any cron schedule in plain English as you type it.