Data and settings

By the end you'll know

How to give the same image different settings, without rebuilding it
Where to keep data so it survives a container being removed
How to share a folder from your laptop with a container, and why that's great for development
How to limit a container's memory and CPU, and what happens when it goes over

One image, different settings

In Lesson 2 you learned that changes inside a container disappear with it, and in Lesson 4 you baked your app into an image. That leaves two questions every real app runs into:

Settings. Your app needs a database address, a log level, maybe a password, and they're different on your laptop and in production. Building a separate image for each place would defeat the point of one image that runs everywhere.
Data. A database writes files. If they live inside the container, removing the container deletes the data.

This lesson answers both. Settings come in from outside through environment variables; data lives outside the container in volumes.

Settings: environment variables

An environment variable is a named value a program can read when it starts, like GREETING=Hello. Docker lets you set them for each container with -e. Most apps, and most official images, are designed to be configured this way.

Make your app from Lesson 4 read one. In your hello-app folder, replace app.py with:

PYTHON

import os
import socket

from flask import Flask

app = Flask(__name__)

GREETING = os.environ.get("GREETING", "Hello")   # read GREETING, or use "Hello" if it isn't set


@app.get("/")
def hello():
    return f"{GREETING} from a container! My name is {socket.gethostname()}.\n"

Build it, then run the same image twice, with different settings:

BASH

docker build -t hello-app:2.0 .
docker run -d -p 8000:8000 --name hello-en hello-app:2.0
docker run -d -p 8001:8000 --name hello-fr -e GREETING=Bonjour hello-app:2.0
curl http://localhost:8000
curl http://localhost:8001

Hello from a container!, then Bonjour from a container!. One image, two behaviors, no rebuilding. When you have several settings, put them in a file instead, one NAME=value per line, and pass it with --env-file:

BASH

echo "GREETING=Hola" > settings.env
docker run --rm -p 8002:8000 --env-file settings.env hello-app:2.0

Open http://localhost:8002 and you'll see Hola. Press Ctrl+C to stop it.

Settings aren't secret just because they're not in the image

Anyone who can run Docker commands on the machine can see a container's environment variables with docker inspect. And an env file with real passwords must never be committed to Git: add it to your .gitignore. Environment variables are the right way to pass settings in; just keep in mind who else can read them. Kubernetes has a dedicated Secret object for sensitive values, covered in the Kubernetes track.

Data: three places it can live

Now data. There are three places a container can write files, and they last very different lengths of time:

Data written inside the container disappears with it. A named volume is kept until you delete it. A bind mount is just a folder on your laptop that the container can also see.

Think of a hotel guest's belongings. Things left in the room are cleared out at checkout: that's the container's own layer. The hotel's luggage room keeps your bags safe between stays, and staff look after it: that's a named volume. Bringing your own suitcase, which you take home and can open anytime, is a bind mount.

Try it: a named volume

Create a volume, and have one container write to it:

BASH

docker volume create notes
docker run --rm -v notes:/data busybox:1.36 sh -c 'echo "written by the first container" >> /data/notes.txt'

-v notes:/data means "attach the volume notes at the folder /data inside the container". That container is gone now (--rm). Start a completely new one with the same volume, and read the file:

BASH

docker run --rm -v notes:/data busybox:1.36 cat /data/notes.txt

written by the first container. The data outlived the container that wrote it. This is how you'd run a database in Docker: its data folder goes on a volume, so you can remove the container, upgrade to a newer image, and start again with your data intact. Lesson 6 does exactly that with a real database.

See your volumes, and how much space they use:

BASH

docker volume ls
docker system df

Try it: a bind mount for development

In Lesson 4, every change to app.py meant rebuilding the image. That's right for shipping, and slow for trying things out. A bind mount shares a folder from your laptop with the container, so the container sees your files as you edit them.

From your hello-app folder:

BASH

docker run --rm -p 8004:8000 -v "$PWD":/app --name hello-dev hello-app:2.0 flask --app app run --host 0.0.0.0 --port 8000 --debug

Three new things in one command:

-v "$PWD":/app shares your current folder ($PWD is your shell's name for it) at /app inside the container, on top of the app.py that was baked into the image.
Everything after the image name replaces the image's CMD for this one container. Here it's the same command plus --debug, which makes Flask reload whenever a file changes.
There's no -d, so the logs stay in your terminal.

Now edit app.py in your editor: change from a container to from my laptop's folder, and save. Flask notices and reloads, and curl http://localhost:8004 in a second terminal shows the new text. No rebuild. That's the everyday development loop with containers: the image provides the right Python and libraries, and your code comes straight from your folder.

Press Ctrl+C when you're done. Remember that the change only exists in your folder: the image hello-app:2.0 still has the old app.py until you build again.

In my real setup

My search for city officials' pages, from Lesson 2's real setup, uses both of this lesson's ideas. It runs on my home server like this:

BASH

docker run --rm -v "$PWD":/w -w /w -e DATABASE_URL -e SLACK_WEBHOOK_CHANGES civicgrid-discovery python discover_mayor_urls_v12.py --render

-v "$PWD":/w bind-mounts my project folder, so the scripts and their output files come straight from my folder; the image holds only Python, the libraries and the browser. -w /w sets the working folder, like WORKDIR. And -e DATABASE_URL with no value passes along the value already set in my server's own environment, so the database password never appears in the command, or in my shell history. The comment at the top of my Dockerfile sums it up: the image never holds code or secrets.

Limits: how much a container may use

By default, a container can use as much memory and CPU as the machine has. One misbehaving container can slow everything else down. Set limits with --memory and --cpus:

BASH

docker run -d -p 8005:8000 --name hello-limited --memory 128m --cpus 0.5 hello-app:2.0
docker stats --no-stream

--memory 128m caps it at 128 megabytes, and --cpus 0.5 at half of one CPU core. docker stats shows each container's actual use next to its limit. If you've done the Kubernetes track, these are the same ideas as requests and limits there.

Break it on purpose

Go over the memory limit. Your hello-app image has Python in it, so you can use it to run a one-line program that tries to use far more memory than it's allowed:

BASH

docker run --name greedy --memory 32m hello-app:2.0 python -c 'x = b"a" * (200 * 1024 * 1024); print("this never prints")'
echo $?

137, and this never prints never prints. The program tried to hold 200 megabytes in memory; the limit was 32. When a container goes over its memory limit, the system doesn't slow it down: it kills it. Ask Docker why it stopped:

BASH

docker inspect greedy --format '{{.State.OOMKilled}}'

true. OOM means "out of memory". Exit code 137 almost always means the container was killed this way. When an app keeps dying with 137, give it more memory or make it use less. (In the Kubernetes track this shows up as OOMKilled, for the same reason.)

Misspell a setting. Run the app with a typo in the variable's name:

BASH

docker run --rm -p 8003:8000 -e GREETNG=Bonjour hello-app:2.0

Open http://localhost:8003: it says Hello, not Bonjour. Docker happily set GREETNG, and the app never looked for it, so it used its default. No error anywhere. When a setting seems to be ignored, compare the exact name with what the app reads. Press Ctrl+C to stop.

Clean up

BASH

docker rm -f hello-en hello-fr hello-limited greedy
docker volume rm notes
docker rmi hello-app:2.0

docker volume rm deletes the volume and its data for good. docker system prune from Lesson 3 doesn't touch volumes unless you add --volumes, so your data isn't removed by accident.

Check yourself

You need your app to use a different database address on your laptop and in production. What's the usual way?
A database container writes its data inside its own writable layer. What happens when you remove the container?
What does -v notes:/data do?
Why is a bind mount handy during development?
A container keeps stopping with exit code 137. What's the most likely cause?
You pass -e GREETNG=Bonjour, and the app still says Hello. Why?

Key terms

Environment variable: a named value a program reads when it starts. Set with -e or --env-file.
Named volume: storage Docker manages, attached to a container at a folder. Kept until you delete it.
Bind mount: a folder from your laptop shared with a container.
-v: attaches a volume or a bind mount: name-or-folder:/path/inside.
--memory / --cpus: limits on how much memory and CPU a container may use.
OOM killed: stopped by the system for going over its memory limit. Usually exit code 137.

Go deeper

Volumes (opens in a new tab) in the official Docker docs.
Bind mounts (opens in a new tab) explains sharing folders, and when to use each.
Resource constraints (opens in a new tab) covers memory and CPU limits.