Building and tagging images

Where you are

Lesson 3's checks guard a gate, and on the other side of it, the build job still just says echo "Would build an image". This lesson makes it real. Your sandbox gets a Dockerfile, GitHub builds the image and pushes it to a registry, and Jenkins builds and tests the same image with a version number of its own.

If you did the Docker track, images, tags and registries will be familiar. This lesson is about what changes when a pipeline does the building, on every merge, without you.

Nothing new to install: Jenkins and its Docker engine from Lesson 3 are all you need. If you stopped them, start them again with docker start jenkins-docker jenkins.

Build once, in the pipeline

Two habits come out of that:

• The pipeline builds the image, not a person's laptop. It starts from a clean checkout of a commit that passed its checks, so the image can only contain what's in the repository. An image built on a laptop can contain anything that happened to be in the folder.
• Build once, use everywhere. The same image moves from testing to production. You don't rebuild it for each environment, because a rebuild can pull in a newer base image or package and come out different. Lesson 8 is about moving that one image through to production.

Tags: labels with a job each

A tag is a label on an image in a registry, and the Docker track's Lesson 3 showed that a tag can be moved to point at a different image. In a pipeline, tags get a job each:

• sha-664d196: the commit the image was built from, using Git's short commit ID. Every commit gets its own, so it never needs to move. This is the one deploys use.
• 1.4.0 or 1.0.27-664d196: a version a person can read and talk about. Useful in release notes and conversations, often alongside the commit tag.
• main: the newest build of a branch. Handy for a test environment that should always run the newest code, and moves on every merge.
• latest: the newest build of the default branch. It moves on every merge, and Docker uses it whenever you don't say which tag you want.

That last point is the trap. A server that asks for app:latest gets whatever latest meant when it last pulled the image. Push a few commits, restart the servers, and see:

Server A was never deployed to. A restart, which happens for all kinds of ordinary reasons, quietly changed what it runs, and two servers asking for the same latest can run different code depending on when each last restarted. Server B runs what someone decided it should run, and the tag says which commit that is. Rolling back means changing one tag to the previous commit's.

If you need an absolute guarantee, the Docker track's digest, the sha256: fingerprint of the image's exact contents, is stronger still: a tag can be moved by someone with access to the registry, a digest can't. Commit tags that your pipeline never reuses are the practical middle ground most teams use, CivicGrid included.

The sandbox gets a Dockerfile

The visit counter needs something to run, so give it a tiny command-line entry point, then a Dockerfile. On your laptop, in ~/Apps/cicd-sandbox:

cd ~/Apps/cicd-sandbox
cat > app/__main__.py <<'EOF'
import sys

from app.visits import clean_name

name = sys.argv[1] if len(sys.argv) > 1 else "  Home "
print(f"{name!r} is counted as {clean_name(name)!r}")
EOF
cat > Dockerfile <<'EOF'
FROM python:3.13-slim
WORKDIR /srv
COPY app/ app/
ENTRYPOINT ["python", "-m", "app"]
EOF
cat > .dockerignore <<'EOF'
.git
.github
.venv*
tests
__pycache__
EOF
docker build -t cicd-sandbox:local .
docker run --rm cicd-sandbox:local "  About Us "

app/__main__.py is what Python runs for python -m app, and ENTRYPOINT makes that the container's command, so anything after the image name, like " About Us ", is passed to it. .dockerignore keeps the Git history, the tests and any virtual environments out of the image: they're not needed to run it, and the less that goes in, the less there is to leak or patch.

'  About Us ' is counted as 'about us'

One detail from the build's own output is worth spotting: it pulled python:3.13-slim@sha256:d98cc29d3266…. That's the same digest Jenkins' Docker engine pulled in Lesson 3. Two different Docker engines asked for the same tag on the same day and got exactly the same image, and the digest is how you can tell.

That's a local build, for checking the Dockerfile works. From here on, the pipelines build the images that count.

In GitHub Actions: build and push to GHCR

GitHub Container Registry, or GHCR, at ghcr.io, comes with every GitHub account. A workflow can push to it using the token GitHub gives every run, so there's no password to set up. Replace the build job at the end of .github/workflows/ci.yml:

python3 - <<'EOF'
from pathlib import Path
p = Path(".github/workflows/ci.yml")
s = p.read_text()
start = s.index("  build:\n")
s = s[:start] + """  build:
    needs: [lint, test]
    runs-on: ubuntu-24.04
    permissions:
      contents: read
      packages: write
    steps:
      - uses: actions/checkout@v7
      - uses: docker/setup-buildx-action@v4
      - name: Log in to GHCR
        if: github.event_name != 'pull_request'
        uses: docker/login-action@v4
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      - name: Work out the tags
        id: meta
        uses: docker/metadata-action@v6
        with:
          images: ghcr.io/${{ github.repository }}
          tags: |
            type=raw,value=latest,enable={{is_default_branch}}
            type=sha,format=short
      - uses: docker/build-push-action@v7
        with:
          context: .
          push: ${{ github.event_name != 'pull_request' }}
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha
          cache-to: type=gha,mode=max
"""
p.write_text(s)
EOF
tail -5 .github/workflows/ci.yml

The build job is the last one in the file, so the script replaces everything from build: to the end. The new job, which mirrors CivicGrid's:

• needs: [lint, test] is the gate from Lesson 3, unchanged. No image is built from code that failed.
• permissions: packages: write lets this one job's token push to GHCR. The rest of the workflow stays read-only, the least-privilege idea Lesson 5 is about.
• docker/setup-buildx-action sets up Buildx, Docker's newer build engine, which the build step and its cache need.
• docker/login-action signs in to ghcr.io with GITHUB_TOKEN, a token GitHub creates for this run and throws away when it ends. The if: skips the login on pull requests, which never push.
• docker/metadata-action works out the tags and labels. Here: sha- plus the short commit ID on every build, and latest only on the default branch. It also adds standard labels, including the repository's address, which is how GHCR knows which repository the image belongs to.
• docker/build-push-action builds the image and pushes it, but only when the event isn't a pull request, so an unmerged branch can never reach the registry. cache-from and cache-to with type=gha keep Docker's layers in GitHub's cache between runs, so later builds skip any step whose inputs didn't change.

Commit everything and push:

git add -A
git status --short
git commit -m "Build and push an image to GHCR"
git push

Wait until gh run list shows a tick rather than *, which takes about a minute, then try running the image GitHub built:

gh run list --limit 1
docker run --rm ghcr.io/kwamib/cicd-sandbox:latest "  Pipeline "

Use your own GitHub username in place of kwamib, in lowercase: image names must be lowercase, which is why metadata-action lowercases Kwamib/cicd-sandbox for you. If you try before the run has finished, the registry answers denied, which it says both for an image that doesn't exist yet and for one you're not allowed to see. A private package also needs docker login ghcr.io first. On my Mac, once the run had finished, it said this instead:

docker: Error response from daemon: no matching manifest for linux/arm64/v8 in the manifest list entries: no match for platform in manifest: not found

Two kinds of computer

That error isn't about permissions: Docker reached the image and read its list of versions. It's about processors. The Docker track's Lesson 3 covered the two common families: amd64, the Intel and AMD chips in most servers, and arm64, the chips in Apple's M-series Macs and a growing number of cloud servers. GitHub's standard runners are amd64; Lesson 2's logs said x64, which is the same thing. My Mac is arm64, so Docker asked for the arm64 version of the image, and there wasn't one. To see what's behind a tag:

docker buildx imagetools inspect ghcr.io/kwamib/cicd-sandbox:latest

Name:      ghcr.io/kwamib/cicd-sandbox:latest
MediaType: application/vnd.oci.image.index.v1+json
Digest:    sha256:ccc282ab3c11da1fa03ec136834f04c32c2a5637618a1743dffd7e674ab714d0

Manifests:
  Name:        ghcr.io/kwamib/cicd-sandbox:latest@sha256:ffa6aa927a5b8177fe2e05a2e9a6000383933e7a1db24ff0e0ca11ce727447fc
  MediaType:   application/vnd.oci.image.manifest.v1+json
  Platform:    linux/amd64

  Name:        ghcr.io/kwamib/cicd-sandbox:latest@sha256:c5067fe0f83ba11ff29424daf2f133c2b257d95223ed2eb964424ffa2d8d77bc
  MediaType:   application/vnd.oci.image.manifest.v1+json
  Platform:    unknown/unknown
  Annotations:
    vnd.docker.reference.digest: sha256:ffa6aa927a5b8177fe2e05a2e9a6000383933e7a1db24ff0e0ca11ce727447fc
    vnd.docker.reference.type:   attestation-manifest

The tag points at an index: a list of images, one per kind of processor, with its own digest. This one has a single real image, linux/amd64. The second entry, unknown/unknown, isn't an image at all: it's an attestation, a record that build-push-action adds by default, saying how and where the image was built. Lesson 7 comes back to why that's useful.

Docker Desktop can still run it, by pretending to be an amd64 computer, which is called emulation. It's slower, but fine for a check like this:

docker run --rm --platform linux/amd64 ghcr.io/kwamib/cicd-sandbox:latest "  Pipeline "

latest: Pulling from kwamib/cicd-sandbox
Digest: sha256:ccc282ab3c11da1fa03ec136834f04c32c2a5637618a1743dffd7e674ab714d0
Status: Downloaded newer image for ghcr.io/kwamib/cicd-sandbox:latest
'  Pipeline ' is counted as 'pipeline'

Notice the digest: ccc282ab…, the index's, the same one imagetools showed. The image GitHub built, tested behind its gate and pushed is exactly the one that ran here.

The lasting fix is to build for the processors your image will run on. build-push-action takes a platforms: setting, like linux/amd64,linux/arm64, and builds one image for each under the same tag. Building arm64 on an amd64 runner uses emulation too, set up with docker/setup-qemu-action, and it's slow; GitHub also offers arm64 runners, which build it natively. So it's a real trade-off, and CivicGrid's ci.yml makes it on purpose:

# amd64 only: the cluster's node is amd64, and the arm64 build doubled build time.
platforms: linux/amd64

Build for where the image runs. If that's only amd64 servers, one platform is the right answer, and --platform linux/amd64 covers the odd check on a Mac. If people pull it onto all kinds of laptops, build both.

In Jenkins: build, tag and smoke-test

Jenkins can build images now, thanks to its Docker engine from Lesson 3. Pushing them to GHCR needs a stored credential, which is Lesson 5, so for now Jenkins builds the image, tags it, and checks that it actually runs. Replace the Jenkinsfile's Build stage:

python3 - <<'EOF'
from pathlib import Path
p = Path("Jenkinsfile")
s = p.read_text()
start = s.index("    stage('Build') {")
end = s.index("\n  }\n}", start)
s = s[:start] + """    stage('Build') {
      steps {
        script {
          env.IMAGE_TAG = "1.0.${env.BUILD_NUMBER}-${env.GIT_COMMIT.take(7)}"
        }
        sh 'docker build -t cicd-sandbox:$IMAGE_TAG .'
        sh 'docker image ls cicd-sandbox'
        sh 'docker run --rm cicd-sandbox:$IMAGE_TAG "  Smoke Test "'
      }
    }""" + s[end:]
p.write_text(s)
EOF
grep -n -A10 "stage('Build')" Jenkinsfile
git commit -am "Jenkins builds, tags and smoke-tests the image"
git push

• env.IMAGE_TAG builds a tag like 1.0.4-2be56ac from the Jenkins build number and the first seven characters of the commit ID. Jenkins sets GIT_COMMIT when it checks out the code. Setting it on env makes it available to the sh steps that follow.
• 'single quotes' with $IMAGE_TAG means Jenkins passes the text through as it is and the shell fills in the value. For a tag that makes no difference, but it's the habit that matters most for secrets in Lesson 5, so it's worth starting here.
• docker run --rm ... " Smoke Test " is a smoke test: the smallest possible check that the image starts and does its one job. If the Dockerfile were broken, the build would still succeed, and this step would fail. --rm removes the container afterwards.

Then Build Now on the sandbox job.

The Build stage from mine, build #3, with Docker's step-by-step output shortened:

[Pipeline] { (Build)
+ docker build -t cicd-sandbox:1.0.3-34d9496 .
#5 [1/3] FROM docker.io/library/python:3.13-slim@sha256:d98cc29d3266e535dd9da1099688d9cb3ca4753813571e071763d17725460962
#6 [2/3] WORKDIR /srv
#7 [3/3] COPY app/ app/
#8 naming to docker.io/library/cicd-sandbox:1.0.3-34d9496 done
+ docker image ls cicd-sandbox
IMAGE                        ID             DISK USAGE   CONTENT SIZE   EXTRA
cicd-sandbox:1.0.3-34d9496   d1e675117c6c        202MB         43.3MB
+ docker run --rm cicd-sandbox:1.0.3-34d9496   Smoke Test 
'  Smoke Test ' is counted as 'smoke test'
Finished: SUCCESS

• 1.0.3-34d9496 is build 3, from commit 34d9496, the one you just pushed. Anyone looking at that tag can find both the Jenkins build and the exact code.
• The same base image again: python:3.13-slim@sha256:d98cc29d…, the digest your laptop pulled earlier.
• DISK USAGE and CONTENT SIZE are the image unpacked and ready to run, and its compressed size, which is what actually travels to and from a registry.
• The smoke test passed: the container started and did its one job. In the + line, the shell trace leaves out the quotes, but the output shows the spaces were passed through as intended.

One more difference from GitHub's image: this one was built on your laptop's Docker engine, inside jenkins-docker, so it's linux/arm64. The same commit produced an amd64 image on GitHub and an arm64 one here, which is exactly why the platform matters when you pick where your pipeline builds.

environment {
  VERSION = "1.0.${BUILD_NUMBER}-${env.GIT_COMMIT?.take(7) ?: 'local'}"
}

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 123456789012.dkr.ecr.us-east-1.amazonaws.com
docker tag clixx-image:${VERSION} 123456789012.dkr.ecr.us-east-1.amazonaws.com/clixx-image:${VERSION}
docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/clixx-image:${VERSION}

Spot the problem

The Friday rebuild. A team builds the image once for testing and, after it passes, builds it again from the same commit for production. A week later, production behaves differently from test, with the same code. How? The two builds weren't the same image. Between them, the base image python:3.13-slim got a new release, and the second build picked it up. Building once and promoting the same image, by its commit tag or digest, means test and production really run the same thing.

Who changed production? A production server runs app:latest. On Monday morning it's running code that nobody deployed, and the change log says nothing happened over the weekend. What did happen? A merge on Friday moved latest, and the server restarted on Sunday, for a host update, and pulled the new latest. Nothing was deployed, and everything changed. Deploying by commit tag would have kept it on the version someone chose.

The image with everything in it. A team's image is 900 megabytes, and a security scan finds an old SSH key inside it. Nobody added a key to the Dockerfile. Where did it come from? COPY . . copied the whole folder, including a key someone had left in it, a .git folder with the full history, and test data. A .dockerignore, and copying only the folders the app needs, as the sandbox's Dockerfile does, keeps images small and keeps leftovers out.

Where you are now

Your pipelines build real images now. GitHub builds behind the test gate, tags with the commit and latest, and pushes to GHCR, but never from a pull request. Jenkins builds with a version tag like 1.0.4-2be56ac and smoke-tests the image before anything else happens to it. You've seen why deploys should pin to a commit rather than trust latest, and why an image built on GitHub may not match your laptop's processor. Lesson 5 is about the secrets a pipeline needs, starting with the one Jenkins needs to push its images: a token, kept out of the Jenkinsfile and out of the logs.

Check yourself

Key terms

• Image: a packaged app with everything it needs to run, built from a Dockerfile.
• Registry: a service that stores images, like GHCR, Docker Hub or Amazon ECR.
• GHCR (GitHub Container Registry): GitHub's registry, at ghcr.io.
• Tag: a label on an image in a registry, which can be moved.
• Commit tag: a tag made from a commit ID, like sha-664d196, used for deploys.
• latest: the tag Docker uses when none is given; in most pipelines, it moves on every merge.
• Digest: the sha256: fingerprint of an image's exact contents, which can't move.
• Buildx: Docker's newer build engine, used for caching and building for several processors.
• Smoke test: a quick check that something starts and does its basic job.
• amd64 and arm64: the two common processor families; Intel and AMD chips, and Apple and many cloud chips.

Go deeper

• The docker/metadata-action README (opens in a new tab), which lists every kind of tag it can make, including the version tags from Git tags that CivicGrid uses.
• GitHub's guide to publishing Docker images (opens in a new tab), covering GHCR and Docker Hub.
• The Docker track's Images and registries, for tags, digests and images that run on more than one kind of processor.