When it doesn't connect: a troubleshooting checklist

By the end you'll know

The five stages every request passes through, in order
How to read curl -v as a progress bar that shows where a request stopped
What the common error messages mean, and which stage each points to
A short checklist of commands that finds the problem in order
The habits that make troubleshooting faster

Where you are

Over eleven lessons you've met a lot of failures, mostly on purpose: Could not resolve host, timeouts, Connection refused, NXDOMAIN, an expired certificate, a handshake failure, 404, 502, 503 and 504, and a NetworkPolicy dropping traffic. Each one was explained where it happened.

This lesson puts them all in one place, in order, so that when something doesn't connect for real, with no lesson telling you what broke, you know exactly where to start. It's the lesson to keep open in a tab.

A phone call that won't connect

When a phone call fails, the way it fails tells you a lot. If the phone says "number not recognised", you dialled wrong. If it just rings and rings, the line works but nobody's picking up. If someone answers but says "wrong number", you reached a real phone, just not the right one. If they answer but you can't understand each other, you got through, and the problem is the conversation. Each failure happens at a different stage, and you'd check different things for each.

Requests are the same. Every request to a website or API passes through the same five stages, always in the same order, and every error you'll ever see happens at one of them:

Name: the name becomes an address. DNS, from Lesson 5.
Reach: data gets across the networks to that address. Routing, NAT, VPCs and firewalls, from Lessons 6 to 8.
Port: a program is listening on the port. Lesson 4.
TLS: the encrypted connection is set up and the certificate checks out. Lesson 9.
HTTP: the request is answered, by the app or by a proxy in front of it. Lessons 4 and 10.

A request can't fail at stage 4 unless stages 1 to 3 worked. So the error you see tells you, for free, that everything before it is fine. That's the whole method: find the stage, ignore everything before it, and check that stage.

curl -v is a progress bar

You've used curl -v all track. Read top to bottom, its lines tick off the five stages one by one. Here are lines you've seen from api.civicgrid.org, in order, matched to their stage:

Stage	The line that says it worked	What failure looks like here
1. Name	`* IPv4: 104.21.69.165, 172.67.210.139`	`Could not resolve host`
2. Reach	`* Trying 104.21.69.165:443...` then a connected line	A long pause, then a timeout
3. Port	`* Connected to` the address and port	`Connection refused`, instantly
4. TLS	`* SSL certificate verify ok.`	`SSL certificate problem` or a handshake failure
5. HTTP	`< HTTP/2 200`	Any status that isn't what you expected

The exact wording varies a little between curl versions, but the order never changes. So the first thing to do with any broken request is run it with -v, find the last line that worked, and you know the stage. One line of caution on stages 2 and 3: a timeout and a refusal both come right after Trying. The difference is the speed: refused is instant, a timeout waits. That difference is worth more than any other single clue in this lesson.

Decode the message

Pick the message you're looking at. The widget shows how far the request got, what the message means, what to run next, and which lesson explains it:

The checklist

When you don't have a clear error, or you want to be thorough, work down this list in order and stop at the first step that fails. Each step has one command. The examples use api.civicgrid.org; put your own name, port and path in their place.

1. Does the name resolve?

BASH

dig +short api.civicgrid.org

You want one or more addresses. Nothing at all, or NXDOMAIN without +short, means the problem is DNS: the spelling, a missing record, or the wrong place (Lesson 5). If the address is one you don't expect, someone may have an old answer cached, or a line in /etc/hosts.

2. Can you reach that address, on that port?

BASH

nc -zv -w 3 api.civicgrid.org 443

succeeded means stages 2 and 3 both work: skip ahead. Connection refused means the machine is there but nothing listens on that port (Lesson 4). timed out means something dropped it: a firewall or security group most often (Lesson 8), or a private address that can't be reached from where you are (Lesson 6). For a timeout, traceroute -n shows how far packets get, remembering that stars don't always mean a problem.

3. Does TLS work?

BASH

curl -sv -o /dev/null https://api.civicgrid.org/ 2>&1 | grep -E "SSL|subject|expire"

You want SSL certificate verify ok. An expired certificate, a name mismatch or a handshake failure means stage 4 (Lesson 9). Check dates and names with openssl, and never "fix" it by turning verification off.

4. What does HTTP say?

BASH

curl -s -o /dev/null -w '%{http_code}\n' https://api.civicgrid.org/

200 means it worked. Anything else, read with Lesson 4's table and Lesson 10's gateway errors: 404 is the path, 401 and 403 are who you are, 502 and 503 are the servers behind a proxy, and 504 is something slow.

5. Inside a cluster, check the layers Kubernetes adds. If the request is between pods, the same stages apply, plus a few cluster pieces from Lesson 11, in this order:

The full Service name, name.namespace.svc.cluster.local, resolves from inside a pod.
kubectl get endpointslices shows pods behind the Service. None means nothing is ready, or the Service's selector doesn't match the pods' labels.
kubectl get networkpolicy shows nothing blocking the route, especially when the error is a timeout.

Run the checklist on a working site

It's worth running the checklist once on something that works, so you know what healthy looks like. Here it is against api.civicgrid.org:

OUTPUT

$ dig +short api.civicgrid.org
104.21.69.165
172.67.210.139

$ nc -zv -w 3 api.civicgrid.org 443
Connection to api.civicgrid.org port 443 [tcp/https] succeeded!

$ curl -sv -o /dev/null https://api.civicgrid.org/ 2>&1 | grep -E "SSL|subject|expire"
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / X25519MLKEM768 / id-ecPublicKey
*  subject: CN=civicgrid.org
*  expire date: Dec  5 01:07:47 2026 GMT
*  subjectAltName: host "api.civicgrid.org" matched cert's "*.civicgrid.org"
*  SSL certificate verify ok.

$ curl -s -o /dev/null -w '%{http_code}\n' https://api.civicgrid.org/
200

Every stage passes. Run the same four commands against a site of your own and compare. The next time something breaks, the first step whose output looks different from this is your stage.

The method in one picture. Find the stage from the error, trust everything to its left, and check that stage with its command.

Habits that save hours

The commands find the stage. These habits stop you going round in circles:

Read the whole error, slowly. Most errors say exactly what's wrong, and people skim past the useful word: refused, expired, NXDOMAIN.
Change one thing at a time. If you change the port, the firewall and the DNS record at once and it starts working, you don't know which fixed it, or whether the other two broke something else.
Compare with something that works. A working request next to a broken one, from the same machine, shows you the difference. That's why you ran the checklist on a healthy site.
Check from somewhere else. Your laptop, a server in the same network, a pod in the cluster. If it works from one place and not another, the problem is between them: a firewall rule, a DNS cache, a private address.
Write down what you tried. A few lines in a notes file, or in the ticket. It stops you repeating checks, and it's half of the write-up afterwards.

In my real setup

The Kubernetes track tells the story of a test I ran on CivicGrid's cluster: I tried reaching the API straight at the gateway's public address, skipping Cloudflare, and it connected. That's this checklist used the other way round, to prove something shouldn't work. After moving to a Cloudflare Tunnel, the gateway has no public address at all, and the same check can't connect, which is exactly what I wanted. Testing that the wrong path fails is as important as testing that the right one works.

Break it on purpose: three mysteries

No commands this time. Each mystery is a message from earlier in the track. Name the stage before you read the answer.

Mystery 1. A teammate says the server is down. They paste: curl: (6) Could not resolve host: api.civicgrid.og. Which stage? Stage 1, and the server is fine: curl never got as far as looking for it. Read the name closely: .og, not .org. Most "the server is down" reports at stage 1 are a typo or a missing DNS record.

Mystery 2. A new app server can't reach the database. nc -zv -w 3 10.0.20.15 5432 waits three seconds, then says timed out. The database is running, and other servers connect to it fine. Which stage, and what would you check? Stage 2: nothing answered, so it's not a missing program, which would have been refused instantly. With other servers fine, the likeliest cause is a firewall rule that allows the old servers but not this one, Lesson 8's "app that lost its database". Check the database's security group, or a NetworkPolicy if it's in a cluster.

Mystery 3. A site loads for you but your colleague gets 502 Bad Gateway. Same URL. Which stage? Stage 5: a proxy answered, so stages 1 to 4 all worked for both of you. A 502 is the server behind the proxy failing. Why only for them? Lesson 10's load balancer, with one backend down, failed every other request. You got a healthy server and they got the broken one. Look at the backends, not the proxy and not your colleague's laptop.

Where you are now

You've finished the Networking track. You can follow a request from a name to an answer, through every stage, and you have a method for finding where it stopped when it doesn't arrive: read the error, find the stage, trust everything before it, and check that stage with its one command.

Where next

Everything here is the ground floor for the rest of the Resources page:

The Kubernetes track puts Services, health checks and the Gateway API to work, and every one of them makes more sense now.
The Terraform track builds infrastructure as code; Lessons 7 and 8 here showed what a VPC and a security group look like in it.

And the best practice of all is your own projects: when something breaks, run the checklist before you search for the error.

Check yourself

curl fails with a certificate error. What does that tell you about the earlier stages?
Which error most strongly suggests a firewall?
nc -zv says succeeded, but the browser shows a certificate error. Where do you look?
Pods can't reach a Service, and kubectl get endpointslices shows no pods. What's likely?
You fix a problem by changing three settings at once. What's the risk?
A site works for you but times out for a colleague elsewhere. What's a good first step?

Key terms

The five stages: name, reach, port, TLS and HTTP. Every request passes them in order.
Timeout: nothing answered. Usually a firewall, a wrong address or a missing route.
Refused: the machine answered at once, with nothing listening on the port.
NXDOMAIN: the name doesn't exist in DNS.
Gateway errors (502, 503, 504): a proxy saying the servers behind it failed, were missing, or were too slow.
Baseline: the output of the checklist on something that works, to compare against.

Go deeper

Everything curl (opens in a new tab), the free book by curl's creator, for every option you'll ever need.
Julia Evans's zine Bite Size Networking (opens in a new tab), a one-sitting refresher of the tools in this lesson.