Resources Networking and DNS Networking basics

Networking basics · Lesson 1 of 12 · about 20 minutes

What happens when you visit a website

The whole journey of one request, from typing an address to seeing a page, in six steps you can watch for yourself with curl. It's the map for the rest of the track.

By the end you'll know

  • The six steps every web request goes through, in order
  • How to watch them happen with curl -v
  • How to read curl's output: what was sent, what came back, and what curl noticed
  • What a redirect looks like
  • Which step failed, from the error you get

The problem: a lot happens in under a second

You type an address, press Enter, and a page appears, usually in less than a second. It feels like one thing. It's actually half a dozen separate steps, each handled by a different part of the internet, and each one can fail on its own.

That's why networking problems feel mysterious: "the site's down" could mean the name couldn't be found, the server couldn't be reached, the security check failed, or the server answered with an error. Knowing the steps turns "it's broken" into "step three failed", and that's most of the way to fixing it.

This lesson walks through all six steps once, as a map. Every later lesson in the track zooms into one of them. And the whole track follows the same real request: to api.civicgrid.org, the API for CivicGrid, my civic-data product, which you can visit yourself.

The six steps

Ordering a takeaway by phone works the same way as visiting a website. You look up the restaurant's number, you call, you make sure it's really them that picked up, you place your order, they tell you it's on its way, and the food arrives. In networking terms:

  1. Look up the address (DNS). Computers don't find each other by name; they use numbers called IP addresses. So first, the name api.civicgrid.org is turned into an address, like a phone book turning a restaurant's name into its number. That's the DNS (Domain Name System), in Lesson 5.
  2. Connect. Your computer opens a connection to that address, on a numbered door called a port. Secure websites use port 443. Addresses are Lessons 2 and 3, ports are Lesson 4, and how the connection finds its way across the internet is Lesson 6.
  3. Make it secure, and check who answered (TLS). Before anything private is sent, the two sides set up encryption, and the server proves it really is api.civicgrid.org, with a certificate. That's what the padlock in your browser means, and it's Lesson 9.
  4. Ask (HTTP request). Your computer sends its request: "please GET me the page at /". That's HTTP (Hypertext Transfer Protocol), in Lesson 4.
  5. Answer (HTTP response). The server replies with a status code, like 200 for OK, some details about the reply, called headers, and then the content itself. Python Lesson 6 met status codes from the other side.
  6. Show it. Your browser turns the content into the page you see, and usually repeats steps 1 to 5 for every image, font and script on it.

In between steps 2 and 5, there are often other machines your request passes through without you knowing, like firewalls (Lesson 8) and load balancers (Lesson 10). Inside a Kubernetes cluster there are even more hops, which is Lesson 11.

The six steps of a web requestSix boxes in a row. One, look up the address with DNS: api.civicgrid.org becomes an IP address, covered in Lesson 5. Two, connect to that address on port 443, Lessons 2 to 4 and 6. Three, secure the connection and check the certificate with TLS, Lesson 9. Four, send the HTTP request, GET slash, Lesson 4. Five, receive the response, status 200 and the content. Six, the browser shows the page.1. Look upname toIP addressDNS2. Connectto the address,on port 443TCP3. Secureencrypt, andcheck the certificateTLS4. AskGET /with headersHTTP5. Answerstatus 200,headers, contentHTTP6. Showthe browserdraws the pageLesson 5Lessons 2 to 4, 6Lesson 9Lesson 4Lesson 4Six steps, every time. Each one can fail on its own, and each lesson in this track zooms into one.
The map for the whole track. When something doesn't load, the question is always: which of these six steps went wrong?

Watch it happen

You don't have to take any of this on trust. curl is a command-line tool for making web requests, and it's already installed on macOS, Linux and Windows. With -v, for verbose, it narrates every step as it goes. Run this:

BASH
curl -sv -o /dev/null https://api.civicgrid.org/
  • -s (silent) hides curl's progress bar, which isn't part of the story.
  • -v (verbose) prints every step.
  • -o /dev/null sends the content itself to /dev/null, a special place on macOS and Linux that throws away anything written to it. You're here to watch the steps, not read the page.

You'll get about 70 lines. Each starts with a symbol that tells you what it is:

  • * is curl telling you what it's doing or what it noticed.
  • > is what curl sent to the server.
  • < is what the server sent back.
  • { and } are data moving in and out, which you can skip.

Here are the parts that matter, from a real run. Your addresses and times may differ.

Step 1, look up the address:

OUTPUT
* Host api.civicgrid.org:443 was resolved.
* IPv6: (none)
* IPv4: 104.21.69.165, 172.67.210.139

The name became two IP addresses. A name can have several addresses, so there's a spare if one is busy. IPv4 and IPv6 are the two kinds of address, which Lesson 2 explains. :443 is the port curl will use, because the address starts with https.

Step 2, connect:

OUTPUT
*   Trying 104.21.69.165:443...

curl picks the first address and opens a connection to port 443.

Step 3, secure it and check who answered:

OUTPUT
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / X25519MLKEM768 / id-ecPublicKey
* Server certificate:
*  subject: CN=civicgrid.org
*  expire date: Dec  5 01:07:47 2026 GMT
*  subjectAltName: host "api.civicgrid.org" matched cert's "*.civicgrid.org"
*  issuer: C=US; O=Let's Encrypt; CN=YE2
*  SSL certificate verify ok.

Above these lines are a dozen TLS handshake lines: the two sides agreeing how to encrypt. Then the important part: the server's certificate, which says it's valid for *.civicgrid.org, meaning any name ending in civicgrid.org. Your request was for api.civicgrid.org, which matched. The certificate was issued by Let's Encrypt, a free certificate authority, and it expires in December. verify ok means curl trusts it. (You'll see "SSL" here too, the old name for what's now called TLS. People still use both.)

Step 4, ask:

OUTPUT
> GET / HTTP/2
> Host: api.civicgrid.org
> User-Agent: curl/8.14.1
> Accept: */*

The request itself. GET / means "give me the page at /", the top of the site. The other lines are request headers: which site you want, Host, what program is asking, and what kinds of answer it accepts.

Step 5, answer:

OUTPUT
< HTTP/2 200
< content-type: application/json
< content-length: 247
< server: cloudflare

200: OK. The content is JSON, the format from Python Lesson 5, and it's 247 bytes long. And server: cloudflare says that whatever answered isn't CivicGrid's own server, which is worth a closer look in a moment.

That's the whole journey, in well under a second.

A site that sends you somewhere else

Now try my website:

BASH
curl -sv -o /dev/null https://kwamib.dev

The first steps look the same: a name, an address, a connection and a certificate, this time for kwamib.dev. But the answer is different:

OUTPUT
< HTTP/2 307
< location: https://www.kwamib.dev/
< server: Vercel

307 is a redirect: "not here, try this other address". The location header says where: the same site, with www. in front. Your browser follows redirects without you noticing, which is why typing kwamib.dev ends up at www.kwamib.dev. curl doesn't, unless you ask with -L:

BASH
curl -sv -L -o /dev/null https://kwamib.dev

Now you'll see the whole journey twice: the redirect, then all six steps again for www.kwamib.dev, ending with a 200. server: Vercel is the company that hosts my site.

In my real setup

server: cloudflare is there because CivicGrid's API doesn't answer the internet directly. Cloudflare sits in front of it: your request reaches one of Cloudflare's servers first, which passes it on to CivicGrid's servers on DigitalOcean, then brings back the answer. That's also why those two IP addresses are Cloudflare's, not mine, which keeps the real servers' address private. Lesson 10 explains this kind of middleman, called a reverse proxy, and the capstone guide follows the request past Cloudflare, all the way to the code that answers it.

Break it on purpose

Each of these breaks a different step. Run it, and see which step the error points to.

Misspell the name. Add an extra g to the end:

BASH
curl -sv -o /dev/null https://api.civicgrid.orgg/
OUTPUT
* Could not resolve host: api.civicgrid.orgg
* shutting down connection #0

Two lines, and it's over. That's step 1: there's no such name, so there's no address to connect to, and nothing after it even started. In a browser, the same problem shows as a page saying the site can't be reached, or its address can't be found.

Connect to a door nobody's behind. Ask for port 12345 instead of 443, and give up after 5 seconds:

BASH
curl -sv --connect-timeout 5 -o /dev/null https://api.civicgrid.org:12345/
OUTPUT
* Host api.civicgrid.org:12345 was resolved.
* IPv6: (none)
* IPv4: 104.21.69.165, 172.67.210.139
*   Trying 104.21.69.165:12345...
* ipv4 connect timeout after 2499ms, move on!
*   Trying 172.67.210.139:12345...
* Connection timed out after 5002 milliseconds

Step 1 worked: the name resolved to the same two addresses as before. Then step 2 failed. curl tried the first address, got no answer, and moved on to the spare, which is exactly what a second address is for. That didn't answer either, and after 5 seconds, it gave up. The addresses are fine; nothing is listening on port 12345. --connect-timeout 5 set that 5-second limit. Without it, curl would wait much longer.

Ask an address for a name it doesn't have. Visit my site by its IP address instead of its name. Use the address your earlier curl -v https://kwamib.dev showed, after IPv4:. At the time of writing, it was 216.198.79.1:

BASH
curl -sv -o /dev/null https://216.198.79.1/

Among the output:

OUTPUT
*  subject: CN=no-sni.vercel-infra.com
*  subjectAltName does not match ipv4 address 216.198.79.1
* SSL: no alternative certificate subject name matches target ipv4 address '216.198.79.1'

There was no name to look up, and the connection opened, so steps 1 and 2 were fine. Step 3 failed. That address is Vercel's, and it hosts thousands of sites, including mine. Normally, your request says which site it wants as part of setting up the secure connection, and Vercel shows that site's certificate. With no name, Vercel had nothing to go on, so it showed a placeholder certificate, no-sni.vercel-infra.com, which doesn't match the address you asked for, and curl refused to go on. Certificates prove names, not addresses. Lesson 9 explains the part of TLS that passes the name along, called SNI (Server Name Indication).

Three errors, three different steps. When something won't load, that's the question to ask first: how far did it get?

Where you are now

You know the six steps every request goes through, and you can watch them with curl -v and tell which one failed. Starting with Lesson 2, the track takes them apart one at a time, beginning with the numbers that step 1 produced: IP addresses.

Check yourself

  1. What's the first thing that happens when you visit api.civicgrid.org?

  2. In curl -v output, what do the lines starting with > show?

  3. Why does curl check the server's certificate?

  4. kwamib.dev answered 307, with a location header. What does that mean?

  5. curl says it could not resolve the host. Which step failed?

  6. Why does the response from api.civicgrid.org say server: cloudflare?

Key terms

  • IP address: the number computers use to find each other, like 104.21.69.165.
  • DNS (Domain Name System): turns names like api.civicgrid.org into IP addresses.
  • Port: a numbered door on an address. Secure websites use 443.
  • TLS: sets up encryption and checks the server's certificate. Often still called SSL.
  • Certificate: proof that a server really is the name you asked for, issued by a trusted authority.
  • HTTP (Hypertext Transfer Protocol): the request and response format the web uses.
  • Status code: the three-digit result of a request, like 200 (OK) or 307 (redirect).
  • Header: a line of detail sent with a request or response, like Host or server.
  • curl: a command-line tool for making web requests. -v shows every step.

Go deeper

From www.kwamib.dev/resources/networking/what-happens-when-you-visit-a-site