Resources Local LLMs Local LLMs with Ollama

Local LLMs with Ollama · Lesson 8 of 13 · about 30 minutes

Checking its answers: catching made-up ones

Two checks between a model's answer and your data: one in plain code, one asking the model a second question. The code check caught every made-up name. The second look said no to everything on a 3B model, which looked perfect until we counted the right answers it threw away. On 7B it caught every wrong person. How to test a check, and why the last check is always a person.

By the end you'll know

  • How to check that a name really appears on the page, in a few lines of code
  • What a second look is: asking the model a separate yes-or-no question
  • Why a check has to be scored on right answers as well as wrong ones
  • Why the same check can be useless on one model and useful on another
  • How to layer checks so the dangerous mistakes reach a person

Where you are

Lessons 5 and 7 kept producing the same dangerous answer. On the Juniper Flats page, Tom Haskell is the mayor and Rosa Delgado is the mayor-elect, who hasn't taken office. Twice, a prompt answered Rosa Delgado: a confident, wrong, real person. Nothing about that answer looks wrong. It's a name, it's on the page, it's in the right shape.

Better prompts reduced these mistakes but never removed them. So this lesson stops trying to make the model never wrong, and starts catching it when it is. Every answer goes through checks before anyone trusts it, and ends up with one of four verdicts:

  • ACCEPT: passed every check.
  • REVIEW: something looks off; a person decides.
  • REJECT: clearly wrong; thrown away.
  • NONE: the model found no mayor; a person checks that too.

Check 1: is it on the page?

The simplest check needs no model at all. If the model was asked to read a name off a page, the name has to be on the page. A made-up name, like the "Emily Mitchell" that Lesson 2's model invented for Maple Falls, won't be.

checks.py in the practice set checks that every word of the name appears in the page text, ignoring case:

PYTHON
def on_page(name, text):
    """True if every word of the name appears in the page, ignoring case."""
    words = re.findall(r"[A-Za-z']+", name)
    return bool(words) and all(
        re.search(rf"\b{re.escape(w)}\b", text, re.IGNORECASE) for w in words
    )

Checking each word, rather than the exact name, matters. Willow Bend's page says Robert "Bob" Chen; checking for the exact string "Robert Chen" would wrongly reject the right answer. Checking that "Robert" and "Chen" both appear lets it through. Every check has to be fair to right answers as well as strict with wrong ones; that comes up again below.

Check 2: a second look

The on-page check can't catch Rosa Delgado. She is on the page; she's just not the mayor. Code can't easily tell "elected mayor and will be sworn in" from "continues to serve". So the second check asks the model a different, narrower question:

TEXT
Read this web page. According to the page, is {name} the person currently serving as mayor or acting mayor right now?
Answer with only yes or no.

It's a second look: the first prompt asked "who?", this one asks "is it this person?" A yes-or-no question about one name is a different task from finding a name, and the hope is that it catches what the first got wrong.

Put together, the checks run in order, cheapest first:

PYTHON
def check(name, text):
    """Return (on_page, second_look, verdict). Second look runs only if on the page."""
    if name == "NONE":
        return None, None, "NONE: send to a person"
    if not on_page(name, text):
        return False, None, "REJECT: not on the page"
    if not second_look(name, text):
        return True, False, "REVIEW: second look said no"
    return True, True, "ACCEPT"

Testing the checks

A check is code, and code needs testing. checks.py has a decoys mode that feeds the checks names where the right verdict is already known. For each page: the right mayor, a real person from the page who isn't the mayor (a former mayor, the one who resigned, the council president, the mayor-elect, another town's mayor), and the made-up "Emily Mitchell". Pull the latest practice set and run it:

BASH
python3 checks.py decoys
OUTPUT
town           kind        name             verdict
Cedar Bend     right       Dana Whitfield   REVIEW: second look said no
Cedar Bend     wrong       Harold Greene    REVIEW: second look said no
Cedar Bend     made up     Emily Mitchell   REJECT: not on the page
Elm Ridge      right       Carla Mendes     REVIEW: second look said no
Elm Ridge      wrong       Raymond Ellis    REVIEW: second look said no
Elm Ridge      made up     Emily Mitchell   REJECT: not on the page
Alder Creek    right       Grace Okafor     REVIEW: second look said no
Alder Creek    wrong       Diego Ramirez    REVIEW: second look said no
Alder Creek    made up     Emily Mitchell   REJECT: not on the page
Birch Hollow   wrong       Margaret Coyle   REVIEW: second look said no
Birch Hollow   made up     Emily Mitchell   REJECT: not on the page
Fox Run        right       Lena Park        REVIEW: second look said no
Fox Run        made up     Emily Mitchell   REJECT: not on the page
Juniper Flats  right       Tom Haskell      REVIEW: second look said no
Juniper Flats  wrong       Rosa Delgado     REVIEW: second look said no
Juniper Flats  made up     Emily Mitchell   REJECT: not on the page
Heron Point    right       Samuel Adeyemi   REVIEW: second look said no
Heron Point    made up     Emily Mitchell   REJECT: not on the page
Quarry Hill    made up     Emily Mitchell   REJECT: not on the page
Willow Bend    right       Robert Chen      REVIEW: second look said no
Willow Bend    made up     Emily Mitchell   REJECT: not on the page
Stone Mill     right       Ana Silva        REVIEW: second look said no
Stone Mill     wrong       Dana Whitfield   REVIEW: second look said no
Stone Mill     made up     Emily Mitchell   REJECT: not on the page

The on-page check is perfect. "Emily Mitchell" was rejected on all ten pages, and every real name got past it, as it should.

The second look said no to everything. Rosa Delgado went to review, as she should. So did Harold Greene, Raymond Ellis and every other wrong person. But so did every right mayor: Dana Whitfield, Carla Mendes, Tom Haskell, all of them.

A check that always says no

Look only at the wrong names, and the second look scored 6 out of 6: it caught every one. That's the trap. A check that rejects everything catches every wrong answer, and also rejects every right one. It isn't checking anything; it's just saying no. Every answer goes to a person, so the check saves no work at all.

A check has to be scored on both columns: how many right answers it lets through, and how many wrong ones it catches. Either number alone can look perfect.

Before blaming the model, it was worth making sure the bug wasn't mine. My code treats any reply that doesn't start with "yes" as a no; a reply like "Based on the page, yes" would count as no. The raw mode prints the exact replies:

BASH
python3 checks.py raw
OUTPUT
pages/cedar-bend.html      Dana Whitfield   'No'
pages/cedar-bend.html      Harold Greene    'No'
pages/juniper-flats.html   Tom Haskell      'No'
pages/juniper-flats.html   Rosa Delgado     'No'

A plain "No", every time. The model really said it.

So the looks mode tries four versions of the second look on all 14 real people: the plain question; the same question with the practice pages' footer removed (every page ends "This town is made up… and does not exist", which I suspected was confusing it); a stricter wording modeled on the one my real CivicGrid check uses, answering in JSON; and that wording without the footer:

BASH
python3 checks.py looks
OUTPUT
== plain: right names passed 0 of 8, wrong names caught 6 of 6
== no-footer: right names passed 0 of 8, wrong names caught 6 of 6
== verifier-style: right names passed 0 of 8, wrong names caught 6 of 6
== verifier-style+no-footer: right names passed 0 of 8, wrong names caught 6 of 6

(Each version also listed the eight right names it rejected; they're left out here.) No wording helped. On a 3B model, this second look doesn't work.

The same check on 7B

checks.py can use a different model for one run. Pulling the 7B model, running the same four versions, then removing it again:

BASH
ollama pull qwen2.5:7b
LLM_MODEL=qwen2.5:7b python3 checks.py looks
ollama rm qwen2.5:7b
OUTPUT
== plain: right names passed 5 of 8, wrong names caught 6 of 6
   Alder Creek right Grace Okafor: said no
   Fox Run right Lena Park: said no
   Juniper Flats right Tom Haskell: said no

== no-footer: right names passed 6 of 8, wrong names caught 6 of 6
   Alder Creek right Grace Okafor: said no
   Juniper Flats right Tom Haskell: said no

== verifier-style: right names passed 4 of 8, wrong names caught 6 of 6
   Elm Ridge right Carla Mendes: said no
   Alder Creek right Grace Okafor: said no
   Fox Run right Lena Park: said no
   Juniper Flats right Tom Haskell: said no

== verifier-style+no-footer: right names passed 5 of 8, wrong names caught 6 of 6
   Elm Ridge right Carla Mendes: said no
   Alder Creek right Grace Okafor: said no
   Juniper Flats right Tom Haskell: said no

A different story. On 7B, every version still caught all six wrong people, Rosa Delgado included, and now passed most of the right ones, up to 6 of 8. When it was wrong, it was always wrong in the same direction: it sent a right name to review, never accepted a wrong one.

Three details are worth noticing:

  • The footer did matter, a little. Removing "This town is made up… and does not exist" passed one more right name in both wordings. Text that has nothing to do with the question can still sway the answer. One name is a small sample, so treat it as a hint, not a rule.
  • The hard cases are hard for a reason. Grace Okafor is a name in one table cell and "Mayor" in another. Tom Haskell's page is mostly about the person replacing him. Those are the pages where a careful person would slow down too.
  • The stricter wording is stricter. It rejected Carla Mendes, the acting mayor, even though this version said acting mayors count. Strict instructions push the model further towards no.

Score every combination yourself:

Try it: score the second look

model
question

Right names passed 0 of 8. Wrong names caught 6 of 6. Every wrong name caught, and every right name rejected too. It said no to everything, so it tells you nothing.

The right mayor: should pass

  • Cedar BendDana Whitfieldsent to review
  • Elm RidgeCarla Mendessent to review
  • Alder CreekGrace Okaforsent to review
  • Fox RunLena Parksent to review
  • Juniper FlatsTom Haskellsent to review
  • Heron PointSamuel Adeyemisent to review
  • Willow BendRobert Chensent to review
  • Stone MillAna Silvasent to review

Someone else on the page: should be caught

  • Cedar BendHarold Greene, former mayorcaught
  • Elm RidgeRaymond Ellis, resignedcaught
  • Alder CreekDiego Ramirez, council presidentcaught
  • Birch HollowMargaret Coyle, select board chaircaught
  • Juniper FlatsRosa Delgado, mayor-electcaught
  • Stone MillDana Whitfield, another town’s mayorcaught

What a good set of checks looks like

Put the results together and the design almost writes itself:

  1. Cheap code checks first. The on-page check costs nothing, never sees a model, and removed every invented name. Anything that fails it is rejected outright.
  2. A model check next, if the model can do it. On 7B, the second look caught every real-but-wrong person. On 3B, it couldn't tell right from wrong at all. Whether a model check works is a measurement, not an assumption.
  3. A person last. Every REVIEW and every NONE goes to someone who reads the page. The checks don't replace that person; they decide which answers the person needs to see.

And choose which way each check should lean. Here, two mistakes are possible, and they don't cost the same:

  • Sending a right name to review costs a person a minute.
  • Accepting a wrong name puts a false fact into your data, where nothing will flag it again.

So a check that leans towards "no", like the second look on 7B, is the right kind for this job. One that leaned towards "yes" would look faster, and quietly let Rosa Delgado through.

In my real setup

CivicGrid's weekly mayor check uses exactly these layers. A code check, which the script calls its verbatim guard, rejects any name that doesn't appear on the page. A second look, on qwen2.5:7b, asks whether that person is the current holder of the office, with a JSON yes-or-no answer. And there's a third check in plain code that this lesson didn't build: it rejects a name when words like "former", "vice", "pro tem" or "acting" sit right in front of it.

Above all of them, the check never changes a leader's name itself. A different name becomes a proposal, and I approve or reject each one on an admin page. Nothing goes into CivicGrid without a person reading it.

This lesson also told me something I didn't know. I had never tested whether that second look actually works. On these pages, at 7B, it does: it caught every wrong person and sent a few right ones to me for review, which is the right way round. On 3B it would have been worthless. That's a measured reason to keep the bigger model, and a test I'll keep, so the next model change has to pass it too.

Spot the problem

Three situations. Work out what's wrong before you read the explanation.

A perfect catch rate. A team adds a model check and reports that it "caught 100% of bad answers" in testing. Six weeks later, reviewers are drowning: every answer is being sent to them. What did the test miss? It only counted the bad answers. A check that rejects everything catches 100% of them too. Scoring the right answers it lets through would have shown it.

Exact-match checking. A script rejects any extracted name that doesn't appear on the page exactly as written. It starts rejecting correct answers like "Robert Chen" from a page that says Robert "Bob" Chen. What's the fix? Check each word of the name, not the exact string, or compare names more loosely. A check has to be fair to right answers, not only strict with wrong ones.

Trusting the second look. A team switches their extraction from a 7B model to a 3B model to save memory, keeping the same second-look check. Everything still runs, and nothing errors. What's the risk? The check may no longer work. In this lesson, the same second look caught every wrong person on 7B and said no to everything on 3B. Any change of model means re-testing the checks, not just the extraction.

Where you are now

Checks sit between a model's answer and your data. A code check that the name is on the page caught every made-up name, at no cost. A second look, a separate yes-or-no question, is the only check here that could catch a real but wrong person like Rosa Delgado. On 3B it said no to everything, which looked perfect until the right answers were counted; on 7B it caught every wrong person and passed most right ones. So score every check on both columns, re-test checks when anything changes, lean them towards review, and let a person make the final call. Lesson 9 asks what happens when the page itself is trying to fool the model. Lesson 10 turns all these test runs into a proper test suite: one command that tells you whether a change to a prompt, a model or Ollama itself made things better or worse.

Check yourself

  1. What does the on-page check catch?

  2. Why check each word of the name, not the exact name?

  3. The 3B second look caught 6 of 6 wrong names. Why is that not good?

  4. What did running the same check on 7B show?

  5. Why should a mayor check lean towards "send to review"?

  6. A team swaps to a smaller model. What must they re-test?

Key terms

  • Check: a test an answer must pass before it's trusted.
  • On-page check: a code check that the answer actually appears in the source.
  • Second look: a separate question to the model about one specific answer, like "is this person the current mayor?"
  • Decoy: a test input where the right verdict is already known, used to test a check.
  • Accept, review, reject: what a check does with an answer: trust it, send it to a person, or throw it away.
  • False alarm: a right answer a check wrongly flags. Cheap, if a person reviews it.
  • Miss: a wrong answer a check wrongly accepts. Costly, because nothing flags it again.

Go deeper

From www.kwamib.dev/resources/local-llms/checking-its-answers