If an AI speaks in your name, test it like a product. Reading a few answers and trusting your gut is not enough, because AI is very good at text that sounds right. I learned this on my own website, where a chat has been answering in my name for months. The test caught the AI failing in four different ways. I expected to find problems with the AI, but not that the test would catch one in me.
I think this applies to any organization that lets a system speak for it. My chat answers around the clock, without me in the room. I had read a handful of answers. They sounded right. Whether they were any good, I had no idea. So I stopped sampling vibes and started measuring.
Reading revealed the failure patterns
It is tempting to start with a metric, but I would resist that. I started the way Hamel Husain teaches it. He calls it an eval: ask real questions, read every answer and label it by hand. Only then do you decide what "bad" even means.
So I sat down with the questions a curious visitor actually asks. What does Adrian do for a living? Does he have experience with GenAI in enterprises? What does he believe about AI adoption? I read the answers slowly, the way a stranger would. I was not skimming for facts I knew were in there, but reading for what each answer actually conveys.
It was sobering. The failures fell into patterns I would never have written checks for in advance.
The chat reliably answered German questions in English. A visitor asks "Was macht Adrian eigentlich beruflich?" and gets a polished English paragraph back. The cause was an integration mistake, not a lack of intelligence. At the end of the prompt, the set of instructions sent to the AI, the website's language setting overrode the visitor's language. It was a single line in the wrong place. So evals find not only quality problems but also integration bugs that no amount of prompt tuning would fix.
It dumped lists where the question asked for a story. "Does he have experience with X?" came back as a bulleted inventory of activities, technically correct and completely lifeless. Nobody asks that question because they want an inventory. They want to know what happened, what was hard and what changed.
It was generic. Some answers could have described, word for word, any AI lead at any German company. I now call this the name test. If you can swap in a different name and the answer still works, the answer says nothing.
It mixed up the perspective. Questions about Adrian sometimes came back answered in the first person. It is a small thing, but it reads very strangely. An AI that loses track of who is speaking does not inspire much confidence in anything else it says.
Every rule has to say when it does not apply
The patterns became four failure modes, which amount to four questions for every answer. Is the chat using the right language? Does it tell experience as a story? Does it give concrete details? Does it speak from the right perspective? Each failure mode got a strict definition.
Deciding when each rule applies took longer than defining it. Every rule needs an explicit answer to one question: when does it NOT apply? "Tell me about his experience" deserves a story, while "How many people has he trained?" deserves a number. Punish the number for not being a story, and you only train the system to produce filler. Without scopes, an automatic judge raises false alarms until you stop trusting it, and then you are back on gut feel.
The vaguest category was "generic". I broke it into three yes-no checks: the name test, a check for differentiating detail, and buzzword density. For the second check, numbers or job titles alone do not count, but concrete decisions and situations do. Vague criteria give you a judge you can't challenge. Clear yes-or-no criteria let you question its verdict.
The judge has to reproduce my verdicts first
Then I automated the grading, with AI as the judge. But trusting a judge you have not tested is still gut feel, just with extra steps.
So the judge had to earn the job. I hand-labelled a golden set: question and answer pairs with clean hits, deliberate failures and edge cases. For each pair I recorded my own verdict on every failure mode. Before the judge could score new answers, it had to reproduce my labels. The first version got four out of five. The judge and I argued about the fifth, and the argument was useful, because it exposed a sloppy definition in my own criteria.
Only once the judge agreed with me on answers I had already labelled did it get to score answers I had not seen.
The fix was in the knowledge base
Once the pipeline was running, a pattern showed up that I would not have predicted. Narrow factual questions passed, broad questions failed. The model, the prompt and the knowledge base were the same for both. The knowledge base is the file the chat draws its answers from.
The cause was the knowledge base itself. I had written it like a database: dense, factual and complete. Every fact was in there, but not how the facts connect. Why did one thing lead to another? What was at stake? What would have happened otherwise? A model that builds answers from isolated facts gives you exactly that back: isolated facts.
So I rewrote the knowledge base story by story. Each got the same skeleton of situation, tension and outcome, plus one detail specific enough to make the name test swing the right way. The eval results told me which story to fix first and whether each revision improved the answers.
The test caught a mistake of mine too
Then came the part I did not see coming. While labelling, I reread one of the source stories. I realised I had written it the way I wished it had gone. It was smoother than reality, with a conflict resolved a little too cleanly. The AI had faithfully repeated my own embellishment. It can write your story, but it cannot know which version of it is true. I corrected the source.
The last two failures had nothing to do with knowledge
After the rewrite, most questions passed reliably. Two kept alternating between passing and failing, with nothing changed in between.
The judge's reasoning showed why. The facts were all there, but the answer was put together wrong. On certain broad question types, the model fell back into listing, even though it had stories to tell. The cause was a gap in the prompt. My rule "answer as a story, not as a CV" listed the phrasings that should trigger it. Those two question types matched none of them, so the rule never fired for them.
This distinction holds well beyond my small website. What a system knows and how it composes an answer are two different layers. They fail in different ways, and the evals' reasoning gives you a hint as to which layer is at fault. Without it, you rewrite content when you should be fixing the prompt, or the other way around. Then everything takes four times as long.
I am responsible for every sentence the chat writes
I do this because of the principle, not because of the website. I believe AI should work autonomously but never be a black box. That is one of three convictions on this site. A chat that answers in my name without being checked is exactly that kind of black box. I am responsible for every sentence it produces, whether I read it or not. Testing it is the price of letting it speak for me.
And honestly, the bar out there is very low. Most chatbots go live after someone asked three questions in a demo and nodded.
Bottom line
The method that fixed my website chat is the same one I practise on production systems at work. It is not hard: read real answers and label them by hand. Define failure precisely, including the cases where a rule does not apply. Make the judge earn your trust before you trust it. Fix the layer the evidence points at.
The goal was never a perfect chatbot. I wanted to know exactly how my system fails before a stranger finds out for me. To me, that is the difference between autonomous and out of control.
When did someone last personally review your AI's answers? And do you know whether your judge can reproduce your own verdicts?
The Numbers
As in my last post, here are the numbers in full.
- Golden set: 8 hand-labelled entries, started with 5. Small, but I personally wrestled with every single one.
- Failure modes: 4, each with an explicit scope and yes-no checks.
- Judge agreement: 4/5 on the first version. After that I fixed the criteria, not the judge.
- Before: 5 of 8 questions passed. The weakest one failed on every single run.
- After: 8 of 8, three runs in a row, 24 out of 24 verdicts.
- Cost per full eval run: cents. The expensive part was hand-labelling, which took a few evenings. The judge runs for less than a coffee.
Rebuild it
This is the "generic" check as it stands in the judge prompt, minus some internal examples. If an AI speaks in your name, you can take it almost as is. Swap the name, adjust the comparison group, done.
F3: Generic
Definition: The answer stays at the level of generic claims and buzzwords instead of concrete, differentiating details that set Adrian apart from any other German AI leader.
Scope: applies to questions about experience, competence or concrete work. Does NOT apply to pure factual questions. If out of scope, return verdict "na".
Verdict determined by three checks. FAIL if 2 or more checks trigger.
Check A, Name Test: Could the answer be word-for-word identical if it were about any other German AI leader (e.g. someone at Otto, Telekom, Allianz, SAP)? If yes, Check A is triggered.
Check B, Differentiating Details Missing: Are there NO concrete, differentiating details that go beyond what a standard LinkedIn profile would say? Numbers alone and named roles alone do NOT count as differentiating. Stakeholder names alone do NOT count. Specific decisions, named constraints, concrete situations, conflicts or unusual choices DO count. If the answer has no differentiating detail, Check B is triggered.
Check C, Buzzword Density: Is the answer heavy with interchangeable management phrases ("strategisch", "nachhaltig", "ganzheitlich", "Change Management", "Transformation", "Hands-on") WITHOUT concrete examples behind them? If yes, Check C is triggered.
Pass: 0 or 1 checks triggered.
Fail: 2 or 3 checks triggered.