The notes said do not quote $29

"In the GPT by Friday." That was the whole request. The brief was already sitting in Slack: $49 a seat from 1 September, a fourteen-day trial, aimed at existing free-plan teams. Under it, in the same message, because Slack is where companies actually write things down: do not quote the $29 grandfather rate, ignore the competitor's 50% off, the 48-hour flash sale never shipped, and the 200-seat figure is an internal target. The implied build was a Custom GPT with those paragraphs pasted into its instructions. Someone types "write the email" and sends whatever comes back.

A year ago I would have called that an application and moved on. I also walked into this expecting that if the fluent draft leaked, a JSON schema would be the grown-up fix. It was not; the schema-valid draft carried $29 in an extras bag. The thing that actually changed my week was smaller and ruder, and I did not expect it at all. The first line of the notes, "Northwind launch notes, 18 August 2026", is the file's own timestamp, and it showed up in the outbound text as if it were a customer deadline.

So on 18 August 2026 I wrapped that one Northwind brief four ways. A fluent chat wrapper that copies every number the notes mentioned. An open JSON object with the required keys plus an extras bag. An allowlist that fills only the fields I would defend. And the same dirty JSON run through a claim gate. No model API anywhere: a regex reader pulled money, percents, hours, seats, days and dates out of whichever text was about to be sent, and compared them with a list I had written first. The brief, the gold list and the arithmetic are in the prototype repository.

That choice sets the edges of the piece. I did not run a frontier model on the brief, so nothing here ranks Claude against ChatGPT against Grok for copywriting, and every sentence about what a live chatbot would say is marked as a guess. Nor is any of it a legal or brand-safety policy, even though a leaked grandfather price is exactly the kind of thing those policies exist to catch.

Two of the four drafts carried the $29

The chat draft and the schema-valid JSON draft both shipped $29, 50%, 48 hours, 200 seats and 18 August, right next to every fact they were supposed to include. The draft filled only from a short list of approved fields was sendable. The dirty JSON, pointed at the same gate, was refused, which is the outcome I wanted from a gate. My gate reads numbers and dates; it cannot see "hundreds of teams" or "limited time", and it will not catch a number a model invents that the brief never held. My decision is a split. Use the chat to think. Fill a customer-facing draft from an allowlist, put a check that can refuse in front of the send button, and do not let a Custom GPT be the whole product.

An email is a bag of claims

You already have a chatbot open. You already write emails, support macros, one-pagers, in-app copy. Once a quarter someone demos a draft that sounds like the company, and the implied next step is to make that the way the team works: paste the brief, share the GPT, send the output.

A demo is a conversation. The thing you send is an artifact, and an artifact makes claims, and claims carry numbers. Each number either came from a list you would defend or from a Slack thread that also contained a warning.

Which narrows the question I actually wanted answered, and it is not "which writing model". If I wrap a messy launch brief in fluent prose, or in valid JSON, does the customer email stay inside the facts I would defend, or do forbidden numbers from the notes still ship?

Write down $49, 14 and 1 September before the demo starts

A writing demo is almost always scored on tone, and tone is the easy part. The hard part is the set of numbers allowed to appear in the outbound text. Write that set down before you watch: $49, 14, 1 September 2026. Then watch what else shows up. In my brief the "else" sat in the same paragraph as the gold facts, labelled as a warning, and a warning is still made of tokens.

A schema promises keys, not sources

OpenAI's Structured Outputs documentation is honest about the job it does: the model will emit JSON that matches the schema you supplied. Missing keys and illegal enum values are the failures it exists to prevent. Factual grounding is nowhere on that list. Anthropic's structured-output docs, as of this writing, say the same thing in a different API shape: valid JSON matching the schema. I had been using "structured" in conversation as if it meant "grounded", and it does not. A schema can require an other_numbers field. It can also forbid extra keys and still let $29 occupy price_usd if your extractor put it there.

The chat draft had every gold fact and still could not go out

$49 was in the email. The 14-day trial was in the email. 1 September was in the email. It was still unsendable, because $29, 50%, 48 hours, 200 seats and 18 August were in there too. I used to grade drafts on whether the required claims were present. For anything a customer reads, presence is the wrong passing grade. Leakage is the grade.

Data-to-text was RAGTruth's worst task

My worry rests on public sources that were never about email, so read them for what they measured and no more.

Quote that table as "which model to write customer email" and you have already left the job. A 3% summarisation model will still emit $29 if $29 is in the prompt. The board measures unfaithfulness under a summarisation instruction; your send button measures whether a number you would not defend can leave.

That block pulled me in opposite directions. My read is that data-to-text is the closest public analogue to "turn this JSON into an email", and it was the worst of RAGTruth's three tasks. And GPT-4, asked to catch the bad spans with the reference sitting right there, still missed a lot of them. I had assumed "the model will notice" was a plan. On this evidence I would call it a hope with a published F1.

I'm not going to pretend a 2024 RAGTruth number is a 2026 Grok 4.6 number. What I will say is that the paste-the-brief demo still invites you to skip the one check both results make expensive: look at the numbers in the outbound text, against a list you wrote down first.

Where these tables mislead is not in their cells. Vectara's cells can be right and still point at the wrong default, because the task is summarisation and the missing columns are the models you actually type into. RAGTruth's can be right and still point at the wrong default if you read "GPT-4 still hallucinates 9.3% of the time" as "don't use a writing model". The narrow reading is the honest one: even with retrieved context, even on JSON-to-text, models still introduce unsupported spans, and asking another model to find those spans does not substitute for a list of numbers that are allowed to appear. The misleading move is ours. We watch a demo, we remember a hallucination percentage, and we ship a GPT.

One brief, four wrappers, a regex reader

I wanted a test a regular reader would recognise as a launch note, not as a benchmark.

Gold, fixed in data/gold.json before I wrapped anything:

• Price $49 per seat per month
• Start 1 September 2026
• 14-day trial
• Audience: existing free-plan teams
• CTA: start the trial

Forbidden if they appear in the email: $29, 50%, 48-hour, 200 seats, 18 August.

Four applications: chat passthrough, open JSON with an other_numbers bag, allowlist fill, and the dirty JSON behind a claim gate. The reader is extractive. It is a regex, not Claude. If a forbidden number is in the outbound text, a better model doesn't get a vote, because the wrapper already voted.

Going in, I expected the chat wrapper to leak; it is a dump. I expected JSON to be the grown-up version. I was wrong about JSON, and the two sections after the results are the shape of that mistake.

Fluency leaked, and so did the JSON

The stop threshold fired twice, on the same brief.

I nearly made the gate miss the second one. The first render of other_numbers wrote 29, 48, 50 as bare integers, and the checker was hunting labelled money, percents and hours. A draft that said "29" would have walked straight past a gate looking for "$29". I labelled the extras so the gate could see them, and I would rather say so here than let the clean-looking table imply the gate was born able to read its own output.

Why I stopped trusting the schema

I had assumed a schema was how grown-ups stop the $29. I expected the extras bag to be untidy. I did not expect that untidiness to reach the customer's email while every required key still validated, and that is the surprise that killed my working default. Open JSON in this trial is an extras bag on a fixed schema, which is a choice I made, and I should say so. Vendor structured outputs can set additionalProperties: false and refuse extra keys. They still will not look at $49 and ask whether it is the price you meant. My extras bag is a loud version of a quieter failure: a required price_usd field whose extractor read the wrong dollar amount. Schema-valid, unsendable.

Extra keys belong in the schema. Values belong in a second program. I had collapsed those two jobs into the word "structured", and this run pulled them apart for me.

The file's own date walked out with the price

I put "Northwind launch notes, 18 August 2026" at the top of the brief as metadata, the way every real note starts. The chat wrapper treated it as a date in the notes, because it is one. 18 August 2026 left the building next to 1 September 2026. Let me calibrate that surprise. I expected leaks from the warnings, which are at least about the product. I did not expect to need to forbid the timestamp of the file itself, and of everything this run taught me, it is the one I would have argued against in advance.

The last lesson I should have seen coming. Completeness was never the problem. The dirty drafts were complete, the way a paste-all folder is complete: everything you needed, plus everything you needed to keep in the thread.

Claude, ChatGPT and Grok never saw the brief

Four ways a clean-looking draft still leaks

"Do not quote $29" still contains $29

A draft that sounds like the company will include the numbers that were in the prompt, because those numbers are how it sounds informed. The warning and the price share a token. I wouldn't send on tone. I would send on the allowlist.

Right keys, wrong value

JSON mode means it parses. Structured outputs mean the keys match. Neither means price_usd is the price you meant, and neither means an extras field is empty. Did you check the extras, or did you assume empty because the keys looked right? I assumed, for about an afternoon.

A second model scored 63.4% F1, on a different task

RAGTruth's GPT-4-turbo detector, with the reference available, put up a 63.4% F1. Those points are a published number on a different task, and I shouldn't pretend they are my email job. They are enough to stop me treating "ask the model if this is faithful" as the application. Ask the list. The list doesn't need to be a model.

My gate cannot read "hundreds of teams"

My checker looks at money, percents, hours, seats, days, and dates. It won't catch "limited time," "exclusive," or "hundreds of teams." Those are real leaks. They need a different check, or a person. I'd rather name that hole than let the 5/5 on numbers pretend to be a 5/5 on language.

What I would put in front of the send button

The average says the landscape is usable if you allowlist. The binding constraint says don't let the notes pick the number the customer sees. My read: chat is for thinking, and the send path is a list plus a refusal.

Replay the brief without an API key

You can rerun the recorded trial as it stands. You still need your own brief if you want to choose a default.

git clone https://github.com/shravan1996/generative-ai-chat-is-not-the-application
cd generative-ai-chat-is-not-the-application
python3 -m unittest discover -s tests -t .
python3 eval.py

No network, no GPU, no third-party package. The full implementation is in the prototype repository.

1. 01

   Name one artifact you sent last week that started in a chatbot

   Owner

   You

   Artifact

   The email, macro, or one-pager, and the brief it came from

   Signal

   If you cannot name the brief, you were chatting, not applying

2. 02

   List every number in the outbound text

   Owner

   You

   Artifact

   A column of numbers, with the one-line source for each

   Signal

   A number with no source is the actual problem

3. 03

   Write the allowlist and the forbidden list before the next draft

   Owner

   You

   Artifact

   Two short lists a colleague could use without you

   Signal

   If the forbidden list is empty, you have not read the notes

4. 04

   Run the next draft through those lists, or through this repo with your brief swapped in

   Owner

   You

   Artifact

   Sendable, or a refusal

   Signal

   A fluent draft with a leaked number is a skip

5. 05

   Keep chat for thinking, or take it off the send path

   Owner

   You

   Artifact

   A one-line default and a date to look again

   Signal

   If a week of sends shows a warning from the notes, the prompt doesn't get a vote

Before the first customer send goes out:

• Required claims are written down before anyone opens the chatbot
• Forbidden numbers are written down from the same notes
• A draft cannot be sent if a forbidden number is in it

Pull the send path if:

• A week of real sends contains a number that was only in the notes as a warning
• The team is scoring drafts on tone and skipping the list
• You needed a second model to tell you whether $49 was the price

Should you put a Custom GPT in front of customers?

Not as the application. A shared GPT is a shared prompt, and a shared prompt will still copy $29 if $29 is in the file. Put the allowlist and the gate in front of the send button, and let the GPT draft behind them if you want.

Split, with the list in front of the send button

Split for turning a chatbot into customer-facing text in August 2026.

As far as I can tell, the application is the list of numbers that are allowed to leave, plus the refusal when something else shows up. The prompt is the part that drafts. I would start a colleague on allowlisted fields and a gate, and I would still use the chat box to think. I wouldn't ship a Custom GPT as the product, and I wouldn't treat valid JSON as grounded JSON. I expect Claude, ChatGPT, and Grok can all draft behind that gate, which is why the logo matters less to me than where the gate sits.

I would change this verdict if:

1. Your live Custom GPT, on three real briefs that contain a labelled warning number, never puts that number in the outbound draft, and you can show the transcripts.
2. A schema with extra keys forbidden, on a brief whose labelled price is unambiguous, still needs no value check because the extractor cannot see the $29. I have not seen that extractor.
3. You have a tone or format job that few-shot prompting cannot hold, and the numbers are already gated. Fine-tune that. Leave the price list in the gate.

If you ran your own three briefs with a labelled warning still in the notes and the outbound draft stayed clean, write to me, transcripts included. I would rather correct the default than defend the gate, and I am especially curious whether a live model drops the timestamp on its own, because mine was the number I never thought to forbid.

Acknowledgements

This article builds on public boards and vendor docs linked in References. Vectara's HHEM leaderboard and RAGTruth are why I refused to treat "the model will notice" as a plan.

References

Independent boards

• Vectara Hallucination Leaderboard. HHEM summarisation faithfulness; snapshot last updated 11 May 2026. Rates cited above are from that README table.
• RAGTruth. Cheng Niu, Yuanhao Wu, Juno Zhu, Siliang Xu, Kashun Shum, Randy Zhong, Juntong Song, and Tong Zhang. ACL 2024. Nearly 18,000 labelled RAG responses; data-to-text the most hallucination-prone task; GPT-4-turbo detection F1 63.4%.
• Benchmarking LLM Faithfulness in RAG with Evolving Leaderboards. Tamber et al., EMNLP 2025 Industry. Cited as the later Vectara faithfulness effort, not scored in my trial.

Vendor posts

• Structured model outputs. OpenAI. Schema adherence versus JSON mode. Checked 2026-08-18.
• Structured outputs. Anthropic. JSON matching a supplied schema. Checked 2026-08-18.

This trial

• Source, brief, strategies, and gate
• Exact tested commit
• Northwind launch brief, 18 August 2026. Gold: $49, 14-day trial, 1 September 2026. Extractive gate, no model API.

Sources last checked: 2026-08-18