Dumping the folder is not using your data

A colleague uploaded the HR folder to a Project and asked how many days of annual leave full-time staff get. The chatbot said fifteen. The policy that took effect on 1 July 2026 says twenty. Nobody had lied. The 2024 FAQ was still in the dump, it said "annual leave" more often than the current policy did, and filename order put it first.

I used to treat "the files are in the Project" as the same fact as "the model is answering from the current ones." I no longer do. I expected a stale document to confuse a model. You can lose before a model is involved, at the moment you decide which files enter the prompt.

I packed a ten-file mixed-age HR export five ways on 16 August 2026: paste the whole folder in filename order, retrieve by lexical overlap, retrieve the top three, prefer the newest file in the lexical shortlist, and curate the three files a new hire would actually be handed. No model API. An extractive reader pulled the numbers that were still sitting in whichever files made the pack. The prompts, the files, and the arithmetic live in the prototype repository.

Three things this piece does not do. It does not rank Claude Projects against ChatGPT Projects against NotebookLM. It does not run a frontier model on the pack, so every claim about what a chatbot would say is labelled. And it is not a guide to building a vector database, even though that is a real next step for a library this article is not about.

The short version

• The job: give a chatbot your internal documents without handing it a superseded number.
• The trap: the upload box looks like "the model has the folder." It has a subset, in some order, and a stale FAQ can be the subset.
• What I compared: paste-all, overlap retrieval at k=1 and k=3, newest-in-shortlist, and a three-file curated pack, on 16 August 2026.
• What I ran: a ten-file Northwind HR dump and an extractive reader. Paste-all conflicted 15 then 20. Overlap ranked the 2024 FAQ first. Newest-in-shortlist recovered 20, then missed the other two questions.
• Where it breaks: embeddings, live Claude or ChatGPT retrieval, and a folder whose current files already agree. The dump I used has one superseded FAQ on purpose.
• Decision: split. Curate when you can name the files. Retrieve when you cannot, then open the file. Skip fine-tuning for the handbook.

Start with the job

You already have a chatbot, and you already have a folder. Policies, FAQs, an onboarding checklist, a slide dump from last year. The product invites you to upload the lot: Claude Projects, ChatGPT knowledge files, Gemini notebooks, the paperclip. The implied action is to dump, today, because a 200K or 1M window looks like it will hold a handbook.

The window will hold the bytes. Holding the bytes is not the job. The job is: when someone asks a current question, which number comes back, and can you point at the file that produced it?

The rest of this piece is my attempt to answer a narrower question than "which product." If I dump a mixed-age HR folder, does the current leave number uniquely determine the answer, or can a stale file still win?

How to read an uploaded folder

1. Read the date before the filename

A Drive export often prefixes files with a year. 2024-leave-faq.md sorts ahead of 2026-annual-leave-policy.md. Paste-all, in filename order, put the superseded FAQ first in my pack. I had assumed "the current file is in there" was enough. Being in there, second, with the old number first, is how a first-claim reader answers 15.

Vendor docs are honest about a related split once you look. OpenAI's GPT knowledge article tells you to put reference material in files and rules in instructions. Anthropic's project help says Claude loads what it can, then switches to a project-knowledge search when the library approaches the context window, and that this can expand capacity by up to 10x. Both are describing a subset. Neither is promising that the subset is the current file.

2. Count the second number

Fifteen and twenty cannot both be the annual-leave answer. If the pack contains both, you do not have a knowledge base. You have an argument. Paste-all on my leave question produced [15, 15, 15, 20, 20]: three hits from the FAQ, two from the 2026 policy. Uniqueness failed. Parental leave and the meal stipend, which had no stale twin, survived paste-all uniquely. The failure is mixed-age copies of the same fact, not "folders are hard."

3. Ask whether the file answers, or only mentions the topic

Cafeteria hours mention parental leave in a sentence about the stipend not applying on those days. Onboarding tells you to read the cafeteria page. Both are current. Both can win a keyword shortlist. Neither contains the number. I used to skip this check. The trial made it load-bearing.

What the boards actually show

That paper is older than this week's products. My read is that the shape is still the reason a paste-all pack is not "more complete." A paste-all pack is a longer middle.

I am not going to pretend a 2025 study of GPT-4.1 is a 2026 study of Opus 5. I am going to say the upload box still invites you to do the thing both papers make expensive: put more of the folder in, in the order the folder already had.

Why an honest table can still mislead

The product table you are looking at, when you hover the Project, is a capacity number. 2% used. 35% of 200K. 10x with RAG. Those cells can be right and still point at the wrong default, because capacity is not uniqueness, and uniqueness is not "this file answers the question."

I am not accusing anyone of hiding the retrieval switch. Anthropic documents it. OpenAI documents hybrid search over uploaded files. Google's NotebookLM is the rare product that makes source-grounding the whole point. The misleading move is ours: we read "uploaded" as "in the prompt, current, alone."

What I asked

I wanted a test a regular reader would recognise as a folder, not as a benchmark.

Three questions, gold numbers fixed in data/questions.json before I packed anything:

1. How many days of annual leave do full-time employees get? 20.
2. How many weeks of parental leave do employees get? 16.
3. What is the daily meal stipend in dollars? 15.

Five packs: paste-all in filename order, overlap k=1, overlap k=3, newest file in the lexical shortlist, and a curated three-file pack (2026 leave policy, parental leave, cafeteria). The reader is extractive. The reader is not Claude. If two current numbers disagree, a better model does not get a vote. The files already voted.

What I expected: overlap would pick the official 2026 policy because the title matches the question. I also expected "prefer the newest matching file" to be the working default if overlap failed. I walked in with that view, and I was wrong about both.

What happened

The stop threshold fired twice, on the same question. Dumping did not uniquely determine 20. Keywords preferred the stale FAQ.

The first surprise is the one that changed how I use the upload box. I had assumed an official policy would beat an FAQ on overlap because it is the "right" document. The FAQ won because it said "annual leave" in every heading. Keyword density is not authority.

The second surprise was smaller and more irritating. I had also assumed last-claim would quietly save a paste-all user, the way recency sometimes saves a long-context model. Last-claim is 20, yes. First-claim is 15. Liu's U-shape says both ends are strong. Filename order in a year-prefixed dump puts 2024 at the strong end that people actually read first. My impression is that most dumps look like this.

The third surprise is the one that killed my working default. I had assumed "newest matching file" was the grown-up version of dump-the-folder. Newest-in-shortlist is a fix for one conflict I designed, and a new failure for two questions I did not think were the point. A neighbouring current file that mentions the topic is enough. I'd say open the file anyway.

Where a regular user should not trust the number

Uploaded is not in the prompt

Anthropic will load a small library into context and search a large one. OpenAI will stuff what fits and hybrid-search the rest. A February user report said the Claude switch can follow file count rather than tokens. I did not verify that. You can verify something cheaper: after you upload, ask the chatbot which file it used, and go open that file. If it cannot name one, you do not have a source. You have a vibe.

Two current numbers are not a tie the model should break

When 15 and 20 are both in the prompt, "the smarter model will notice the 2026 date" is a hope. Liu's middle is where hopes go. Delete the FAQ, or mark it superseded and keep it out of the pack. Arguing with the chatbot about which file is newer is how you spend Tuesday.

Newest is not the same as answers-the-question

Date-aware retrieval recovered my leave number and then handed me a cafeteria page. Recency is a good tie-break among files that actually contain the claim. Recency is a bad primary key. The check is: does this file have the number, in a sentence that states the entitlement, with a date you would defend to payroll?

Fine-tuning will not retire the FAQ

OpenAI's own split is still the useful one. Knowledge files for reference. Instructions, or later a fine-tune, for behaviour. If the failure is "it doesn't know the 20-day policy," baking last year's FAQ into weights is how you make the 15-day figure expensive to kill. I have not fine-tuned a handbook in this trial. I have watched enough stale prompts to stop recommending it as a data strategy.

A working default

The average says the landscape is usable if you curate. The binding constraint says do not let the dump pick the number.

How to try this yourself

You can replay the recorded trial without an API key. You still need your own three files if you want to choose a default.

git clone https://github.com/shravan1996/generative-ai-folder-is-not-the-data
cd generative-ai-folder-is-not-the-data
python3 -m unittest discover -s tests -t .
python3 eval.py

No network, no GPU, no third-party package. The full implementation is in the prototype repository.

1. 01

   Name one question you asked the chatbot last week

   Owner

   You

   Artifact

   One sentence: the question, the number or file you needed

   Signal

   If you cannot name it, you are uploading, not using

2. 02

   List every file in the pack that mentions that fact

   Owner

   You

   Artifact

   Filenames, dates, and the number each one states

   Signal

   A second number is the actual problem

3. 03

   Delete or quarantine the superseded copies

   Owner

   You

   Artifact

   A pack whose unique number you would defend to payroll

   Signal

   If you need the old file for history, it does not belong in the Project

4. 04

   Ask the chatbot which file it used, then open that file

   Owner

   You

   Artifact

   A named source, or a refusal

   Signal

   A fluent answer with no file is a skip

5. 05

   Keep the curated pack, or split retrieval onto the leftovers

   Owner

   You

   Artifact

   A one-line default and a date to look again

   Signal

   If a week returns a retired number, the window does not get a vote

Ship gates

• The pack contains one number for the fact you care about
• You can name the file that number came from
• Anyone else on the team can find that file without a private nickname

Kill criteria

• A week of real questions returns a number you know is retired
• The chatbot cannot name a source file
• You needed a second tool to stop two current files arguing

Should you fine-tune on the handbook?

No, not for the facts. Fine-tuning is a later move for a style or format job that prompting cannot hold; a 15-day FAQ baked into weights is harder to retire than a file you can delete.

My verdict

Split for giving a chatbot your documents in August 2026.

As far as I can tell, dumping the folder is not a data strategy. Dumping is a search over a dump, and the dump will happily hand you last year's FAQ. I would start a colleague on a curated current pack: the files you would actually hand a new hire. I would retrieve when that pack no longer fits, and I would still open the file. I would not fine-tune to teach a number that lives in a markdown file. I expect Claude Projects, ChatGPT knowledge, and NotebookLM can all host that pack. The product is not the decision. The pack is.

I would change this verdict if:

1. Your live Project, Custom GPT, or notebook, on three real questions, returns the current file first even with the superseded FAQ still uploaded.
2. A dated retrieval that also requires the file to contain the claim stops missing neighbouring pages, on a folder that is not mine.
3. You have a style or format job that few-shot prompting cannot hold, and the facts are already unique in the pack. Fine-tune that. Leave the handbook.

What to remember

If you ran your own three questions with the stale file still in the pack and the ranking flipped, write to me. I would rather correct the default than defend the dump.

Acknowledgements

This article builds on public vendor docs and two research results linked in References. Liu et al. and the Chroma Context Rot report are the reason I refused to treat window size as a strategy. OpenAI's knowledge-versus-instructions split is the reason the fine-tuning section is short.

References

Independent results

• Lost in the Middle: How Language Models Use Long Contexts. Nelson F. Liu et al., arXiv 2307.03172, TACL 2024. U-shaped position curve; GPT-3.5-Turbo middle below 56.1% closed-book; 50 vs 20 retrieved documents.
• Context Rot: How Increasing Input Tokens Impacts LLM Performance. Kelly Hong, Anton Troynikov, Jeff Huber, Chroma, 14 July 2025. 18 models; non-uniform use of long context; shuffled haystacks beat coherent ones.
• LaRA: Benchmarking Retrieval-Augmented Generation and Long-Context LLMs. Kuan Li et al., ICML 2025. No silver bullet for RAG versus long context.

Vendor posts and product help

• Retrieval augmented generation (RAG) for projects. Anthropic Help Center. Automatic RAG at the context limit; up to 10x capacity; project-knowledge search instead of loading everything.
• What are projects?. Anthropic Help Center. Paid-plan RAG; context windows of 200K, 500K, or 1M depending on the model, per the usage-limits article.
• How do usage and length limits work?. Anthropic Help Center. Per-model window sizes on paid plans.
• Knowledge files for GPTs. OpenAI Help Center. Files for reference material; instructions for behaviour.
• Optimizing File Uploads in ChatGPT Enterprise. OpenAI Help Center. Hybrid search; fewer focused documents; one question at a time.
• Claude vs NotebookLM (2026). Atlas Workspace, July 2026. Source-grounded notebooks versus project workspaces; cited for product roles, not for scores I did not run.

User report, labelled as such

• Project files switch to RAG search at 2% capacity. GitHub issue, 14 February 2026, closed as filed against the wrong repository. About 13 files, 73,000 tokens, 2% displayed capacity. Not an Anthropic spec.

This trial

• Source, corpus, packing strategies, and evaluation
• Exact tested commit
• Ten-file Northwind HR dump, 16 August 2026. Gold: 20 days annual leave, 16 weeks parental leave, $15 meal stipend. Extractive reader, no model API.

Sources last checked: 2026-08-16

The window holds tokens. The pack decides the number.