What the AI Can and Can't See

Start here, because it explains most of the rest. A language model does not remember your last conversation. It does not remember your project between sessions. Every single time you press enter, the tool assembles a package of text and sends the whole thing — and that package is the entire world as far as the model is concerned.

That package usually contains:

1. 1
   A system prompt

   Instructions from the tool vendor you never see.

2. 2
   Your rules file

   AGENTS.md, CLAUDE.md, or .cursorrules, if one exists.

3. 3
   Some of your code

   The open file, files you referenced, and whatever the tool's search decided was relevant.

4. 4
   The conversation so far

   Earlier messages in this session, until they get too long and start being dropped.

5. 5
   Your actual prompt

   The smallest part, and the only one you fully control.

When the model “forgets” something you said twenty minutes ago, nothing mystical happened. That text stopped being included in the package.

The chat window makes this harder to see, because it is designed to look continuous. Your messages stay on screen, scrolling upward, reading like a conversation with someone who remembers what you said earlier. Nothing about the interface tells you that underneath, each message you send triggers a completely fresh request — the model wakes up, is handed the transcript so far as plain text, answers, and then, in every sense that matters, stops existing until the next message arrives.

“Remembering” what you said five minutes ago is not memory in any sense a person would recognise. It is re-reading. The transcript is included again, in full, every single time — right up until it gets too large to fit, at which point the tool starts quietly leaving parts of it out, usually the oldest parts first. You experience that as the model forgetting. What actually happened is closer to someone being handed a shorter and shorter set of notes to read before answering.

There is a hard ceiling on how much text can go in that package, measured in tokens — roughly ¾ of a word each, so 1,000 tokens is about 750 words. Modern models take a lot: hundreds of thousands of tokens. That sounds limitless until you notice a mid-sized codebase is millions.

So the tool is constantly choosing what to include and what to leave out. It is usually good at this. It is never perfect. And critically: it will not tell you what it left out.

“Hundreds of thousands of tokens” is an abstract number until you put your own project next to it. Roughly speaking a token is three-quarters of a word, and the maths runs in a direction most people do not expect: a paragraph of prose is cheap, a single file is still cheap, and a whole codebase adds up far faster than either.

The jump from “one file” to “the whole repo” is where budgets actually run out. A single component fits with room to spare. A directory of forty related files starts competing with everything else that also needs a place in the package: the system prompt, your rules file, the conversation so far. Something gives, and it is rarely the part you would have chosen yourself.

This is the behaviour people find most alarming, and it has a mundane explanation. The model predicts plausible text. When it cannot see your actual formatDate helper, it does not think “I lack information.” It thinks “what would a formatDate helper look like in a project like this?” — and writes that.

The output is confident because plausible text is confident text. There is no internal signal that distinguishes “I read this” from “I inferred this.”

There are two entirely different sources a model draws on, and mixing them up is most of why hallucination feels unpredictable. The first is what it learned during training: the shape of idiomatic code, common library APIs, the usual name for the usual pattern, absorbed from a huge amount of text it cannot point back to and cannot quote a source for.

The second is what is sitting in the context window right now — your actual files, in this actual request. The model does not experience these as different in kind, so it does not warn you when it switches from one to the other. It writes your project’s formatDate helper using the second source, and a plausible-looking substitute using the first, in exactly the same confident voice.

Rather than guessing, ask. This costs one message and tells you more about your setup than any amount of theory:

Before answering anything else: list the files you can currentlysee in this project, and tell me which parts of my codebase youcannot see. Do not guess — if you are unsure, say so.

Run this in each tool you use. A repo-aware agent lists real files. An autocomplete extension names your open file and little else. A browser chat says it sees nothing but what you pasted. All three answers are correct, and knowing which one you are holding changes how you prompt.

1. 1

   Reference files explicitly

   Do not make the tool guess which of your 200 files matters. Naming two files is faster than any search it can run.

2. 2

   Start fresh when the thread wanders

   A new conversation with a good opening prompt beats a long one that has drifted. You are not losing progress — the code is on disk.

3. 3

   Put durable facts in a rules file

   Anything you find yourself repeating every session belongs somewhere the tool loads automatically.

   That is Chapter 15, where you put repeated context in a rules file.

4. 4

   Paste errors in full

   Your summary of an error drops the stack frame that identified the file. The raw text costs you nothing.