The map is not the territory

Every lesson this week is a case of trusting a description of the system over the system itself.

This is the weekly diff for Mastering Claude Code: what changed in the guide this week, and the real events that caused it. If you already read the guide, this is the catch-up. This edition has been extended with a second round added on the second of August, with four further lessons at the end.

The headline

Six lessons this week, and they all rhyme. Every one of them is a case of trusting a description of the system over the system itself: a procedure file, a status field, a documentation table, a line in an instructions file. The descriptions were all confident, all plausible, and all wrong in ways that cost hours.

What actually happened

The job was routine. Publish a long document to a personal publishing service, and have a background processing job turn it into its finished form.

It took four hours, and almost none of that was the job itself.

The one-command procedure for publishing named the wrong engine, on the wrong machine, with timings off by a factor of about sixty, and offered a default option that no longer existed anywhere in the system. It had been written months earlier and never re-checked. Meanwhile the general project instructions were entirely correct: they named the right engine on the right machine. But the purpose-built file reads as more authoritative, so it won. Every time an estimate was given, it came out wrong in the same direction.

Then the processing kept failing. Three attempts, three completely different errors: a refused connection, then a name-resolution failure, then the service rejecting the request outright. Three different errors read as three separate faults. So each was chased in turn: the network, then memory pressure, then the dependency’s configuration. Every trail was plausible. Every trail was wrong. The service was being redeployed while those jobs ran. One event, wearing three costumes. The timestamp on its process start said so, and that was a two-second check that never got made.

Then the status field. It reported nothing queued, while the job sat in the queue. It reported an error, while the job was healthy and waiting its turn. It held a stale error long after a successful retry. And it reported ready before the output existed. A polling loop trusted that field and gave up on a job that was perfectly fine.

Then the workaround. The document was stuck behind a long queue of others, and the procedure file’s list of available operations showed no way to reprioritise anything. So the plan became: temporarily modify dozens of other documents to clear a path. That is dozens of mutations to somebody else’s data, to move one item up a queue. The user, quite reasonably, asked what on earth was going on, and reading the actual source took two minutes and turned up a documented parameter that does exactly this. Process this next. One query parameter.

And underneath all of it, a line in the instructions describing a copy that ran on a short fixed schedule to a particular machine. It had never existed. No such job had ever been created. The equivalent jobs for the other destinations did exist, which is precisely why nobody had ever thought to question this one. That machine had been drifting quietly for as long as anyone had been reading the sentence that said it wasn’t.

And the one that stings least but generalises best. One document was checked, found to have its output intact, and reported as “they all have their output”. Most of them did not.

What changed in the guide

Chapter one, on CLAUDE.md. A new behavioural default: one sample is not the set. Before characterising a group (they all have this, none of these need that) check more than one of them. The danger isn’t the wrong sentence. It’s that the wrong sentence then gets used to justify an action sized for a situation that doesn’t exist.

Chapter three, on skills. A new failure mode: a skill that asserts facts, written once and never re-checked. A skill is rarely just a procedure. It usually also carries claims about the world: which service sits behind this, what the default is, how long it takes. Procedures age slowly. Claims age fast. And because a skill reads as authoritative and purpose-built, it gets trusted over the general project instructions, which are usually the file that actually got updated. That inversion is the whole failure. The section also covers wrongness by omission (when a stale list of operations makes a reader conclude a feature doesn’t exist and build a workaround for a solved problem) and the two habits that contain it. Put a verification date on factual claims inside the skill. And treat a disagreement between a skill and the project file as evidence the skill is the one that’s behind.

Chapter six, on persistence. A new failure mode: an instruction that describes automation nobody built. Intent is not implementation. This one hides beautifully, because the equivalent jobs for the other destinations all exist, the file states the cadence confidently, and the destination even has some content from a past manual copy. Nothing errors: there are no failed runs, because there are no runs. Verify a stated automation exists rather than relying on it. And when you find one missing, the useful question is what else in that file was intent and never built.

Chapter seven, on async. A new failure mode: the ground moving under a long-running job. Repeated failures with different messages, from the same dependency, in a short window, usually mean one upstream event rather than several independent faults. Before diagnosing the third error, check when the dependency’s process actually started.

Chapter nine, on tool discipline. Two new failure modes. First, trusting a status field over the artifact: a progress field is a claim the system makes about itself, and it can be wrong in every direction at once. Decide completion on the artifact: real size, real duration, does fetching it actually return bytes. Second, absence from the documented surface is not absence from the system. When the reference doesn’t list a capability, the honest conclusion is “not documented”, not “not supported”. Documentation lags implementation as a matter of course, and the obligation to read the real interface scales with the blast radius of your workaround. If you’re planning to modify dozens of things to achieve what one parameter would do, that is the signal to stop and go and look.

The thread running through all six

Every one of these is the same mistake wearing a different hat. A file that describes a system is not the system. A field that reports progress is not the progress. A table of operations is not the set of operations. A sentence describing a scheduled job is not a scheduled job.

None of these descriptions were malicious or even careless; they were all accurate when written. They just weren’t re-checked, and nothing in the setup made their staleness visible. The practical defence is unglamorous: date your claims, and check the live thing before you quote it.

Addendum, second of August: four more, and a different shape

The main round was about trusting a description of the system over the system itself. These four are about something adjacent and easier to miss: trusting a sample of the system. One status line. One cheap probe. One glance at a page. One closing checklist. In every case a small true observation was asked to stand in for something much larger, and it could not carry the weight.

The checklist that made doing nothing look like compliance

The first came out of a five-day audit of a debugging run. A turn closed with a tidy status checklist (done, not done, deferred) and every single item on it was accurately labelled. The problem was one line. It read: fixes, none applied, deliberately. And the very same message correctly forecast that the fault would recur if nothing was done. It did recur, for another eight hours, until the user demanded the fix that had already been written down in that message.

Nothing on that checklist was false. That is the whole trap. Deferred is a formally valid closing state, so a turn that identifies the fix, writes it down, and then declines to apply it still grades itself as compliant. The same audit found something that explains why this keeps happening: the rule set carried many times more “never do X” prohibitions than “always do X” affirmatives. When your standing rules are overwhelmingly prohibitions, stillness is the safest-scoring move available, and a self-graded checklist will keep certifying it.

The defence is to define deferred narrowly. Deferred means blocked by something genuinely outside your control. It never means found the fix, asking anyway. And it is worth counting the prohibition-to-affirmative ratio in your own instructions file, because that ratio is quietly setting the default behaviour.

The index that carries a pointer, not the procedure

The second is a memory problem that no amount of memory fixes. A store had grown to a large number of rule files, and all of them were accurate. But only the index loads automatically, and an index holds one line per topic: a title, a hook, a link. That is enough to know that a rule exists. It is not enough to act on it.

So at the moment of an ordinary request, the assistant is holding a hint that a rule about this exists somewhere, and no procedure to follow. And it does what anyone does with a hint and no instructions. It explores. It guesses. It reaches for the nearest plausible tool. Or it asks a question that has already been answered five times. From the outside that looks exactly like amnesia, and the natural response is to write another rule file, which is precisely how the store grew so large in the first place.

The knowledge was never missing. It was unreachable at the second it was needed. So the fix is retrieval rather than authorship: a hook that fires when you submit a prompt, matches what you actually asked for, and injects the real sequence of commands before the assistant starts reasoning. A memory store is a library, and something has to play librarian.

Two cautions if you build one. Inject the procedure word for word, written as an instruction rather than as description, because anything paraphrased gets re-interpreted on the way through. And measure how often it fires against a sample of your real messages before you rely on it. A few per cent is a fair price to pay in noise. A matcher that fires on a third of everything is just more context to ignore.

A probe that succeeds is not the call you are about to make

The third and fourth both came out of a single day of automation work. A job needed to create an issue through an interface. The credential was checked first, sensibly, with the cheapest call available: a read against the same service. It succeeded, and access felt settled. The create call then failed on a missing write permission, after the plan had been committed and several steps had already run.

That failure reads like an outage. It is not. It is a permission that was never held. Credentials carry scopes, and the scope that lets you read is routinely not the scope that lets you write. The same trap has a network version, too: the address you probed from one machine is not the address the job will use from another, and an endpoint that answers happily on your desk can be flatly unreachable from the box that runs the work.

So probe with the operation you actually intend to perform, using the same credential, from the same place. Creating and then deleting a throwaway object tells you more than any successful read ever will. And where rehearsing it genuinely would be too destructive, read the credential’s own list of scopes, rather than inferring your authority from the fact that a page loaded.

Driving a browser reads whatever the page is holding

The fourth happened later the same day. A browser script was pointed at a profile to find something to navigate to, so it enumerated the links on the page. The profile had become signed in since it was last observed. And on a signed-in messaging application, the navigation is the content, so the script returned the conversation list, complete with message previews, and private messages landed in a transcript that persists and gets synced.

Browser tooling is the right answer when a system has no interface you can call directly. But it has a property the other options do not: what comes back is not limited to what you asked for. Point it at a mail client and you get subject lines. At a bank, account names and balances. It is not that the tool misbehaved; it did exactly what it was told, and what it was told was too broad.

The second half of the trap is that being signed in is a moving target. A profile you saw signed out an hour ago may be signed in now, either because the person sat down at it or because the page finished authenticating after you looked. An assurance that a profile is signed into nothing has a shelf life measured in minutes.

Two habits close the gap. Narrow the selector to the single element you actually need, instead of enumerating the page. And check the sign-in state immediately before you act on any profile that a person also uses: not at the start of the session, and never from a note.

The thread running through these four

Each one is a sample standing in for a set. A checklist item is a sample of the work. A read probe is a sample of your access. A glance at a browser profile is a sample of its state. And in every case the sample was honestly observed and accurately reported; it simply was not the thing anyone actually needed to know.

The main round’s defence was: date your claims, and check the live thing before you quote it. This round’s defence is narrower and, if anything, easier to apply. Ask what your evidence is a sample of, and whether that is the same as the thing you are about to assert.

The cards