The Moats

Everything in the last chapter fell in about twenty months. It would be reasonable to open this one braced for consolation prizes — the soft, unfalsifiable comforts about creativity and the human touch that fill airport books on this subject.

This chapter offers none of that, because it was compiled under an anti-romantic rule: every human edge below was attacked before it was admitted. Each candidate was hunted for counter-evidence, dated, and — where the evidence went the other way — demoted to Chapter 12 or 14. What survived the hunt is short, specific, and far better defended than the folk version ever was.

The tag [durable] here means exactly this: on the evidence as of July 2026, a five-year human lead is a defensible bet. Defensible — not guaranteed, not sacred. Five moats qualify. Then comes the flip: a result about expert taste that this chapter refuses to soften, because the moats are only credible if the losses are counted at full price.

Moat 1 — The world pushes back [durable]

Start with the cleanest measurement. Meta's IntPhys 2 benchmark (2025) shows video models judging physically impossible events — the kind that violate object permanence, solidity, continuity — at chance level, roughly 50–60%. Humans score above 95% ⚖. The physics a one-year-old enforces before acquiring a single word is, as of 2025–26, missing from systems that write passable sonnets. This is Moravec's paradox — hard things easy, easy things hard — and it remains the best-supported human edge on the books.

The hardware side tells the same story with money attached. Robot dexterity is genuinely moving: 1X's NEO humanoid carries 22-degree-of-freedom tendon-driven hands; Physical Intelligence's π0.7 foundation model (April 2026) folds laundry and assembles boxes; its π*0.6 line learns from real-world reinforcement. And yet the same 2026 technical literature calls the field "nowhere near human capabilities" (Construction Physics, 2026; a running Science Robotics debate, 2026). Deployment numbers agree: Figure's 02 robot logged about 1,250 operating hours across eleven months at a BMW plant; Tesla missed its 2025 Optimus production target by roughly 90%, with Musk conceding on the Q4 2025 earnings call that the robots were "not doing useful work" yet; credible estimates put productive autonomous humanoids at hundreds to low thousands worldwide by end-2026. Not zero. Also not a workforce.

Why this moat holds, mechanically ⚙: two structural reasons, not one hopeful one. Hardware iterates on physical time — a gripper that breaks cannot be patched tonight and pushed to a million users tomorrow. And physical training data is scarce: there is no internet of touch. The field calls this the physical data wall. Text fell first because centuries of text were already piled up, labeled, and cheap to check; the world hands over its data one grasp at a time.

Now the honest exception, because this moat has a hole with a company logo on it. Waymo's driverless fleet passed 220.6 million rider-only miles by March 2026, with 91% fewer serious-injury crashes than matched human drivers ⚖ — in a domain narrowed by geo-fencing, high-definition maps, and a decade of industrialized data collection. Even there the long tail bites: stopped school buses and dead traffic lights produced failures through 2025–26. The rule the exception teaches ✦: where a physical domain can be fenced, mapped, and data-farmed, it falls like a benchmark. What stands is the open world — the unfamiliar kitchen, the cluttered job site, the task nobody fenced in advance. General-purpose embodied common sense is a defensible five-year bet as of July 2026. A five-year bet on any particular fenced-off slice of it is not.

Moat 2 — Twenty watts and a hundred million words [durable]

Two numbers, and neither has moved despite years of well-funded trying.

First: a child hears fewer than 100 million words by age thirteen. Frontier language models train on three to four orders of magnitude more — a thousand to ten thousand times the linguistic experience — and still need it (BabyLM 2026 call for papers). The BabyLM challenge has spent four years hunting for training tricks that close the gap at child scale; the results help at small scale and approach nothing like a child's sample efficiency on open-ended learning ⚖.

Second: the brain runs on about 20 watts. Per watt, published estimates put biological cognition somewhere between a thousand and a million times more efficient than current AI hardware. Meanwhile, single AI campuses draw 100–750 megawatts (2026) — the top figure is the resting draw of tens of millions of human brains — and more than 10 gigawatts of new AI capacity was announced in 2025 alone.

Read the two numbers together and the point sharpens ⚙: this gap is architectural, not a matter of scale, because brute scale is precisely what the gap measures. Every added gigawatt widens it. Whatever brains do to pull deep structure out of thin data has not been replicated, and it is not for lack of budget.

One honest qualification. In-context learning — a few examples shown inside the prompt — narrows the practical gap for the task at hand, and it genuinely works. But it evaporates when the session ends. Few-shot permanent learning did not exist as of July 2026. That distinction is a moat in its own right, and it gets the next section.

Why this belongs in a puzzle book: sample-efficient learning is the puzzle habit. One rope, one fuse, one properly digested solution becomes a Move you own for the rest of your life. That efficiency — a single example turning into a permanent skill — is exactly what Parts II and III train, one puzzle at a time.

Moat 3 — Learning that sticks [durable]

Ask the field itself, in 2026, which single capability is most conspicuously missing, and the answer is unusually consistent: continual learning. It headlines "AGI's Last Bottlenecks" (AI Frontiers, 2026); it is why Karpathy called this a "decade of agents" rather than a year (2025); it fuels the "continual-learning overhang" debate (2026) ⚖. The field's own clinical image: models wake up with anterograde amnesia.

Here is what that means in practice. Everything a model learns inside a session — your corrections, its own discoveries, the bug it finally understood — evaporates when the session ends. The colleague who was brilliant yesterday arrives today having never met you. Every morning, the first morning.

The workarounds are real, and they are not learning ⚙. Persistent agents keep external memory — wiki files, notes-to-self, retrieval stores — and these help the way a stranger's notebook helps the stranger who inherits it: the skill never moves into the system itself; it sits in a file that the next session may or may not read correctly. Meanwhile the mirror-image problem gained production teeth: catastrophic forgetting — train in the new, damage the old — became a working engineer's headache in 2025–26 as agents became persistent services ⚖.

Against all this, the human baseline barely needs stating: skills compound across decades. Your ten-thousandth conversation informs your ten-thousand-and-first. A reader who absorbs this book's Moves in 2026 will still own them in 2046, no retrieval store required. Chapter 15 builds directly on this asymmetry — the mistake journal, upsolving, spaced review — because a compounding learner should invest where compounding pays.

Grade it honestly, though. This is the single most-attacked research target in AI ⚠: robots already learn from real-world experience in narrow settings (π*0.6, 2026), and the incentive to crack the general case is measured in trillions. The [durable] here is pinned to today's architectures, and today has a date on it: July 2026. If one moat in this chapter falls to an architectural change rather than to scale, expect it to be this one.

Moat 4 — Knowing you don't know [durable]

Start with the part that is eroding, so the durable part earns your belief.

Raw error rates are collapsing. Hallucination in grounded summarization: 3–8% in 2023, down to 1.0–2.5% by 2026 (HHEM figures). Faithfulness errors in retrieval-augmented setups: 4–9%. Closed-book factual error on SimpleQA-style tests: roughly 4–10% for frontier models, with extended-thinking modes roughly halving errors (April 2026; aggregator figures) ⚠. OpenAI's September 2025 paper "Why Language Models Hallucinate" argues the root cause is benchmark incentives that reward guessing over abstaining — an argument that the whole phenomenon is fixable ⚠. Anyone building a human moat out of AI error rates is building on sand, and this book declines to.

The moat sits one layer down, and it has not moved. Studies through 2026 find language models show no signs of individuated metacognition ⚖: the model's stated confidence does not track what that particular model actually knows, and its willingness to say "I don't know" turns out to be a prompt artifact — rephrase the "Unknown" option and the abstention rate shifts, a bias the model cannot detect in itself (arXiv, 2026). A model can be prompted to perform doubt. It does not have doubt ⚙. The difference sounds philosophical, right up until it meets a courtroom.

So meet the courtroom. By April 2026 a running tracker had counted 1,313 documented court proceedings involving AI-fabricated content — invented cases, phantom citations — 496 of them involving licensed attorneys ⚖. US courts issued more than $145,000 in sanctions in the first quarter of 2026 alone. Nebraska delivered the first indefinite suspension of a law license over AI-fabricated filings. Note the anatomy of these failures: nothing exotic. A professional asked a fluent system a question, received a fluent answer, and signed it.

The economics land exactly where the case law does ⚙: when generation becomes nearly free, verification becomes the scarce input. Calibrated doubt — knowing which of your own answers deserve checking, and saying "I don't know" at the right moments for the right reasons — was, as of July 2026, a human craft. It is also a trainable one; Part V drills it.

Mark the shape of this moat carefully. Its metacognitive core is durable while its outer wall — raw accuracy — erodes every quarter. Do not paraphrase this section as "AI makes things up." It makes things up less every year. It still does not know when it is doing it.

Moat 5 — The signature [durable]

The Nebraska suspension points at something larger than error rates: someone had to be the one who signed.

The doctrine that crystallized in the 2026 sanctions wave is blunt: a signature "creates liability for its contents regardless of who drafted it" ⚖. The courts did not ask whether the machine was clever. They asked who signed. From August 2026, the EU AI Act's human-oversight duties for high-risk systems write the same assignment into statute. Accountability, it turns out, is not a cognitive capability that machines lack and might one day acquire. It is a role institutions assign to entities that can be sanctioned, disbarred, sued, and shamed ⚙. There is no benchmark to climb, because it is not a performance — it is a position. Durable by construction.

Goal-ownership is the same moat seen from the front. Reviews of agentic "AI scientists" in 2026 find deployment running technology-push: the systems execute impressively while problem selection, high-level strategy, and standards of proof remain human-set ⚖ (arXiv reviews and case studies, 2026). Chapter 12's mathematics pattern repeats everywhere: machines supply technique; humans decide what is worth proving and what counts as proven.

Log the erosion at full price, though. Judges trained on 700,000 citation-matched paper pairs can learn something that behaves like scientific taste (March 2026), and one frontier lab reports that in live research sessions its model's proposed next step now beats the human researcher's proposal about half the time (2026; secondary report, ~75% confidence) ⚠. Proposing the next step is not choosing the mountain. But the distance between those two is smaller than it sounds, and this paragraph may age quickly.

Two things stay put even so. First, longitudinal social reasoning: models pass static mental-state tests (Chapter 12) yet fail to adapt their behavior to a mind tracked across time (DialToM, 2026) ⚖. Second, and deeper: wanting. A system can optimize a goal handed to it; as of July 2026, nothing is at stake for the system itself. It cannot be fired, bankrupted, embarrassed, or proud. Skin in the game is not a metaphor here — it is the mechanism the entire accountability structure runs on ⚙. The right to be the one who decides, and the obligation to answer for the decision, is a human monopoly not because machines are dim, but because institutions are made of consequences.

The flip — expert taste and eighty-one dollars

This chapter owes you one story told at full strength, because it is the one that would be easiest to soften.

The setup (CHI, published April 2026): twenty-eight MFA-trained writers — people whose profession is telling good prose from almost-good — judged passages by language models emulating fifty acclaimed authors. Under ordinary prompting, the experts preferred the human-written prose 82.7% of the time. The moat held. Connoisseurship could tell.

Then the researchers spent about $81 fine-tuning a model on an author's actual corpus. The experts' preference flipped: 62% for the machine. Lay readers had preferred the machine all along, in both conditions. The study's debriefs record expert "identity crisis." The human labor being emulated was estimated at roughly $25,000 of writing time. The flip cost eighty-one dollars ⚖.

Say the unsettling part plainly, because pretending otherwise would poison the rest of the chapter. Expert taste, pointed at typical-quality work, was supposed to be deep — the accumulated, embodied judgment of a career. As a detector, it was defeated for the price of a good dinner, and the experts did not merely fail to spot the machine; they preferred it. If a professional edge amounts to "I can tell good from almost-good," that edge is not a moat. It is a fine-tuning run that has not happened yet.

Nor is this confined to prose. OpenAI's GDPval (September 2025, updated since) rated frontier-model deliverables equal to or better than those of professionals averaging fourteen years' experience on about half of 220 real occupational tasks — roughly a hundred times faster and cheaper.

Now what survives — because something does, and it is better evidenced than the thing that fell.

The tails. Pools of human writing are 2–8× more semantically diverse than pools of model writing, and the homogenization persists even after deliberate diversity-boosting interventions (Doshi & Hauser, Science Advances, 2024; further results 2025–26) ⚖. Models regress to the cultural center ⚙ — they are, nearly by construction, engines of the expected. The center is exactly where they now beat the median professional. But the tails are where new genres, new markets, and new science come from, and the tails remained human as of 2026. One excellent machine writer is impressive; a million of them are, collectively, the same writer.

The choosing. Taste-as-discrimination fell at the center. Taste-as-selection — which problem deserves your year, which standard of proof counts, which of a thousand adequate drafts is worth staking a name on — runs through Moat 5, and the 2026 evidence still finds it human-set ⚖.

One caveat and one instruction. The caveat: outlier taste is durable partly because nobody has built a benchmark for it yet, and protection by unmeasurability is a weaker kind of safety than protection by evidence ⚠ — expect this section to be stress-tested before most others. The instruction, for anyone whose work is judgment: move your value off the center ✦. Median discrimination is falling. Oddness, selection, and the willingness to sign are not — as of July 2026.

What the moats share

Stand back from the five moats and the flip, and Chapter 12's fallen list reads like a photographic negative of this one.

What fell was static, fully specified, and cheap to verify. What stands is the opposite in every coordinate: a world that pushes back mid-task (Moat 1); learning that compounds instead of evaporating (Moats 2 and 3); doubt that is owned rather than performed (Moat 4); goals that are chosen and signed for (Moat 5); tails instead of centers (the flip's survivors). The machines mastered the closed loop ⚙. Every moat is an open one — no fence around the world, no goal handed down, no free verifier, and consequences that land on the agent who acts.

None of this is a promise. Chapter 12 showed that falls are sudden and one-way; Moat 3 is under direct assault by the best-funded research effort in history; the flip happened to professionals at least as confident as any reader of this page. What the moats offer is not safety but direction: they mark where human effort still compounds, and Part V spends that information.

Between the fallen and the durable runs a strip of ground that is neither — the front line itself, where the lead is real and the clock on it is public. That is the next chapter.

Sources

All accessed July 16, 2026. Press and aggregator items are marked; treat their figures with the confidence noted in the text.

• IntPhys 2, Meta (2025): arxiv.org/abs/2506.09849
• Robot dexterity assessment (2026) — press: construction-physics.com/p/robot-dexterity-still-seems-hard
• Humanoid deployment reality check, Q2 2026 — press: 1023jack.com/news/the-humanoid-robotics-reality-check-q2-2026-pilot-to-production-status/
• Waymo Safety Impact, 220.6M rider-only miles (March 2026): waymo.com/safety/impact/
• BabyLM 2026 call for papers (child data-efficiency gap): arxiv.org/abs/2602.20092
• AI Frontiers, "AGI's Last Bottlenecks" (2026): ai-frontiers.org/articles/agis-last-bottlenecks
• Karpathy on agent memory and the "decade of agents" (2025–26) — press: inovabeing.com/blog/karpathy-llm-wiki-ai-agent-memory-2026
• No individuated metacognition (2026): arxiv.org/pdf/2605.24299 · abstention as prompt artifact (2026): arxiv.org/pdf/2507.16199 and arXiv 2604.17073
• OpenAI, "Why Language Models Hallucinate" (September 2025) — cited by title; hallucination-rate figures from HHEM (2026) and aggregator data (April 2026), secondary
• Legal tracker of AI-fabricated content in court — 1,313 proceedings by April 2026: gc.ai/blog/ai-hallucination-legal-cases
• Fine-tuned AI writing versus MFA-trained experts, CHI (April 2026): dl.acm.org/doi/10.1145/3772318.3791276
• GDPval, OpenAI (September 2025): openai.com/index/gdpval/
• "AI Can Learn Scientific Taste" (March 2026): arxiv.org/abs/2603.14473
• Agentic AI scientists — deployment critique (2026): arxiv.org/pdf/2605.08956
• Homogenization of creative diversity: Doshi & Hauser, Science Advances (2024); replication and extension (2025): sciencedirect.com/science/article/pii/S294988212500091X
• DialToM (2026): arxiv.org/pdf/2604.20443