TypeSafe's Founder Published Coding-Agent Notes. The KV-Cache Math Is the Part Worth Reading.

Diogo Almeida — TypeSafe AI's founder, a co-author of the RLHF work behind ChatGPT, and the subject of explainx.ai's own profile on his "weird detour" argument — published a long, unusually candid notes document on September 21, 2026, sketching what a Jev-centric coding agent could look like. He's explicit that it's speculative: "we likely will never have time (ever again) to play ourselves, but hope the community goes WILD (and makes me look like a naive idiot)."

Most of the document is brainstorming, and this post doesn't try to relitigate every idea in it. But one section is different from the rest — a worked cost comparison about model routing that's concrete enough to check, and that runs against a piece of conventional agent-engineering wisdom worth examining directly: the assumption that routing easy tasks to a cheaper model and hard ones back to a stronger model is close to a free efficiency win.

TL;DR: What people are asking

The KV-cache argument, worked through

Almeida's core claim starts from a genuinely useful question: "how would you design a coding agent if LLMs had no KV cache?" — a thought experiment meant to expose which parts of current agent design only make sense because of KV-cache reuse, and which parts people assume are efficient without actually accounting for it.

His routing example uses rough, self-described "vibed" cost assumptions for two models — a pricier one (he labels it Opus-like, at roughly 5/25 for input/output pricing) and a cheaper one (Sonnet-like, at roughly 3/15) — and compares two paths for a long agent session with a large existing context:

• Path 1: stay on the expensive model the whole time.
• Path 2: switch to the cheap model partway through, then switch back to the expensive model to finish.

The catch in Path 2: when the expensive model resumes, it has no access to the cheap model's KV cache from the intervening turns, so it has to reprocess the entire accumulated context — the original context plus everything generated while the cheap model was active — from scratch, at the expensive model's input-token rate. Under Almeida's assumed proportions (roughly 65% of tokens being reused context, 12% being genuinely new generated output, 23% being additional tool-generated tokens like command output or file reads), his math comes out with the pure-expensive-model path costing about two-thirds of the switch-and-return path.

The precise ratio depends entirely on his assumed proportions, and he says as much — this is a worked illustration, not a universal constant. But the underlying mechanism is real and often left out of "route to a cheaper model for easy steps" pitches: the savings from a cheaper model's lower rate have to be weighed against the cost of losing KV-cache continuity on whichever model resumes the session, and in a long session with a large existing context, that reprocessing cost can dominate. This is a sharper, more mechanistic version of a caution explainx.ai's own harness-engineering coverage has gestured at more generally — that agent efficiency claims need to specify exactly what's being measured, not just cite a per-token price difference.

Five more things Almeida says the "tyranny of the KV cache" explains

The routing argument is the most checkable claim in the document, but Almeida frames it as one instance of a broader pattern — several common agent-design choices that make sense specifically because reprocessing a fresh context is expensive, not because they're independently optimal:

• Tool calling is a "weird tradeoff." Every tool has to be specified upfront in the system message whether or not it ends up relevant to a given task, consuming context on every single turn. His stated hypothesis is that models struggle with some combination of high option cardinality and "off-policy" tool selection — plausibly, he suggests, part of why narrower, dynamically loaded Claude-style skills have an edge over an always-present tool list.
• Compaction assumes a single shared future state is worth preserving perfectly, but general-purpose compression is hard, and — his argument — likely worse than compression that knows what the next query actually needs, rather than trying to preserve everything indiscriminately.
• Subagents underperform expectations partly because managing what context to pass in and what to merge back is itself expensive relative to whatever parallelism gain subagents are supposed to provide.
• Session restarts make sense only if you assume corrupted state can't be selectively salvaged — an assumption Almeida questions rather than accepts as given.
• The "batteries included" debate — frictionless agents like OpenClaw on one end, power-user-configurable ones like Claude Code or Codex on the other — is, in his framing, downstream of how expensive it is to load in optional capability without it permanently taxing every future turn's context.

Where Jev fits, per the notes

The document sketches several concrete roles for a Jev-style decision model inside a coding agent, ranging from immediately plausible to explicitly speculative:

• Permission and approval gating — a Noul-style yes/no check on whether a command should run, extended to reading a file's contents before executing something like a Python or bash script, the same tool-risk gating pattern explainx.ai has already covered for Jev.
• MCP and tool-call routing — using Jev calls to select the best matching tool, or a top-K shortlist, from a high-level text description of intent, rather than keeping every tool's full schema permanently loaded in context.
• "Meta-attention" — the most speculative idea in the document: instead of treating loaded context as static once it's in, continuously re-score it. In its simplest form, a Noul call per context "chunk" deciding keep-or-drop; in a fancier version, a Score call choosing between don't-show, a short summary, a longer summary, or the full chunk. This directly extends the context-engineering framing explainx.ai has covered for tool definitions and schema design into something more dynamic and query-aware than a fixed system prompt.
• Security-aware routing — flagging which files or tasks are sensitive enough that they shouldn't be routed to a cheaper or geopolitically riskier model, a genuinely different axis from the cost/difficulty routing decisions agent frameworks usually optimize for.

None of this is described as built or shipping. It's explicitly a list of directions Almeida finds interesting enough to write down and publish, not a roadmap.

A real, if informal, data point from the replies

One reply to the thread, from a developer posting as ickas, reports an independent test worth noting precisely because it's a negative result in one configuration and a positive one in another: a single Jev Choice call over roughly 90 nearly identically labeled Battleship grid cells lost to a simple 50-line heuristic, while giving Jev a shortlist of just 16 clearly described options matched their own best existing solver's performance. That's consistent with the architectural picture in explainx.ai's coverage of Kev, Jev's most fully documented open-source counterpart — decision models built on the Choice primitive are structured to discriminate between meaningfully described options, not to search a large space of near-identical, minimally differentiated ones the way a purpose-built heuristic or search algorithm can.

Honest limitations

• This is a personal notes document, not a product announcement or roadmap. Its own author calls the underlying assumptions "vibed" in places and explicitly hopes to be proven wrong by the community.
• The KV-cache routing math uses self-described rough proportions, not measured production data from an actual agent workload — treat the exact 2:3 cost ratio as illustrative of a real mechanism, not a number to plug into your own infrastructure decisions unchanged.
• The "meta-attention" and security-aware routing ideas are the least developed parts of the document — genuinely interesting directions, with no described implementation, evaluation, or even a rough architecture beyond the one-paragraph sketch covered here.
• The Battleship benchmark cited above is a single informal report from one developer, not a controlled study, and shouldn't be read as a general statement about Jev's or Kev's Choice-primitive accuracy on structured search tasks.

What this means for builders

The specific, actionable idea here isn't "adopt Jev for your agent" — it's the KV-cache accounting question itself. If your agent framework routes between models to save cost, the routing math needs to account for what happens to context continuity when you switch back, not just compare per-token rates between the two models in isolation. That's a checkable thing to audit in your own pipeline this week, independent of whether TypeSafe, Almeida, or anyone in the community ends up building the more speculative parts of this notes document.

Related on explainx.ai

• He co-invented ChatGPT. Now he says it was a "weird detour." — explainx.ai's profile of Diogo Almeida
• Kev's real numbers: inside the open-source Jev clone's 0.8B/4B/9B family
• How to wire Jev into your agent pipeline for routing decisions
• Jev Ultrafast: Browser Use puts Jev in the browser agent loop
• What is harness engineering? Complete guide
• Tool definitions and schema design: a context-engineering guide
• DeepSeek V4.1 Flash cuts KV cache HBM by ~75% — what changed
• What are agent skills? Complete guide

This post is sourced to Diogo Almeida's publicly shared notes document and its accompanying X thread as of September 21, 2026. The document is explicitly framed by its author as speculative personal notes, not a confirmed TypeSafe product roadmap; treat every proposed feature accordingly.