Where the Training Data Runs Out: Why Domain CLIs Need a Playbook

Hook

I was building the back end for a tool I'd been meaning to build for years — a color pair iterator, what I now call CPQI. Front-end-only Contrast Chaser had taken me as far as it could; the next move was a real backend, somewhere the accessibility logic could live where it could be tested, extended, and — the new wrinkle — called by an agent.

This was when MCP was peaking. I read the posts. I built the server.

And the rabbit hole opened up. It was an NPM module. Then I had to choose a transport. Then the streaming connection kept dropping. Then I had to figure out auth. MCP became its own product before it became a useful tool. I shipped it anyway. It worked.

What I couldn't fix was Claude Code never reaching for it. The token burn was real — I could ration around that — but the part that nagged was that the model, dropped into a session, would do accessibility checks the wrong way unless I nudged it toward my server. Putting "use the MCP server for accessibility" in CLAUDE.md helped. It also felt like a hack. The instinct wasn't there.

Around then, the CLI-vs-MCP debate hit critical mass and I tried the other path. Wrote the documentation. Wrote a workflow playbook alongside the binary — explicit workflows for the tasks I expected an agent to need, such as building a palette from a hero color or generating dark-mode equivalents. Each workflow names the situation, gives the reasoning, lays out the command chain. The first thing I noticed wasn't the token reduction everyone writes about — it was that the agent always reached for the CLI without being told. The second thing I noticed was that the playbook actually got read. I'd expected to have to babysit the agent through the workflows. Instead it picked the right one without prompting and chained the steps with a fluency I hadn't budgeted for.

The CLI vs MCP debate has been written to death since August 2025. Mario Zechner's benchmarks circulate everywhere, "MCP is dead, long live the CLI" was on Hacker News for two days, and at this point any senior engineer can quote the schema-overhead numbers from memory. I'll recap it briefly so the pivot has somewhere to stand.

I want to talk about the case the debate has barely covered.

The Settled Part

The case for CLIs over MCP for agentic work, when it applies, is closed. It boils down to four points:

1. Schema overhead. A typical MCP server pushes its full tool catalogue into the agent's context on initialization. The Atlassian MCP server is around 73 tools and roughly 40% of a 200K context budget before the agent does anything. The GitHub MCP server is 93 tools at about 55,000 tokens. CLI commands cost zero up front because the model already knows them.

2. Model familiarity. Frontier models are trained on billions of lines of terminal interaction. They know gh, git, docker, kubectl, jq the way a senior engineer does. No schema needed.

3. Composability. Shell pipes compose in one runtime call. MCP can compose too — the agent chains tool calls in sequence — but each step costs a model turn. A five-step pipeline is five turns; a shell pipe is one.

4. Reliability. The benchmarks consistently show higher task-completion rates with CLI transports for the same model.

This applies cleanly to the cases the prior posts cover: agents calling tools the model already understands. The existing CLI wins; the debate is over.

The case I am writing about is different.

The Case Nobody Is Writing About

What happens when the agent calls a tool the model has no training on?

Frontier models know gh because GitHub Actions logs and Stack Overflow answers are everywhere in the training data. They don't know how to evaluate whether a foreground/background pair is readable for a user with moderate contrast sensitivity loss. They know the WCAG 2 contrast formula because it's documented in 2,000 blog posts. They know critiques of the formula exist. What they don't know is how to act on them: which competing algorithm to trust for which text size, in which context, against which design intent. Picking that requires practitioner judgment the training data doesn't carry.

When I wired CPQI's accessibility back end into an agent loop, I wasn't connecting it to a tool the model already understood. I was injecting expertise the model didn't have.

That changes the question. The benchmarks everyone cites measure how to wrap tools the model already knows. The domain-specific question is about how to deliver new judgment into the agent's loop. The tradeoffs invert.

What Inverts

Discoverability inverts. Bash binaries are reachable by default — the model knows gh is callable because the model knows what gh is. An MCP server is not reachable by default. The model has to be told it exists, told what it does, and reminded to use it during the actual task. CLAUDE.md instructions are a workaround — agents shouldn't need sticky notes. Once I had a CLI binary, the prompt-engineering disappeared. The agent reached for it the way it reaches for git.

The playbook composes; the agent picks. MCP's mental model is a catalog of atomic operations — contrast check, color conversion, gamut probe — each a discrete tool the agent can call. The agent's job in that model is to invent the right combination. A CLI plus an agent playbook flips that. CPQI's playbook does the composing up front; the agent's task changes from inventing a combination of atomic tools to picking the right documented workflow and running it. The judgment about when to use what lives in prose the agent reads. That costs flexibility and buys reliability — and for domain expertise the model lacks, reliability is the trade you want.

The playbook is also the part of the tool that's hardest to reproduce. The algorithm is a number. The CLI is a binary. The playbook is the encoded judgment about how the work gets done — which workflow fits which situation, which order of operations protects the user, which shortcut to refuse. Shipping the binary without the playbook would be like shipping a power tool without the manual.

Capability stack compounds. Not an inversion — the dividend the inversions paid, and the one that surprised me. Once the CLI was reliable, it surfaced the next problem — the existing perceptual contrast algorithms (bPCA, APCA) had license terms ruling out closed-source commercial use, which meant I couldn't ship them inside a CLI I wanted to monetize. So I built OKCA. OKCA inside the CLI then made design-token decisions I couldn't have made with either tool alone. Those decisions made a design system feasible at a velocity I hadn't planned for. The CLI was supposed to be the destination. It turned out to be the first node in a stack where each layer reduced the work of the next.

Receipts

OKCA shipped on npm 2026-03-15. Eight days later, I committed Candor v1.0.0 — a 26-component design system with a published accessibility conformance statement against WCAG 2.1 AA, validated by CPQI CLI against three contrast algorithms in parallel. Five weeks after OKCA, Candor 3.0.0 shipped. I wasn't trying to build a design system this fast — the stack made each step short.

The decision that anchored the system: a three-tier OKCA contrast model for text at 14px and below.

• Reading text: 9.5 regular / 6.5 bold (Tier 1)
• Functional UI: 6.5 / 4.5 (Tier 2)
• Supplementary text whose meaning is also redundantly coded: 4.5 / 4.5 (Tier 3)

Passing OKCA also passes WCAG, so there's no false-pass surface to worry about. I'd been circling that tier system for years. With the CLI calling OKCA in the loop, it took an afternoon. The result is more coherent and more implementable than I could have anticipated, because the agent could test every combination against the algorithm before I committed to a number.

Without the integration, that decision would have been an essay. With it, it's a tool the agent uses to validate every token pair Candor ships.

Four downstream products run on the same stack:

• dev-den v0.4.0 — full Candor adoption, Phosphor icons, button hierarchy enforced from the design system rather than reasserted per app.
• Color Pair Quick Iterator — its own accessibility overhaul ran through the CPQI CLI it spawned. Modal patterns and a screen-reader audit landed in two days.
• whisper-electron v1.1.5 — Candor-driven UI, navy primary CTAs validated against the tier system, native overlay controls.
• The blog you're reading this on — full visual relaunch April 2026; every color pair OKCA-verified before merge.

That's five products on a foundation that didn't exist two months ago.

cpqi-cli and dev-den are still in private development. Release details when they're ready.

The Proprietary Choice

Most "build a CLI for your agent" posts assume the CLI is open source. Mine is not. The choice has a specific origin.

I went looking for an existing perceptual contrast algorithm to bake into the CLI — the one decision that would let the tool make accessibility calls a frontier model can't make on its own. I found two candidates that practitioners take seriously: bPCA and APCA, both Andrew Somers' work. I read the licenses. Both have terms designed to keep the algorithms open — a defensible IP choice. It just didn't fit my goal of shipping a closed, monetizable tool. The choice was to open-source the CLI, license a third-party algorithm in a way that didn't make sense at this stage, or build my own algorithm. I built my own — OKCA, OKLCH-derived and independent of APCA.

Then I split the stack. OKCA shipped on npm under MIT — the algorithm is for the world; the goal there is adoption, eventually a path to W3C / AGWG. The CLI that wraps OKCA stays closed.

• The algorithm earns trust by being open. People can read it, test it, point at false passes, fork it. That's how a contrast algorithm becomes credible.
• The CLI earns trust by being judged on its output. The agentic interface, the editorial verdict layer, the tier guidance, the integration with downstream design tokens — that's a decade of practitioner judgment, and it's the part that compounds.

Open algorithm, closed tool.

What This Opens

A user with moderate contrast sensitivity loss isn't well-represented in the data the model was trained on. The judgment about what works for that user lives in a different population — practitioners who have spent years making accessibility calls on their behalf. The CLI moves that practitioner judgment into the agent's loop. The playbook documents the workflows. OKCA encodes the perceptual call. Each layer is a route by which a perspective the model lacks gets a seat at the agent's reasoning.

The MCP-vs-CLI debate is a transport question. The domain-CLI question is a governance one.

Whose expertise travels through your agent? Who supplies the judgment your tool encodes? Who has authority to change it? Who audits it? When the agent makes an accessibility call on behalf of a user, what's the chain of accountability that ends in a person?

Generic CLIs ducked this question because the expertise was diffuse — the model picked up git from a million GitHub repos, no single author owns it. Domain CLIs surface the question because somebody chose, on purpose, what the tool says.

Zoom out. The model's priors come from a small group of trainers — a narrow slice of the population the model will eventually serve. Tools the agent reaches for can carry perspective the trainer pool didn't. (That holds for MCPs too — the alignment argument doesn't depend on which transport you pick.) That's not the whole alignment problem, not even most of it. But it is a layer where someone outside the trainer pool can put their judgment in the loop, deliberately. The work ahead is uphill.

Close

The answer to "CLI or MCP" depends on what you're moving through the agent.

If the agent needs a tool the model already knows — gh, git, kubectl, jq — let it use that tool directly. Don't wrap it in MCP. The benchmarks back this; the prior posts have it covered.

If the agent needs expertise the model doesn't have — build a domain CLI and write it a playbook. The binary handles invocation; the playbook handles composition.

The architecture isn't new — the case is.

So: whose voice is in the layer you're building? I'd like to hear how you're thinking about it. The most useful thing you could leave me through the about page is a have you considered — something that pushes past where the argument lands today. This is work I'd rather do in conversation than alone.