We Split One Skill Into 14 Files. Then We Raced Them.

🤔 The Question Nobody Could Answer

One-Shot Beta is a 400-line execution protocol spread across 4 files. It works. It ships code that passes its own rubric 92% of the time.

But what if each phase lived in its own file? Would the AI follow instructions better when they arrive one script at a time — or worse?

We had strong opinions on both sides. So we stopped arguing and ran the experiment.

⚙️ The Setup

Contestant A: The Monolith

/one-shot-beta — the production skill. Four files: SKILL.md (the orchestrator), execution-protocol.md, scoring-and-assessment.md, and delivery-format.md. All phases live together in one file.

Contestant B: The Modular Version

/one-shot-scripts — same protocol, decomposed into 14 standalone scripts. One file per phase, loaded on demand. The orchestrator is a slim routing document: “Read scripts/phase-2-build.md and execute.”

🔬 The Protocol (and the Round We Threw Out)

We actually ran 4 rounds. The first one got scrapped.

During Round 1’s second session, one of the skills found the safe-parse directory already written by the first session’s run — and used it. Instead of building from scratch, it read the existing code, “improved” it, and claimed credit. A contaminated test.

Rounds 1–3 below used isolated directories: safe-parse-a-monolith/ and safe-parse-b-modular/. No cross-contamination.

Each valid round followed the same process:

Control variables: same prompt (word for word), fresh sessions (no prior context), same model (Opus 4.6), same rubric (8 dimensions, 0.03 significance threshold).

1️⃣ Round 1: Small — safe-parse Utility

The task: Build a Node.js module with 3 parse functions (safeJsonParse, safeIntParse, safeDateParse), a full test suite, and TypeScript definitions. No dependencies.

The gap: B produced 75 tests vs A’s 36. B had a 1MB JSON size limit, epoch range validation, structured ParseError with source tracking, and readonly on all result fields. A had none of that — including no size limit on a library called “safe-parse.”

2️⃣ Round 2: Medium — Rate Limiter Middleware

The task: Express middleware with fixed window and sliding window algorithms, per-IP and per-API-key limiting, 429 responses with Retry-After headers, in-memory store, and concurrent request tests.

A’s edge: 67% faster (4m37s vs 7m44s) and upgraded its mutex from spin-wait to FIFO queue during hardening. Genuine runtime improvement.

B’s edge: 46 tests vs A’s 23. Constructor validation with RangeError (A silently accepted capacity=0). Dedicated adversarial test suite with null bytes, XSS payloads, and special characters. A had zero adversarial tests.

3️⃣ Round 3: Large — Markdown Link Checker CLI

The task: CLI tool that validates markdown files for broken links, checks image references, validates YAML frontmatter, supports glob patterns, has a --fix mode, and outputs in multiple formats. Full test suite with fixtures.

This round broke it open. Output A had zero unit tests. No test framework. No test files. No test script in package.json. Only manual CLI verification against fixtures. It self-scored testing at 0.88, claiming this was “acceptable for a CLI tool of this scope.”

Output B produced 81 tests across 6 suites — parser, checker, fixer, scanner, CLI args, and config loading — plus adversarial inputs for null bytes, deeply nested brackets, and 10,000-character URLs. All passing.

📊 The Cumulative Scorecard

CUMULATIVE: Monolith 0 — 3 Modular

Round 1 (Small):  B wins +0.15  (0.82 vs 0.97)
Round 2 (Medium): B wins +0.09  (0.80 vs 0.89)
Round 3 (Large):  B wins +0.21  (0.70 vs 0.91)

Four patterns held across every round:

💡 Why Does the Same Content Perform Differently?

The instructions are identical. Word for word. So why does the format matter?

When all phases live in one file, Claude can skim. It reads the full protocol, forms a plan, and executes from memory. Phases it considers “less important” for the task — like adversarial testing on a CLI tool — get silently dropped.

When each phase is a separate file, Claude has to explicitly load it. The act of reading scripts/phase-3-test.md puts that phase’s full checklist into working memory right when it’s needed. Harder to skip what you just read.

⚖️ The Tradeoff Is Real

Modular isn’t strictly better. There’s a genuine cost:

If you need fast iteration and plan to review the output yourself, the monolith’s speed advantage matters. If you need to ship what it produces, the modular version’s verification pays for itself.

🧰 What This Means for Skill Authors

If your skill has phases, steps, or checklists that must all execute — split them into separate files. The file boundary acts as a natural attention gate. Same content, better follow-through.

Four concrete rules from this experiment:

1. One concern per file. Testing instructions in one file, security review in another. Don’t bundle them.
2. The orchestrator stays slim. It routes to scripts. It doesn’t contain instructions itself. If the orchestrator has checklists, they’ll get skimmed.
3. Accept the speed cost. More file reads means slower execution. That’s the price of reliability.
4. Verify your own skill’s self-scoring. Both skills inflated their scores. Build an independent grader. Trust the grader, not the skill.