Our Free Skill Failed Its Own Audit
🔎 The setup: We audit code for a living. This time we pointed the process at
godmode-lite, the free skill anyone can download from us.💥 The worst find: Only 1 of its 4 scanning commands refused to run on your home directory. One wrong starting folder and its polish step hands your entire home folder to code formatters that rewrite files.
🧾 The count: 15 findings. 3 severe, 5 medium, 7 low. Every one reproduced live before we believed it.
✅ The fix: v2.4.0 closes all 15, re-proved with 11 live-fire checks before the download was rebuilt.
godmode-lite is our free tier: a small, deterministic Node runner that walks Claude through four layers of discipline. Discover the code, finish the work, run the tests, tidy up.
It is also the version running on machines we know nothing about. That makes it the one with the least room for a nasty surprise, so we gave it the same treatment we give everything else: a full adversarial audit.
Why audit the thing we give away?
A paying customer who hits a bug files a report. A free user who hits a bug just deletes the folder, and the lesson leaves with them.
The runner also feels safe in a way that invites complacency. It is a zero-dependency CLI that does the same thing every time, but deterministic only means it makes the same mistake every time.
1 · Discover
Scans the project and ranks the files related to your task, so nothing gets edited unread.
2 · Check
Scans the files the run touched for stubs, TODOs, and placeholder code before tests are allowed.
3 · Test
Detects the project's test runner and executes the real suite, recording the exit code.
4 · Polish
Detects the project's formatter and linter and runs them. This is the layer that writes to disk.
Three of those layers only read. One of them rewrites files, which is exactly where the audit found the scariest thing.
The finding that could eat a home folder
Every command takes an optional path and falls back to wherever the run started. The discover command refused to scan your home directory or a drive root, and we remembered that guard as covering everything.
It covered one command out of four. check, test, and polish accepted any root you handed them, and polish is the one that runs prettier --write ., eslint --fix ., black ., and gofmt -w . across the whole tree.
Start a run from your home directory and layer 4 becomes four different formatters sweeping every project, config, and stray script you own. Not malicious. Worse: helpful.
Think of it like hiring a cleaning crew for one room. Because nobody wrote down which room, they got keys to the entire house and tidied every drawer they could reach. Nothing is stolen, but everything has been rearranged.
Nobody got hurt: the guard held on our machines and no user report exists. But the audit's job is the sentence before "we got lucky."
The gate that waved runs through
The runner's last command, end, is supposed to be the bouncer. It checks the gates and only reports a clean finish when the protocol was actually satisfied.
The old logic only failed a run on evidence that something went wrong: markers still present, a recorded failing test. The absence of evidence counted as innocence, so a run that skipped tests entirely ended with a clean bill of health.
A second bug stacked on top: end trusted the most recent completeness scan. Any code written after that clean scan was never looked at again.
It was a security guard signing the patrol sheet at dawn because no break-in was reported overnight, without checking whether anyone had actually patrolled. The new gate reads the patrol log first, then walks the halls one more time itself.
Here is that gate as a toy you can poke. Toggle which layers ran, then compare what each version of the skill tells you.
click a layer to toggle whether it ran:
With tests skipped, v2.3.1 stamps the run clean. v2.4.0 refuses, and it also distinguishes hard failures from warnings: a project with no test suite at all can still finish, it just finishes loudly.
The stale-scan bug died in the same release. end now re-runs the completeness scan itself, so code written after the last clean check still gets caught.
The test suite that didn't exist
Finding three is the one most likely to bite a beginner. When npm scaffolds a new project, it writes a placeholder test script into package.json. This one:
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
}The old detector saw scripts.test and concluded the project had a real test suite. Running it fails by design, so the run recorded a permanent test failure, and the skill's advice made it worse: "fix the failures, do not delete the failing tests."
There were no tests. There were no failures. The loop had no exit.
↓
🚫 placeholder → skip it, keep looking
↓
📦 test framework in dependencies? (vitest, jest, mocha)
↓
🔍 other ecosystems? (pytest, go test, cargo test, rspec)
↓
⚙️ each candidate checked: is the binary actually installed?
↓
📢 nothing usable → report "no_runner" honestly: write tests, then re-run
The same family of fixes covered the neighbors we found on the way. A configured-but-not-installed pytest no longer counts as a runner, and pytest's "I ran but collected zero tests" exit code now reads as "write tests" instead of "tests failed."
What did the audit actually find?
The audit of godmode-lite produced 15 findings: 3 severe, 5 medium, and 7 low. The severe three were the home-folder formatter sweep, an end gate that passed runs whose tests never ran, and npm's placeholder script detected as a real test suite. Version 2.4.0 fixes all 15.
| ID | Severity | Finding | In v2.4.0 |
|---|---|---|---|
| H1 | High | Only 1 of 4 scanning commands refused the home directory; polish would format whatever tree it was pointed at | All 4 commands refuse home and filesystem roots |
| H2 | High | end reported a clean pass when tests never ran | Skipped check or test now fails the gate |
| H3 | High | npm's placeholder test script detected as a real suite, causing a permanent false failure | Placeholder skipped; detection falls through to real runners |
| M1 | Medium | Two runs in different projects clobbered each other's shared state pointer | Per-project map; concurrent runs stay isolated |
| M2 | Medium | Code written after a clean scan was never rescanned | end re-runs the completeness scan itself |
| M3 | Medium | Polish formatted the whole tree while check judged only the run's own files | Polish scoped to run-modified files, capped at 40 |
| M4 | Medium | The file walk silently stopped at 4,000 files, so big repos got partial scans without warning | Truncation is flagged in the result and the narration |
| M5 | Medium | Tools were "detected" that weren't installed, producing confusing spawn errors | Every detection branch checks the binary exists first |
| L1–L7 | Low | Seven smaller issues: state mismatches in refusal messages, duplicate keywords, stale help text, four hard-coded version strings, under-described consent copy, an unbounded runs folder, stack traces in user output | All seven fixed |
Proof over reading
None of those findings came from reading the code and nodding gravely. Each one was fired at a scratch project until the bad behavior actually appeared on disk, because a finding you can't reproduce is an opinion.
↓
🔫 fire the real CLI command at it
↓
📨 read the JSON envelope the runner emits
↓
💾 read what actually changed on disk
↓
✅ only then: believe the finding, and later, believe the fix
The fixes earned their way back in the same way, with 11 live-fire checks. Two concurrent runs kept their own state. A back-dated ugly file survived polish untouched while the file the run created came back formatted.
Do
Reproduce a finding before you report it, and re-fire the exact same shot after the fix. The bug and its funeral should use the same test.
Don't
Audit by code-reading alone, and don't verify a fix "by inspection." Both of the gate bugs looked fine in the source and only showed up when a real run ended.
What shipped in v2.4.0
A receipt, not a brag. This is the free skill, so the last line matters most.
| Metric | Value |
|---|---|
| Skill | godmode-lite (free tier) |
| Version | 2.3.1 → 2.4.0 |
| Findings fixed | 15 (3 high, 5 medium, 7 low) |
| New guards | 4 commands refuse home + filesystem roots |
| Polish blast radius | whole tree → run-modified files only (cap 40) |
| Live re-verification | 11 scenario checks, all green |
| Price | still $0 |
The honest notes: this is not a sandbox. The runner now refuses the two catastrophic roots and scopes its writes to your diff, but guardrails are not guarantees, and the 4,000-file walk cap still exists; it just tells you when it binds instead of staying silent.
The free tier gets the paid tier's audit because the free tier is the pitch. If the thing we hand out for $0 quietly holds a loaded formatter, nothing we charge for deserves trust. This is the third self-audit we've published, after the tool that vanished and the quality-loop audit. Same rule every time: prove it live, fix it, prove the fix.
Run the hardened free skill
godmode-lite v2.4.0 drives Claude Code through discover, check, test, and polish, and now it refuses to finish a run that skipped the hard parts.
Get Godmode Lite free See the full 8-layer version