AI Architect Academy

Architecture notes · build-in-public

What the content↔bundle invariant test caught

In short

Adding a module to this course takes two steps: seed its metadata into the database, and bundle its lesson body into the Worker. A batch of new modules got step one but not step two — so they were seeded but unbundled, and rendered with empty bodies. The quiz still worked, the page still loaded, and the gap slipped straight past manual QA.

The fix wasn't to be more careful. It was a filesystem-driven test that walks every module on disk and asserts each has a non-empty bundled body. A module missing its bundle import now fails npm test instead of shipping blank.

Why a module needs two writes

Content authoring and content serving live on two sides of a boundary. On disk, each module is a folder under content/<track>/<module>/ with lesson.json, objectives.json, and items.json. A seed build (scripts/build-seed.mjs) discovers those folders and emits the rows that land in D1. That's step one — it puts the module's metadata in the database.

But the lesson body the learner reads is served from a static import list compiled into the Worker (src/lib/content.ts), looked up by getLessonBody(id). If a module is seeded but its lesson.json was never added to that import list, getLessonBody returns null and the body renders empty. Two writes, two places, one easy-to-forget second step.

How it actually went wrong

The history shows it plainly. A commit bundling lesson bodies "for all 34 new modules" describes the exact failure: the static-import list "stopped at trackD m6 + trackG, so the newly seeded Track B/C/D/E/F modules rendered with an empty body." Thirty-four modules had been authored and seeded; their bodies weren't in the bundle. Every one of them would have served a blank lesson.

Why manual QA missed it
A seeded-but-unbundled module isn't obviously broken. The catalog lists it, the page route resolves, the objectives and the quiz come from the database and work fine. Only the prose is missing. Unless a reviewer opens that specific module and scrolls to where the body should be, nothing looks wrong. The defect is invisible exactly where a human spot-check doesn't look.

The test that closed the gap

The guard is deliberately not a hard-coded list of modules — that would just be a third place to forget to update. Instead the test reads the content/ directory the same way the seed builder does, discovering every module that has all three required files, then asserts each one has a non-empty bundled body:

// discover every module the seed pipeline would emit, from disk
const seeded = discoverSeededModules();

it.each(seeded)('bundles a non-empty lesson body for $id', ({ id, label }) => {
  const body = getLessonBody(id);
  expect(body,
    `${label}: getLessonBody('${id}') returned null` +
    ` — add its import to src/lib/content.ts`).not.toBeNull();
  expect(body!.blocks.length).toBeGreaterThan(0);
});

Because the test source-of-truth is the filesystem, a newly authored module is covered with no test edits: drop the folder in, and if you forget to bundle it, the test fails — and the failure message tells you the exact fix ("add its import to src/lib/content.ts"). The test file says so in its own comment: the bug it guards is "a module seeded into D1 but missing from content.ts's static import list," which "renders with an EMPTY body … so it slips past manual QA. This makes the gap a failing test instead — part of the Definition of Done."

The principle: make the invisible failure loud

The real lesson here isn't about content bundling specifically. It's that the dangerous bugs are the ones that don't announce themselves — the page loads, the test suite is green, and a whole subsystem is quietly empty. The durable fix for that class of bug is to encode the invariant ("every seeded module has a body") as a check that can't be skipped, and to derive the check from the same source of truth the system uses, so it can't drift. The same instinct shows up elsewhere in the repo: the public health endpoint was enriched to report per-module hasBody so a production deploy can be verified to have actually landed content, not just metadata.

This is the project's premise in miniature. "The build is the curriculum" — and the build keeps surfacing the exact engineering judgement the course teaches: a process with two manual steps will eventually run one of them, so make the second one's omission impossible to ship silently.

This is the senior skill the course is about.

Spotting the silent-failure class of bug and encoding invariants that can't drift is the kind of judgement the Academy builds — taught on the same Anthropic, AWS, and Cloudflare stack this platform runs on.

Provenance — drawn entirely from this repo
  • test/content.test.tsdiscoverSeededModules() (filesystem-driven), the per-module non-empty-body assertion, and the comment describing the seeded-but-unbundled bug it guards.
  • src/lib/content.tsgetLessonBody(id) and the static lesson-body import list.
  • scripts/build-seed.mjs — discovers content/<track>/<module>/ folders and emits the D1 seed; the test mirrors its discovery.
  • Commit b525922 "fix(content): bundle lesson bodies for all 34 new modules" — "the static-import list stopped at trackD m6 + trackG, so the newly seeded … modules rendered with an empty body."
  • Commit e817ce3 "docs(content): note new modules need both seed AND content.ts bundle + deploy."
  • Commit 626d199 "test+tooling: make end-to-end verification possible without Clerk creds" — added the filesystem-driven test and enriched GET /api/health with per-module hasBody.
  • Commit 1c8b301 "feat(progression): …" — earlier content-bundling work the import list extended from.

Build-in-public note, grounded entirely in this repository. Spot a mistake? hello@aiarch.dev.