Two-stage validation: let the schema accept drafts, let the gate decide URLs

A content schema with required fields creates a dilemma you cannot design your way out of with one validation step.

Make the demanding fields required, and half-finished research cannot be committed. It lives in a document, or a spreadsheet, or nowhere. You lose the ability to review it, to see it in a diff, to have two people work on it.

Make them optional, and half-finished records publish. You get exactly the thin page the required fields existed to prevent.

Teams generally pick the second and add a rule that people are supposed to remember.

Two steps instead

Split validation by what each step is actually for.

Stage one is the schema, and its job is to accept drafts. Every field that can only be written by someone who did first-hand work is optional here. A competitor record with a name, a homepage and a sitemap analysis parses fine. It can be committed, reviewed, and left alone for three weeks.

Stage two is the publication gate, and its job is to decide whether a record has earned a URL. It runs in the route’s path generation, so a record that does not pass produces no page at all.

The record moves between these states on its own timeline. Nothing is lost and nothing leaks.

What this buys

Research becomes reviewable. A pull request adding eleven competitor records with sitemap data and no hands-on evaluation is a useful pull request. It is the groundwork. It just is not eleven pages.

The bar cannot drift. With one validation step, the pressure to publish shows up as pressure to relax a required field, and relaxing it affects everything retroactively. With two, the pressure has nowhere to go except into doing the work, or into an explicit commit that changes the gate.

The system can explain itself. Because the gate is a function that returns what is missing rather than a boolean, we can ask it. It says things like outputTranscriptExcerpt — real narration from a real run and failureModes — at least 2, observed not inferred. That is a work queue, generated from the actual state of the data.

The part that took a second attempt

Our first version had the gate check status === 'published' and nothing else, with the real requirements in the schema. That is one-stage validation wearing a two-stage costume: the schema still refused drafts, and the status field was decoration.

The rule we settled on is that the schema may only assert things about shape — this is a string, it is between 40 and 200 characters, it references a real source. The gate asserts things about completeness and evidence. Shape is knowable when the record is created. Evidence is knowable only after someone has gone and got it.

Keeping that line clean is what makes the two stages genuinely different rather than the same check in two places.