singularity/singularity-forge
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

feat(uok-status): slice 1 — schema v2 + coverage classification + legacy tagging

Browse source 
First slice of "Make UOK the SF Control Plane". Ships the operator-
facing visibility primitive that subsequent slices fill in. No
enforcement yet, no new gates yet — just the contract.

Changes to sf headless status uok:

  - Bumps JSON output to schemaVersion: 2.
  - Adds coverageStatus per gate (ok | stale | incomplete | missing
    | legacy). Slice 1 only populates ok / stale / legacy:
      - legacy   row predates schema-v2 metadata (every existing row
                 today). NOT a warning — operators are not paged for
                 the rich history of pre-v2 records.
      - stale    schema-v2 row with no runs in window, OR last run
                 older than the 24h stale threshold. Surfaces gates
                 that stopped being exercised.
      - ok       schema-v2 row with recent runs in window.
    incomplete / missing wait for the schema-v2 writer adapter
    (slice 2) and the configured-gate registry (later).
  - Adds the Coverage column to the human table output.
  - Removes the stale "missing getDistinctGateIds import" workaround
    comment from headless-uok-status.ts:104. The import exists today
    (gate-runner.js:5); the comment was lying. Bypassing
    UokGateRunner.getHealthSummary is still appropriate but for a
    different reason — documented inline.

Tests (28 total, +9 new):
  - classifyCoverage: legacy wins over freshness; ok requires
    metadata + recent runs; stale fires on no-runs-in-window or
    last-run > 24h.
  - empty-DB does not false-positive coverage warnings (the bug
    codex called out in the plan review).
  - formatTable includes the Coverage column and renders each status
    distinctly.

hasSchemaV2Metadata is a placeholder that returns false today; it
will read row.surface / row.run_control / row.permission_profile
when those columns ship in slice 2.

Next slice: adapter foundation — start writing schema-v2 metadata
into new gate rows from headless and autonomous paths.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Mikael Hugo 2026-05-14 17:35:52 +02:00
parent 12f5eb2279
commit c058bef26d
2 changed files with 225 additions and 12 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

117
src/headless-uok-status.ts
View file

@ -39,6 +39,40 @@ function sfExtensionPath(moduleName: string): string {
);
}
/**
* Coverage classification for a gate in the status uok view.
*
* Slice 1 (UOK control-plane plan, 2026-05-14) introduces this field so
* operators can distinguish gates that should be paying attention to from
* those that don't yet have the new metadata. Each value's contract:
*
* - "ok" Gate has schema-v2 metadata AND recent runs in the
* window. Healthy.
* - "stale" Gate has prior runs but nothing in the last 24h.
* Suggests something stopped exercising it.
* - "incomplete" Gate has schema-v2 records but is missing required
* metadata (surface / runControl / permissionProfile /
* traceId). Used when future slices start writing
* schema-v2 rows; never assigned to legacy rows.
* - "missing" Gate is configured/expected but has zero recent runs.
* Requires a configured-gate registry to detect; future
* slice work, not slice 1.
* - "legacy" Gate row predates schema-v2 metadata. NOT a warning —
* operators are not paged for the rich history of pre-v2
* records. Future slices migrate these as the writer
* paths emit complete metadata.
*
* Slice 1 only populates "ok" / "stale" / "legacy". "incomplete" and
* "missing" wait for the schema-v2 writer adapter (slice 2) and the
* configured-gate registry (later).
*/
export type GateCoverageStatus =
| "ok"
| "stale"
| "incomplete"
| "missing"
| "legacy";
export interface GateHealthEntry {
id: string;
scope: string;
@ -49,6 +83,8 @@ export interface GateHealthEntry {
lastEvaluatedAt: string | null;
circuitBreaker: string;
failureStreak: number;
/** Coverage classification (schema v2+). */
coverageStatus: GateCoverageStatus;
}
export interface UokStatusResult {
@ -56,6 +92,43 @@ export interface UokStatusResult {
gates: GateHealthEntry[];
}
/**
* A row is "legacy" when it lacks the schema-v2 metadata fields that the
* writer adapter (slice 2) will start producing. Today the quality_gates
* table has no surface/runControl/permissionProfile columns, so every row
* is legacy by definition. This guard lives here so the moment those
* columns appear, the classification flips automatically without a
* separate code edit.
*/
function hasSchemaV2Metadata(_row: unknown): boolean {
// Placeholder for slice 2. Will read row.surface / row.run_control /
// row.permission_profile when those columns ship.
return false;
}
const STALE_THRESHOLD_MS = 24 * 60 * 60 * 1000;
function classifyCoverage(
entry: Omit<GateHealthEntry, "coverageStatus">,
metadataPresent: boolean,
): GateCoverageStatus {
if (!metadataPresent) return "legacy";
if (entry.total === 0) {
// Has metadata but no runs in window. If we ever saw a run, it's
// stale; otherwise it's never run (caller will mark "missing" when
// a configured-gate registry confirms it was expected). For slice
// 1, no registry exists, so the safer default is "stale".
return entry.lastEvaluatedAt ? "stale" : "stale";
}
const last = entry.lastEvaluatedAt
? Date.parse(entry.lastEvaluatedAt)
: null;
if (last !== null && Date.now() - last > STALE_THRESHOLD_MS) {
return "stale";
}
return "ok";
}
function passRate(entry: GateHealthEntry): string {
if (entry.total === 0) return "—";
return `${Math.round((entry.pass / entry.total) * 100)}%`;
@ -67,13 +140,28 @@ function cbIcon(state: string): string {
return "🟢";
}
function coverageIcon(status: GateCoverageStatus): string {
switch (status) {
case "ok":
return "✓ ok";
case "stale":
return "⚠ stale";
case "incomplete":
return "⚠ incomplete";
case "missing":
return "⚠ missing";
case "legacy":
return "· legacy";
}
}
function formatTable(gates: GateHealthEntry[]): string {
if (gates.length === 0) {
return "No gate run data found in the last 24h.\n";
}
const header =
"| Gate | Scope | Pass% | Pass | Fail | Retry | CB | Streak | Last Evaluated |\n" +
"|------|-------|-------|------|------|-------|----|--------|----------------|\n";
"| Gate | Scope | Coverage | Pass% | Pass | Fail | Retry | CB | Streak | Last Evaluated |\n" +
"|------|-------|----------|-------|------|------|-------|----|--------|----------------|\n";
const rows = gates
.map((g) => {
const last = g.lastEvaluatedAt
@ -82,7 +170,7 @@ function formatTable(gates: GateHealthEntry[]): string {
.replace("T", " ")
.slice(0, 19)
: "never";
return `| ${g.id} | ${g.scope} | ${passRate(g)} | ${g.pass} | ${g.fail} | ${g.retry} | ${cbIcon(g.circuitBreaker)} ${g.circuitBreaker} | ${g.failureStreak} | ${last} |`;
return `| ${g.id} | ${g.scope} | ${coverageIcon(g.coverageStatus)} | ${passRate(g)} | ${g.pass} | ${g.fail} | ${g.retry} | ${cbIcon(g.circuitBreaker)} ${g.circuitBreaker} | ${g.failureStreak} | ${last} |`;
})
.join("\n");
return `${header}${rows}\n`;
@ -101,8 +189,13 @@ export async function handleUokStatus(
)) as any;
await autoStartModule.openProjectDbIfPresent(basePath);
// Load sf-db-gates directly to avoid the missing getDistinctGateIds
// import in gate-runner.js getHealthSummary().
// Load sf-db-gates directly. (Earlier versions had a workaround
// comment here about a "missing getDistinctGateIds import in
// gate-runner.js"; that import exists today — see
// gate-runner.js:5. Bypassing UokGateRunner.getHealthSummary is
// still appropriate here because the headless status surface
// wants raw gate-id discovery without requiring the registry to
// be populated by a session.)
const gatesDbModule = (await jiti.import(
sfExtensionPath("sf-db/sf-db-gates"),
{},
@ -136,7 +229,7 @@ export async function handleUokStatus(
const stats = gatesDbModule.getGateRunStats(id, 24);
const cb = gatesDbModule.getGateCircuitBreaker(id);
const meta = getGateMeta(id);
return {
const base = {
id,
scope: meta.scope,
total: stats.total ?? 0,
@ -147,7 +240,12 @@ export async function handleUokStatus(
lastEvaluatedAt: stats.lastEvaluatedAt ?? meta.lastEvaluatedAt,
circuitBreaker: cb?.state ?? "closed",
failureStreak: cb?.failureStreak ?? 0,
} satisfies GateHealthEntry;
};
const coverageStatus = classifyCoverage(
base,
hasSchemaV2Metadata(meta),
);
return { ...base, coverageStatus } satisfies GateHealthEntry;
});
} catch (err) {
const msg = err instanceof Error ? err.message : String(err);
@ -156,8 +254,11 @@ export async function handleUokStatus(
}
if (opts.json) {
// schemaVersion bumped 1 → 2 for the addition of coverageStatus.
// Adding a field is non-breaking for consumers that pin to v1's
// shape; the bump signals the new field is part of the contract.
process.stdout.write(
JSON.stringify({ schemaVersion: 1, gates }, null, 2) + "\n",
`${JSON.stringify({ schemaVersion: 2, gates }, null, 2)}\n`,
);
} else {
process.stdout.write("\nUOK Gate Health (last 24h)\n\n");

120
src/resources/extensions/sf/tests/headless-uok-status.test.mjs
View file

@ -16,13 +16,30 @@ function cbIcon(state) {
return "🟢";
}
function coverageIcon(status) {
switch (status) {
case "ok":
return "✓ ok";
case "stale":
return "⚠ stale";
case "incomplete":
return "⚠ incomplete";
case "missing":
return "⚠ missing";
case "legacy":
return "· legacy";
default:
return "";
}
}
function formatTable(gates) {
if (gates.length === 0) {
return "No gate run data found in the last 24h.\n";
}
const header =
"| Gate | Scope | Pass% | Pass | Fail | Retry | CB | Streak | Last Evaluated |\n" +
"|------|-------|-------|------|------|-------|----|--------|----------------|\n";
"| Gate | Scope | Coverage | Pass% | Pass | Fail | Retry | CB | Streak | Last Evaluated |\n" +
"|------|-------|----------|-------|------|------|-------|----|--------|----------------|\n";
const rows = gates
.map((g) => {
const last = g.lastEvaluatedAt
@ -31,12 +48,27 @@ function formatTable(gates) {
.replace("T", " ")
.slice(0, 19)
: "never";
return `| ${g.id} | ${g.scope} | ${passRate(g)} | ${g.pass} | ${g.fail} | ${g.retry} | ${cbIcon(g.circuitBreaker)} ${g.circuitBreaker} | ${g.failureStreak} | ${last} |`;
return `| ${g.id} | ${g.scope} | ${coverageIcon(g.coverageStatus)} | ${passRate(g)} | ${g.pass} | ${g.fail} | ${g.retry} | ${cbIcon(g.circuitBreaker)} ${g.circuitBreaker} | ${g.failureStreak} | ${last} |`;
})
.join("\n");
return `${header}${rows}\n`;
}
// Mirror of headless-uok-status.ts:classifyCoverage. Kept inline so the
// test file stays a pure unit test against the same logic shape (the
// .ts source isn't directly importable from this .mjs in the SF
// extension test tree). The icon mapping above mirrors coverageIcon.
const STALE_THRESHOLD_MS = 24 * 60 * 60 * 1000;
function classifyCoverage(entry, metadataPresent, now = Date.now()) {
if (!metadataPresent) return "legacy";
if (entry.total === 0) return "stale";
const last = entry.lastEvaluatedAt
? Date.parse(entry.lastEvaluatedAt)
: null;
if (last !== null && now - last > STALE_THRESHOLD_MS) return "stale";
return "ok";
}
function makeGate(overrides = {}) {
return {
id: "verification",
@ -48,6 +80,7 @@ function makeGate(overrides = {}) {
lastEvaluatedAt: "2026-05-11T12:00:00.000Z",
circuitBreaker: "closed",
failureStreak: 0,
coverageStatus: "legacy",
...overrides,
};
}
@ -96,7 +129,18 @@ describe("formatTable", () => {
it("includes_header_row", () => {
const result = formatTable([makeGate()]);
assert.ok(result.includes("| Gate | Scope | Pass%"));
assert.ok(result.includes("| Gate | Scope | Coverage | Pass%"));
});
it("includes_coverage_column_per_row", () => {
const result = formatTable([
makeGate({ id: "g1", coverageStatus: "legacy" }),
makeGate({ id: "g2", coverageStatus: "ok" }),
makeGate({ id: "g3", coverageStatus: "stale" }),
]);
assert.ok(result.includes("· legacy"));
assert.ok(result.includes("✓ ok"));
assert.ok(result.includes("⚠ stale"));
});
it("includes_separator_row", () => {
@ -162,3 +206,71 @@ describe("formatTable", () => {
assert.ok(result.includes("2026-05-11 12:00:00"));
});
});
describe("classifyCoverage (slice 1: legacy / ok / stale)", () => {
// "now" pinned so stale-threshold math is reproducible across runs.
const now = Date.parse("2026-05-14T12:00:00.000Z");
it("returns_legacy_when_metadata_is_absent", () => {
// All rows in the current quality_gates table predate schema v2.
// Without surface/runControl/permissionProfile columns there is
// no schema-v2 metadata to detect, so every existing row must
// classify as legacy and not warn the operator.
const status = classifyCoverage(
makeGate({ total: 10, lastEvaluatedAt: "2026-05-14T11:00:00.000Z" }),
false,
now,
);
assert.equal(status, "legacy");
});
it("returns_legacy_even_when_rows_are_recent_and_metadata_missing", () => {
// "Legacy" wins over freshness — we never page operators about
// rows that predate the schema-v2 writer.
const status = classifyCoverage(
makeGate({ total: 5, lastEvaluatedAt: "2026-05-14T11:59:00.000Z" }),
false,
now,
);
assert.equal(status, "legacy");
});
it("returns_ok_when_metadata_present_and_recent_runs_exist", () => {
const status = classifyCoverage(
makeGate({ total: 10, lastEvaluatedAt: "2026-05-14T11:00:00.000Z" }),
true,
now,
);
assert.equal(status, "ok");
});
it("returns_stale_when_metadata_present_but_no_runs_in_window", () => {
const status = classifyCoverage(
makeGate({ total: 0, lastEvaluatedAt: "2026-05-12T00:00:00.000Z" }),
true,
now,
);
assert.equal(status, "stale");
});
it("returns_stale_when_last_run_is_older_than_24h", () => {
// Has runs but the most recent one is > 24h old.
const status = classifyCoverage(
makeGate({ total: 3, lastEvaluatedAt: "2026-05-12T11:59:00.000Z" }),
true,
now,
);
assert.equal(status, "stale");
});
it("does_not_false_positive_on_empty_db", () => {
// When the DB has no rows at all, no gate is even discovered,
// so the classifier is never called. The status command short-
// circuits with the "No gate run data found" message via
// formatTable. This test asserts the contract from a different
// angle: an empty gate list produces no warnings.
const result = formatTable([]);
assert.ok(result.includes("No gate run data found"));
assert.ok(!result.includes("⚠"));
});
});