feat(detectors): wire gate-deadlock-classifier into the autonomous loop
Some checks are pending
CI / detect-changes (push) Waiting to run
CI / docs-check (push) Blocked by required conditions
CI / lint (push) Blocked by required conditions
CI / build (push) Blocked by required conditions
CI / integration-tests (push) Blocked by required conditions
CI / windows-portability (push) Blocked by required conditions
CI / rtk-portability (linux, blacksmith-4vcpu-ubuntu-2404) (push) Blocked by required conditions
CI / rtk-portability (macos, macos-15) (push) Blocked by required conditions
CI / rtk-portability (windows, blacksmith-4vcpu-windows-2025) (push) Blocked by required conditions
Some checks are pending
CI / detect-changes (push) Waiting to run
CI / docs-check (push) Blocked by required conditions
CI / lint (push) Blocked by required conditions
CI / build (push) Blocked by required conditions
CI / integration-tests (push) Blocked by required conditions
CI / windows-portability (push) Blocked by required conditions
CI / rtk-portability (linux, blacksmith-4vcpu-ubuntu-2404) (push) Blocked by required conditions
CI / rtk-portability (macos, macos-15) (push) Blocked by required conditions
CI / rtk-portability (windows, blacksmith-4vcpu-windows-2025) (push) Blocked by required conditions
Three changes that close the gap between the gate-deadlock-classifier
landed in ab2c99686 and a working detection signal.
(1) Detector wrapper now returns outcome=manual-attention (not fail) when
a deadlock fires. The whole point of detecting the deadlock is to
escape it — returning `fail` would add another refusal and compound
the lockout. Same precedent as periodicDetectorSweepGate.
(2) New auto/gate-refusal-recorder.js — in-process ring buffer (cap 32,
TTL 30 min) that records UokGate refusals from the dispatcher.
Storage is intentionally in-memory; refusals are operational signals,
not durable state.
(3) auto/run-unit.js — calls recordGateRefusal() at the inline-route-refused
branch, passing the rationale (already includes `[gate-id]` prefix +
R-id status fragments the detector parses) plus unitType/unitId.
(4) detectors/periodic-runner.js — adds a `gate-deadlock` entry to the
default detector list, pulling ctx.gateRefusals from the caller OR
falling back to recentGateRefusals() from the recorder. ctx can also
override requirementCoverageByMilestone + resolveMilestoneId for tests.
After this change, an inline-route refusal flows:
inlineRuntimeGate.execute → outcome=fail
→ run-unit.js records the refusal in gate-refusal-recorder
→ periodic-runner sweep picks it up via recentGateRefusals()
→ detectGateDeadlock cross-references against milestone coverage
→ if overlap: detectorsFired includes {name:"gate-deadlock", signature}
→ periodicDetectorSweepGate surfaces as manual-attention
Tests: 16 detector + 10 existing periodic-runner = 26/26 pass. The
existing periodic-runner test exercises the default detector list, so
adding the new entry is implicitly validated.
Follow-up still open: have the periodic sweep file a self_feedback entry
when the gate-deadlock detector fires, so the operator and SF's autonomous
triage both see the signal without polling logs. That belongs in the
sweep handler, not the detector — separate commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
ab2c996866
commit
a3469f2334
5 changed files with 109 additions and 16 deletions
70
src/resources/extensions/sf/auto/gate-refusal-recorder.js
Normal file
70
src/resources/extensions/sf/auto/gate-refusal-recorder.js
Normal file
|
|
@ -0,0 +1,70 @@
|
|||
/**
|
||||
* gate-refusal-recorder.js — in-process ring buffer of recent UokGate refusals.
|
||||
*
|
||||
* Purpose: collect gate-refusal events from the dispatcher (run-unit.js inline
|
||||
* route refusal + future gate-refusal sites) so the gate-deadlock-classifier
|
||||
* Wiggums detector has something to look at without each detector site needing
|
||||
* to scrape rationale text from disparate places.
|
||||
*
|
||||
* Consumer: src/resources/extensions/sf/auto/run-unit.js (recordGateRefusal on
|
||||
* inline-route-refused) and src/resources/extensions/sf/detectors/periodic-runner.js
|
||||
* (recentGateRefusals as ctx input).
|
||||
*
|
||||
* Storage is intentionally in-memory; gate refusals are operational signals,
|
||||
* not durable state. The dispatcher restarts (auto-reload, crash) reset the
|
||||
* buffer — that's fine, the detector fires from the live signal, not history.
|
||||
*/
|
||||
|
||||
const MAX_REFUSALS = 32;
|
||||
const ENTRY_TTL_MS = 30 * 60 * 1000; // 30 min — recently-seen refusals only
|
||||
|
||||
/** @type {Array<{ts: number, rationale: string, unitType: string|null, unitId: string|null, gateId: string|null}>} */
|
||||
const buffer = [];
|
||||
|
||||
/**
|
||||
* Record a UokGate refusal observed by the dispatcher.
|
||||
*
|
||||
* Purpose: make the refusal observable to the Wiggums detector without forcing
|
||||
* the dispatcher to know about detector internals.
|
||||
*
|
||||
* @param {{ rationale: string, unitType?: string|null, unitId?: string|null, gateId?: string|null, ts?: number }} event
|
||||
*/
|
||||
export function recordGateRefusal(event) {
|
||||
if (
|
||||
!event ||
|
||||
typeof event.rationale !== "string" ||
|
||||
event.rationale.length === 0
|
||||
) {
|
||||
return;
|
||||
}
|
||||
buffer.push({
|
||||
ts: typeof event.ts === "number" ? event.ts : Date.now(),
|
||||
rationale: event.rationale,
|
||||
unitType: event.unitType ?? null,
|
||||
unitId: event.unitId ?? null,
|
||||
gateId: event.gateId ?? null,
|
||||
});
|
||||
// Trim from the front when over capacity.
|
||||
while (buffer.length > MAX_REFUSALS) buffer.shift();
|
||||
}
|
||||
|
||||
/**
|
||||
* Return the recent (within TTL) gate-refusal events.
|
||||
*
|
||||
* Purpose: feed the gate-deadlock-classifier with a bounded, time-filtered list.
|
||||
*
|
||||
* @param {number} [now] — current timestamp (default Date.now()), for testability.
|
||||
* @returns {Array<{ts: number, rationale: string, unitType: string|null, unitId: string|null, gateId: string|null}>}
|
||||
*/
|
||||
export function recentGateRefusals(now = Date.now()) {
|
||||
const cutoff = now - ENTRY_TTL_MS;
|
||||
// Drop expired entries lazily (every call) so the buffer doesn't accumulate
|
||||
// stale events across long-lived processes.
|
||||
while (buffer.length > 0 && buffer[0].ts < cutoff) buffer.shift();
|
||||
return buffer.slice();
|
||||
}
|
||||
|
||||
/** Test-only — reset the buffer. */
|
||||
export function _resetGateRefusalBuffer() {
|
||||
buffer.length = 0;
|
||||
}
|
||||
|
|
@ -31,6 +31,7 @@ import { DispatchLayer } from "../dispatch/dispatch-layer.js";
|
|||
import { isInlineEligible } from "../dispatch/run-unit-inline.js";
|
||||
import { swarmDispatchAndWait } from "../uok/swarm-dispatch.js";
|
||||
import { inlineRuntimeGate } from "../uok/inline-runtime-gate.js";
|
||||
import { recordGateRefusal } from "./gate-refusal-recorder.js";
|
||||
|
||||
/**
|
||||
* #M010/S05: Try inline-scope dispatch via DispatchLayer.
|
||||
|
|
@ -67,6 +68,14 @@ async function tryInlineDispatch(ctx, s, unitType, unitId, _prompt, options) {
|
|||
unitId,
|
||||
rationale: gateResult.rationale,
|
||||
});
|
||||
// Feed the gate-deadlock Wiggums detector. The rationale already includes
|
||||
// the [<gate-id>] prefix and R-id status fragments the detector parses.
|
||||
recordGateRefusal({
|
||||
rationale: `[inline-runtime-gate] ${gateResult.rationale}`,
|
||||
gateId: "inline-runtime-gate",
|
||||
unitType,
|
||||
unitId,
|
||||
});
|
||||
return {
|
||||
status: "cancelled",
|
||||
errorContext: {
|
||||
|
|
|
|||
|
|
@ -40,7 +40,9 @@ export function parseGateRefusal(rationale) {
|
|||
if (typeof rationale !== "string" || rationale.length === 0) return null;
|
||||
const gateMatch = rationale.match(/\[([a-z][a-z0-9-]+(?:-gate)?)\]/);
|
||||
if (!gateMatch) return null;
|
||||
const reqMatches = rationale.match(/\bR\d{2,}\b(?==active|=pending|=deferred)/g);
|
||||
const reqMatches = rationale.match(
|
||||
/\bR\d{2,}\b(?==active|=pending|=deferred)/g,
|
||||
);
|
||||
if (!reqMatches || reqMatches.length === 0) return null;
|
||||
const seen = new Set();
|
||||
const refusedRequirements = [];
|
||||
|
|
@ -182,8 +184,14 @@ export const gateDeadlockClassifierGate = {
|
|||
async execute(ctx = {}) {
|
||||
const result = detectGateDeadlock(ctx, ctx.options ?? {});
|
||||
if (result.stuck) {
|
||||
// Deliberate: emit `manual-attention`, not `fail`. The whole point of
|
||||
// detecting a gate-deadlock is to surface it so SF can route around
|
||||
// (audited bypass, spawn-mode retry, operator notification). Returning
|
||||
// `fail` would compound the deadlock by adding another refusal to the
|
||||
// chain. Same precedent as `periodicDetectorSweepGate` — detector
|
||||
// firings ARE NOT dispatch refusals.
|
||||
return {
|
||||
outcome: "fail",
|
||||
outcome: "manual-attention",
|
||||
failureClass: "verification",
|
||||
rationale: `gate-deadlock on ${result.signature.gateId}: refused units ${result.signature.refusedUnits.join(", ")} need ${result.signature.deadlockedRequirements.join(", ")} which the gate is waiting for`,
|
||||
findings: result.signature,
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@
|
|||
*/
|
||||
import { detectArtifactFlap } from "./artifact-flap.js";
|
||||
import { detectCrashLoop } from "./crash-loop-classifier.js";
|
||||
import { detectGateDeadlock } from "./gate-deadlock-classifier.js";
|
||||
import { detectProductionPlateau } from "./production-plateau.js";
|
||||
import { detectRepeatedFeedbackKind } from "./repeated-feedback-kind.js";
|
||||
import { detectSameUnitLoop } from "./same-unit-loop.js";
|
||||
|
|
@ -15,6 +16,7 @@ import { detectServerDirectionDrift } from "./server-direction-drift.js";
|
|||
import { detectStaleLock } from "./stale-lock.js";
|
||||
import { detectStatusCompletionDrift } from "./status-completion-drift.js";
|
||||
import { detectZeroProgress } from "./zero-progress.js";
|
||||
import { recentGateRefusals } from "../auto/gate-refusal-recorder.js";
|
||||
|
||||
export const SWEEP_CADENCE_MS = 60 * 1000;
|
||||
|
||||
|
|
@ -79,6 +81,18 @@ function defaultDetectors(ctx, options) {
|
|||
name: "server-direction-drift",
|
||||
run: () => detectServerDirectionDrift(ctx, options),
|
||||
},
|
||||
{
|
||||
name: "gate-deadlock",
|
||||
run: () =>
|
||||
detectGateDeadlock(
|
||||
{
|
||||
gateRefusals: ctx?.gateRefusals ?? recentGateRefusals(),
|
||||
requirementCoverageByMilestone: ctx?.requirementCoverageByMilestone,
|
||||
resolveMilestoneId: ctx?.resolveMilestoneId,
|
||||
},
|
||||
options,
|
||||
),
|
||||
},
|
||||
];
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -99,9 +99,7 @@ describe("detectGateDeadlock", () => {
|
|||
|
||||
it("does NOT fire when milestone coverage map is empty for the unit's milestone", () => {
|
||||
const result = detectGateDeadlock({
|
||||
gateRefusals: [
|
||||
{ rationale: INLINE_REFUSAL, unitId: "M048/S05" },
|
||||
],
|
||||
gateRefusals: [{ rationale: INLINE_REFUSAL, unitId: "M048/S05" }],
|
||||
requirementCoverageByMilestone: {},
|
||||
});
|
||||
expect(result.stuck).toBe(false);
|
||||
|
|
@ -110,9 +108,7 @@ describe("detectGateDeadlock", () => {
|
|||
it("throttles repeat firings for the same gate within throttleMs", () => {
|
||||
const throttleState = new Map();
|
||||
const ctx = {
|
||||
gateRefusals: [
|
||||
{ rationale: INLINE_REFUSAL, unitId: "M048/S05" },
|
||||
],
|
||||
gateRefusals: [{ rationale: INLINE_REFUSAL, unitId: "M048/S05" }],
|
||||
requirementCoverageByMilestone: { M048: ["R066"] },
|
||||
throttleState,
|
||||
};
|
||||
|
|
@ -126,9 +122,7 @@ describe("detectGateDeadlock", () => {
|
|||
|
||||
it("custom resolveMilestoneId overrides the default M-prefix resolver", () => {
|
||||
const result = detectGateDeadlock({
|
||||
gateRefusals: [
|
||||
{ rationale: INLINE_REFUSAL, unitId: "slice-xyz" },
|
||||
],
|
||||
gateRefusals: [{ rationale: INLINE_REFUSAL, unitId: "slice-xyz" }],
|
||||
requirementCoverageByMilestone: { CUSTOM: ["R020"] },
|
||||
resolveMilestoneId: () => "CUSTOM",
|
||||
});
|
||||
|
|
@ -168,14 +162,12 @@ describe("gateDeadlockClassifierGate", () => {
|
|||
expect(result.rationale).toContain("no gate-deadlock");
|
||||
});
|
||||
|
||||
it("execute returns fail with structured findings when deadlock detected", async () => {
|
||||
it("execute returns manual-attention (not fail) when deadlock detected — refusing would compound the deadlock", async () => {
|
||||
const result = await gateDeadlockClassifierGate.execute({
|
||||
gateRefusals: [
|
||||
{ rationale: INLINE_REFUSAL, unitId: "M048/S05" },
|
||||
],
|
||||
gateRefusals: [{ rationale: INLINE_REFUSAL, unitId: "M048/S05" }],
|
||||
requirementCoverageByMilestone: { M048: ["R066"] },
|
||||
});
|
||||
expect(result.outcome).toBe("fail");
|
||||
expect(result.outcome).toBe("manual-attention");
|
||||
expect(result.failureClass).toBe("verification");
|
||||
expect(result.rationale).toContain("inline-runtime-gate");
|
||||
expect(result.rationale).toContain("R066");
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue