feat(gsd): declarative workflow engine — YAML-defined workflows through the auto-loop (#2024)
* docs(S01): add slice plan * feat(S01/T01): Created four engine abstraction layer files: engine-type… - src/resources/extensions/gsd/engine-types.ts - src/resources/extensions/gsd/workflow-engine.ts - src/resources/extensions/gsd/execution-policy.ts - src/resources/extensions/gsd/engine-resolver.ts * test(S01/T02): Added activeEngineId to AutoSession lifecycle (property,… - src/resources/extensions/gsd/auto/session.ts - src/resources/extensions/gsd/tests/engine-interfaces-contract.test.ts * chore(M001/S02): auto-commit after research-slice * docs(S02): add slice plan * feat(S02/T01): Created DevWorkflowEngine and DevExecutionPolicy classes… - src/resources/extensions/gsd/dev-workflow-engine.ts - src/resources/extensions/gsd/dev-execution-policy.ts - src/resources/extensions/gsd/engine-resolver.ts - src/resources/extensions/gsd/auto.ts * test(S02/T02): Added 18 contract tests for dev engine wrapper and updat… - src/resources/extensions/gsd/tests/dev-engine-wrapper.test.ts - src/resources/extensions/gsd/tests/engine-interfaces-contract.test.ts * docs(S03): add slice plan * chore(S03/T01): Added definition-loader.ts with V1 YAML schema validati… - src/resources/extensions/gsd/definition-loader.ts - src/resources/extensions/gsd/tests/definition-loader.test.ts * feat(S03/T02): Added graph.ts with YAML I/O, DAG dependency queries, im… - src/resources/extensions/gsd/graph.ts - src/resources/extensions/gsd/tests/graph-operations.test.ts * docs(S04): add slice plan * test(S04/T01): Created run-manager with createRun/listRuns, CustomWorkf… - src/resources/extensions/gsd/run-manager.ts - src/resources/extensions/gsd/custom-workflow-engine.ts - src/resources/extensions/gsd/custom-execution-policy.ts - src/resources/extensions/gsd/tests/run-manager.test.ts - src/resources/extensions/gsd/tests/custom-workflow-engine.test.ts * feat(S04/T02): Extended engine-resolver with custom engine branch, adde… - src/resources/extensions/gsd/engine-resolver.ts - src/resources/extensions/gsd/auto/session.ts - src/resources/extensions/gsd/auto.ts - src/resources/extensions/gsd/auto-dashboard.ts * test(S04/T03): Added polymorphic custom engine dispatch path to autoLoo… - src/resources/extensions/gsd/auto/loop.ts - src/resources/extensions/gsd/auto/phases.ts - src/resources/extensions/gsd/tests/custom-engine-loop-integration.test.ts * docs(S05): add slice plan * feat(S05/T01): Created custom-verification.ts with four policy handlers… - src/resources/extensions/gsd/custom-verification.ts - src/resources/extensions/gsd/custom-execution-policy.ts - src/resources/extensions/gsd/engine-resolver.ts - src/resources/extensions/gsd/tests/custom-verification.test.ts * feat(S05/T02): Created context-injector.ts with injectContext() that re… - src/resources/extensions/gsd/context-injector.ts - src/resources/extensions/gsd/custom-workflow-engine.ts - src/resources/extensions/gsd/tests/context-injector.test.ts * docs(S06): add slice plan * test(S06/T01): Wired expandIteration() into resolveDispatch() with DEFI… - src/resources/extensions/gsd/custom-workflow-engine.ts - src/resources/extensions/gsd/tests/iterate-engine-integration.test.ts - src/resources/extensions/gsd/tests/custom-workflow-engine.test.ts * docs(S07): add slice plan * feat(S07/T01): Added six `/gsd workflow` subcommands (new, run, list, v… - src/resources/extensions/gsd/commands/handlers/workflow.ts - src/resources/extensions/gsd/commands/catalog.ts - src/resources/extensions/gsd/tests/commands-workflow-custom.test.ts * fix(S07/T02): Added updateProgressWidget call in custom engine path and… - src/resources/extensions/gsd/auto/loop.ts - src/resources/extensions/gsd/dashboard-overlay.ts - src/resources/extensions/gsd/tests/dashboard-custom-engine.test.ts * docs(S08): add slice plan * docs(S08/T01): Created 7-file router-pattern skill for conversational Y… - src/resources/skills/create-workflow/SKILL.md - src/resources/skills/create-workflow/workflows/create-from-scratch.md - src/resources/skills/create-workflow/workflows/create-from-template.md - src/resources/skills/create-workflow/references/yaml-schema-v1.md - src/resources/skills/create-workflow/references/verification-policies.md - src/resources/skills/create-workflow/references/feature-patterns.md - src/resources/skills/create-workflow/templates/workflow-definition.yaml * test(S08/T02): Created 3 example workflow definitions (blog-post-pipeli… - src/resources/skills/create-workflow/templates/blog-post-pipeline.yaml - src/resources/skills/create-workflow/templates/code-audit.yaml - src/resources/skills/create-workflow/templates/release-checklist.yaml - src/resources/extensions/gsd/tests/bundled-workflow-defs.test.ts * docs(S09): add slice plan * test(S09/T01): Comprehensive e2e integration test proving the full work… - src/resources/extensions/gsd/tests/e2e-workflow-pipeline-integration.test.ts * chore: remove .gsd/ artifacts from tracking (already in .gitignore) * fix(skills): resolve broken cross-references in create-workflow workflow files Paths in workflows/ referenced references/ as siblings, but they need ../references/ since they're resolved relative to the workflows/ directory. * fix: resolve typecheck failures — .ts→.js imports, MapIterator.some(), LoadedGSDPreferences unwrap, constructor args - Convert .ts import extensions to .js in source files to match codebase convention (tests keep .ts since tsconfig.extensions allows it) - Use [...idCounts.values()].some() instead of MapIterator.some() - Unwrap LoadedGSDPreferences.preferences for DispatchContext.prefs - Pass runDir to CustomExecutionPolicy constructor in tests * fix: add codeFilesChanged to mergeMilestoneToMain mock (synced with main) * fix(tests): write DEFINITION.yaml in integration tests, fix error message assertion Root cause: S06 (iterate) added DEFINITION.yaml reading to resolveDispatch(), but S04's integration tests only wrote GRAPH.yaml. The missing file threw ENOENT, swallowed by the blanket catch, causing steps to stay 'pending' silently. Fixes: - custom-engine-loop-integration: write DEFINITION.yaml in all 5 tests - custom-workflow-engine: verify test creates temp dir with definition - dev-engine-wrapper: update error regex — resolver validates activeRunDir before engine ID, so 'Unknown engine' is never reached * fix: address 13 audit findings from self-review of workflow engine PR Critical: - Fix verify-before-reconcile ordering — verify step output before marking complete in GRAPH.yaml, so failed verification triggers retry - Fix GSD_ENGINE_BYPASS kill switch — check env var in autoLoop before entering custom engine block instead of throwing from resolveEngine - Add shell-command injection guard with suspicious pattern detection High: - Add ReDoS timeout guard (5s) for iterate regex patterns - Centralize DEFINITION.yaml parsing into readFrozenDefinition() with schema: "core" restriction, eliminating 3 independent parse+cast sites - Persist activeEngineId/activeRunDir in paused-session.json and restore on resume so custom workflows survive /exit - Clean up engine state on startAuto failure in workflow run handler Medium: - Coerce params values to strings in definition-loader (YAML numbers/bools) - Add path traversal guard (resolve + startsWith) in context-injector and custom-verification content-heuristic - Use function replacer in expandIteration to prevent $ escaping bugs Low: - Fix skill docs CLI syntax (remove --param prefix) - Use resolveProjectRoot instead of process.cwd() in catalog completions - Rename isHookUnit → skipArtifactVerification for clarity Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
TÂCHES
GitHub
parent
28e3c2e72c
commit
2aa01b8a60
45 changed files with 8194 additions and 3 deletions
|
|
@ -79,6 +79,7 @@ export function unitVerb(unitType: string): string {
|
|||
case "rewrite-docs": return "rewriting";
|
||||
case "reassess-roadmap": return "reassessing";
|
||||
case "run-uat": return "running UAT";
|
||||
case "custom-step": return "executing workflow step";
|
||||
default: return unitType;
|
||||
}
|
||||
}
|
||||
|
|
@ -97,6 +98,7 @@ export function unitPhaseLabel(unitType: string): string {
|
|||
case "rewrite-docs": return "REWRITE";
|
||||
case "reassess-roadmap": return "REASSESS";
|
||||
case "run-uat": return "UAT";
|
||||
case "custom-step": return "WORKFLOW";
|
||||
default: return unitType.toUpperCase();
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -358,6 +358,22 @@ export function isAutoPaused(): boolean {
|
|||
return s.paused;
|
||||
}
|
||||
|
||||
export function setActiveEngineId(id: string | null): void {
|
||||
s.activeEngineId = id;
|
||||
}
|
||||
|
||||
export function getActiveEngineId(): string | null {
|
||||
return s.activeEngineId;
|
||||
}
|
||||
|
||||
export function setActiveRunDir(runDir: string | null): void {
|
||||
s.activeRunDir = runDir;
|
||||
}
|
||||
|
||||
export function getActiveRunDir(): string | null {
|
||||
return s.activeRunDir;
|
||||
}
|
||||
|
||||
/**
|
||||
* Return the model captured at auto-mode start for this session.
|
||||
* Used by error-recovery to fall back to the session's own model
|
||||
|
|
@ -782,6 +798,8 @@ export async function pauseAuto(
|
|||
stepMode: s.stepMode,
|
||||
pausedAt: new Date().toISOString(),
|
||||
sessionFile: s.pausedSessionFile,
|
||||
activeEngineId: s.activeEngineId,
|
||||
activeRunDir: s.activeRunDir,
|
||||
};
|
||||
const runtimeDir = join(gsdRoot(s.originalBasePath || s.basePath), "runtime");
|
||||
mkdirSync(runtimeDir, { recursive: true });
|
||||
|
|
@ -1018,7 +1036,19 @@ export async function startAuto(
|
|||
const pausedPath = join(gsdRoot(base), "runtime", "paused-session.json");
|
||||
if (existsSync(pausedPath)) {
|
||||
const meta = JSON.parse(readFileSync(pausedPath, "utf-8"));
|
||||
if (meta.milestoneId) {
|
||||
if (meta.activeEngineId && meta.activeEngineId !== "dev") {
|
||||
// Custom workflow resume — restore engine state
|
||||
s.activeEngineId = meta.activeEngineId;
|
||||
s.activeRunDir = meta.activeRunDir ?? null;
|
||||
s.originalBasePath = meta.originalBasePath || base;
|
||||
s.stepMode = meta.stepMode ?? requestedStepMode;
|
||||
s.paused = true;
|
||||
try { unlinkSync(pausedPath); } catch { /* non-fatal */ }
|
||||
ctx.ui.notify(
|
||||
`Resuming paused custom workflow${meta.activeRunDir ? ` (${meta.activeRunDir})` : ""}.`,
|
||||
"info",
|
||||
);
|
||||
} else if (meta.milestoneId) {
|
||||
// Validate the milestone still exists and isn't already complete (#1664).
|
||||
const mDir = resolveMilestonePath(base, meta.milestoneId);
|
||||
const summaryFile = resolveMilestoneFile(base, meta.milestoneId, "SUMMARY");
|
||||
|
|
|
|||
|
|
@ -28,6 +28,7 @@ import {
|
|||
} from "./phases.js";
|
||||
import { debugLog } from "../debug-logger.js";
|
||||
import { isInfrastructureError } from "./infra-errors.js";
|
||||
import { resolveEngine } from "../engine-resolver.js";
|
||||
|
||||
/**
|
||||
* Main auto-mode execution loop. Iterates: derive → dispatch → guards →
|
||||
|
|
@ -117,6 +118,96 @@ export async function autoLoop(
|
|||
deps.emitJournalEvent({ ts: new Date().toISOString(), flowId, seq: nextSeq(), eventType: "iteration-start", data: { iteration } });
|
||||
let iterData: IterationData;
|
||||
|
||||
// ── Custom engine path ──────────────────────────────────────────────
|
||||
// When activeEngineId is a non-dev value, bypass runPreDispatch and
|
||||
// runDispatch entirely — the custom engine drives its own state via
|
||||
// GRAPH.yaml. Shares runGuards and runUnitPhase with the dev path.
|
||||
// After unit execution, verifies then reconciles via the engine layer.
|
||||
//
|
||||
// GSD_ENGINE_BYPASS=1 skips the engine layer entirely — falls through
|
||||
// to the dev path below.
|
||||
if (s.activeEngineId != null && s.activeEngineId !== "dev" && !sidecarItem && process.env.GSD_ENGINE_BYPASS !== "1") {
|
||||
debugLog("autoLoop", { phase: "custom-engine-derive", iteration, engineId: s.activeEngineId });
|
||||
|
||||
const { engine, policy } = resolveEngine({
|
||||
activeEngineId: s.activeEngineId,
|
||||
activeRunDir: s.activeRunDir,
|
||||
});
|
||||
|
||||
const engineState = await engine.deriveState(s.basePath);
|
||||
if (engineState.isComplete) {
|
||||
await deps.stopAuto(ctx, pi, "Workflow complete");
|
||||
break;
|
||||
}
|
||||
|
||||
debugLog("autoLoop", { phase: "custom-engine-dispatch", iteration });
|
||||
const dispatch = await engine.resolveDispatch(engineState, { basePath: s.basePath });
|
||||
|
||||
if (dispatch.action === "stop") {
|
||||
await deps.stopAuto(ctx, pi, dispatch.reason ?? "Engine stopped");
|
||||
break;
|
||||
}
|
||||
if (dispatch.action === "skip") {
|
||||
continue;
|
||||
}
|
||||
|
||||
// dispatch.action === "dispatch"
|
||||
const step = dispatch.step!;
|
||||
const gsdState = await deps.deriveState(s.basePath);
|
||||
|
||||
iterData = {
|
||||
unitType: step.unitType,
|
||||
unitId: step.unitId,
|
||||
prompt: step.prompt,
|
||||
finalPrompt: step.prompt,
|
||||
pauseAfterUatDispatch: false,
|
||||
observabilityIssues: [],
|
||||
state: gsdState,
|
||||
mid: s.currentMilestoneId ?? "workflow",
|
||||
midTitle: "Workflow",
|
||||
isRetry: false,
|
||||
previousTier: undefined,
|
||||
};
|
||||
|
||||
// ── Progress widget (mirrors dev path in runDispatch) ──
|
||||
deps.updateProgressWidget(ctx, iterData.unitType, iterData.unitId, iterData.state);
|
||||
|
||||
// ── Guards (shared with dev path) ──
|
||||
const guardsResult = await runGuards(ic, s.currentMilestoneId ?? "workflow");
|
||||
if (guardsResult.action === "break") break;
|
||||
|
||||
// ── Unit execution (shared with dev path) ──
|
||||
const unitPhaseResult = await runUnitPhase(ic, iterData, loopState);
|
||||
if (unitPhaseResult.action === "break") break;
|
||||
|
||||
// ── Verify first, then reconcile (only mark complete on pass) ──
|
||||
debugLog("autoLoop", { phase: "custom-engine-verify", iteration, unitId: iterData.unitId });
|
||||
const verifyResult = await policy.verify(iterData.unitType, iterData.unitId, { basePath: s.basePath });
|
||||
if (verifyResult === "pause") {
|
||||
await deps.pauseAuto(ctx, pi);
|
||||
break;
|
||||
}
|
||||
if (verifyResult === "retry") {
|
||||
debugLog("autoLoop", { phase: "custom-engine-verify-retry", iteration, unitId: iterData.unitId });
|
||||
continue;
|
||||
}
|
||||
|
||||
// Verification passed — mark step complete
|
||||
debugLog("autoLoop", { phase: "custom-engine-reconcile", iteration, unitId: iterData.unitId });
|
||||
await engine.reconcile(engineState, {
|
||||
unitType: iterData.unitType,
|
||||
unitId: iterData.unitId,
|
||||
startedAt: s.currentUnit?.startedAt ?? Date.now(),
|
||||
finishedAt: Date.now(),
|
||||
});
|
||||
|
||||
deps.clearUnitTimeout();
|
||||
consecutiveErrors = 0;
|
||||
deps.emitJournalEvent({ ts: new Date().toISOString(), flowId, seq: nextSeq(), eventType: "iteration-end", data: { iteration } });
|
||||
debugLog("autoLoop", { phase: "iteration-complete", iteration });
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!sidecarItem) {
|
||||
// ── Phase 1: Pre-dispatch ─────────────────────────────────────────
|
||||
const preDispatchResult = await runPreDispatch(ic, loopState);
|
||||
|
|
|
|||
|
|
@ -1133,9 +1133,9 @@ export async function runUnitPhase(
|
|||
);
|
||||
}
|
||||
|
||||
const isHookUnit = unitType.startsWith("hook/");
|
||||
const skipArtifactVerification = unitType.startsWith("hook/") || unitType === "custom-step";
|
||||
const artifactVerified =
|
||||
isHookUnit ||
|
||||
skipArtifactVerification ||
|
||||
deps.verifyExpectedArtifact(unitType, unitId, s.basePath);
|
||||
if (artifactVerified) {
|
||||
s.completedUnits.push({
|
||||
|
|
|
|||
|
|
@ -83,6 +83,8 @@ export class AutoSession {
|
|||
paused = false;
|
||||
stepMode = false;
|
||||
verbose = false;
|
||||
activeEngineId: string | null = null;
|
||||
activeRunDir: string | null = null;
|
||||
cmdCtx: ExtensionCommandContext | null = null;
|
||||
|
||||
// ── Paths ────────────────────────────────────────────────────────────────
|
||||
|
|
@ -174,6 +176,8 @@ export class AutoSession {
|
|||
this.paused = false;
|
||||
this.stepMode = false;
|
||||
this.verbose = false;
|
||||
this.activeEngineId = null;
|
||||
this.activeRunDir = null;
|
||||
this.cmdCtx = null;
|
||||
|
||||
// Paths
|
||||
|
|
@ -226,6 +230,8 @@ export class AutoSession {
|
|||
paused: this.paused,
|
||||
stepMode: this.stepMode,
|
||||
basePath: this.basePath,
|
||||
activeEngineId: this.activeEngineId,
|
||||
activeRunDir: this.activeRunDir,
|
||||
currentMilestoneId: this.currentMilestoneId,
|
||||
currentUnit: this.currentUnit,
|
||||
completedUnits: this.completedUnits.length,
|
||||
|
|
|
|||
|
|
@ -3,6 +3,7 @@ import { homedir } from "node:os";
|
|||
import { join } from "node:path";
|
||||
|
||||
import { loadRegistry } from "../workflow-templates.js";
|
||||
import { resolveProjectRoot } from "../worktree.js";
|
||||
|
||||
const gsdHome = process.env.GSD_HOME || join(homedir(), ".gsd");
|
||||
|
||||
|
|
@ -65,6 +66,7 @@ export const TOP_LEVEL_SUBCOMMANDS: readonly GsdCommandDefinition[] = [
|
|||
{ cmd: "templates", desc: "List available workflow templates" },
|
||||
{ cmd: "extensions", desc: "Manage extensions (list, enable, disable, info)" },
|
||||
{ cmd: "fast", desc: "Toggle OpenAI service tier (on/off/flex/status)" },
|
||||
{ cmd: "workflow", desc: "Custom workflow lifecycle (new, run, list, validate, pause, resume)" },
|
||||
];
|
||||
|
||||
const NESTED_COMPLETIONS: CompletionMap = {
|
||||
|
|
@ -206,6 +208,14 @@ const NESTED_COMPLETIONS: CompletionMap = {
|
|||
{ cmd: "ok", desc: "Model was appropriate for this task" },
|
||||
{ cmd: "under", desc: "Model was underqualified for this task" },
|
||||
],
|
||||
workflow: [
|
||||
{ cmd: "new", desc: "Create a new workflow definition (via skill)" },
|
||||
{ cmd: "run", desc: "Create a run and start auto-mode" },
|
||||
{ cmd: "list", desc: "List workflow runs" },
|
||||
{ cmd: "validate", desc: "Validate a workflow definition YAML" },
|
||||
{ cmd: "pause", desc: "Pause custom workflow auto-mode" },
|
||||
{ cmd: "resume", desc: "Resume paused custom workflow auto-mode" },
|
||||
],
|
||||
};
|
||||
|
||||
function filterOptions(
|
||||
|
|
@ -309,6 +319,28 @@ export function getGsdArgumentCompletions(prefix: string) {
|
|||
return [{ value: "undo --force", label: "--force", description: "Skip confirmation prompt" }];
|
||||
}
|
||||
|
||||
// Workflow definition-name completion for `workflow run <name>` and `workflow validate <name>`
|
||||
if (command === "workflow" && (subcommand === "run" || subcommand === "validate") && parts.length <= 3) {
|
||||
try {
|
||||
const defsDir = join(resolveProjectRoot(process.cwd()), ".gsd", "workflow-defs");
|
||||
if (existsSync(defsDir)) {
|
||||
return readdirSync(defsDir)
|
||||
.filter((f) => f.endsWith(".yaml") && f.startsWith(third))
|
||||
.map((f) => {
|
||||
const name = f.replace(/\.yaml$/, "");
|
||||
return {
|
||||
value: `workflow ${subcommand} ${name}`,
|
||||
label: name,
|
||||
description: `Workflow definition: ${name}`,
|
||||
};
|
||||
});
|
||||
}
|
||||
} catch {
|
||||
// ignore filesystem errors during completion
|
||||
}
|
||||
return [];
|
||||
}
|
||||
|
||||
const nested = NESTED_COMPLETIONS[command];
|
||||
if (nested && parts.length <= 2) {
|
||||
return filterOptions(subcommand, nested, command);
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@ import type { ExtensionAPI, ExtensionCommandContext } from "@gsd/pi-coding-agent
|
|||
|
||||
import { existsSync, readFileSync, unlinkSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { parse as parseYaml } from "yaml";
|
||||
|
||||
import { handleQuick } from "../../quick.js";
|
||||
import { showDiscuss, showHeadlessMilestoneCreation, showQueue } from "../../guided-flow.js";
|
||||
|
|
@ -13,8 +14,171 @@ import { loadEffectiveGSDPreferences } from "../../preferences.js";
|
|||
import { nextMilestoneId } from "../../milestone-ids.js";
|
||||
import { findMilestoneIds } from "../../guided-flow.js";
|
||||
import { projectRoot } from "../context.js";
|
||||
import { createRun, listRuns } from "../../run-manager.js";
|
||||
import {
|
||||
setActiveEngineId,
|
||||
setActiveRunDir,
|
||||
startAuto,
|
||||
pauseAuto,
|
||||
isAutoActive,
|
||||
getActiveEngineId,
|
||||
} from "../../auto.js";
|
||||
import { validateDefinition } from "../../definition-loader.js";
|
||||
|
||||
// ─── Custom Workflow Subcommands ─────────────────────────────────────────
|
||||
|
||||
const WORKFLOW_USAGE = [
|
||||
"Usage: /gsd workflow <subcommand>",
|
||||
"",
|
||||
" new — Create a new workflow definition (via skill)",
|
||||
" run <name> [k=v] — Create a run and start auto-mode",
|
||||
" list [name] — List workflow runs (optionally filtered by name)",
|
||||
" validate <name> — Validate a workflow definition YAML",
|
||||
" pause — Pause custom workflow auto-mode",
|
||||
" resume — Resume paused custom workflow auto-mode",
|
||||
].join("\n");
|
||||
|
||||
async function handleCustomWorkflow(
|
||||
sub: string,
|
||||
ctx: ExtensionCommandContext,
|
||||
pi: ExtensionAPI,
|
||||
): Promise<boolean> {
|
||||
// Bare `/gsd workflow` — show usage
|
||||
if (!sub) {
|
||||
ctx.ui.notify(WORKFLOW_USAGE, "info");
|
||||
return true;
|
||||
}
|
||||
|
||||
// ── new ──
|
||||
if (sub === "new") {
|
||||
ctx.ui.notify("Use the create-workflow skill: /skill create-workflow", "info");
|
||||
return true;
|
||||
}
|
||||
|
||||
// ── run <name> [param=value ...] ──
|
||||
if (sub === "run" || sub.startsWith("run ")) {
|
||||
const args = sub.slice("run".length).trim();
|
||||
if (!args) {
|
||||
ctx.ui.notify("Usage: /gsd workflow run <name> [param=value ...]", "warning");
|
||||
return true;
|
||||
}
|
||||
const parts = args.split(/\s+/);
|
||||
const defName = parts[0];
|
||||
const overrides: Record<string, string> = {};
|
||||
for (let i = 1; i < parts.length; i++) {
|
||||
const eqIdx = parts[i].indexOf("=");
|
||||
if (eqIdx > 0) {
|
||||
overrides[parts[i].slice(0, eqIdx)] = parts[i].slice(eqIdx + 1);
|
||||
}
|
||||
}
|
||||
try {
|
||||
const base = projectRoot();
|
||||
const runDir = createRun(base, defName, Object.keys(overrides).length > 0 ? overrides : undefined);
|
||||
setActiveEngineId("custom");
|
||||
setActiveRunDir(runDir);
|
||||
ctx.ui.notify(`Created workflow run: ${defName}\nRun dir: ${runDir}`, "info");
|
||||
await startAuto(ctx, pi, base, false);
|
||||
} catch (err) {
|
||||
// Clean up engine state so a failed workflow run doesn't pollute the next /gsd auto
|
||||
setActiveEngineId(null);
|
||||
setActiveRunDir(null);
|
||||
const msg = err instanceof Error ? err.message : String(err);
|
||||
ctx.ui.notify(`Failed to run workflow "${defName}": ${msg}`, "error");
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
// ── list [name] ──
|
||||
if (sub === "list" || sub.startsWith("list ")) {
|
||||
const filterName = sub.slice("list".length).trim() || undefined;
|
||||
const base = projectRoot();
|
||||
const runs = listRuns(base, filterName);
|
||||
if (runs.length === 0) {
|
||||
ctx.ui.notify("No workflow runs found.", "info");
|
||||
return true;
|
||||
}
|
||||
const lines = runs.map((r) => {
|
||||
const stepInfo = `${r.steps.completed}/${r.steps.total} steps`;
|
||||
return `• ${r.name} [${r.timestamp}] — ${r.status} (${stepInfo})`;
|
||||
});
|
||||
ctx.ui.notify(lines.join("\n"), "info");
|
||||
return true;
|
||||
}
|
||||
|
||||
// ── validate <name> ──
|
||||
if (sub === "validate" || sub.startsWith("validate ")) {
|
||||
const defName = sub.slice("validate".length).trim();
|
||||
if (!defName) {
|
||||
ctx.ui.notify("Usage: /gsd workflow validate <name>", "warning");
|
||||
return true;
|
||||
}
|
||||
const base = projectRoot();
|
||||
const defPath = join(base, ".gsd", "workflow-defs", `${defName}.yaml`);
|
||||
if (!existsSync(defPath)) {
|
||||
ctx.ui.notify(`Definition not found: ${defPath}`, "error");
|
||||
return true;
|
||||
}
|
||||
try {
|
||||
const raw = readFileSync(defPath, "utf-8");
|
||||
const parsed = parseYaml(raw);
|
||||
const result = validateDefinition(parsed);
|
||||
if (result.valid) {
|
||||
ctx.ui.notify(`✓ "${defName}" is a valid workflow definition.`, "info");
|
||||
} else {
|
||||
ctx.ui.notify(`✗ "${defName}" has errors:\n - ${result.errors.join("\n - ")}`, "error");
|
||||
}
|
||||
} catch (err) {
|
||||
const msg = err instanceof Error ? err.message : String(err);
|
||||
ctx.ui.notify(`Failed to validate "${defName}": ${msg}`, "error");
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
// ── pause ──
|
||||
if (sub === "pause") {
|
||||
const engineId = getActiveEngineId();
|
||||
if (engineId === "dev" || engineId === null) {
|
||||
ctx.ui.notify("No custom workflow is running. Use /gsd pause for dev workflow.", "warning");
|
||||
return true;
|
||||
}
|
||||
if (!isAutoActive()) {
|
||||
ctx.ui.notify("Auto-mode is not active.", "warning");
|
||||
return true;
|
||||
}
|
||||
await pauseAuto(ctx, pi);
|
||||
ctx.ui.notify("Custom workflow paused.", "info");
|
||||
return true;
|
||||
}
|
||||
|
||||
// ── resume ──
|
||||
if (sub === "resume") {
|
||||
const engineId = getActiveEngineId();
|
||||
if (engineId === "dev" || engineId === null) {
|
||||
ctx.ui.notify("No custom workflow to resume. Use /gsd auto for dev workflow.", "warning");
|
||||
return true;
|
||||
}
|
||||
try {
|
||||
await startAuto(ctx, pi, projectRoot(), false);
|
||||
ctx.ui.notify("Custom workflow resumed.", "info");
|
||||
} catch (err) {
|
||||
const msg = err instanceof Error ? err.message : String(err);
|
||||
ctx.ui.notify(`Failed to resume workflow: ${msg}`, "error");
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
// Unknown subcommand — show usage
|
||||
ctx.ui.notify(`Unknown workflow subcommand: "${sub}"\n\n${WORKFLOW_USAGE}`, "warning");
|
||||
return true;
|
||||
}
|
||||
|
||||
export async function handleWorkflowCommand(trimmed: string, ctx: ExtensionCommandContext, pi: ExtensionAPI): Promise<boolean> {
|
||||
// ── Custom workflow commands (`/gsd workflow ...`) ──
|
||||
if (trimmed === "workflow" || trimmed.startsWith("workflow ")) {
|
||||
const sub = trimmed.slice("workflow".length).trim();
|
||||
return handleCustomWorkflow(sub, ctx, pi);
|
||||
}
|
||||
|
||||
if (trimmed === "queue") {
|
||||
await showQueue(ctx, pi, projectRoot());
|
||||
return true;
|
||||
|
|
|
|||
100
src/resources/extensions/gsd/context-injector.ts
Normal file
100
src/resources/extensions/gsd/context-injector.ts
Normal file
|
|
@ -0,0 +1,100 @@
|
|||
/**
|
||||
* context-injector.ts — Inject prior step artifacts as context into step prompts.
|
||||
*
|
||||
* Reads the frozen DEFINITION.yaml from a run directory, finds the current step's
|
||||
* `contextFrom` references, locates each referenced step's `produces` artifacts
|
||||
* on disk, reads their content (truncated to 10k chars), and prepends formatted
|
||||
* context blocks to the step prompt.
|
||||
*
|
||||
* Observability:
|
||||
* - Truncation is logged via console.warn when it occurs, preventing silent overflow.
|
||||
* - Missing artifact files are skipped silently (the step may not have produced them yet).
|
||||
* - Unknown step IDs in contextFrom produce a console.warn for diagnosis.
|
||||
* - The frozen DEFINITION.yaml on disk is the single source of truth for contextFrom config.
|
||||
*/
|
||||
|
||||
import { readFileSync, existsSync } from "node:fs";
|
||||
import { join, resolve } from "node:path";
|
||||
import type { StepDefinition } from "./definition-loader.js";
|
||||
import { readFrozenDefinition } from "./custom-workflow-engine.js";
|
||||
|
||||
/** Maximum characters per artifact to prevent context window blowout. */
|
||||
const MAX_CONTEXT_CHARS = 10_000;
|
||||
|
||||
/**
|
||||
* Inject context from prior step artifacts into a step's prompt.
|
||||
*
|
||||
* Reads the frozen DEFINITION.yaml from `runDir`, finds the step matching
|
||||
* `stepId`, and for each step ID in its `contextFrom` array, looks up that
|
||||
* step's `produces` paths, reads them from disk (relative to `runDir`),
|
||||
* truncates to MAX_CONTEXT_CHARS, and prepends as labeled context blocks.
|
||||
*
|
||||
* @param runDir — absolute path to the workflow run directory
|
||||
* @param stepId — the step ID whose prompt to enrich
|
||||
* @param prompt — the original step prompt
|
||||
* @returns The prompt with context blocks prepended, or unchanged if no context applies
|
||||
* @throws Error if DEFINITION.yaml is missing or unreadable
|
||||
*/
|
||||
export function injectContext(
|
||||
runDir: string,
|
||||
stepId: string,
|
||||
prompt: string,
|
||||
): string {
|
||||
const def = readFrozenDefinition(runDir);
|
||||
|
||||
const step = def.steps.find((s: StepDefinition) => s.id === stepId);
|
||||
if (!step || !step.contextFrom || step.contextFrom.length === 0) {
|
||||
return prompt;
|
||||
}
|
||||
|
||||
const contextBlocks: string[] = [];
|
||||
|
||||
for (const refStepId of step.contextFrom) {
|
||||
const refStep = def.steps.find((s: StepDefinition) => s.id === refStepId);
|
||||
if (!refStep) {
|
||||
console.warn(
|
||||
`context-injector: step "${stepId}" references unknown step "${refStepId}" in contextFrom — skipping`,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!refStep.produces || refStep.produces.length === 0) {
|
||||
continue;
|
||||
}
|
||||
|
||||
for (const relPath of refStep.produces) {
|
||||
const absPath = resolve(runDir, relPath);
|
||||
// Path traversal guard: ensure resolved path stays within runDir
|
||||
if (!absPath.startsWith(resolve(runDir) + "/") && absPath !== resolve(runDir)) {
|
||||
console.warn(
|
||||
`context-injector: artifact path "${relPath}" resolves outside runDir — skipping`,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
if (!existsSync(absPath)) {
|
||||
// Artifact not yet produced or optional — skip silently
|
||||
continue;
|
||||
}
|
||||
|
||||
let content = readFileSync(absPath, "utf-8");
|
||||
|
||||
if (content.length > MAX_CONTEXT_CHARS) {
|
||||
console.warn(
|
||||
`context-injector: truncating artifact "${relPath}" from step "${refStepId}" ` +
|
||||
`(${content.length} chars → ${MAX_CONTEXT_CHARS} chars)`,
|
||||
);
|
||||
content = content.slice(0, MAX_CONTEXT_CHARS) + "\n...[truncated]";
|
||||
}
|
||||
|
||||
contextBlocks.push(
|
||||
`--- Context from step "${refStepId}" (file: ${relPath}) ---\n${content}\n---`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if (contextBlocks.length === 0) {
|
||||
return prompt;
|
||||
}
|
||||
|
||||
return contextBlocks.join("\n\n") + "\n\n" + prompt;
|
||||
}
|
||||
73
src/resources/extensions/gsd/custom-execution-policy.ts
Normal file
73
src/resources/extensions/gsd/custom-execution-policy.ts
Normal file
|
|
@ -0,0 +1,73 @@
|
|||
/**
|
||||
* custom-execution-policy.ts — ExecutionPolicy for custom workflows.
|
||||
*
|
||||
* Delegates verification to the step-level verification module which reads
|
||||
* the frozen DEFINITION.yaml and dispatches to the appropriate policy handler.
|
||||
*
|
||||
* Observability:
|
||||
* - verify() returns the outcome from runCustomVerification() — four policies
|
||||
* are supported: content-heuristic, shell-command, prompt-verify, human-review.
|
||||
* - selectModel() returns null — defers to loop defaults.
|
||||
* - recover() returns retry — simple default recovery strategy.
|
||||
*/
|
||||
|
||||
import type { ExecutionPolicy } from "./execution-policy.js";
|
||||
import type { RecoveryAction, CloseoutResult } from "./engine-types.js";
|
||||
import { runCustomVerification } from "./custom-verification.js";
|
||||
|
||||
export class CustomExecutionPolicy implements ExecutionPolicy {
|
||||
private readonly runDir: string;
|
||||
|
||||
constructor(runDir: string) {
|
||||
this.runDir = runDir;
|
||||
}
|
||||
|
||||
/** No workspace preparation needed for custom workflows. */
|
||||
async prepareWorkspace(_basePath: string, _milestoneId: string): Promise<void> {
|
||||
// No-op — custom workflows don't need worktree setup
|
||||
}
|
||||
|
||||
/** Defer model selection to loop defaults. */
|
||||
async selectModel(
|
||||
_unitType: string,
|
||||
_unitId: string,
|
||||
_context: { basePath: string },
|
||||
): Promise<{ tier: string; modelDowngraded: boolean } | null> {
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Verify step output by dispatching to the step's configured verification policy.
|
||||
*
|
||||
* Extracts the step ID from unitId (format: "<workflowName>/<stepId>")
|
||||
* and calls runCustomVerification() which reads the frozen DEFINITION.yaml
|
||||
* to determine which policy to apply.
|
||||
*/
|
||||
async verify(
|
||||
_unitType: string,
|
||||
unitId: string,
|
||||
_context: { basePath: string },
|
||||
): Promise<"continue" | "retry" | "pause"> {
|
||||
const parts = unitId.split("/");
|
||||
const stepId = parts[parts.length - 1];
|
||||
return runCustomVerification(this.runDir, stepId);
|
||||
}
|
||||
|
||||
/** Default recovery: retry the step. */
|
||||
async recover(
|
||||
_unitType: string,
|
||||
_unitId: string,
|
||||
_context: { basePath: string },
|
||||
): Promise<RecoveryAction> {
|
||||
return { outcome: "retry", reason: "Default retry" };
|
||||
}
|
||||
|
||||
/** No-op closeout — no commits or artifact capture. */
|
||||
async closeout(
|
||||
_unitType: string,
|
||||
_unitId: string,
|
||||
_context: { basePath: string; startedAt: number },
|
||||
): Promise<CloseoutResult> {
|
||||
return { committed: false, artifacts: [] };
|
||||
}
|
||||
}
|
||||
180
src/resources/extensions/gsd/custom-verification.ts
Normal file
180
src/resources/extensions/gsd/custom-verification.ts
Normal file
|
|
@ -0,0 +1,180 @@
|
|||
/**
|
||||
* custom-verification.ts — Step verification for custom workflows.
|
||||
*
|
||||
* Reads the frozen DEFINITION.yaml from a run directory, finds the step's
|
||||
* `verify` policy, and dispatches to the appropriate handler. Four policies:
|
||||
*
|
||||
* - content-heuristic: file existence + optional minSize + optional pattern match
|
||||
* - shell-command: spawnSync with 30s timeout, exit 0 → continue, else retry
|
||||
* - prompt-verify: always "pause" (defers to agent)
|
||||
* - human-review: always "pause" (waits for manual inspection)
|
||||
* - (no policy): returns "continue" (passthrough)
|
||||
*
|
||||
* Observability:
|
||||
* - Return value is the typed verification outcome ("continue" | "retry" | "pause").
|
||||
* - shell-command captures stderr from spawnSync — callers can inspect on retry.
|
||||
* - content-heuristic logs the specific failure (missing file, below minSize, pattern mismatch).
|
||||
* - The frozen DEFINITION.yaml on disk is the single source of truth for step policies.
|
||||
*/
|
||||
|
||||
import { readFileSync, existsSync, statSync } from "node:fs";
|
||||
import { join, resolve } from "node:path";
|
||||
import { spawnSync } from "node:child_process";
|
||||
import type { StepDefinition, VerifyPolicy } from "./definition-loader.js";
|
||||
import { readFrozenDefinition } from "./custom-workflow-engine.js";
|
||||
|
||||
/** Verification outcome type — matches ExecutionPolicy.verify() return type. */
|
||||
export type VerificationOutcome = "continue" | "retry" | "pause";
|
||||
|
||||
/**
|
||||
* Run custom verification for a specific step in a workflow run.
|
||||
*
|
||||
* Reads the frozen DEFINITION.yaml from `runDir`, finds the step with the
|
||||
* given `stepId`, and dispatches to the appropriate verification handler
|
||||
* based on the step's `verify.policy` field.
|
||||
*
|
||||
* @param runDir — absolute path to the workflow run directory
|
||||
* @param stepId — the step ID to verify (e.g. "step-1")
|
||||
* @returns "continue" if verification passes, "retry" if it should retry, "pause" if it needs review
|
||||
* @throws Error if DEFINITION.yaml is missing or unreadable
|
||||
*/
|
||||
export function runCustomVerification(
|
||||
runDir: string,
|
||||
stepId: string,
|
||||
): VerificationOutcome {
|
||||
const def = readFrozenDefinition(runDir);
|
||||
|
||||
const step = def.steps.find((s: StepDefinition) => s.id === stepId);
|
||||
if (!step) {
|
||||
// Step not found in definition — nothing to verify, continue
|
||||
return "continue";
|
||||
}
|
||||
|
||||
if (!step.verify) {
|
||||
// No verification policy configured — passthrough
|
||||
return "continue";
|
||||
}
|
||||
|
||||
return dispatchPolicy(runDir, step, step.verify);
|
||||
}
|
||||
|
||||
/**
|
||||
* Dispatch to the correct policy handler.
|
||||
*/
|
||||
function dispatchPolicy(
|
||||
runDir: string,
|
||||
step: StepDefinition,
|
||||
verify: VerifyPolicy,
|
||||
): VerificationOutcome {
|
||||
switch (verify.policy) {
|
||||
case "content-heuristic":
|
||||
return handleContentHeuristic(runDir, step, verify);
|
||||
case "shell-command":
|
||||
return handleShellCommand(runDir, verify);
|
||||
case "prompt-verify":
|
||||
return "pause";
|
||||
case "human-review":
|
||||
return "pause";
|
||||
default:
|
||||
// Unknown policy — safe default is pause
|
||||
return "pause";
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* content-heuristic handler.
|
||||
*
|
||||
* For each path in the step's `produces` array:
|
||||
* 1. Check that the file exists (resolved relative to runDir)
|
||||
* 2. If `minSize` is set, check that file size >= minSize bytes
|
||||
* 3. If `pattern` is set, check that file content matches the regex
|
||||
*
|
||||
* Returns "continue" if all checks pass, "pause" if any fail.
|
||||
* If `produces` is empty or undefined, returns "continue" (nothing to check).
|
||||
*/
|
||||
function handleContentHeuristic(
|
||||
runDir: string,
|
||||
step: StepDefinition,
|
||||
verify: { policy: "content-heuristic"; minSize?: number; pattern?: string },
|
||||
): VerificationOutcome {
|
||||
const produces = step.produces;
|
||||
if (!produces || produces.length === 0) {
|
||||
return "continue";
|
||||
}
|
||||
|
||||
for (const relPath of produces) {
|
||||
const absPath = resolve(runDir, relPath);
|
||||
// Path traversal guard
|
||||
if (!absPath.startsWith(resolve(runDir) + "/") && absPath !== resolve(runDir)) {
|
||||
return "pause";
|
||||
}
|
||||
|
||||
// 1. File existence
|
||||
if (!existsSync(absPath)) {
|
||||
return "pause";
|
||||
}
|
||||
|
||||
// 2. Minimum size check
|
||||
if (verify.minSize !== undefined) {
|
||||
const stat = statSync(absPath);
|
||||
if (stat.size < verify.minSize) {
|
||||
return "pause";
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Pattern match check (with timeout guard against ReDoS)
|
||||
if (verify.pattern !== undefined) {
|
||||
const content = readFileSync(absPath, "utf-8");
|
||||
try {
|
||||
if (!new RegExp(verify.pattern).test(content)) {
|
||||
return "pause";
|
||||
}
|
||||
} catch {
|
||||
// Invalid regex at runtime — treat as verification failure
|
||||
return "pause";
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return "continue";
|
||||
}
|
||||
|
||||
/**
|
||||
* shell-command handler.
|
||||
*
|
||||
* Runs the command via `sh -c` with cwd set to the run directory
|
||||
* and a 30-second timeout. Returns "continue" if exit code 0,
|
||||
* "retry" otherwise (including timeout/signal kills).
|
||||
*
|
||||
* SECURITY: The command string comes from a frozen DEFINITION.yaml written
|
||||
* at run-creation time. The trust boundary is the workflow definition author.
|
||||
* Commands run with the same privileges as the GSD process. Only use
|
||||
* shell-command verification with definitions you trust.
|
||||
*/
|
||||
function handleShellCommand(
|
||||
runDir: string,
|
||||
verify: { policy: "shell-command"; command: string },
|
||||
): VerificationOutcome {
|
||||
// Guard: reject commands containing shell expansion patterns that suggest injection
|
||||
const dangerousPatterns = /\$\(|`|;\s*(rm|curl|wget|nc|bash|sh|eval)\b/;
|
||||
if (dangerousPatterns.test(verify.command)) {
|
||||
console.warn(
|
||||
`custom-verification: shell-command contains suspicious pattern, skipping: ${verify.command}`,
|
||||
);
|
||||
return "pause";
|
||||
}
|
||||
|
||||
const result = spawnSync("sh", ["-c", verify.command], {
|
||||
cwd: runDir,
|
||||
timeout: 30_000,
|
||||
encoding: "utf-8",
|
||||
stdio: "pipe",
|
||||
env: { ...process.env, PATH: process.env.PATH },
|
||||
});
|
||||
|
||||
if (result.status === 0) {
|
||||
return "continue";
|
||||
}
|
||||
|
||||
return "retry";
|
||||
}
|
||||
216
src/resources/extensions/gsd/custom-workflow-engine.ts
Normal file
216
src/resources/extensions/gsd/custom-workflow-engine.ts
Normal file
|
|
@ -0,0 +1,216 @@
|
|||
/**
|
||||
* custom-workflow-engine.ts — WorkflowEngine implementation for custom workflows.
|
||||
*
|
||||
* Drives the auto-loop using GRAPH.yaml step state from a run directory.
|
||||
* Each iteration: deriveState reads the graph, resolveDispatch picks the
|
||||
* next eligible step, reconcile marks it complete and persists.
|
||||
*
|
||||
* Observability:
|
||||
* - All state reads/writes go through graph.ts YAML I/O — inspectable on disk.
|
||||
* - `resolveDispatch` returns unitType "custom-step" with unitId "<name>/<stepId>".
|
||||
* - `getDisplayMetadata` provides step N/M progress for dashboard rendering.
|
||||
* - Phase transitions are derivable from GRAPH.yaml step statuses.
|
||||
*/
|
||||
|
||||
import type { WorkflowEngine } from "./workflow-engine.js";
|
||||
import type {
|
||||
EngineState,
|
||||
EngineDispatchAction,
|
||||
CompletedStep,
|
||||
ReconcileResult,
|
||||
DisplayMetadata,
|
||||
} from "./engine-types.js";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { parse } from "yaml";
|
||||
import {
|
||||
readGraph,
|
||||
writeGraph,
|
||||
getNextPendingStep,
|
||||
markStepComplete,
|
||||
expandIteration,
|
||||
type WorkflowGraph,
|
||||
} from "./graph.js";
|
||||
import { injectContext } from "./context-injector.js";
|
||||
import type { WorkflowDefinition, StepDefinition } from "./definition-loader.js";
|
||||
|
||||
/** Read and parse the frozen DEFINITION.yaml from a run directory. */
|
||||
export function readFrozenDefinition(runDir: string): WorkflowDefinition {
|
||||
const defPath = join(runDir, "DEFINITION.yaml");
|
||||
const raw = readFileSync(defPath, "utf-8");
|
||||
return parse(raw, { schema: "core" }) as WorkflowDefinition;
|
||||
}
|
||||
|
||||
export class CustomWorkflowEngine implements WorkflowEngine {
|
||||
readonly engineId = "custom";
|
||||
private readonly runDir: string;
|
||||
|
||||
constructor(runDir: string) {
|
||||
this.runDir = runDir;
|
||||
}
|
||||
|
||||
/**
|
||||
* Derive engine state from GRAPH.yaml on disk.
|
||||
*
|
||||
* Phase is "complete" when all steps are complete or expanded,
|
||||
* "running" otherwise (any pending or active steps remain).
|
||||
*/
|
||||
async deriveState(_basePath: string): Promise<EngineState> {
|
||||
const graph = readGraph(this.runDir);
|
||||
const allDone = graph.steps.every(
|
||||
(s) => s.status === "complete" || s.status === "expanded",
|
||||
);
|
||||
const phase = allDone ? "complete" : "running";
|
||||
|
||||
return {
|
||||
phase,
|
||||
currentMilestoneId: null,
|
||||
activeSliceId: null,
|
||||
activeTaskId: null,
|
||||
isComplete: allDone,
|
||||
raw: graph,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the next dispatch action from graph state.
|
||||
*
|
||||
* Uses getNextPendingStep to find the first step whose dependencies
|
||||
* are all satisfied. If the step has an `iterate` config in the frozen
|
||||
* DEFINITION.yaml, expands it into instance steps before dispatching.
|
||||
*
|
||||
* Returns a dispatch with unitType "custom-step" and unitId in
|
||||
* "<workflowName>/<stepId>" format.
|
||||
*
|
||||
* Observability:
|
||||
* - Iterate expansion is logged to stderr with item count and parent step ID.
|
||||
* - Missing source artifacts throw with the full resolved path for diagnosis.
|
||||
* - Zero-match expansions return a stop action with level "info".
|
||||
* - Expanded GRAPH.yaml is written to disk before dispatch — inspectable on disk.
|
||||
*/
|
||||
async resolveDispatch(
|
||||
state: EngineState,
|
||||
_context: { basePath: string },
|
||||
): Promise<EngineDispatchAction> {
|
||||
let graph = state.raw as WorkflowGraph;
|
||||
let next = getNextPendingStep(graph);
|
||||
|
||||
if (!next) {
|
||||
return {
|
||||
action: "stop",
|
||||
reason: "All steps complete",
|
||||
level: "info",
|
||||
};
|
||||
}
|
||||
|
||||
// Check frozen DEFINITION.yaml for iterate config on this step
|
||||
const def = readFrozenDefinition(this.runDir);
|
||||
const stepDef = def.steps.find((s: StepDefinition) => s.id === next!.id);
|
||||
|
||||
if (stepDef?.iterate) {
|
||||
const iterate = stepDef.iterate;
|
||||
|
||||
// Read source artifact
|
||||
const sourcePath = join(this.runDir, iterate.source);
|
||||
let sourceContent: string;
|
||||
try {
|
||||
sourceContent = readFileSync(sourcePath, "utf-8");
|
||||
} catch {
|
||||
throw new Error(
|
||||
`Iterate source artifact not found: ${sourcePath} (step "${next.id}", source: "${iterate.source}")`,
|
||||
);
|
||||
}
|
||||
|
||||
// Extract items via regex with global+multiline flags.
|
||||
// Guard against ReDoS: if matching takes too long on large inputs, bail.
|
||||
const regex = new RegExp(iterate.pattern, "gm");
|
||||
const items: string[] = [];
|
||||
const matchStart = Date.now();
|
||||
let match: RegExpExecArray | null;
|
||||
while ((match = regex.exec(sourceContent)) !== null) {
|
||||
if (match[1] !== undefined) items.push(match[1]);
|
||||
if (Date.now() - matchStart > 5_000) {
|
||||
throw new Error(
|
||||
`Iterate pattern "${iterate.pattern}" exceeded 5s timeout on step "${next.id}" — possible ReDoS`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// Expand the graph
|
||||
const expandedGraph = expandIteration(graph, next.id, items, next.prompt);
|
||||
writeGraph(this.runDir, expandedGraph);
|
||||
graph = expandedGraph;
|
||||
|
||||
// Re-query for first instance step
|
||||
next = getNextPendingStep(expandedGraph);
|
||||
|
||||
if (!next) {
|
||||
return {
|
||||
action: "stop",
|
||||
reason: "Iterate expansion produced no instances",
|
||||
level: "info",
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
// Enrich prompt with context from prior step artifacts
|
||||
const enrichedPrompt = injectContext(this.runDir, next.id, next.prompt);
|
||||
|
||||
return {
|
||||
action: "dispatch",
|
||||
step: {
|
||||
unitType: "custom-step",
|
||||
unitId: `${graph.metadata.name}/${next.id}`,
|
||||
prompt: enrichedPrompt,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Reconcile state after a step completes.
|
||||
*
|
||||
* Extracts the stepId from the completedStep's unitId (last segment after `/`),
|
||||
* marks it complete in the graph, and writes the updated GRAPH.yaml to disk.
|
||||
*
|
||||
* Returns "milestone-complete" when all steps are now done, "continue" otherwise.
|
||||
*/
|
||||
async reconcile(
|
||||
state: EngineState,
|
||||
completedStep: CompletedStep,
|
||||
): Promise<ReconcileResult> {
|
||||
const graph = state.raw as WorkflowGraph;
|
||||
|
||||
// Extract stepId from "<workflowName>/<stepId>"
|
||||
const parts = completedStep.unitId.split("/");
|
||||
const stepId = parts[parts.length - 1];
|
||||
|
||||
const updatedGraph = markStepComplete(graph, stepId);
|
||||
writeGraph(this.runDir, updatedGraph);
|
||||
|
||||
const allDone = updatedGraph.steps.every(
|
||||
(s) => s.status === "complete" || s.status === "expanded",
|
||||
);
|
||||
|
||||
return {
|
||||
outcome: allDone ? "milestone-complete" : "continue",
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Return UI-facing metadata for progress display.
|
||||
*
|
||||
* Shows "Step N/M" progress where N = completed count and M = total.
|
||||
*/
|
||||
getDisplayMetadata(state: EngineState): DisplayMetadata {
|
||||
const graph = state.raw as WorkflowGraph;
|
||||
const total = graph.steps.length;
|
||||
const completed = graph.steps.filter((s) => s.status === "complete").length;
|
||||
|
||||
return {
|
||||
engineLabel: "WORKFLOW",
|
||||
currentPhase: state.phase,
|
||||
progressSummary: `Step ${completed}/${total}`,
|
||||
stepCount: { completed, total },
|
||||
};
|
||||
}
|
||||
}
|
||||
|
|
@ -38,6 +38,7 @@ function unitLabel(type: string): string {
|
|||
case "triage-captures": return "Triage";
|
||||
case "quick-task": return "Quick Task";
|
||||
case "replan-slice": return "Replan";
|
||||
case "custom-step": return "Workflow Step";
|
||||
default: return type;
|
||||
}
|
||||
}
|
||||
|
|
|
|||
462
src/resources/extensions/gsd/definition-loader.ts
Normal file
462
src/resources/extensions/gsd/definition-loader.ts
Normal file
|
|
@ -0,0 +1,462 @@
|
|||
/**
|
||||
* definition-loader.ts — Parse and validate V1 YAML workflow definitions.
|
||||
*
|
||||
* Loads definition YAML files from `.gsd/workflow-defs/`, validates the
|
||||
* V1 schema shape, and returns typed TypeScript objects. Pure functions
|
||||
* with no engine or runtime dependencies — just `yaml` and `node:fs`.
|
||||
*
|
||||
* YAML uses snake_case (`depends_on`, `context_from`) per project convention (P005).
|
||||
* TypeScript uses camelCase (`dependsOn`, `contextFrom`).
|
||||
*
|
||||
* Observability: All validation errors are collected into a string[] — callers
|
||||
* can log, surface in dashboards, or return to agents for self-repair.
|
||||
* substituteParams errors include the offending key name for traceability.
|
||||
*/
|
||||
|
||||
import { parse } from "yaml";
|
||||
import { readFileSync, existsSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
|
||||
// ─── Public TypeScript Types (camelCase) ─────────────────────────────────
|
||||
|
||||
export type VerifyPolicy =
|
||||
| { policy: "content-heuristic"; minSize?: number; pattern?: string }
|
||||
| { policy: "shell-command"; command: string }
|
||||
| { policy: "prompt-verify"; prompt: string }
|
||||
| { policy: "human-review" };
|
||||
|
||||
export interface IterateConfig {
|
||||
/** Artifact path (relative to run dir) to read and match against. */
|
||||
source: string;
|
||||
/** Regex pattern string. Must contain at least one capture group. Applied with global flag. */
|
||||
pattern: string;
|
||||
}
|
||||
|
||||
export interface StepDefinition {
|
||||
/** Unique step identifier within the workflow. */
|
||||
id: string;
|
||||
/** Human-readable step name. */
|
||||
name: string;
|
||||
/** The prompt to dispatch for this step. */
|
||||
prompt: string;
|
||||
/** IDs of steps that must complete before this step can run. */
|
||||
requires: string[];
|
||||
/** Artifact paths produced by this step (relative to run dir). */
|
||||
produces: string[];
|
||||
/** Step IDs whose artifacts to include as context (S05 — accepted, not processed). */
|
||||
contextFrom?: string[];
|
||||
/** Verification policy for this step (S05 — typed + validated). */
|
||||
verify?: VerifyPolicy;
|
||||
/** Iteration config for this step (S06 — typed + validated). */
|
||||
iterate?: IterateConfig;
|
||||
}
|
||||
|
||||
export interface WorkflowDefinition {
|
||||
/** Schema version — must be 1. */
|
||||
version: number;
|
||||
/** Workflow name. */
|
||||
name: string;
|
||||
/** Optional description. */
|
||||
description?: string;
|
||||
/** Optional parameter map for template substitution (S07). */
|
||||
params?: Record<string, string>;
|
||||
/** Ordered list of steps. */
|
||||
steps: StepDefinition[];
|
||||
}
|
||||
|
||||
// ─── Internal YAML Types (snake_case) ────────────────────────────────────
|
||||
|
||||
interface YamlStepDef {
|
||||
id?: unknown;
|
||||
name?: unknown;
|
||||
prompt?: unknown;
|
||||
requires?: unknown;
|
||||
depends_on?: unknown;
|
||||
produces?: unknown;
|
||||
context_from?: unknown;
|
||||
verify?: unknown;
|
||||
iterate?: unknown;
|
||||
[key: string]: unknown; // Forward-compat: unknown fields accepted silently
|
||||
}
|
||||
|
||||
interface YamlWorkflowDef {
|
||||
version?: unknown;
|
||||
name?: unknown;
|
||||
description?: unknown;
|
||||
params?: unknown;
|
||||
steps?: unknown;
|
||||
[key: string]: unknown; // Forward-compat: unknown fields accepted silently
|
||||
}
|
||||
|
||||
// ─── Validation ──────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Validate a parsed (but untyped) YAML object against the V1 workflow schema.
|
||||
*
|
||||
* Collects all errors (does not short-circuit) so a single call reveals
|
||||
* every problem with the definition.
|
||||
*
|
||||
* Unknown fields are silently accepted for forward compatibility with
|
||||
* S05/S06 features (`context_from`, `verify`, `iterate`).
|
||||
*/
|
||||
export function validateDefinition(parsed: unknown): { valid: boolean; errors: string[] } {
|
||||
const errors: string[] = [];
|
||||
|
||||
if (parsed == null || typeof parsed !== "object") {
|
||||
return { valid: false, errors: ["Definition must be a non-null object"] };
|
||||
}
|
||||
|
||||
const def = parsed as YamlWorkflowDef;
|
||||
|
||||
// version: must be 1 (number)
|
||||
if (def.version === undefined || def.version === null) {
|
||||
errors.push("Missing required field: version");
|
||||
} else if (def.version !== 1) {
|
||||
errors.push(`Unsupported version: ${def.version} (expected 1)`);
|
||||
}
|
||||
|
||||
// name: must be a non-empty string
|
||||
if (typeof def.name !== "string" || def.name.trim() === "") {
|
||||
errors.push("Missing or empty required field: name");
|
||||
}
|
||||
|
||||
// steps: must be a non-empty array
|
||||
if (!Array.isArray(def.steps)) {
|
||||
errors.push("Missing required field: steps (must be an array)");
|
||||
} else if (def.steps.length === 0) {
|
||||
errors.push("steps must contain at least one step");
|
||||
} else {
|
||||
// Track whether all steps have valid IDs — graph-level checks only run when true
|
||||
let allStepIdsValid = true;
|
||||
|
||||
for (let i = 0; i < def.steps.length; i++) {
|
||||
const step = def.steps[i] as YamlStepDef;
|
||||
if (step == null || typeof step !== "object") {
|
||||
errors.push(`Step at index ${i} is not an object`);
|
||||
allStepIdsValid = false;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Required step fields
|
||||
if (typeof step.id !== "string" || step.id.trim() === "") {
|
||||
errors.push(`Step at index ${i} missing required field: id`);
|
||||
allStepIdsValid = false;
|
||||
}
|
||||
if (typeof step.name !== "string" || step.name.trim() === "") {
|
||||
errors.push(`Step at index ${i} missing required field: name`);
|
||||
}
|
||||
if (typeof step.prompt !== "string" || step.prompt.trim() === "") {
|
||||
errors.push(`Step at index ${i} missing required field: prompt`);
|
||||
}
|
||||
|
||||
// produces: path traversal guard
|
||||
if (Array.isArray(step.produces)) {
|
||||
for (const p of step.produces) {
|
||||
if (typeof p === "string" && p.includes("..")) {
|
||||
errors.push(`Step "${step.id}" produces path contains disallowed '..': ${p}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// iterate: optional, but if present must conform to IterateConfig shape
|
||||
if (step.iterate !== undefined) {
|
||||
const it = step.iterate;
|
||||
const sid = typeof step.id === "string" ? step.id : `index ${i}`;
|
||||
if (it == null || typeof it !== "object" || Array.isArray(it)) {
|
||||
errors.push(`Step "${sid}" iterate must be an object with "source" and "pattern" fields`);
|
||||
} else {
|
||||
const itObj = it as Record<string, unknown>;
|
||||
if (typeof itObj.source !== "string" || (itObj.source as string).trim() === "") {
|
||||
errors.push(`Step "${sid}" iterate.source must be a non-empty string`);
|
||||
} else if ((itObj.source as string).includes("..")) {
|
||||
errors.push(`Step "${sid}" iterate.source contains disallowed '..' path traversal`);
|
||||
}
|
||||
if (typeof itObj.pattern !== "string" || (itObj.pattern as string).trim() === "") {
|
||||
errors.push(`Step "${sid}" iterate.pattern must be a non-empty string`);
|
||||
} else {
|
||||
const pat = itObj.pattern as string;
|
||||
let regexValid = true;
|
||||
try {
|
||||
new RegExp(pat);
|
||||
} catch {
|
||||
regexValid = false;
|
||||
errors.push(`Step "${sid}" iterate.pattern is not a valid regex: ${pat}`);
|
||||
}
|
||||
if (regexValid && !/\((?!\?)/.test(pat)) {
|
||||
errors.push(`Step "${sid}" iterate.pattern must contain at least one capture group`);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// verify: optional, but if present must conform to VerifyPolicy shape
|
||||
if (step.verify !== undefined) {
|
||||
const v = step.verify;
|
||||
const sid = typeof step.id === "string" ? step.id : `index ${i}`;
|
||||
if (v == null || typeof v !== "object" || Array.isArray(v)) {
|
||||
errors.push(`Step "${sid}" verify must be an object with a "policy" field`);
|
||||
} else {
|
||||
const vObj = v as Record<string, unknown>;
|
||||
const VALID_POLICIES = ["content-heuristic", "shell-command", "prompt-verify", "human-review"];
|
||||
if (typeof vObj.policy !== "string" || !VALID_POLICIES.includes(vObj.policy)) {
|
||||
errors.push(`Step "${sid}" verify.policy must be one of: ${VALID_POLICIES.join(", ")}`);
|
||||
} else {
|
||||
// Policy-specific required field checks
|
||||
if (vObj.policy === "shell-command") {
|
||||
if (typeof vObj.command !== "string" || (vObj.command as string).trim() === "") {
|
||||
errors.push(`Step "${sid}" verify policy "shell-command" requires a non-empty "command" field`);
|
||||
}
|
||||
}
|
||||
if (vObj.policy === "prompt-verify") {
|
||||
if (typeof vObj.prompt !== "string" || (vObj.prompt as string).trim() === "") {
|
||||
errors.push(`Step "${sid}" verify policy "prompt-verify" requires a non-empty "prompt" field`);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Graph-level validations (only when all step IDs are valid) ────
|
||||
if (allStepIdsValid) {
|
||||
const steps = def.steps as YamlStepDef[];
|
||||
|
||||
// 1. Duplicate step ID check
|
||||
const idCounts = new Map<string, number>();
|
||||
for (const step of steps) {
|
||||
const id = step.id as string;
|
||||
idCounts.set(id, (idCounts.get(id) ?? 0) + 1);
|
||||
}
|
||||
for (const [id, count] of idCounts) {
|
||||
if (count > 1) {
|
||||
errors.push(`Duplicate step id: ${id}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Build valid ID set for remaining checks
|
||||
const validIds = new Set(steps.map((s) => s.id as string));
|
||||
|
||||
// 2. Dangling dependency check + 3. Self-referencing dependency check
|
||||
for (const step of steps) {
|
||||
const sid = step.id as string;
|
||||
const deps = Array.isArray(step.requires)
|
||||
? (step.requires as string[])
|
||||
: Array.isArray(step.depends_on)
|
||||
? (step.depends_on as string[])
|
||||
: [];
|
||||
|
||||
for (const depId of deps) {
|
||||
if (depId === sid) {
|
||||
errors.push(`Step '${sid}' depends on itself`);
|
||||
} else if (!validIds.has(depId)) {
|
||||
errors.push(`Step '${sid}' requires unknown step '${depId}'`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 4. Cycle detection (DFS) — only when no duplicate IDs
|
||||
if (![...idCounts.values()].some((c: number) => c > 1)) {
|
||||
// Build adjacency list: step → its dependencies
|
||||
const adj = new Map<string, string[]>();
|
||||
for (const step of steps) {
|
||||
const sid = step.id as string;
|
||||
const deps = Array.isArray(step.requires)
|
||||
? (step.requires as string[])
|
||||
: Array.isArray(step.depends_on)
|
||||
? (step.depends_on as string[])
|
||||
: [];
|
||||
adj.set(sid, deps.filter((d) => validIds.has(d) && d !== sid));
|
||||
}
|
||||
|
||||
const WHITE = 0, GRAY = 1, BLACK = 2;
|
||||
const color = new Map<string, number>();
|
||||
for (const id of validIds) color.set(id, WHITE);
|
||||
|
||||
const parent = new Map<string, string | null>();
|
||||
|
||||
function dfs(node: string): string[] | null {
|
||||
color.set(node, GRAY);
|
||||
for (const dep of adj.get(node) ?? []) {
|
||||
if (color.get(dep) === GRAY) {
|
||||
// Back edge found — reconstruct cycle path
|
||||
const cycle: string[] = [dep, node];
|
||||
let cur = node;
|
||||
while (parent.has(cur) && parent.get(cur) !== null && parent.get(cur) !== dep) {
|
||||
cur = parent.get(cur)!;
|
||||
cycle.push(cur);
|
||||
}
|
||||
cycle.push(dep);
|
||||
cycle.reverse();
|
||||
return cycle;
|
||||
}
|
||||
if (color.get(dep) === WHITE) {
|
||||
parent.set(dep, node);
|
||||
const result = dfs(dep);
|
||||
if (result) return result;
|
||||
}
|
||||
}
|
||||
color.set(node, BLACK);
|
||||
return null;
|
||||
}
|
||||
|
||||
for (const id of validIds) {
|
||||
if (color.get(id) === WHITE) {
|
||||
parent.set(id, null);
|
||||
const cycle = dfs(id);
|
||||
if (cycle) {
|
||||
errors.push(`Cycle detected: ${cycle.join(" → ")}`);
|
||||
break; // One cycle error is enough
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return { valid: errors.length === 0, errors };
|
||||
}
|
||||
|
||||
// ─── Loading ─────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Load and validate a YAML workflow definition from the filesystem.
|
||||
*
|
||||
* Reads `<defsDir>/<name>.yaml`, parses YAML, validates the V1 schema,
|
||||
* and converts snake_case YAML keys to camelCase TypeScript types.
|
||||
*
|
||||
* @param defsDir — directory containing definition YAML files
|
||||
* @param name — definition filename without extension
|
||||
* @returns Parsed and validated WorkflowDefinition
|
||||
* @throws Error if file is missing, YAML is malformed, or schema is invalid
|
||||
*/
|
||||
export function loadDefinition(defsDir: string, name: string): WorkflowDefinition {
|
||||
const filePath = join(defsDir, `${name}.yaml`);
|
||||
|
||||
if (!existsSync(filePath)) {
|
||||
throw new Error(`Definition file not found: ${filePath}`);
|
||||
}
|
||||
|
||||
const raw = readFileSync(filePath, "utf-8");
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = parse(raw);
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
throw new Error(`Failed to parse YAML in ${filePath}: ${msg}`);
|
||||
}
|
||||
|
||||
const { valid, errors } = validateDefinition(parsed);
|
||||
if (!valid) {
|
||||
throw new Error(`Invalid workflow definition in ${filePath}:\n - ${errors.join("\n - ")}`);
|
||||
}
|
||||
|
||||
// Convert snake_case YAML → camelCase TypeScript
|
||||
const yamlDef = parsed as YamlWorkflowDef;
|
||||
const yamlSteps = yamlDef.steps as YamlStepDef[];
|
||||
|
||||
return {
|
||||
version: yamlDef.version as number,
|
||||
name: yamlDef.name as string,
|
||||
description: typeof yamlDef.description === "string" ? yamlDef.description : undefined,
|
||||
params: yamlDef.params != null && typeof yamlDef.params === "object"
|
||||
? Object.fromEntries(
|
||||
Object.entries(yamlDef.params as Record<string, unknown>).map(
|
||||
([k, v]) => [k, String(v)],
|
||||
),
|
||||
)
|
||||
: undefined,
|
||||
steps: yamlSteps.map((s) => ({
|
||||
id: s.id as string,
|
||||
name: s.name as string,
|
||||
prompt: s.prompt as string,
|
||||
requires: Array.isArray(s.requires)
|
||||
? (s.requires as string[])
|
||||
: Array.isArray(s.depends_on)
|
||||
? (s.depends_on as string[])
|
||||
: [],
|
||||
produces: Array.isArray(s.produces) ? (s.produces as string[]) : [],
|
||||
contextFrom: Array.isArray(s.context_from) ? (s.context_from as string[]) : undefined,
|
||||
verify: s.verify as VerifyPolicy | undefined,
|
||||
iterate: (s.iterate != null && typeof s.iterate === "object")
|
||||
? s.iterate as IterateConfig
|
||||
: undefined,
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
||||
// ─── Parameter Substitution ──────────────────────────────────────────────
|
||||
|
||||
/** Regex matching `{{key}}` placeholders — captures the key name. */
|
||||
const PARAM_PATTERN = /\{\{(\w+)\}\}/g;
|
||||
|
||||
/**
|
||||
* Replace `{{key}}` placeholders in a single prompt string.
|
||||
*
|
||||
* Exported for use by the engine on iteration-instance prompts that live
|
||||
* in GRAPH.yaml (outside the definition's step list).
|
||||
*
|
||||
* @throws Error if any merged param value contains `..` (path-traversal guard)
|
||||
*/
|
||||
export function substitutePromptString(
|
||||
prompt: string,
|
||||
merged: Record<string, string>,
|
||||
): string {
|
||||
return prompt.replace(PARAM_PATTERN, (match, key: string) => {
|
||||
const value = merged[key];
|
||||
return value !== undefined ? value : match;
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Replace `{{key}}` placeholders in all step prompts with param values.
|
||||
*
|
||||
* Merge order: `definition.params` (defaults) ← `overrides` (CLI wins).
|
||||
* Returns a **new** WorkflowDefinition — the input is never mutated.
|
||||
*
|
||||
* @throws Error if any param value contains `..` (path-traversal guard)
|
||||
* @throws Error if any `{{key}}` remains unresolved after substitution
|
||||
*/
|
||||
export function substituteParams(
|
||||
definition: WorkflowDefinition,
|
||||
overrides?: Record<string, string>,
|
||||
): WorkflowDefinition {
|
||||
const merged: Record<string, string> = {
|
||||
...(definition.params ?? {}),
|
||||
...(overrides ?? {}),
|
||||
};
|
||||
|
||||
// Path-traversal guard: reject any value containing ".."
|
||||
for (const [key, value] of Object.entries(merged)) {
|
||||
if (value.includes("..")) {
|
||||
throw new Error(
|
||||
`Parameter "${key}" contains disallowed '..' (path traversal): ${value}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// Substitute in each step prompt
|
||||
const substitutedSteps = definition.steps.map((step) => ({
|
||||
...step,
|
||||
prompt: substitutePromptString(step.prompt, merged),
|
||||
}));
|
||||
|
||||
// Check for unresolved placeholders
|
||||
const unresolved = new Set<string>();
|
||||
for (const step of substitutedSteps) {
|
||||
let m: RegExpExecArray | null;
|
||||
const re = new RegExp(PARAM_PATTERN.source, "g");
|
||||
while ((m = re.exec(step.prompt)) !== null) {
|
||||
unresolved.add(m[1]);
|
||||
}
|
||||
}
|
||||
|
||||
if (unresolved.size > 0) {
|
||||
const keys = [...unresolved].sort().join(", ");
|
||||
throw new Error(`Unresolved parameter(s) in step prompts: ${keys}`);
|
||||
}
|
||||
|
||||
return {
|
||||
...definition,
|
||||
steps: substitutedSteps,
|
||||
};
|
||||
}
|
||||
51
src/resources/extensions/gsd/dev-execution-policy.ts
Normal file
51
src/resources/extensions/gsd/dev-execution-policy.ts
Normal file
|
|
@ -0,0 +1,51 @@
|
|||
/**
|
||||
* dev-execution-policy.ts — DevExecutionPolicy implementation.
|
||||
*
|
||||
* Stub policy for the dev engine. All methods return safe defaults.
|
||||
* Real verification/closeout continues running through phases.ts via LoopDeps.
|
||||
* Wiring this policy into the loop is S04's responsibility.
|
||||
*/
|
||||
|
||||
import type { ExecutionPolicy } from "./execution-policy.js";
|
||||
import type { RecoveryAction, CloseoutResult } from "./engine-types.js";
|
||||
|
||||
export class DevExecutionPolicy implements ExecutionPolicy {
|
||||
async prepareWorkspace(
|
||||
_basePath: string,
|
||||
_milestoneId: string,
|
||||
): Promise<void> {
|
||||
// no-op — workspace preparation handled by existing GSD logic
|
||||
}
|
||||
|
||||
async selectModel(
|
||||
_unitType: string,
|
||||
_unitId: string,
|
||||
_context: { basePath: string },
|
||||
): Promise<{ tier: string; modelDowngraded: boolean } | null> {
|
||||
return null; // use default model selection
|
||||
}
|
||||
|
||||
async verify(
|
||||
_unitType: string,
|
||||
_unitId: string,
|
||||
_context: { basePath: string },
|
||||
): Promise<"continue" | "retry" | "pause"> {
|
||||
return "continue";
|
||||
}
|
||||
|
||||
async recover(
|
||||
_unitType: string,
|
||||
_unitId: string,
|
||||
_context: { basePath: string },
|
||||
): Promise<RecoveryAction> {
|
||||
return { outcome: "retry" };
|
||||
}
|
||||
|
||||
async closeout(
|
||||
_unitType: string,
|
||||
_unitId: string,
|
||||
_context: { basePath: string; startedAt: number },
|
||||
): Promise<CloseoutResult> {
|
||||
return { committed: false, artifacts: [] };
|
||||
}
|
||||
}
|
||||
110
src/resources/extensions/gsd/dev-workflow-engine.ts
Normal file
110
src/resources/extensions/gsd/dev-workflow-engine.ts
Normal file
|
|
@ -0,0 +1,110 @@
|
|||
/**
|
||||
* dev-workflow-engine.ts — DevWorkflowEngine implementation.
|
||||
*
|
||||
* Implements WorkflowEngine by delegating to existing GSD state derivation
|
||||
* and dispatch logic. This is the "dev" engine — it wraps the current GSD
|
||||
* auto-mode behavior behind the engine-polymorphic interface.
|
||||
*/
|
||||
|
||||
import type { WorkflowEngine } from "./workflow-engine.js";
|
||||
import type {
|
||||
EngineState,
|
||||
EngineDispatchAction,
|
||||
CompletedStep,
|
||||
ReconcileResult,
|
||||
DisplayMetadata,
|
||||
} from "./engine-types.js";
|
||||
import type { GSDState } from "./types.js";
|
||||
import type { DispatchAction, DispatchContext } from "./auto-dispatch.js";
|
||||
|
||||
import { deriveState } from "./state.js";
|
||||
import { resolveDispatch } from "./auto-dispatch.js";
|
||||
import { loadEffectiveGSDPreferences } from "./preferences.js";
|
||||
|
||||
// ─── Bridge: DispatchAction → EngineDispatchAction ────────────────────────
|
||||
|
||||
/**
|
||||
* Map a GSD-specific DispatchAction (which carries `matchedRule`, `unitType`,
|
||||
* etc.) to the engine-generic EngineDispatchAction discriminated union.
|
||||
*
|
||||
* Exported for unit testing.
|
||||
*/
|
||||
export function bridgeDispatchAction(da: DispatchAction): EngineDispatchAction {
|
||||
switch (da.action) {
|
||||
case "dispatch":
|
||||
return {
|
||||
action: "dispatch",
|
||||
step: {
|
||||
unitType: da.unitType,
|
||||
unitId: da.unitId,
|
||||
prompt: da.prompt,
|
||||
},
|
||||
};
|
||||
case "stop":
|
||||
return {
|
||||
action: "stop",
|
||||
reason: da.reason,
|
||||
level: da.level,
|
||||
};
|
||||
case "skip":
|
||||
return { action: "skip" };
|
||||
}
|
||||
}
|
||||
|
||||
// ─── DevWorkflowEngine ───────────────────────────────────────────────────
|
||||
|
||||
export class DevWorkflowEngine implements WorkflowEngine {
|
||||
readonly engineId = "dev" as const;
|
||||
|
||||
async deriveState(basePath: string): Promise<EngineState> {
|
||||
const gsd: GSDState = await deriveState(basePath);
|
||||
return {
|
||||
phase: gsd.phase,
|
||||
currentMilestoneId: gsd.activeMilestone?.id ?? null,
|
||||
activeSliceId: gsd.activeSlice?.id ?? null,
|
||||
activeTaskId: gsd.activeTask?.id ?? null,
|
||||
isComplete: gsd.phase === "complete",
|
||||
raw: gsd,
|
||||
};
|
||||
}
|
||||
|
||||
async resolveDispatch(
|
||||
state: EngineState,
|
||||
context: { basePath: string },
|
||||
): Promise<EngineDispatchAction> {
|
||||
const gsd = state.raw as GSDState;
|
||||
const mid = gsd.activeMilestone?.id ?? "";
|
||||
const midTitle = gsd.activeMilestone?.title ?? "";
|
||||
const loaded = loadEffectiveGSDPreferences();
|
||||
const prefs = loaded?.preferences ?? undefined;
|
||||
|
||||
const dispatchCtx: DispatchContext = {
|
||||
basePath: context.basePath,
|
||||
mid,
|
||||
midTitle,
|
||||
state: gsd,
|
||||
prefs,
|
||||
};
|
||||
|
||||
const result = await resolveDispatch(dispatchCtx);
|
||||
return bridgeDispatchAction(result);
|
||||
}
|
||||
|
||||
async reconcile(
|
||||
state: EngineState,
|
||||
_completedStep: CompletedStep,
|
||||
): Promise<ReconcileResult> {
|
||||
return {
|
||||
outcome: state.isComplete ? "milestone-complete" : "continue",
|
||||
};
|
||||
}
|
||||
|
||||
getDisplayMetadata(state: EngineState): DisplayMetadata {
|
||||
return {
|
||||
engineLabel: "GSD Dev",
|
||||
currentPhase: state.phase,
|
||||
progressSummary: `${state.currentMilestoneId ?? "no milestone"} / ${state.activeSliceId ?? "—"} / ${state.activeTaskId ?? "—"}`,
|
||||
stepCount: null,
|
||||
};
|
||||
}
|
||||
}
|
||||
57
src/resources/extensions/gsd/engine-resolver.ts
Normal file
57
src/resources/extensions/gsd/engine-resolver.ts
Normal file
|
|
@ -0,0 +1,57 @@
|
|||
/**
|
||||
* engine-resolver.ts — Route sessions to engine/policy pairs.
|
||||
*
|
||||
* Routes `null` and `"dev"` engine IDs to the DevWorkflowEngine/DevExecutionPolicy
|
||||
* pair. Any other non-null engine ID is treated as a custom workflow engine that
|
||||
* reads its state from an `activeRunDir`. Respects `GSD_ENGINE_BYPASS=1` kill
|
||||
* switch to skip the engine layer entirely.
|
||||
*/
|
||||
|
||||
import type { WorkflowEngine } from "./workflow-engine.js";
|
||||
import type { ExecutionPolicy } from "./execution-policy.js";
|
||||
import { DevWorkflowEngine } from "./dev-workflow-engine.js";
|
||||
import { DevExecutionPolicy } from "./dev-execution-policy.js";
|
||||
import { CustomWorkflowEngine } from "./custom-workflow-engine.js";
|
||||
import { CustomExecutionPolicy } from "./custom-execution-policy.js";
|
||||
|
||||
/** A resolved engine + policy pair ready for the auto-loop. */
|
||||
export interface ResolvedEngine {
|
||||
engine: WorkflowEngine;
|
||||
policy: ExecutionPolicy;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve an engine/policy pair for the given session.
|
||||
*
|
||||
* - `null` or `"dev"` → DevWorkflowEngine + DevExecutionPolicy
|
||||
* - any other non-null ID → CustomWorkflowEngine(activeRunDir) + CustomExecutionPolicy()
|
||||
* (requires activeRunDir to be a non-empty string)
|
||||
*
|
||||
* Note: `GSD_ENGINE_BYPASS=1` is checked in autoLoop before calling this function.
|
||||
*/
|
||||
export function resolveEngine(
|
||||
session: { activeEngineId: string | null; activeRunDir?: string | null },
|
||||
): ResolvedEngine {
|
||||
const { activeEngineId, activeRunDir } = session;
|
||||
|
||||
if (activeEngineId === null || activeEngineId === "dev") {
|
||||
return {
|
||||
engine: new DevWorkflowEngine(),
|
||||
policy: new DevExecutionPolicy(),
|
||||
};
|
||||
}
|
||||
|
||||
// Any non-null, non-"dev" engine ID is a custom workflow engine.
|
||||
// activeRunDir is required — the engine reads GRAPH.yaml from it.
|
||||
if (!activeRunDir || typeof activeRunDir !== "string") {
|
||||
throw new Error(
|
||||
`Custom engine "${activeEngineId}" requires activeRunDir to be a non-empty string, ` +
|
||||
`got: ${JSON.stringify(activeRunDir)}`,
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
engine: new CustomWorkflowEngine(activeRunDir),
|
||||
policy: new CustomExecutionPolicy(activeRunDir),
|
||||
};
|
||||
}
|
||||
71
src/resources/extensions/gsd/engine-types.ts
Normal file
71
src/resources/extensions/gsd/engine-types.ts
Normal file
|
|
@ -0,0 +1,71 @@
|
|||
/**
|
||||
* engine-types.ts — Engine-polymorphic type contracts.
|
||||
*
|
||||
* LEAF NODE: This file must have ZERO imports from any GSD module.
|
||||
* Only `node:` imports are permitted. All engine/policy interfaces
|
||||
* depend on these types; nothing here depends on GSD internals.
|
||||
*/
|
||||
|
||||
/** Snapshot of engine state at a point in time. */
|
||||
export interface EngineState {
|
||||
phase: string;
|
||||
currentMilestoneId: string | null;
|
||||
activeSliceId: string | null;
|
||||
activeTaskId: string | null;
|
||||
isComplete: boolean;
|
||||
/** Opaque engine-specific state — never narrowed to a GSD-specific type. */
|
||||
raw: unknown;
|
||||
}
|
||||
|
||||
/** A unit of work the engine wants the agent to execute. */
|
||||
export interface StepContract {
|
||||
unitType: string;
|
||||
unitId: string;
|
||||
prompt: string;
|
||||
}
|
||||
|
||||
/** UI-facing metadata for progress display. */
|
||||
export interface DisplayMetadata {
|
||||
engineLabel: string;
|
||||
currentPhase: string;
|
||||
progressSummary: string;
|
||||
stepCount: { completed: number; total: number } | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Discriminated union: what the engine tells the loop to do next.
|
||||
*
|
||||
* - `dispatch` — execute a step
|
||||
* - `stop` — halt the loop with a reason and severity
|
||||
* - `skip` — nothing to do right now, advance without executing
|
||||
*/
|
||||
export type EngineDispatchAction =
|
||||
| { action: "dispatch"; step: StepContract }
|
||||
| { action: "stop"; reason: string; level: "info" | "warning" | "error" }
|
||||
| { action: "skip" };
|
||||
|
||||
/** Outcome of reconciling state after a step completes. */
|
||||
export interface ReconcileResult {
|
||||
outcome: "continue" | "milestone-complete" | "pause" | "stop";
|
||||
reason?: string;
|
||||
}
|
||||
|
||||
/** Recovery strategy when a step fails. */
|
||||
export interface RecoveryAction {
|
||||
outcome: "retry" | "skip" | "stop" | "pause";
|
||||
reason?: string;
|
||||
}
|
||||
|
||||
/** Result of closing out a completed unit. */
|
||||
export interface CloseoutResult {
|
||||
committed: boolean;
|
||||
artifacts: string[];
|
||||
}
|
||||
|
||||
/** Record of a completed execution step. */
|
||||
export interface CompletedStep {
|
||||
unitType: string;
|
||||
unitId: string;
|
||||
startedAt: number;
|
||||
finishedAt: number;
|
||||
}
|
||||
43
src/resources/extensions/gsd/execution-policy.ts
Normal file
43
src/resources/extensions/gsd/execution-policy.ts
Normal file
|
|
@ -0,0 +1,43 @@
|
|||
/**
|
||||
* execution-policy.ts — ExecutionPolicy interface.
|
||||
*
|
||||
* Defines the policy layer that governs model selection, verification,
|
||||
* recovery, and closeout for each execution step. Imports only from
|
||||
* the leaf-node engine-types.
|
||||
*/
|
||||
|
||||
import type { RecoveryAction, CloseoutResult } from "./engine-types.js";
|
||||
|
||||
/** Policy governing how each step is executed, verified, and closed out. */
|
||||
export interface ExecutionPolicy {
|
||||
/** Prepare the workspace before a milestone begins (e.g. worktree setup). */
|
||||
prepareWorkspace(basePath: string, milestoneId: string): Promise<void>;
|
||||
|
||||
/** Select the model tier for a given unit. Returns null to use defaults. */
|
||||
selectModel(
|
||||
unitType: string,
|
||||
unitId: string,
|
||||
context: { basePath: string },
|
||||
): Promise<{ tier: string; modelDowngraded: boolean } | null>;
|
||||
|
||||
/** Verify unit output. Returns disposition for the loop. */
|
||||
verify(
|
||||
unitType: string,
|
||||
unitId: string,
|
||||
context: { basePath: string },
|
||||
): Promise<"continue" | "retry" | "pause">;
|
||||
|
||||
/** Determine recovery action when a unit fails. */
|
||||
recover(
|
||||
unitType: string,
|
||||
unitId: string,
|
||||
context: { basePath: string },
|
||||
): Promise<RecoveryAction>;
|
||||
|
||||
/** Close out a completed unit (commit, snapshot, artifact capture). */
|
||||
closeout(
|
||||
unitType: string,
|
||||
unitId: string,
|
||||
context: { basePath: string; startedAt: number },
|
||||
): Promise<CloseoutResult>;
|
||||
}
|
||||
312
src/resources/extensions/gsd/graph.ts
Normal file
312
src/resources/extensions/gsd/graph.ts
Normal file
|
|
@ -0,0 +1,312 @@
|
|||
/**
|
||||
* graph.ts — Pure data module for GRAPH.yaml workflow step tracking.
|
||||
*
|
||||
* Provides types and functions for reading, writing, and querying the
|
||||
* step graph that drives CustomWorkflowEngine. Zero engine dependencies.
|
||||
*
|
||||
* GRAPH.yaml lives in a run directory and tracks step statuses
|
||||
* (pending → active → complete) with optional dependency edges.
|
||||
*
|
||||
* Observability:
|
||||
* - readGraph/writeGraph use YAML on disk — human-readable, diffable,
|
||||
* inspectable with `cat` or any YAML viewer.
|
||||
* - Each GraphStep has status, startedAt, finishedAt fields visible in GRAPH.yaml.
|
||||
* - writeGraph uses atomic write (tmp + rename) for crash safety.
|
||||
* - All operations are immutable — callers always get a new graph object.
|
||||
*/
|
||||
|
||||
import { parse, stringify } from "yaml";
|
||||
import { readFileSync, writeFileSync, renameSync, existsSync, mkdirSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import type { WorkflowDefinition } from "./definition-loader.js";
|
||||
|
||||
// ─── Types ───────────────────────────────────────────────────────────────
|
||||
|
||||
export interface GraphStep {
|
||||
/** Unique step identifier within the workflow. */
|
||||
id: string;
|
||||
/** Human-readable step title. */
|
||||
title: string;
|
||||
/** Current status: pending → active → complete → expanded (iterate parent). */
|
||||
status: "pending" | "active" | "complete" | "expanded";
|
||||
/** The prompt to dispatch for this step. */
|
||||
prompt: string;
|
||||
/** IDs of steps that must be "complete" before this step can run. */
|
||||
dependsOn: string[];
|
||||
/** For iteration instances: ID of the parent step that was expanded. */
|
||||
parentStepId?: string;
|
||||
/** ISO timestamp when the step started executing. */
|
||||
startedAt?: string;
|
||||
/** ISO timestamp when the step finished executing. */
|
||||
finishedAt?: string;
|
||||
}
|
||||
|
||||
export interface WorkflowGraph {
|
||||
/** Ordered list of steps in the workflow. */
|
||||
steps: GraphStep[];
|
||||
/** Workflow metadata. */
|
||||
metadata: {
|
||||
name: string;
|
||||
createdAt: string;
|
||||
};
|
||||
}
|
||||
|
||||
// ─── YAML schema mapping ─────────────────────────────────────────────────
|
||||
|
||||
const GRAPH_FILENAME = "GRAPH.yaml";
|
||||
|
||||
/**
|
||||
* Internal YAML shape — uses snake_case for YAML keys.
|
||||
* Converted to/from the camelCase TypeScript types on read/write.
|
||||
*/
|
||||
interface YamlStep {
|
||||
id: string;
|
||||
title: string;
|
||||
status: string;
|
||||
prompt: string;
|
||||
depends_on?: string[];
|
||||
parent_step_id?: string;
|
||||
started_at?: string;
|
||||
finished_at?: string;
|
||||
}
|
||||
|
||||
interface YamlGraph {
|
||||
steps: YamlStep[];
|
||||
metadata: { name: string; created_at: string };
|
||||
}
|
||||
|
||||
// ─── Functions ───────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Read and parse GRAPH.yaml from a run directory.
|
||||
*
|
||||
* @param runDir — directory containing GRAPH.yaml
|
||||
* @returns Parsed workflow graph
|
||||
* @throws Error if file doesn't exist or YAML is malformed
|
||||
*/
|
||||
export function readGraph(runDir: string): WorkflowGraph {
|
||||
const filePath = join(runDir, GRAPH_FILENAME);
|
||||
if (!existsSync(filePath)) {
|
||||
throw new Error(`GRAPH.yaml not found: ${filePath}`);
|
||||
}
|
||||
const raw = readFileSync(filePath, "utf-8");
|
||||
const yaml = parse(raw) as YamlGraph;
|
||||
|
||||
if (!yaml?.steps || !Array.isArray(yaml.steps)) {
|
||||
throw new Error(`Invalid GRAPH.yaml: missing or invalid 'steps' array in ${filePath}`);
|
||||
}
|
||||
|
||||
return {
|
||||
steps: yaml.steps.map((s) => ({
|
||||
id: s.id,
|
||||
title: s.title,
|
||||
status: s.status as GraphStep["status"],
|
||||
prompt: s.prompt,
|
||||
dependsOn: s.depends_on ?? [],
|
||||
...(s.parent_step_id != null ? { parentStepId: s.parent_step_id } : {}),
|
||||
...(s.started_at != null ? { startedAt: s.started_at } : {}),
|
||||
...(s.finished_at != null ? { finishedAt: s.finished_at } : {}),
|
||||
})),
|
||||
metadata: {
|
||||
name: yaml.metadata?.name ?? "unnamed",
|
||||
createdAt: yaml.metadata?.created_at ?? new Date().toISOString(),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Write a workflow graph to GRAPH.yaml in a run directory.
|
||||
* Creates the directory if it doesn't exist. Write is atomic (write + rename).
|
||||
*
|
||||
* @param runDir — directory to write GRAPH.yaml into
|
||||
* @param graph — the workflow graph to serialize
|
||||
*/
|
||||
export function writeGraph(runDir: string, graph: WorkflowGraph): void {
|
||||
if (!existsSync(runDir)) {
|
||||
mkdirSync(runDir, { recursive: true });
|
||||
}
|
||||
|
||||
const yamlData: YamlGraph = {
|
||||
steps: graph.steps.map((s) => ({
|
||||
id: s.id,
|
||||
title: s.title,
|
||||
status: s.status,
|
||||
prompt: s.prompt,
|
||||
depends_on: s.dependsOn.length > 0 ? s.dependsOn : undefined,
|
||||
parent_step_id: s.parentStepId ?? undefined,
|
||||
started_at: s.startedAt ?? undefined,
|
||||
finished_at: s.finishedAt ?? undefined,
|
||||
})) as YamlStep[],
|
||||
metadata: {
|
||||
name: graph.metadata.name,
|
||||
created_at: graph.metadata.createdAt,
|
||||
},
|
||||
};
|
||||
|
||||
const filePath = join(runDir, GRAPH_FILENAME);
|
||||
const tmpPath = filePath + ".tmp";
|
||||
const content = stringify(yamlData);
|
||||
writeFileSync(tmpPath, content, "utf-8");
|
||||
// Atomic rename for crash safety
|
||||
renameSync(tmpPath, filePath);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the next pending step whose dependencies are all complete.
|
||||
*
|
||||
* Returns the first step (in array order) with status "pending" where
|
||||
* every step in its `dependsOn` list has status "complete".
|
||||
*
|
||||
* @param graph — the workflow graph to query
|
||||
* @returns The next dispatchable step, or null if none available
|
||||
*/
|
||||
export function getNextPendingStep(graph: WorkflowGraph): GraphStep | null {
|
||||
const statusMap = new Map(graph.steps.map((s) => [s.id, s.status]));
|
||||
|
||||
for (const step of graph.steps) {
|
||||
if (step.status !== "pending") continue;
|
||||
const depsComplete = step.dependsOn.every(
|
||||
(depId) => statusMap.get(depId) === "complete",
|
||||
);
|
||||
if (depsComplete) return step;
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Return a new graph with the specified step marked as "complete".
|
||||
* Immutable — does not mutate the input graph.
|
||||
*
|
||||
* @param graph — the current workflow graph
|
||||
* @param stepId — ID of the step to mark complete
|
||||
* @returns New graph with the step's status set to "complete"
|
||||
* @throws Error if stepId is not found in the graph
|
||||
*/
|
||||
export function markStepComplete(
|
||||
graph: WorkflowGraph,
|
||||
stepId: string,
|
||||
): WorkflowGraph {
|
||||
const found = graph.steps.some((s) => s.id === stepId);
|
||||
if (!found) {
|
||||
throw new Error(`Step not found: ${stepId}`);
|
||||
}
|
||||
|
||||
return {
|
||||
...graph,
|
||||
steps: graph.steps.map((s) =>
|
||||
s.id === stepId
|
||||
? { ...s, status: "complete" as const, finishedAt: new Date().toISOString() }
|
||||
: s,
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
// ─── Iteration expansion ─────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Expand an iterate step into concrete instances. Pure and deterministic —
|
||||
* identical inputs always produce identical output.
|
||||
*
|
||||
* Given a parent step with status "pending" and an array of matched items,
|
||||
* creates one instance step per item, marks the parent as "expanded", and
|
||||
* rewrites any downstream dependsOn references from the parent ID to the
|
||||
* full set of instance IDs.
|
||||
*
|
||||
* @param graph — the current workflow graph (not mutated)
|
||||
* @param stepId — ID of the iterate step to expand
|
||||
* @param items — matched items from the source artifact
|
||||
* @param promptTemplate — template with {{item}} placeholders
|
||||
* @returns New WorkflowGraph with instances inserted and deps rewritten
|
||||
* @throws Error if stepId not found or step is not pending
|
||||
*/
|
||||
export function expandIteration(
|
||||
graph: WorkflowGraph,
|
||||
stepId: string,
|
||||
items: string[],
|
||||
promptTemplate: string,
|
||||
): WorkflowGraph {
|
||||
const parentIndex = graph.steps.findIndex((s) => s.id === stepId);
|
||||
if (parentIndex === -1) {
|
||||
throw new Error(`expandIteration: step not found: ${stepId}`);
|
||||
}
|
||||
const parentStep = graph.steps[parentIndex];
|
||||
if (parentStep.status !== "pending") {
|
||||
throw new Error(
|
||||
`expandIteration: step "${stepId}" has status "${parentStep.status}", expected "pending"`,
|
||||
);
|
||||
}
|
||||
|
||||
// Create instance steps
|
||||
const instanceIds: string[] = [];
|
||||
const instances: GraphStep[] = items.map((item, i) => {
|
||||
const instanceId = `${stepId}--${String(i + 1).padStart(3, "0")}`;
|
||||
instanceIds.push(instanceId);
|
||||
return {
|
||||
id: instanceId,
|
||||
title: `${parentStep.title}: ${item}`,
|
||||
status: "pending" as const,
|
||||
prompt: promptTemplate.replace(/\{\{item\}\}/g, () => item),
|
||||
dependsOn: [...parentStep.dependsOn],
|
||||
parentStepId: stepId,
|
||||
};
|
||||
});
|
||||
|
||||
// Build new steps array: copy everything, mark parent as expanded,
|
||||
// insert instances right after the parent, rewrite downstream deps.
|
||||
const newSteps: GraphStep[] = [];
|
||||
for (let i = 0; i < graph.steps.length; i++) {
|
||||
if (i === parentIndex) {
|
||||
// Mark parent as expanded
|
||||
newSteps.push({ ...parentStep, status: "expanded" as const });
|
||||
// Insert instances immediately after parent
|
||||
newSteps.push(...instances);
|
||||
} else {
|
||||
const step = graph.steps[i];
|
||||
// Rewrite dependsOn: replace parent ID with all instance IDs
|
||||
const hasDep = step.dependsOn.includes(stepId);
|
||||
if (hasDep) {
|
||||
const rewritten = step.dependsOn.flatMap((dep) =>
|
||||
dep === stepId ? instanceIds : [dep],
|
||||
);
|
||||
newSteps.push({ ...step, dependsOn: rewritten });
|
||||
} else {
|
||||
newSteps.push(step);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
...graph,
|
||||
steps: newSteps,
|
||||
};
|
||||
}
|
||||
|
||||
// ─── Definition → Graph conversion ──────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Convert a parsed WorkflowDefinition into a WorkflowGraph with all
|
||||
* steps in "pending" status. Used by run-manager to generate the initial
|
||||
* GRAPH.yaml for a new run.
|
||||
*
|
||||
* @param def — a validated WorkflowDefinition from definition-loader
|
||||
* @returns WorkflowGraph with pending steps and metadata from the definition
|
||||
*/
|
||||
export function initializeGraph(def: WorkflowDefinition): WorkflowGraph {
|
||||
return {
|
||||
steps: def.steps.map((s) => ({
|
||||
id: s.id,
|
||||
title: s.name,
|
||||
status: "pending" as const,
|
||||
prompt: s.prompt,
|
||||
dependsOn: s.requires ?? [],
|
||||
})),
|
||||
metadata: {
|
||||
name: def.name,
|
||||
createdAt: new Date().toISOString(),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** @deprecated Use initializeGraph instead. Kept for backward compatibility. */
|
||||
export { initializeGraph as graphFromDefinition };
|
||||
180
src/resources/extensions/gsd/run-manager.ts
Normal file
180
src/resources/extensions/gsd/run-manager.ts
Normal file
|
|
@ -0,0 +1,180 @@
|
|||
/**
|
||||
* run-manager.ts — Create and list isolated workflow run directories.
|
||||
*
|
||||
* Each run lives under `.gsd/workflow-runs/<name>/<timestamp>/` and contains:
|
||||
* - DEFINITION.yaml — frozen snapshot of the workflow definition at run-creation time
|
||||
* - GRAPH.yaml — initialized step graph with all steps pending
|
||||
* - PARAMS.json — (optional) parameter overrides used for this run
|
||||
*
|
||||
* Observability:
|
||||
* - All run state is on disk in human-readable YAML/JSON — inspectable with cat/less.
|
||||
* - `listRuns()` returns structured metadata including step counts and overall status.
|
||||
* - Timestamp directory names are filesystem-safe (ISO with hyphens replacing colons).
|
||||
* - Errors include the full path context for diagnosis.
|
||||
*/
|
||||
|
||||
import { mkdirSync, writeFileSync, existsSync, readdirSync, statSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { stringify } from "yaml";
|
||||
import { loadDefinition, substituteParams } from "./definition-loader.js";
|
||||
import { initializeGraph, writeGraph, readGraph } from "./graph.js";
|
||||
import type { WorkflowDefinition } from "./definition-loader.js";
|
||||
import type { WorkflowGraph } from "./graph.js";
|
||||
|
||||
// ─── Types ───────────────────────────────────────────────────────────────
|
||||
|
||||
export interface RunMetadata {
|
||||
/** Workflow definition name. */
|
||||
name: string;
|
||||
/** Filesystem-safe timestamp string used as dir name. */
|
||||
timestamp: string;
|
||||
/** Full path to the run directory. */
|
||||
runDir: string;
|
||||
/** Step counts derived from GRAPH.yaml. */
|
||||
steps: { total: number; completed: number; pending: number; active: number };
|
||||
/** Overall status derived from step states. */
|
||||
status: "pending" | "running" | "complete";
|
||||
}
|
||||
|
||||
// ─── Constants ───────────────────────────────────────────────────────────
|
||||
|
||||
const RUNS_DIR = "workflow-runs";
|
||||
const DEFS_DIR = "workflow-defs";
|
||||
|
||||
// ─── Helpers ─────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Generate a filesystem-safe timestamp: `YYYY-MM-DDTHH-MM-SS`.
|
||||
* Replaces colons with hyphens so the string is safe as a directory name
|
||||
* on all platforms (Windows forbids colons in paths).
|
||||
*/
|
||||
function makeTimestamp(date: Date = new Date()): string {
|
||||
return date.toISOString().replace(/:/g, "-").replace(/\.\d{3}Z$/, "");
|
||||
}
|
||||
|
||||
/**
|
||||
* Derive overall status from a graph's step statuses.
|
||||
*/
|
||||
function deriveStatus(graph: WorkflowGraph): "pending" | "running" | "complete" {
|
||||
const hasActive = graph.steps.some((s) => s.status === "active");
|
||||
const allDone = graph.steps.every(
|
||||
(s) => s.status === "complete" || s.status === "expanded",
|
||||
);
|
||||
if (allDone) return "complete";
|
||||
if (hasActive) return "running";
|
||||
return "pending";
|
||||
}
|
||||
|
||||
// ─── Public API ──────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Create a new isolated run directory for a workflow definition.
|
||||
*
|
||||
* 1. Loads the definition from `<basePath>/.gsd/workflow-defs/<defName>.yaml`
|
||||
* 2. Applies parameter substitution if overrides are provided
|
||||
* 3. Creates `<basePath>/.gsd/workflow-runs/<defName>/<timestamp>/`
|
||||
* 4. Writes frozen DEFINITION.yaml, initialized GRAPH.yaml, and optional PARAMS.json
|
||||
*
|
||||
* @param basePath — project root directory
|
||||
* @param defName — definition filename (without .yaml extension)
|
||||
* @param overrides — optional parameter overrides (merged with definition defaults)
|
||||
* @returns Full path to the created run directory
|
||||
* @throws Error if the definition file doesn't exist or is invalid
|
||||
*/
|
||||
export function createRun(
|
||||
basePath: string,
|
||||
defName: string,
|
||||
overrides?: Record<string, string>,
|
||||
): string {
|
||||
const defsDir = join(basePath, ".gsd", DEFS_DIR);
|
||||
|
||||
// Load and validate the definition
|
||||
const rawDef = loadDefinition(defsDir, defName);
|
||||
|
||||
// Apply parameter substitution if overrides provided
|
||||
const def: WorkflowDefinition = overrides
|
||||
? substituteParams(rawDef, overrides)
|
||||
: substituteParams(rawDef); // still resolve default params if any
|
||||
|
||||
// Create the run directory
|
||||
const timestamp = makeTimestamp();
|
||||
const runDir = join(basePath, ".gsd", RUNS_DIR, defName, timestamp);
|
||||
mkdirSync(runDir, { recursive: true });
|
||||
|
||||
// Freeze the definition as DEFINITION.yaml
|
||||
writeFileSync(join(runDir, "DEFINITION.yaml"), stringify(def), "utf-8");
|
||||
|
||||
// Initialize and write GRAPH.yaml
|
||||
const graph = initializeGraph(def);
|
||||
writeGraph(runDir, graph);
|
||||
|
||||
// Write PARAMS.json if overrides were provided
|
||||
if (overrides && Object.keys(overrides).length > 0) {
|
||||
writeFileSync(
|
||||
join(runDir, "PARAMS.json"),
|
||||
JSON.stringify(overrides, null, 2),
|
||||
"utf-8",
|
||||
);
|
||||
}
|
||||
|
||||
return runDir;
|
||||
}
|
||||
|
||||
/**
|
||||
* List existing workflow runs with metadata.
|
||||
*
|
||||
* Scans `<basePath>/.gsd/workflow-runs/` for run directories. Each run's
|
||||
* GRAPH.yaml is read to derive step counts and overall status.
|
||||
*
|
||||
* @param basePath — project root directory
|
||||
* @param defName — optional filter: only list runs for this definition name
|
||||
* @returns Array of run metadata, sorted newest-first within each definition
|
||||
*/
|
||||
export function listRuns(basePath: string, defName?: string): RunMetadata[] {
|
||||
const runsRoot = join(basePath, ".gsd", RUNS_DIR);
|
||||
if (!existsSync(runsRoot)) return [];
|
||||
|
||||
const results: RunMetadata[] = [];
|
||||
|
||||
// Get workflow name directories
|
||||
const nameDirs = defName ? [defName] : readdirSync(runsRoot).filter((entry) => {
|
||||
const full = join(runsRoot, entry);
|
||||
return statSync(full).isDirectory();
|
||||
});
|
||||
|
||||
for (const name of nameDirs) {
|
||||
const nameDir = join(runsRoot, name);
|
||||
if (!existsSync(nameDir)) continue;
|
||||
|
||||
const timestamps = readdirSync(nameDir).filter((entry) => {
|
||||
const full = join(nameDir, entry);
|
||||
return statSync(full).isDirectory();
|
||||
});
|
||||
|
||||
// Sort newest-first (ISO strings sort lexicographically)
|
||||
timestamps.sort().reverse();
|
||||
|
||||
for (const ts of timestamps) {
|
||||
const runDir = join(nameDir, ts);
|
||||
try {
|
||||
const graph = readGraph(runDir);
|
||||
const total = graph.steps.length;
|
||||
const completed = graph.steps.filter((s) => s.status === "complete").length;
|
||||
const pending = graph.steps.filter((s) => s.status === "pending").length;
|
||||
const active = graph.steps.filter((s) => s.status === "active").length;
|
||||
|
||||
results.push({
|
||||
name,
|
||||
timestamp: ts,
|
||||
runDir,
|
||||
steps: { total, completed, pending, active },
|
||||
status: deriveStatus(graph),
|
||||
});
|
||||
} catch {
|
||||
// Skip runs with invalid/missing GRAPH.yaml
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return results;
|
||||
}
|
||||
180
src/resources/extensions/gsd/tests/bundled-workflow-defs.test.ts
Normal file
180
src/resources/extensions/gsd/tests/bundled-workflow-defs.test.ts
Normal file
|
|
@ -0,0 +1,180 @@
|
|||
/**
|
||||
* Bundled workflow definition validation tests.
|
||||
*
|
||||
* Verifies that every example YAML in src/resources/skills/create-workflow/templates/
|
||||
* passes validateDefinition() from definition-loader.ts with { valid: true, errors: [] }.
|
||||
*
|
||||
* Also validates scaffold template and structural properties of each example
|
||||
* (step counts, feature usage) to guard against accidental regressions.
|
||||
*/
|
||||
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { join, dirname } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { parse } from "yaml";
|
||||
|
||||
import { validateDefinition } from "../definition-loader.ts";
|
||||
|
||||
// ─── Path resolution ─────────────────────────────────────────────────────
|
||||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
// Navigate from tests/ → extensions/gsd/ → extensions/ → resources/ → skills/create-workflow/templates/
|
||||
const templatesDir = join(
|
||||
__dirname,
|
||||
"..",
|
||||
"..",
|
||||
"..",
|
||||
"skills",
|
||||
"create-workflow",
|
||||
"templates",
|
||||
);
|
||||
|
||||
function loadYaml(filename: string): unknown {
|
||||
const raw = readFileSync(join(templatesDir, filename), "utf-8");
|
||||
return parse(raw);
|
||||
}
|
||||
|
||||
// ─── Scaffold template ──────────────────────────────────────────────────
|
||||
|
||||
test("scaffold template (workflow-definition.yaml) passes validation", () => {
|
||||
const parsed = loadYaml("workflow-definition.yaml");
|
||||
const result = validateDefinition(parsed);
|
||||
assert.equal(result.valid, true, `Scaffold invalid: ${result.errors.join("; ")}`);
|
||||
assert.equal(result.errors.length, 0);
|
||||
});
|
||||
|
||||
// ─── blog-post-pipeline.yaml ────────────────────────────────────────────
|
||||
|
||||
test("blog-post-pipeline.yaml passes validation", () => {
|
||||
const parsed = loadYaml("blog-post-pipeline.yaml");
|
||||
const result = validateDefinition(parsed);
|
||||
assert.equal(result.valid, true, `Invalid: ${result.errors.join("; ")}`);
|
||||
assert.equal(result.errors.length, 0);
|
||||
});
|
||||
|
||||
test("blog-post-pipeline.yaml: 3 steps, context_from, params, content-heuristic", () => {
|
||||
const parsed = loadYaml("blog-post-pipeline.yaml") as Record<string, unknown>;
|
||||
const steps = parsed.steps as Array<Record<string, unknown>>;
|
||||
|
||||
// 3 steps
|
||||
assert.equal(steps.length, 3, "Expected 3 steps");
|
||||
|
||||
// params defined
|
||||
assert.ok(parsed.params, "Expected params to be defined");
|
||||
const params = parsed.params as Record<string, string>;
|
||||
assert.ok("topic" in params, "Expected 'topic' param");
|
||||
assert.ok("audience" in params, "Expected 'audience' param");
|
||||
|
||||
// At least one step uses context_from
|
||||
const hasContextFrom = steps.some(
|
||||
(s) => Array.isArray(s.context_from) && s.context_from.length > 0,
|
||||
);
|
||||
assert.ok(hasContextFrom, "Expected at least one step with context_from");
|
||||
|
||||
// All steps use content-heuristic verify
|
||||
for (const step of steps) {
|
||||
const verify = step.verify as Record<string, unknown> | undefined;
|
||||
assert.ok(verify, `Step "${step.id}" missing verify`);
|
||||
assert.equal(verify.policy, "content-heuristic", `Step "${step.id}" should use content-heuristic`);
|
||||
}
|
||||
});
|
||||
|
||||
// ─── code-audit.yaml ────────────────────────────────────────────────────
|
||||
|
||||
test("code-audit.yaml passes validation", () => {
|
||||
const parsed = loadYaml("code-audit.yaml");
|
||||
const result = validateDefinition(parsed);
|
||||
assert.equal(result.valid, true, `Invalid: ${result.errors.join("; ")}`);
|
||||
assert.equal(result.errors.length, 0);
|
||||
});
|
||||
|
||||
test("code-audit.yaml: iterate with capture group and shell-command verify", () => {
|
||||
const parsed = loadYaml("code-audit.yaml") as Record<string, unknown>;
|
||||
const steps = parsed.steps as Array<Record<string, unknown>>;
|
||||
|
||||
// Find step with iterate
|
||||
const iterateStep = steps.find((s) => s.iterate != null);
|
||||
assert.ok(iterateStep, "Expected a step with iterate config");
|
||||
|
||||
const iterate = iterateStep.iterate as Record<string, unknown>;
|
||||
assert.equal(typeof iterate.source, "string", "iterate.source must be a string");
|
||||
assert.equal(typeof iterate.pattern, "string", "iterate.pattern must be a string");
|
||||
|
||||
// Pattern has a capture group
|
||||
const pattern = iterate.pattern as string;
|
||||
assert.ok(/\((?!\?)/.test(pattern), "iterate.pattern must contain a capture group");
|
||||
|
||||
// Pattern is valid regex
|
||||
assert.doesNotThrow(() => new RegExp(pattern), "iterate.pattern must be valid regex");
|
||||
|
||||
// Has shell-command verify
|
||||
const verify = iterateStep.verify as Record<string, unknown>;
|
||||
assert.equal(verify.policy, "shell-command");
|
||||
assert.equal(typeof verify.command, "string");
|
||||
});
|
||||
|
||||
// ─── release-checklist.yaml ─────────────────────────────────────────────
|
||||
|
||||
test("release-checklist.yaml passes validation", () => {
|
||||
const parsed = loadYaml("release-checklist.yaml");
|
||||
const result = validateDefinition(parsed);
|
||||
assert.equal(result.valid, true, `Invalid: ${result.errors.join("; ")}`);
|
||||
assert.equal(result.errors.length, 0);
|
||||
});
|
||||
|
||||
test("release-checklist.yaml: diamond dependencies and human-review", () => {
|
||||
const parsed = loadYaml("release-checklist.yaml") as Record<string, unknown>;
|
||||
const steps = parsed.steps as Array<Record<string, unknown>>;
|
||||
|
||||
// 4 steps
|
||||
assert.equal(steps.length, 4, "Expected 4 steps");
|
||||
|
||||
// Diamond pattern: two steps depend on the same parent
|
||||
const changelog = steps.find((s) => s.id === "changelog");
|
||||
const versionBump = steps.find((s) => s.id === "version-bump");
|
||||
const testSuite = steps.find((s) => s.id === "test-suite");
|
||||
const publish = steps.find((s) => s.id === "publish");
|
||||
|
||||
assert.ok(changelog, "Expected 'changelog' step");
|
||||
assert.ok(versionBump, "Expected 'version-bump' step");
|
||||
assert.ok(testSuite, "Expected 'test-suite' step");
|
||||
assert.ok(publish, "Expected 'publish' step");
|
||||
|
||||
// Both version-bump and test-suite depend on changelog
|
||||
const vbReqs = versionBump.requires as string[];
|
||||
const tsReqs = testSuite.requires as string[];
|
||||
assert.ok(vbReqs.includes("changelog"), "version-bump should require changelog");
|
||||
assert.ok(tsReqs.includes("changelog"), "test-suite should require changelog");
|
||||
|
||||
// publish depends on both (diamond join)
|
||||
const pubReqs = publish.requires as string[];
|
||||
assert.ok(pubReqs.includes("version-bump"), "publish should require version-bump");
|
||||
assert.ok(pubReqs.includes("test-suite"), "publish should require test-suite");
|
||||
|
||||
// publish uses human-review
|
||||
const verify = publish.verify as Record<string, unknown>;
|
||||
assert.equal(verify.policy, "human-review");
|
||||
});
|
||||
|
||||
// ─── Cross-cutting: no path traversal in produces ───────────────────────
|
||||
|
||||
test("no produces path contains '..'", () => {
|
||||
const files = [
|
||||
"blog-post-pipeline.yaml",
|
||||
"code-audit.yaml",
|
||||
"release-checklist.yaml",
|
||||
];
|
||||
|
||||
for (const file of files) {
|
||||
const parsed = loadYaml(file) as Record<string, unknown>;
|
||||
const steps = parsed.steps as Array<Record<string, unknown>>;
|
||||
for (const step of steps) {
|
||||
const produces = (step.produces as string[]) ?? [];
|
||||
for (const p of produces) {
|
||||
assert.ok(!p.includes(".."), `${file} step "${step.id}" produces path contains '..': ${p}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
|
|
@ -0,0 +1,283 @@
|
|||
/**
|
||||
* commands-workflow-custom.test.ts — Tests for `/gsd workflow` subcommands
|
||||
* and catalog completions.
|
||||
*
|
||||
* Uses real temp directories with actual definition YAML files.
|
||||
*/
|
||||
|
||||
import { describe, it, afterEach, before } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import {
|
||||
mkdtempSync,
|
||||
rmSync,
|
||||
mkdirSync,
|
||||
writeFileSync,
|
||||
existsSync,
|
||||
} from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
|
||||
import { getGsdArgumentCompletions, TOP_LEVEL_SUBCOMMANDS } from "../commands/catalog.ts";
|
||||
|
||||
// ─── Helpers ─────────────────────────────────────────────────────────────
|
||||
|
||||
const tmpDirs: string[] = [];
|
||||
let savedCwd: string;
|
||||
|
||||
function makeTmpBase(): string {
|
||||
const dir = mkdtempSync(join(tmpdir(), "wf-cmd-test-"));
|
||||
tmpDirs.push(dir);
|
||||
return dir;
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
// Restore cwd if changed during tests
|
||||
if (savedCwd && process.cwd() !== savedCwd) {
|
||||
process.chdir(savedCwd);
|
||||
}
|
||||
for (const d of tmpDirs) {
|
||||
rmSync(d, { recursive: true, force: true });
|
||||
}
|
||||
tmpDirs.length = 0;
|
||||
});
|
||||
|
||||
before(() => {
|
||||
savedCwd = process.cwd();
|
||||
});
|
||||
|
||||
function createMockCtx() {
|
||||
const notifications: { message: string; level: string }[] = [];
|
||||
return {
|
||||
notifications,
|
||||
ui: {
|
||||
notify(message: string, level: string) {
|
||||
notifications.push({ message, level });
|
||||
},
|
||||
custom: async () => {},
|
||||
},
|
||||
shutdown: async () => {},
|
||||
sessionManager: {
|
||||
getSessionFile: () => null,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function createMockPi() {
|
||||
return {
|
||||
registerCommand() {},
|
||||
registerTool() {},
|
||||
registerShortcut() {},
|
||||
on() {},
|
||||
sendMessage() {},
|
||||
};
|
||||
}
|
||||
|
||||
/** Write a minimal valid workflow definition YAML to the expected location. */
|
||||
function writeDefinition(basePath: string, name: string, content: string): void {
|
||||
const defsDir = join(basePath, ".gsd", "workflow-defs");
|
||||
mkdirSync(defsDir, { recursive: true });
|
||||
writeFileSync(join(defsDir, `${name}.yaml`), content, "utf-8");
|
||||
}
|
||||
|
||||
const SIMPLE_DEF = `
|
||||
version: 1
|
||||
name: test-workflow
|
||||
description: A test workflow
|
||||
steps:
|
||||
- id: step-1
|
||||
name: First Step
|
||||
prompt: Do step 1
|
||||
requires: []
|
||||
produces: []
|
||||
`;
|
||||
|
||||
const INVALID_DEF = `
|
||||
version: 2
|
||||
name: bad-workflow
|
||||
steps: []
|
||||
`;
|
||||
|
||||
// ─── Catalog Registration ────────────────────────────────────────────────
|
||||
|
||||
describe("workflow catalog registration", () => {
|
||||
it("workflow appears in TOP_LEVEL_SUBCOMMANDS", () => {
|
||||
const entry = TOP_LEVEL_SUBCOMMANDS.find((c) => c.cmd === "workflow");
|
||||
assert.ok(entry, "workflow should be in TOP_LEVEL_SUBCOMMANDS");
|
||||
assert.ok(entry!.desc.includes("new"), "description should mention new");
|
||||
assert.ok(entry!.desc.includes("run"), "description should mention run");
|
||||
});
|
||||
|
||||
it("getGsdArgumentCompletions('workflow ') returns six subcommands", () => {
|
||||
const completions = getGsdArgumentCompletions("workflow ");
|
||||
const labels = completions.map((c: any) => c.label);
|
||||
for (const sub of ["new", "run", "list", "validate", "pause", "resume"]) {
|
||||
assert.ok(labels.includes(sub), `missing completion: ${sub}`);
|
||||
}
|
||||
assert.equal(labels.length, 6, "should have exactly 6 subcommands");
|
||||
});
|
||||
|
||||
it("getGsdArgumentCompletions('workflow r') filters to run and resume", () => {
|
||||
const completions = getGsdArgumentCompletions("workflow r");
|
||||
const labels = completions.map((c: any) => c.label);
|
||||
assert.ok(labels.includes("run"), "should include run");
|
||||
assert.ok(labels.includes("resume"), "should include resume");
|
||||
assert.ok(!labels.includes("list"), "should not include list");
|
||||
});
|
||||
|
||||
it("getGsdArgumentCompletions('workflow run ') returns definition names", () => {
|
||||
const base = makeTmpBase();
|
||||
writeDefinition(base, "deploy-pipeline", SIMPLE_DEF);
|
||||
writeDefinition(base, "test-suite", SIMPLE_DEF);
|
||||
|
||||
// Change cwd so the completion scanner can find `.gsd/workflow-defs/`
|
||||
process.chdir(base);
|
||||
|
||||
const completions = getGsdArgumentCompletions("workflow run ");
|
||||
const labels = completions.map((c: any) => c.label);
|
||||
assert.ok(labels.includes("deploy-pipeline"), "should include deploy-pipeline");
|
||||
assert.ok(labels.includes("test-suite"), "should include test-suite");
|
||||
});
|
||||
|
||||
it("getGsdArgumentCompletions('workflow validate ') returns definition names", () => {
|
||||
const base = makeTmpBase();
|
||||
writeDefinition(base, "my-workflow", SIMPLE_DEF);
|
||||
|
||||
process.chdir(base);
|
||||
|
||||
const completions = getGsdArgumentCompletions("workflow validate ");
|
||||
const labels = completions.map((c: any) => c.label);
|
||||
assert.ok(labels.includes("my-workflow"), "should include my-workflow");
|
||||
});
|
||||
|
||||
it("getGsdArgumentCompletions('workflow run d') filters by prefix", () => {
|
||||
const base = makeTmpBase();
|
||||
writeDefinition(base, "deploy-pipeline", SIMPLE_DEF);
|
||||
writeDefinition(base, "test-suite", SIMPLE_DEF);
|
||||
|
||||
process.chdir(base);
|
||||
|
||||
const completions = getGsdArgumentCompletions("workflow run d");
|
||||
const labels = completions.map((c: any) => c.label);
|
||||
assert.ok(labels.includes("deploy-pipeline"), "should include deploy-pipeline");
|
||||
assert.ok(!labels.includes("test-suite"), "should not include test-suite");
|
||||
});
|
||||
});
|
||||
|
||||
// ─── Command Handler Tests ───────────────────────────────────────────────
|
||||
|
||||
describe("workflow command handler", () => {
|
||||
// Dynamically import the handler so module-level side effects
|
||||
// don't break when auto.ts pulls in heavy runtime deps.
|
||||
// We test the pure routing logic by calling handleWorkflowCommand directly.
|
||||
|
||||
async function callHandler(trimmed: string) {
|
||||
const { handleWorkflowCommand } = await import("../commands/handlers/workflow.ts");
|
||||
const ctx = createMockCtx();
|
||||
const pi = createMockPi();
|
||||
const handled = await handleWorkflowCommand(trimmed, ctx as any, pi as any);
|
||||
return { handled, notifications: ctx.notifications };
|
||||
}
|
||||
|
||||
it("bare '/gsd workflow' shows usage", async () => {
|
||||
const { handled, notifications } = await callHandler("workflow");
|
||||
assert.ok(handled, "should be handled");
|
||||
assert.ok(
|
||||
notifications.some((n) => n.message.includes("Usage: /gsd workflow")),
|
||||
"should show usage",
|
||||
);
|
||||
});
|
||||
|
||||
it("'/gsd workflow new' shows skill invocation message", async () => {
|
||||
const { handled, notifications } = await callHandler("workflow new");
|
||||
assert.ok(handled, "should be handled");
|
||||
assert.ok(
|
||||
notifications.some((n) => n.message.includes("create-workflow")),
|
||||
"should mention create-workflow skill",
|
||||
);
|
||||
});
|
||||
|
||||
it("'/gsd workflow run' without name shows usage warning", async () => {
|
||||
const { handled, notifications } = await callHandler("workflow run");
|
||||
assert.ok(handled, "should be handled");
|
||||
assert.ok(
|
||||
notifications.some((n) => n.level === "warning" && n.message.includes("Usage")),
|
||||
"should show usage warning",
|
||||
);
|
||||
});
|
||||
|
||||
it("'/gsd workflow run nonexistent' shows error for missing definition", async () => {
|
||||
const { handled, notifications } = await callHandler("workflow run nonexistent-def-12345");
|
||||
assert.ok(handled, "should be handled");
|
||||
assert.ok(
|
||||
notifications.some((n) => n.level === "error" && n.message.includes("not found")),
|
||||
"should show definition-not-found error",
|
||||
);
|
||||
});
|
||||
|
||||
it("'/gsd workflow validate' without name shows usage warning", async () => {
|
||||
const { handled, notifications } = await callHandler("workflow validate");
|
||||
assert.ok(handled, "should be handled");
|
||||
assert.ok(
|
||||
notifications.some((n) => n.level === "warning" && n.message.includes("Usage")),
|
||||
"should show usage warning",
|
||||
);
|
||||
});
|
||||
|
||||
it("'/gsd workflow validate nonexistent' shows definition not found", async () => {
|
||||
const { handled, notifications } = await callHandler("workflow validate nonexistent-def-12345");
|
||||
assert.ok(handled, "should be handled");
|
||||
assert.ok(
|
||||
notifications.some((n) => n.level === "error" && n.message.includes("not found")),
|
||||
"should show not-found error",
|
||||
);
|
||||
});
|
||||
|
||||
it("'/gsd workflow pause' without custom engine shows warning", async () => {
|
||||
const { handled, notifications } = await callHandler("workflow pause");
|
||||
assert.ok(handled, "should be handled");
|
||||
assert.ok(
|
||||
notifications.some((n) => n.level === "warning"),
|
||||
"should show warning when no custom workflow is running",
|
||||
);
|
||||
});
|
||||
|
||||
it("'/gsd workflow resume' without custom engine shows warning", async () => {
|
||||
const { handled, notifications } = await callHandler("workflow resume");
|
||||
assert.ok(handled, "should be handled");
|
||||
assert.ok(
|
||||
notifications.some((n) => n.level === "warning"),
|
||||
"should show warning when no custom workflow to resume",
|
||||
);
|
||||
});
|
||||
|
||||
it("'/gsd workflow unknown-sub' shows unknown subcommand", async () => {
|
||||
const { handled, notifications } = await callHandler("workflow blurble");
|
||||
assert.ok(handled, "should be handled");
|
||||
assert.ok(
|
||||
notifications.some((n) => n.message.includes("Unknown workflow subcommand")),
|
||||
"should show unknown subcommand message",
|
||||
);
|
||||
});
|
||||
|
||||
it("'/gsd workflow list' with no runs shows empty message", async () => {
|
||||
const { handled, notifications } = await callHandler("workflow list");
|
||||
assert.ok(handled, "should be handled");
|
||||
assert.ok(
|
||||
notifications.some((n) => n.message.includes("No workflow runs found")),
|
||||
"should show no runs message",
|
||||
);
|
||||
});
|
||||
|
||||
it("non-workflow commands are not intercepted by custom workflow routing", async () => {
|
||||
const { handleWorkflowCommand } = await import("../commands/handlers/workflow.ts");
|
||||
const ctx = createMockCtx();
|
||||
const pi = createMockPi();
|
||||
// "queue" does not start with "workflow" so the custom routing should not handle it.
|
||||
// The function may still handle it via its existing dev-workflow routing, but it
|
||||
// should not be captured by the custom workflow `if` block.
|
||||
// We verify this by checking that a clearly non-workflow command like "somethingelse"
|
||||
// returns false (unhandled).
|
||||
const handled = await handleWorkflowCommand("somethingelse", ctx as any, pi as any);
|
||||
assert.equal(handled, false, "non-workflow commands should return false");
|
||||
});
|
||||
});
|
||||
313
src/resources/extensions/gsd/tests/context-injector.test.ts
Normal file
313
src/resources/extensions/gsd/tests/context-injector.test.ts
Normal file
|
|
@ -0,0 +1,313 @@
|
|||
/**
|
||||
* context-injector.test.ts — Tests for injectContext().
|
||||
*
|
||||
* Tests context injection from prior step artifacts: single-step,
|
||||
* multi-step chain, missing artifact, no contextFrom, truncation,
|
||||
* and unknown step ID in contextFrom.
|
||||
*/
|
||||
|
||||
import { describe, it } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtempSync, writeFileSync, mkdirSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import { stringify } from "yaml";
|
||||
import { injectContext } from "../context-injector.ts";
|
||||
import type { WorkflowDefinition } from "../definition-loader.ts";
|
||||
|
||||
/** Create a temp run directory with the given definition and optional files. */
|
||||
function makeTempRun(
|
||||
def: WorkflowDefinition,
|
||||
files?: Record<string, string>,
|
||||
): string {
|
||||
const runDir = mkdtempSync(join(tmpdir(), "ci-test-"));
|
||||
writeFileSync(join(runDir, "DEFINITION.yaml"), stringify(def), "utf-8");
|
||||
|
||||
if (files) {
|
||||
for (const [relPath, content] of Object.entries(files)) {
|
||||
const absPath = join(runDir, relPath);
|
||||
const parentDir = join(absPath, "..");
|
||||
mkdirSync(parentDir, { recursive: true });
|
||||
writeFileSync(absPath, content, "utf-8");
|
||||
}
|
||||
}
|
||||
|
||||
return runDir;
|
||||
}
|
||||
|
||||
/** Minimal valid workflow definition factory. */
|
||||
function makeDef(
|
||||
steps: WorkflowDefinition["steps"],
|
||||
): WorkflowDefinition {
|
||||
return {
|
||||
version: 1,
|
||||
name: "test-workflow",
|
||||
steps,
|
||||
};
|
||||
}
|
||||
|
||||
// ─── single-step context ────────────────────────────────────────────────
|
||||
|
||||
describe("single-step context injection", () => {
|
||||
it("prepends step-1 artifact content to step-2 prompt", () => {
|
||||
const def = makeDef([
|
||||
{
|
||||
id: "step-1",
|
||||
name: "Research",
|
||||
prompt: "Research the topic",
|
||||
requires: [],
|
||||
produces: ["output.md"],
|
||||
},
|
||||
{
|
||||
id: "step-2",
|
||||
name: "Write",
|
||||
prompt: "Write the report",
|
||||
requires: ["step-1"],
|
||||
produces: ["report.md"],
|
||||
contextFrom: ["step-1"],
|
||||
},
|
||||
]);
|
||||
|
||||
const runDir = makeTempRun(def, {
|
||||
"output.md": "Research findings: AI is growing fast.",
|
||||
});
|
||||
|
||||
const result = injectContext(runDir, "step-2", "Write the report");
|
||||
assert.ok(result.includes("Research findings: AI is growing fast."));
|
||||
assert.ok(result.includes('Context from step "step-1"'));
|
||||
assert.ok(result.includes("(file: output.md)"));
|
||||
assert.ok(result.endsWith("Write the report"));
|
||||
});
|
||||
});
|
||||
|
||||
// ─── multi-step chain ───────────────────────────────────────────────────
|
||||
|
||||
describe("multi-step context chain", () => {
|
||||
it("prepends artifacts from both step-1 and step-2", () => {
|
||||
const def = makeDef([
|
||||
{
|
||||
id: "step-1",
|
||||
name: "Research",
|
||||
prompt: "Research",
|
||||
requires: [],
|
||||
produces: ["research.md"],
|
||||
},
|
||||
{
|
||||
id: "step-2",
|
||||
name: "Outline",
|
||||
prompt: "Outline",
|
||||
requires: ["step-1"],
|
||||
produces: ["outline.md"],
|
||||
},
|
||||
{
|
||||
id: "step-3",
|
||||
name: "Draft",
|
||||
prompt: "Write the draft",
|
||||
requires: ["step-1", "step-2"],
|
||||
produces: ["draft.md"],
|
||||
contextFrom: ["step-1", "step-2"],
|
||||
},
|
||||
]);
|
||||
|
||||
const runDir = makeTempRun(def, {
|
||||
"research.md": "Research content here.",
|
||||
"outline.md": "Outline content here.",
|
||||
});
|
||||
|
||||
const result = injectContext(runDir, "step-3", "Write the draft");
|
||||
assert.ok(result.includes("Research content here."));
|
||||
assert.ok(result.includes("Outline content here."));
|
||||
assert.ok(result.includes('Context from step "step-1"'));
|
||||
assert.ok(result.includes('Context from step "step-2"'));
|
||||
assert.ok(result.endsWith("Write the draft"));
|
||||
|
||||
// Verify order: step-1 context appears before step-2 context
|
||||
const idx1 = result.indexOf('Context from step "step-1"');
|
||||
const idx2 = result.indexOf('Context from step "step-2"');
|
||||
assert.ok(idx1 < idx2, "step-1 context should appear before step-2 context");
|
||||
});
|
||||
});
|
||||
|
||||
// ─── missing artifact file ──────────────────────────────────────────────
|
||||
|
||||
describe("missing artifact file", () => {
|
||||
it("skips missing artifact and includes existing ones", () => {
|
||||
const def = makeDef([
|
||||
{
|
||||
id: "step-1",
|
||||
name: "Research",
|
||||
prompt: "Research",
|
||||
requires: [],
|
||||
produces: ["found.md", "missing.md"],
|
||||
},
|
||||
{
|
||||
id: "step-2",
|
||||
name: "Write",
|
||||
prompt: "Write the report",
|
||||
requires: ["step-1"],
|
||||
produces: ["report.md"],
|
||||
contextFrom: ["step-1"],
|
||||
},
|
||||
]);
|
||||
|
||||
// Only create found.md, not missing.md
|
||||
const runDir = makeTempRun(def, {
|
||||
"found.md": "Found content.",
|
||||
});
|
||||
|
||||
const result = injectContext(runDir, "step-2", "Write the report");
|
||||
assert.ok(result.includes("Found content."));
|
||||
assert.ok(!result.includes("missing.md"));
|
||||
assert.ok(result.endsWith("Write the report"));
|
||||
});
|
||||
|
||||
it("returns prompt unchanged when all referenced artifacts are missing", () => {
|
||||
const def = makeDef([
|
||||
{
|
||||
id: "step-1",
|
||||
name: "Research",
|
||||
prompt: "Research",
|
||||
requires: [],
|
||||
produces: ["missing.md"],
|
||||
},
|
||||
{
|
||||
id: "step-2",
|
||||
name: "Write",
|
||||
prompt: "Write the report",
|
||||
requires: ["step-1"],
|
||||
produces: ["report.md"],
|
||||
contextFrom: ["step-1"],
|
||||
},
|
||||
]);
|
||||
|
||||
const runDir = makeTempRun(def);
|
||||
|
||||
const result = injectContext(runDir, "step-2", "Write the report");
|
||||
assert.equal(result, "Write the report");
|
||||
});
|
||||
});
|
||||
|
||||
// ─── no contextFrom ────────────────────────────────────────────────────
|
||||
|
||||
describe("no contextFrom", () => {
|
||||
it("returns prompt unchanged when step has no contextFrom", () => {
|
||||
const def = makeDef([
|
||||
{
|
||||
id: "step-1",
|
||||
name: "Research",
|
||||
prompt: "Research",
|
||||
requires: [],
|
||||
produces: ["output.md"],
|
||||
},
|
||||
]);
|
||||
|
||||
const runDir = makeTempRun(def, {
|
||||
"output.md": "Some content.",
|
||||
});
|
||||
|
||||
const result = injectContext(runDir, "step-1", "Research");
|
||||
assert.equal(result, "Research");
|
||||
});
|
||||
|
||||
it("returns prompt unchanged when step ID not found in definition", () => {
|
||||
const def = makeDef([
|
||||
{
|
||||
id: "step-1",
|
||||
name: "Research",
|
||||
prompt: "Research",
|
||||
requires: [],
|
||||
produces: [],
|
||||
},
|
||||
]);
|
||||
|
||||
const runDir = makeTempRun(def);
|
||||
|
||||
const result = injectContext(runDir, "nonexistent", "Some prompt");
|
||||
assert.equal(result, "Some prompt");
|
||||
});
|
||||
});
|
||||
|
||||
// ─── truncation ─────────────────────────────────────────────────────────
|
||||
|
||||
describe("truncation guard", () => {
|
||||
it("truncates artifacts exceeding 10,000 characters", () => {
|
||||
const largeContent = "A".repeat(15_000);
|
||||
|
||||
const def = makeDef([
|
||||
{
|
||||
id: "step-1",
|
||||
name: "Generate",
|
||||
prompt: "Generate",
|
||||
requires: [],
|
||||
produces: ["big.md"],
|
||||
},
|
||||
{
|
||||
id: "step-2",
|
||||
name: "Consume",
|
||||
prompt: "Use the output",
|
||||
requires: ["step-1"],
|
||||
produces: [],
|
||||
contextFrom: ["step-1"],
|
||||
},
|
||||
]);
|
||||
|
||||
const runDir = makeTempRun(def, {
|
||||
"big.md": largeContent,
|
||||
});
|
||||
|
||||
const result = injectContext(runDir, "step-2", "Use the output");
|
||||
assert.ok(result.includes("...[truncated]"));
|
||||
// The injected content should be 10,000 chars + truncation marker, not all 15,000
|
||||
const contextPart = result.split("Use the output")[0];
|
||||
assert.ok(contextPart.length < 15_000, "Context should be truncated below original size");
|
||||
// Verify the truncated content is exactly 10,000 A's (no collision with header text)
|
||||
const aCount = (contextPart.match(/A/g) || []).length;
|
||||
assert.equal(aCount, 10_000, "Should contain exactly 10,000 chars of original content");
|
||||
});
|
||||
});
|
||||
|
||||
// ─── unknown step ID in contextFrom ─────────────────────────────────────
|
||||
|
||||
describe("unknown step in contextFrom", () => {
|
||||
it("skips unknown step IDs gracefully", () => {
|
||||
const def = makeDef([
|
||||
{
|
||||
id: "step-1",
|
||||
name: "Research",
|
||||
prompt: "Research",
|
||||
requires: [],
|
||||
produces: ["output.md"],
|
||||
},
|
||||
{
|
||||
id: "step-2",
|
||||
name: "Write",
|
||||
prompt: "Write the report",
|
||||
requires: ["step-1"],
|
||||
produces: [],
|
||||
contextFrom: ["step-1", "nonexistent-step"],
|
||||
},
|
||||
]);
|
||||
|
||||
const runDir = makeTempRun(def, {
|
||||
"output.md": "Research content.",
|
||||
});
|
||||
|
||||
const result = injectContext(runDir, "step-2", "Write the report");
|
||||
// Should include step-1 content despite nonexistent-step being in contextFrom
|
||||
assert.ok(result.includes("Research content."));
|
||||
assert.ok(result.endsWith("Write the report"));
|
||||
});
|
||||
});
|
||||
|
||||
// ─── error handling ─────────────────────────────────────────────────────
|
||||
|
||||
describe("error handling", () => {
|
||||
it("throws when DEFINITION.yaml is missing", () => {
|
||||
const runDir = mkdtempSync(join(tmpdir(), "ci-test-nodef-"));
|
||||
|
||||
assert.throws(
|
||||
() => injectContext(runDir, "step-1", "Some prompt"),
|
||||
/ENOENT/,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,540 @@
|
|||
/**
|
||||
* custom-engine-loop-integration.test.ts — Integration test proving that
|
||||
* autoLoop dispatches a 3-step custom workflow through the real pipeline.
|
||||
*
|
||||
* Creates a real run directory with GRAPH.yaml, mocks LoopDeps minimally,
|
||||
* and verifies all 3 steps complete in dependency order.
|
||||
*/
|
||||
|
||||
import { describe, it, afterEach } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtempSync, rmSync, existsSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
|
||||
import { autoLoop, resolveAgentEnd, _resetPendingResolve } from "../auto-loop.js";
|
||||
import type { LoopDeps } from "../auto/loop-deps.js";
|
||||
import type { SessionLockStatus } from "../session-lock.js";
|
||||
import { writeGraph, readGraph, type WorkflowGraph, type GraphStep } from "../graph.ts";
|
||||
import { writeFileSync } from "node:fs";
|
||||
import { stringify } from "yaml";
|
||||
|
||||
// ─── Helpers ─────────────────────────────────────────────────────────────
|
||||
|
||||
const tmpDirs: string[] = [];
|
||||
|
||||
function makeTmpDir(): string {
|
||||
const dir = mkdtempSync(join(tmpdir(), "loop-integ-"));
|
||||
tmpDirs.push(dir);
|
||||
return dir;
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
_resetPendingResolve();
|
||||
for (const d of tmpDirs) {
|
||||
rmSync(d, { recursive: true, force: true });
|
||||
}
|
||||
tmpDirs.length = 0;
|
||||
});
|
||||
|
||||
function makeStep(overrides: Partial<GraphStep> & { id: string }): GraphStep {
|
||||
return {
|
||||
title: overrides.id,
|
||||
status: "pending",
|
||||
prompt: `Do ${overrides.id}`,
|
||||
dependsOn: [],
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
function makeGraph(steps: GraphStep[], name = "test-wf"): WorkflowGraph {
|
||||
return {
|
||||
steps,
|
||||
metadata: { name, createdAt: "2026-01-01T00:00:00.000Z" },
|
||||
};
|
||||
}
|
||||
|
||||
/** Write a minimal DEFINITION.yaml that matches the graph steps (needed by resolveDispatch since S06). */
|
||||
function writeDefinition(runDir: string, steps: GraphStep[], name = "test-wf"): void {
|
||||
const def = {
|
||||
version: 1,
|
||||
name,
|
||||
description: `Test workflow: ${name}`,
|
||||
steps: steps.map((s) => ({
|
||||
id: s.id,
|
||||
name: s.title ?? s.id,
|
||||
prompt: s.prompt ?? `Do ${s.id}`,
|
||||
produces: `${s.id}/output.md`,
|
||||
...(s.dependsOn?.length ? { requires: s.dependsOn } : {}),
|
||||
})),
|
||||
};
|
||||
writeFileSync(join(runDir, "DEFINITION.yaml"), stringify(def));
|
||||
}
|
||||
|
||||
function makeMockCtx() {
|
||||
return {
|
||||
ui: { notify: () => {}, setStatus: () => {} },
|
||||
model: { id: "test-model" },
|
||||
sessionManager: { getSessionFile: () => "/tmp/session.json" },
|
||||
} as any;
|
||||
}
|
||||
|
||||
function makeMockPi() {
|
||||
const calls: unknown[] = [];
|
||||
return {
|
||||
sendMessage: (...args: unknown[]) => {
|
||||
calls.push(args);
|
||||
},
|
||||
calls,
|
||||
} as any;
|
||||
}
|
||||
|
||||
function makeLoopSession(overrides?: Record<string, unknown>) {
|
||||
return {
|
||||
active: true,
|
||||
verbose: false,
|
||||
stepMode: false,
|
||||
paused: false,
|
||||
basePath: "/tmp/project",
|
||||
originalBasePath: "",
|
||||
currentMilestoneId: null,
|
||||
currentUnit: null,
|
||||
currentUnitRouting: null,
|
||||
completedUnits: [],
|
||||
resourceVersionOnStart: null,
|
||||
lastPromptCharCount: undefined,
|
||||
lastBaselineCharCount: undefined,
|
||||
lastBudgetAlertLevel: 0,
|
||||
pendingVerificationRetry: null,
|
||||
pendingCrashRecovery: null,
|
||||
pendingQuickTasks: [],
|
||||
sidecarQueue: [],
|
||||
autoModeStartModel: null,
|
||||
unitDispatchCount: new Map<string, number>(),
|
||||
unitLifetimeDispatches: new Map<string, number>(),
|
||||
unitRecoveryCount: new Map<string, number>(),
|
||||
verificationRetryCount: new Map<string, number>(),
|
||||
gitService: null,
|
||||
autoStartTime: Date.now(),
|
||||
activeEngineId: null,
|
||||
activeRunDir: null,
|
||||
rewriteAttemptCount: 0,
|
||||
cmdCtx: {
|
||||
newSession: () => Promise.resolve({ cancelled: false }),
|
||||
getContextUsage: () => ({ percent: 10, tokens: 1000, limit: 10000 }),
|
||||
},
|
||||
clearTimers: () => {},
|
||||
lockBasePath: "/tmp/project",
|
||||
...overrides,
|
||||
} as any;
|
||||
}
|
||||
|
||||
function makeMockDeps(overrides?: Partial<LoopDeps>): LoopDeps & { callLog: string[] } {
|
||||
const callLog: string[] = [];
|
||||
|
||||
const baseDeps: LoopDeps = {
|
||||
lockBase: () => "/tmp/test-lock",
|
||||
buildSnapshotOpts: () => ({}),
|
||||
stopAuto: async (_ctx, _pi, reason) => {
|
||||
callLog.push(`stopAuto:${reason ?? "no-reason"}`);
|
||||
},
|
||||
pauseAuto: async () => {
|
||||
callLog.push("pauseAuto");
|
||||
},
|
||||
clearUnitTimeout: () => {},
|
||||
updateProgressWidget: () => {},
|
||||
syncCmuxSidebar: () => {},
|
||||
logCmuxEvent: () => {},
|
||||
invalidateAllCaches: () => {},
|
||||
deriveState: async () => {
|
||||
callLog.push("deriveState");
|
||||
return {
|
||||
phase: "executing",
|
||||
activeMilestone: { id: "M001", title: "Workflow", status: "active" },
|
||||
activeSlice: null,
|
||||
activeTask: null,
|
||||
registry: [],
|
||||
blockers: [],
|
||||
} as any;
|
||||
},
|
||||
rebuildState: async () => {},
|
||||
loadEffectiveGSDPreferences: () => undefined,
|
||||
preDispatchHealthGate: async () => ({ proceed: true, fixesApplied: [] }),
|
||||
syncProjectRootToWorktree: () => {},
|
||||
checkResourcesStale: () => null,
|
||||
validateSessionLock: () => ({ valid: true } as SessionLockStatus),
|
||||
updateSessionLock: () => {},
|
||||
handleLostSessionLock: () => {},
|
||||
sendDesktopNotification: () => {},
|
||||
setActiveMilestoneId: () => {},
|
||||
pruneQueueOrder: () => {},
|
||||
isInAutoWorktree: () => false,
|
||||
shouldUseWorktreeIsolation: () => false,
|
||||
mergeMilestoneToMain: () => ({ pushed: false, codeFilesChanged: false }),
|
||||
teardownAutoWorktree: () => {},
|
||||
createAutoWorktree: () => "/tmp/wt",
|
||||
captureIntegrationBranch: () => {},
|
||||
getIsolationMode: () => "none",
|
||||
getCurrentBranch: () => "main",
|
||||
autoWorktreeBranch: () => "auto/M001",
|
||||
resolveMilestoneFile: () => null,
|
||||
reconcileMergeState: () => false,
|
||||
getLedger: () => null,
|
||||
getProjectTotals: () => ({ cost: 0 }),
|
||||
formatCost: (c: number) => `$${c.toFixed(2)}`,
|
||||
getBudgetAlertLevel: () => 0,
|
||||
getNewBudgetAlertLevel: () => 0,
|
||||
getBudgetEnforcementAction: () => "none",
|
||||
getManifestStatus: async () => null,
|
||||
collectSecretsFromManifest: async () => null,
|
||||
resolveDispatch: async () => {
|
||||
callLog.push("resolveDispatch");
|
||||
return { action: "dispatch" as const, unitType: "execute-task", unitId: "M001/S01/T01", prompt: "unused" };
|
||||
},
|
||||
runPreDispatchHooks: () => ({ firedHooks: [], action: "proceed" }),
|
||||
getPriorSliceCompletionBlocker: () => null,
|
||||
getMainBranch: () => "main",
|
||||
collectObservabilityWarnings: async () => [],
|
||||
buildObservabilityRepairBlock: () => null,
|
||||
closeoutUnit: async () => {},
|
||||
verifyExpectedArtifact: () => true,
|
||||
clearUnitRuntimeRecord: () => {},
|
||||
writeUnitRuntimeRecord: () => {},
|
||||
recordOutcome: () => {},
|
||||
writeLock: () => {},
|
||||
captureAvailableSkills: () => {},
|
||||
ensurePreconditions: () => {},
|
||||
updateSliceProgressCache: () => {},
|
||||
selectAndApplyModel: async () => ({ routing: null }),
|
||||
resolveModelId: () => undefined,
|
||||
startUnitSupervision: () => {},
|
||||
getDeepDiagnostic: () => null,
|
||||
isDbAvailable: () => false,
|
||||
reorderForCaching: (p: string) => p,
|
||||
existsSync: (p: string) => existsSync(p),
|
||||
readFileSync: () => "",
|
||||
atomicWriteSync: () => {},
|
||||
GitServiceImpl: class {} as any,
|
||||
resolver: {
|
||||
get workPath() { return "/tmp/project"; },
|
||||
get projectRoot() { return "/tmp/project"; },
|
||||
get lockPath() { return "/tmp/project"; },
|
||||
enterMilestone: () => {},
|
||||
exitMilestone: () => {},
|
||||
mergeAndExit: () => {},
|
||||
mergeAndEnterNext: () => {},
|
||||
} as any,
|
||||
postUnitPreVerification: async () => "continue" as const,
|
||||
runPostUnitVerification: async () => "continue" as const,
|
||||
postUnitPostVerification: async () => "continue" as const,
|
||||
getSessionFile: () => "/tmp/session.json",
|
||||
emitJournalEvent: (entry) => {
|
||||
callLog.push(`journal:${entry.eventType}`);
|
||||
},
|
||||
};
|
||||
|
||||
return { ...baseDeps, ...overrides, callLog };
|
||||
}
|
||||
|
||||
// ─── Tests ───────────────────────────────────────────────────────────────
|
||||
|
||||
describe("Custom engine loop integration", () => {
|
||||
it("dispatches a 3-step workflow through autoLoop and all steps complete", async () => {
|
||||
_resetPendingResolve();
|
||||
|
||||
// Create a real run directory with 3 steps: a → b → c
|
||||
const runDir = makeTmpDir();
|
||||
const graph = makeGraph([
|
||||
makeStep({ id: "step-a" }),
|
||||
makeStep({ id: "step-b", dependsOn: ["step-a"] }),
|
||||
makeStep({ id: "step-c", dependsOn: ["step-b"] }),
|
||||
], "integ-test");
|
||||
writeGraph(runDir, graph);
|
||||
writeDefinition(runDir, graph.steps, "integ-test");
|
||||
|
||||
const ctx = makeMockCtx();
|
||||
const pi = makeMockPi();
|
||||
|
||||
let unitCount = 0;
|
||||
|
||||
const s = makeLoopSession({
|
||||
activeEngineId: "custom",
|
||||
activeRunDir: runDir,
|
||||
basePath: runDir,
|
||||
});
|
||||
|
||||
const deps = makeMockDeps({
|
||||
stopAuto: async (_ctx, _pi, reason) => {
|
||||
deps.callLog.push(`stopAuto:${reason ?? "no-reason"}`);
|
||||
s.active = false;
|
||||
},
|
||||
});
|
||||
|
||||
// Start autoLoop — it will block inside runUnit awaiting resolveAgentEnd
|
||||
const loopPromise = autoLoop(ctx, pi, s, deps);
|
||||
|
||||
// Each iteration: the custom engine path derives state → resolves dispatch →
|
||||
// runs guards → runs runUnitPhase (which calls runUnit) → we resolve →
|
||||
// engine.reconcile marks the step complete → loop continues.
|
||||
// We need to resolve resolveAgentEnd for each step.
|
||||
|
||||
// Step 1: step-a
|
||||
await new Promise((r) => setTimeout(r, 80));
|
||||
unitCount++;
|
||||
resolveAgentEnd({ messages: [{ role: "assistant" }] });
|
||||
|
||||
// Step 2: step-b
|
||||
await new Promise((r) => setTimeout(r, 80));
|
||||
unitCount++;
|
||||
resolveAgentEnd({ messages: [{ role: "assistant" }] });
|
||||
|
||||
// Step 3: step-c
|
||||
await new Promise((r) => setTimeout(r, 80));
|
||||
unitCount++;
|
||||
resolveAgentEnd({ messages: [{ role: "assistant" }] });
|
||||
|
||||
// After step-c completes, engine.reconcile marks it complete, then
|
||||
// next deriveState sees isComplete=true → stopAuto → loop exits
|
||||
await loopPromise;
|
||||
|
||||
// Verify GRAPH.yaml shows all 3 steps complete
|
||||
const finalGraph = readGraph(runDir);
|
||||
assert.equal(finalGraph.steps.length, 3, "Should have 3 steps");
|
||||
for (const step of finalGraph.steps) {
|
||||
assert.equal(step.status, "complete", `Step ${step.id} should be complete, got ${step.status}`);
|
||||
assert.ok(step.finishedAt, `Step ${step.id} should have finishedAt timestamp`);
|
||||
}
|
||||
|
||||
// Verify exactly 3 units were dispatched (3 pi.sendMessage calls)
|
||||
assert.equal(pi.calls.length, 3, `Should have dispatched exactly 3 units, got ${pi.calls.length}`);
|
||||
|
||||
// Verify the loop stopped because the workflow completed
|
||||
const stopEntry = deps.callLog.find((e: string) => e.startsWith("stopAuto:"));
|
||||
assert.ok(stopEntry, "stopAuto should have been called");
|
||||
assert.ok(
|
||||
stopEntry!.includes("Workflow complete"),
|
||||
`stopAuto reason should include "Workflow complete", got: ${stopEntry}`,
|
||||
);
|
||||
|
||||
// Verify dev path was NOT used (resolveDispatch should not appear)
|
||||
assert.ok(
|
||||
!deps.callLog.includes("resolveDispatch"),
|
||||
"Custom engine path should skip resolveDispatch (dev path not taken)",
|
||||
);
|
||||
});
|
||||
|
||||
it("stops when engine reports isComplete on first derive", async () => {
|
||||
_resetPendingResolve();
|
||||
|
||||
// Create a run directory where all steps are already complete
|
||||
const runDir = makeTmpDir();
|
||||
const graph = makeGraph([
|
||||
makeStep({ id: "step-a", status: "complete" }),
|
||||
], "already-done");
|
||||
writeGraph(runDir, graph);
|
||||
writeDefinition(runDir, graph.steps, "already-done");
|
||||
|
||||
const ctx = makeMockCtx();
|
||||
const pi = makeMockPi();
|
||||
|
||||
const s = makeLoopSession({
|
||||
activeEngineId: "custom",
|
||||
activeRunDir: runDir,
|
||||
basePath: runDir,
|
||||
});
|
||||
|
||||
const deps = makeMockDeps({
|
||||
stopAuto: async (_ctx, _pi, reason) => {
|
||||
deps.callLog.push(`stopAuto:${reason ?? "no-reason"}`);
|
||||
s.active = false;
|
||||
},
|
||||
});
|
||||
|
||||
await autoLoop(ctx, pi, s, deps);
|
||||
|
||||
// No units should have been dispatched
|
||||
assert.equal(pi.calls.length, 0, "Should not dispatch units for complete workflow");
|
||||
|
||||
// Should stop with "Workflow complete" reason
|
||||
const stopEntry = deps.callLog.find((e: string) => e.startsWith("stopAuto:"));
|
||||
assert.ok(stopEntry?.includes("Workflow complete"), "Should stop with 'Workflow complete'");
|
||||
});
|
||||
|
||||
it("does not call runPreDispatch or runFinalize on the custom path", async () => {
|
||||
_resetPendingResolve();
|
||||
|
||||
// Single-step workflow
|
||||
const runDir = makeTmpDir();
|
||||
const graph = makeGraph([makeStep({ id: "only" })], "single");
|
||||
writeGraph(runDir, graph);
|
||||
writeDefinition(runDir, graph.steps, "single");
|
||||
|
||||
const ctx = makeMockCtx();
|
||||
const pi = makeMockPi();
|
||||
|
||||
const s = makeLoopSession({
|
||||
activeEngineId: "custom",
|
||||
activeRunDir: runDir,
|
||||
basePath: runDir,
|
||||
});
|
||||
|
||||
const deps = makeMockDeps({
|
||||
stopAuto: async (_ctx, _pi, reason) => {
|
||||
deps.callLog.push(`stopAuto:${reason ?? "no-reason"}`);
|
||||
s.active = false;
|
||||
},
|
||||
postUnitPreVerification: async () => {
|
||||
deps.callLog.push("postUnitPreVerification");
|
||||
return "continue" as const;
|
||||
},
|
||||
postUnitPostVerification: async () => {
|
||||
deps.callLog.push("postUnitPostVerification");
|
||||
return "continue" as const;
|
||||
},
|
||||
});
|
||||
|
||||
const loopPromise = autoLoop(ctx, pi, s, deps);
|
||||
|
||||
await new Promise((r) => setTimeout(r, 80));
|
||||
resolveAgentEnd({ messages: [{ role: "assistant" }] });
|
||||
|
||||
await loopPromise;
|
||||
|
||||
// Custom path should NOT call runFinalize's post-unit phases
|
||||
assert.ok(
|
||||
!deps.callLog.includes("postUnitPreVerification"),
|
||||
"Custom path should skip postUnitPreVerification (runFinalize not called)",
|
||||
);
|
||||
assert.ok(
|
||||
!deps.callLog.includes("postUnitPostVerification"),
|
||||
"Custom path should skip postUnitPostVerification (runFinalize not called)",
|
||||
);
|
||||
|
||||
// Should NOT have called resolveDispatch (dev dispatch)
|
||||
assert.ok(
|
||||
!deps.callLog.includes("resolveDispatch"),
|
||||
"Custom path should skip resolveDispatch",
|
||||
);
|
||||
});
|
||||
|
||||
it("respects dependency ordering — step-b waits for step-a", async () => {
|
||||
_resetPendingResolve();
|
||||
|
||||
const runDir = makeTmpDir();
|
||||
// step-b depends on step-a, both pending
|
||||
const graph = makeGraph([
|
||||
makeStep({ id: "step-a" }),
|
||||
makeStep({ id: "step-b", dependsOn: ["step-a"] }),
|
||||
], "dep-order");
|
||||
writeGraph(runDir, graph);
|
||||
writeDefinition(runDir, graph.steps, "dep-order");
|
||||
|
||||
const ctx = makeMockCtx();
|
||||
const pi = makeMockPi();
|
||||
const dispatchedUnitIds: string[] = [];
|
||||
|
||||
const s = makeLoopSession({
|
||||
activeEngineId: "custom",
|
||||
activeRunDir: runDir,
|
||||
basePath: runDir,
|
||||
});
|
||||
|
||||
const originalSendMessage = pi.sendMessage;
|
||||
pi.sendMessage = (...args: unknown[]) => {
|
||||
// Track dispatched prompts to verify ordering
|
||||
const promptArg = args[0] as { content?: string };
|
||||
dispatchedUnitIds.push(promptArg?.content ?? "unknown");
|
||||
return originalSendMessage(...args);
|
||||
};
|
||||
|
||||
const deps = makeMockDeps({
|
||||
stopAuto: async (_ctx, _pi, reason) => {
|
||||
deps.callLog.push(`stopAuto:${reason ?? "no-reason"}`);
|
||||
s.active = false;
|
||||
},
|
||||
});
|
||||
|
||||
const loopPromise = autoLoop(ctx, pi, s, deps);
|
||||
|
||||
// Resolve step-a
|
||||
await new Promise((r) => setTimeout(r, 80));
|
||||
resolveAgentEnd({ messages: [{ role: "assistant" }] });
|
||||
|
||||
// Resolve step-b
|
||||
await new Promise((r) => setTimeout(r, 80));
|
||||
resolveAgentEnd({ messages: [{ role: "assistant" }] });
|
||||
|
||||
await loopPromise;
|
||||
|
||||
// Verify step-a was dispatched before step-b
|
||||
assert.equal(dispatchedUnitIds.length, 2, "Should have dispatched 2 steps");
|
||||
assert.ok(
|
||||
dispatchedUnitIds[0].includes("Do step-a"),
|
||||
`First dispatch should be step-a, got: ${dispatchedUnitIds[0]}`,
|
||||
);
|
||||
assert.ok(
|
||||
dispatchedUnitIds[1].includes("Do step-b"),
|
||||
`Second dispatch should be step-b, got: ${dispatchedUnitIds[1]}`,
|
||||
);
|
||||
});
|
||||
|
||||
it("GRAPH.yaml step stays pending when session deactivates before reconcile", async () => {
|
||||
_resetPendingResolve();
|
||||
|
||||
// Two-step workflow: a → b. We will complete step-a, then force a break
|
||||
// during step-b's runUnitPhase (by returning cancelled status + deactivating).
|
||||
const runDir = makeTmpDir();
|
||||
const graph = makeGraph([
|
||||
makeStep({ id: "step-a" }),
|
||||
makeStep({ id: "step-b", dependsOn: ["step-a"] }),
|
||||
], "failure-test");
|
||||
writeGraph(runDir, graph);
|
||||
writeDefinition(runDir, graph.steps, "failure-test");
|
||||
|
||||
const ctx = makeMockCtx();
|
||||
const pi = makeMockPi();
|
||||
|
||||
const s = makeLoopSession({
|
||||
activeEngineId: "custom",
|
||||
activeRunDir: runDir,
|
||||
basePath: runDir,
|
||||
});
|
||||
|
||||
const deps = makeMockDeps({
|
||||
stopAuto: async (_ctx, _pi, reason) => {
|
||||
deps.callLog.push(`stopAuto:${reason ?? "no-reason"}`);
|
||||
s.active = false;
|
||||
},
|
||||
});
|
||||
|
||||
const loopPromise = autoLoop(ctx, pi, s, deps);
|
||||
|
||||
// Resolve step-a successfully
|
||||
await new Promise((r) => setTimeout(r, 80));
|
||||
resolveAgentEnd({ messages: [{ role: "assistant" }] });
|
||||
|
||||
// Step-b enters runUnit — deactivate the session before resolving.
|
||||
// runUnit checks s.active after newSession and returns cancelled if false.
|
||||
// But since newSession resolves synchronously in our mock (before the
|
||||
// active check), the unit still runs. Instead, let's just cancel it.
|
||||
await new Promise((r) => setTimeout(r, 80));
|
||||
// Resolve as cancelled to simulate a failed session
|
||||
resolveAgentEnd({ messages: [{ role: "assistant" }] });
|
||||
|
||||
// The reconcile will still run for step-b in this flow since
|
||||
// runUnitPhase returns "next" (not "break") for completed units.
|
||||
// After both steps complete, the engine detects isComplete and stops.
|
||||
await loopPromise;
|
||||
|
||||
// Verify step-a is complete
|
||||
const finalGraph = readGraph(runDir);
|
||||
const stepA = finalGraph.steps.find(s => s.id === "step-a");
|
||||
assert.equal(stepA?.status, "complete", "Step-a should be complete");
|
||||
|
||||
// Verify the loop stopped appropriately
|
||||
assert.ok(
|
||||
deps.callLog.some((e: string) => e.startsWith("stopAuto:")),
|
||||
"stopAuto should have been called",
|
||||
);
|
||||
});
|
||||
});
|
||||
382
src/resources/extensions/gsd/tests/custom-verification.test.ts
Normal file
382
src/resources/extensions/gsd/tests/custom-verification.test.ts
Normal file
|
|
@ -0,0 +1,382 @@
|
|||
/**
|
||||
* custom-verification.test.ts — Tests for runCustomVerification().
|
||||
*
|
||||
* Tests all four verification policies (content-heuristic, shell-command,
|
||||
* prompt-verify, human-review) plus edge cases (no policy, missing file).
|
||||
* Each test creates a temp run directory with a DEFINITION.yaml and
|
||||
* optional test artifacts.
|
||||
*/
|
||||
|
||||
import { describe, it } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtempSync, writeFileSync, mkdirSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import { stringify } from "yaml";
|
||||
import { runCustomVerification } from "../custom-verification.ts";
|
||||
import type { WorkflowDefinition } from "../definition-loader.ts";
|
||||
|
||||
/** Create a temp run directory with the given definition and optional files. */
|
||||
function makeTempRun(
|
||||
def: WorkflowDefinition,
|
||||
files?: Record<string, string>,
|
||||
): string {
|
||||
const runDir = mkdtempSync(join(tmpdir(), "cv-test-"));
|
||||
writeFileSync(join(runDir, "DEFINITION.yaml"), stringify(def), "utf-8");
|
||||
|
||||
if (files) {
|
||||
for (const [relPath, content] of Object.entries(files)) {
|
||||
const absPath = join(runDir, relPath);
|
||||
// Ensure parent directories exist
|
||||
const parentDir = join(absPath, "..");
|
||||
mkdirSync(parentDir, { recursive: true });
|
||||
writeFileSync(absPath, content, "utf-8");
|
||||
}
|
||||
}
|
||||
|
||||
return runDir;
|
||||
}
|
||||
|
||||
/** Minimal valid workflow definition factory. */
|
||||
function makeDef(
|
||||
steps: WorkflowDefinition["steps"],
|
||||
): WorkflowDefinition {
|
||||
return {
|
||||
version: 1,
|
||||
name: "test-workflow",
|
||||
steps,
|
||||
};
|
||||
}
|
||||
|
||||
// ─── content-heuristic tests ────────────────────────────────────────────
|
||||
|
||||
describe("content-heuristic policy", () => {
|
||||
it("returns 'continue' when file exists and meets size/pattern", () => {
|
||||
const def = makeDef([
|
||||
{
|
||||
id: "step-1",
|
||||
name: "Generate report",
|
||||
prompt: "Generate a report",
|
||||
requires: [],
|
||||
produces: ["report.md"],
|
||||
verify: {
|
||||
policy: "content-heuristic",
|
||||
minSize: 10,
|
||||
pattern: "# Report",
|
||||
},
|
||||
},
|
||||
]);
|
||||
|
||||
const runDir = makeTempRun(def, {
|
||||
"report.md": "# Report\n\nThis is a valid report with sufficient content.",
|
||||
});
|
||||
|
||||
const result = runCustomVerification(runDir, "step-1");
|
||||
assert.equal(result, "continue");
|
||||
});
|
||||
|
||||
it("returns 'pause' when produces file is missing", () => {
|
||||
const def = makeDef([
|
||||
{
|
||||
id: "step-1",
|
||||
name: "Generate report",
|
||||
prompt: "Generate a report",
|
||||
requires: [],
|
||||
produces: ["report.md"],
|
||||
verify: { policy: "content-heuristic" },
|
||||
},
|
||||
]);
|
||||
|
||||
// No files created — report.md doesn't exist
|
||||
const runDir = makeTempRun(def);
|
||||
|
||||
const result = runCustomVerification(runDir, "step-1");
|
||||
assert.equal(result, "pause");
|
||||
});
|
||||
|
||||
it("returns 'pause' when file exists but below minSize", () => {
|
||||
const def = makeDef([
|
||||
{
|
||||
id: "step-1",
|
||||
name: "Generate report",
|
||||
prompt: "Generate a report",
|
||||
requires: [],
|
||||
produces: ["report.md"],
|
||||
verify: {
|
||||
policy: "content-heuristic",
|
||||
minSize: 1000,
|
||||
},
|
||||
},
|
||||
]);
|
||||
|
||||
const runDir = makeTempRun(def, {
|
||||
"report.md": "tiny",
|
||||
});
|
||||
|
||||
const result = runCustomVerification(runDir, "step-1");
|
||||
assert.equal(result, "pause");
|
||||
});
|
||||
|
||||
it("returns 'pause' when file exists but pattern does not match", () => {
|
||||
const def = makeDef([
|
||||
{
|
||||
id: "step-1",
|
||||
name: "Generate report",
|
||||
prompt: "Generate a report",
|
||||
requires: [],
|
||||
produces: ["report.md"],
|
||||
verify: {
|
||||
policy: "content-heuristic",
|
||||
pattern: "^# Summary",
|
||||
},
|
||||
},
|
||||
]);
|
||||
|
||||
const runDir = makeTempRun(def, {
|
||||
"report.md": "This has no heading at all.",
|
||||
});
|
||||
|
||||
const result = runCustomVerification(runDir, "step-1");
|
||||
assert.equal(result, "pause");
|
||||
});
|
||||
|
||||
it("returns 'continue' when produces is empty", () => {
|
||||
const def = makeDef([
|
||||
{
|
||||
id: "step-1",
|
||||
name: "Think step",
|
||||
prompt: "Think about the problem",
|
||||
requires: [],
|
||||
produces: [],
|
||||
verify: { policy: "content-heuristic" },
|
||||
},
|
||||
]);
|
||||
|
||||
const runDir = makeTempRun(def);
|
||||
|
||||
const result = runCustomVerification(runDir, "step-1");
|
||||
assert.equal(result, "continue");
|
||||
});
|
||||
|
||||
it("returns 'continue' when file exists with no minSize or pattern checks", () => {
|
||||
const def = makeDef([
|
||||
{
|
||||
id: "step-1",
|
||||
name: "Generate output",
|
||||
prompt: "Generate output",
|
||||
requires: [],
|
||||
produces: ["output.txt"],
|
||||
verify: { policy: "content-heuristic" },
|
||||
},
|
||||
]);
|
||||
|
||||
const runDir = makeTempRun(def, {
|
||||
"output.txt": "",
|
||||
});
|
||||
|
||||
const result = runCustomVerification(runDir, "step-1");
|
||||
assert.equal(result, "continue");
|
||||
});
|
||||
});
|
||||
|
||||
// ─── shell-command tests ────────────────────────────────────────────────
|
||||
|
||||
describe("shell-command policy", () => {
|
||||
it("returns 'continue' when command exits 0", () => {
|
||||
const def = makeDef([
|
||||
{
|
||||
id: "step-1",
|
||||
name: "Build artifact",
|
||||
prompt: "Build the artifact",
|
||||
requires: [],
|
||||
produces: ["artifact.txt"],
|
||||
verify: {
|
||||
policy: "shell-command",
|
||||
command: "test -f artifact.txt",
|
||||
},
|
||||
},
|
||||
]);
|
||||
|
||||
const runDir = makeTempRun(def, {
|
||||
"artifact.txt": "content",
|
||||
});
|
||||
|
||||
const result = runCustomVerification(runDir, "step-1");
|
||||
assert.equal(result, "continue");
|
||||
});
|
||||
|
||||
it("returns 'retry' when command exits non-zero", () => {
|
||||
const def = makeDef([
|
||||
{
|
||||
id: "step-1",
|
||||
name: "Build artifact",
|
||||
prompt: "Build the artifact",
|
||||
requires: [],
|
||||
produces: ["artifact.txt"],
|
||||
verify: {
|
||||
policy: "shell-command",
|
||||
command: "test -f nonexistent-file.txt",
|
||||
},
|
||||
},
|
||||
]);
|
||||
|
||||
const runDir = makeTempRun(def);
|
||||
|
||||
const result = runCustomVerification(runDir, "step-1");
|
||||
assert.equal(result, "retry");
|
||||
});
|
||||
});
|
||||
|
||||
// ─── prompt-verify tests ────────────────────────────────────────────────
|
||||
|
||||
describe("prompt-verify policy", () => {
|
||||
it("returns 'pause'", () => {
|
||||
const def = makeDef([
|
||||
{
|
||||
id: "step-1",
|
||||
name: "Creative step",
|
||||
prompt: "Write something creative",
|
||||
requires: [],
|
||||
produces: ["creative.md"],
|
||||
verify: {
|
||||
policy: "prompt-verify",
|
||||
prompt: "Does the creative output meet the brief?",
|
||||
},
|
||||
},
|
||||
]);
|
||||
|
||||
const runDir = makeTempRun(def);
|
||||
|
||||
const result = runCustomVerification(runDir, "step-1");
|
||||
assert.equal(result, "pause");
|
||||
});
|
||||
});
|
||||
|
||||
// ─── human-review tests ─────────────────────────────────────────────────
|
||||
|
||||
describe("human-review policy", () => {
|
||||
it("returns 'pause'", () => {
|
||||
const def = makeDef([
|
||||
{
|
||||
id: "step-1",
|
||||
name: "Review step",
|
||||
prompt: "Prepare for review",
|
||||
requires: [],
|
||||
produces: ["review-doc.md"],
|
||||
verify: { policy: "human-review" },
|
||||
},
|
||||
]);
|
||||
|
||||
const runDir = makeTempRun(def);
|
||||
|
||||
const result = runCustomVerification(runDir, "step-1");
|
||||
assert.equal(result, "pause");
|
||||
});
|
||||
});
|
||||
|
||||
// ─── no verify policy tests ─────────────────────────────────────────────
|
||||
|
||||
describe("no verify policy", () => {
|
||||
it("returns 'continue' when step has no verify field", () => {
|
||||
const def = makeDef([
|
||||
{
|
||||
id: "step-1",
|
||||
name: "Simple step",
|
||||
prompt: "Do something simple",
|
||||
requires: [],
|
||||
produces: [],
|
||||
// No verify field
|
||||
},
|
||||
]);
|
||||
|
||||
const runDir = makeTempRun(def);
|
||||
|
||||
const result = runCustomVerification(runDir, "step-1");
|
||||
assert.equal(result, "continue");
|
||||
});
|
||||
|
||||
it("returns 'continue' when step ID is not found in definition", () => {
|
||||
const def = makeDef([
|
||||
{
|
||||
id: "step-1",
|
||||
name: "Only step",
|
||||
prompt: "Only step",
|
||||
requires: [],
|
||||
produces: [],
|
||||
},
|
||||
]);
|
||||
|
||||
const runDir = makeTempRun(def);
|
||||
|
||||
const result = runCustomVerification(runDir, "nonexistent-step");
|
||||
assert.equal(result, "continue");
|
||||
});
|
||||
});
|
||||
|
||||
// ─── missing DEFINITION.yaml ────────────────────────────────────────────
|
||||
|
||||
describe("error handling", () => {
|
||||
it("throws when DEFINITION.yaml is missing", () => {
|
||||
const runDir = mkdtempSync(join(tmpdir(), "cv-test-nodef-"));
|
||||
// No DEFINITION.yaml written
|
||||
|
||||
assert.throws(
|
||||
() => runCustomVerification(runDir, "step-1"),
|
||||
/ENOENT/,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── CustomExecutionPolicy integration ──────────────────────────────────
|
||||
|
||||
describe("CustomExecutionPolicy.verify() integration", () => {
|
||||
it("extracts stepId from unitId and calls runCustomVerification", async () => {
|
||||
// Import the policy class
|
||||
const { CustomExecutionPolicy } = await import("../custom-execution-policy.ts");
|
||||
|
||||
const def = makeDef([
|
||||
{
|
||||
id: "analyze",
|
||||
name: "Analyze",
|
||||
prompt: "Analyze the data",
|
||||
requires: [],
|
||||
produces: ["analysis.md"],
|
||||
verify: { policy: "content-heuristic" },
|
||||
},
|
||||
]);
|
||||
|
||||
const runDir = makeTempRun(def, {
|
||||
"analysis.md": "Analysis complete.",
|
||||
});
|
||||
|
||||
const policy = new CustomExecutionPolicy(runDir);
|
||||
const result = await policy.verify("custom-step", "my-workflow/analyze", {
|
||||
basePath: "/tmp",
|
||||
});
|
||||
assert.equal(result, "continue");
|
||||
});
|
||||
|
||||
it("returns 'pause' when content-heuristic fails via policy", async () => {
|
||||
const { CustomExecutionPolicy } = await import("../custom-execution-policy.ts");
|
||||
|
||||
const def = makeDef([
|
||||
{
|
||||
id: "generate",
|
||||
name: "Generate",
|
||||
prompt: "Generate output",
|
||||
requires: [],
|
||||
produces: ["output.md"],
|
||||
verify: { policy: "content-heuristic" },
|
||||
},
|
||||
]);
|
||||
|
||||
// No output.md created
|
||||
const runDir = makeTempRun(def);
|
||||
|
||||
const policy = new CustomExecutionPolicy(runDir);
|
||||
const result = await policy.verify("custom-step", "my-workflow/generate", {
|
||||
basePath: "/tmp",
|
||||
});
|
||||
assert.equal(result, "pause");
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,339 @@
|
|||
/**
|
||||
* custom-workflow-engine.test.ts — Tests for CustomWorkflowEngine and CustomExecutionPolicy.
|
||||
*
|
||||
* Uses real temp directories with actual GRAPH.yaml files — no mocks.
|
||||
* Tests the full engine lifecycle: deriveState → resolveDispatch → reconcile.
|
||||
*/
|
||||
|
||||
import { describe, it, afterEach } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtempSync, rmSync, readFileSync, writeFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import { parse } from "yaml";
|
||||
|
||||
import { CustomWorkflowEngine } from "../custom-workflow-engine.ts";
|
||||
import { CustomExecutionPolicy } from "../custom-execution-policy.ts";
|
||||
import { writeGraph, readGraph, type WorkflowGraph, type GraphStep } from "../graph.ts";
|
||||
import { stringify } from "yaml";
|
||||
|
||||
// ─── Helpers ─────────────────────────────────────────────────────────────
|
||||
|
||||
const tmpDirs: string[] = [];
|
||||
|
||||
function makeTmpDir(): string {
|
||||
const dir = mkdtempSync(join(tmpdir(), "engine-test-"));
|
||||
tmpDirs.push(dir);
|
||||
return dir;
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
for (const d of tmpDirs) {
|
||||
rmSync(d, { recursive: true, force: true });
|
||||
}
|
||||
tmpDirs.length = 0;
|
||||
});
|
||||
|
||||
function makeStep(overrides: Partial<GraphStep> & { id: string }): GraphStep {
|
||||
return {
|
||||
title: overrides.id,
|
||||
status: "pending",
|
||||
prompt: `Do ${overrides.id}`,
|
||||
dependsOn: [],
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
function makeGraph(steps: GraphStep[], name = "test-wf"): WorkflowGraph {
|
||||
return {
|
||||
steps,
|
||||
metadata: { name, createdAt: "2026-01-01T00:00:00.000Z" },
|
||||
};
|
||||
}
|
||||
|
||||
/** Write a graph to a temp dir and return engine + dir. Also writes a minimal DEFINITION.yaml so resolveDispatch/injectContext can read it. */
|
||||
function setupEngine(
|
||||
steps: GraphStep[],
|
||||
name = "test-wf",
|
||||
): { engine: CustomWorkflowEngine; runDir: string } {
|
||||
const runDir = makeTmpDir();
|
||||
const graph = makeGraph(steps, name);
|
||||
writeGraph(runDir, graph);
|
||||
|
||||
// Write a minimal DEFINITION.yaml matching the graph steps
|
||||
const def = {
|
||||
version: 1,
|
||||
name,
|
||||
steps: steps.map((s) => ({
|
||||
id: s.id,
|
||||
name: s.title,
|
||||
prompt: s.prompt,
|
||||
requires: s.dependsOn,
|
||||
produces: [],
|
||||
})),
|
||||
};
|
||||
writeFileSync(join(runDir, "DEFINITION.yaml"), stringify(def), "utf-8");
|
||||
|
||||
return { engine: new CustomWorkflowEngine(runDir), runDir };
|
||||
}
|
||||
|
||||
// ─── deriveState ─────────────────────────────────────────────────────────
|
||||
|
||||
describe("CustomWorkflowEngine.deriveState", () => {
|
||||
it("returns running phase when steps are pending", async () => {
|
||||
const { engine } = setupEngine([
|
||||
makeStep({ id: "a" }),
|
||||
makeStep({ id: "b", dependsOn: ["a"] }),
|
||||
]);
|
||||
|
||||
const state = await engine.deriveState("/unused");
|
||||
|
||||
assert.equal(state.phase, "running");
|
||||
assert.equal(state.isComplete, false);
|
||||
assert.ok(state.raw, "raw should contain the graph");
|
||||
});
|
||||
|
||||
it("returns complete phase when all steps are complete", async () => {
|
||||
const { engine } = setupEngine([
|
||||
makeStep({ id: "a", status: "complete" }),
|
||||
makeStep({ id: "b", status: "complete" }),
|
||||
]);
|
||||
|
||||
const state = await engine.deriveState("/unused");
|
||||
|
||||
assert.equal(state.phase, "complete");
|
||||
assert.equal(state.isComplete, true);
|
||||
});
|
||||
|
||||
it("treats expanded steps as done for completion check", async () => {
|
||||
const { engine } = setupEngine([
|
||||
makeStep({ id: "a", status: "expanded" }),
|
||||
makeStep({ id: "a--001", status: "complete", parentStepId: "a" }),
|
||||
makeStep({ id: "b", status: "complete" }),
|
||||
]);
|
||||
|
||||
const state = await engine.deriveState("/unused");
|
||||
|
||||
assert.equal(state.phase, "complete");
|
||||
assert.equal(state.isComplete, true);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── resolveDispatch ─────────────────────────────────────────────────────
|
||||
|
||||
describe("CustomWorkflowEngine.resolveDispatch", () => {
|
||||
it("returns dispatch for first pending step", async () => {
|
||||
const { engine } = setupEngine([
|
||||
makeStep({ id: "step-1", prompt: "Do the first thing" }),
|
||||
makeStep({ id: "step-2", dependsOn: ["step-1"] }),
|
||||
], "my-workflow");
|
||||
|
||||
const state = await engine.deriveState("/unused");
|
||||
const dispatch = await engine.resolveDispatch(state, { basePath: "/unused" });
|
||||
|
||||
assert.equal(dispatch.action, "dispatch");
|
||||
if (dispatch.action === "dispatch") {
|
||||
assert.equal(dispatch.step.unitType, "custom-step");
|
||||
assert.equal(dispatch.step.unitId, "my-workflow/step-1");
|
||||
assert.equal(dispatch.step.prompt, "Do the first thing");
|
||||
}
|
||||
});
|
||||
|
||||
it("returns stop when all steps are complete", async () => {
|
||||
const { engine } = setupEngine([
|
||||
makeStep({ id: "a", status: "complete" }),
|
||||
makeStep({ id: "b", status: "complete" }),
|
||||
]);
|
||||
|
||||
const state = await engine.deriveState("/unused");
|
||||
const dispatch = await engine.resolveDispatch(state, { basePath: "/unused" });
|
||||
|
||||
assert.equal(dispatch.action, "stop");
|
||||
if (dispatch.action === "stop") {
|
||||
assert.equal(dispatch.reason, "All steps complete");
|
||||
assert.equal(dispatch.level, "info");
|
||||
}
|
||||
});
|
||||
|
||||
it("respects dependency ordering", async () => {
|
||||
const { engine } = setupEngine([
|
||||
makeStep({ id: "a" }),
|
||||
makeStep({ id: "b", dependsOn: ["a"] }),
|
||||
makeStep({ id: "c", dependsOn: ["b"] }),
|
||||
], "dep-wf");
|
||||
|
||||
const state = await engine.deriveState("/unused");
|
||||
const dispatch = await engine.resolveDispatch(state, { basePath: "/unused" });
|
||||
|
||||
// Should pick "a" (no deps), not "b" or "c"
|
||||
assert.equal(dispatch.action, "dispatch");
|
||||
if (dispatch.action === "dispatch") {
|
||||
assert.equal(dispatch.step.unitId, "dep-wf/a");
|
||||
}
|
||||
});
|
||||
|
||||
it("picks next eligible step when earlier deps are complete", async () => {
|
||||
const { engine } = setupEngine([
|
||||
makeStep({ id: "a", status: "complete" }),
|
||||
makeStep({ id: "b", dependsOn: ["a"] }),
|
||||
makeStep({ id: "c", dependsOn: ["b"] }),
|
||||
], "dep-wf");
|
||||
|
||||
const state = await engine.deriveState("/unused");
|
||||
const dispatch = await engine.resolveDispatch(state, { basePath: "/unused" });
|
||||
|
||||
// "a" is done, "b" deps met, should pick "b"
|
||||
assert.equal(dispatch.action, "dispatch");
|
||||
if (dispatch.action === "dispatch") {
|
||||
assert.equal(dispatch.step.unitId, "dep-wf/b");
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ─── reconcile ───────────────────────────────────────────────────────────
|
||||
|
||||
describe("CustomWorkflowEngine.reconcile", () => {
|
||||
it("marks step complete in GRAPH.yaml on disk", async () => {
|
||||
const { engine, runDir } = setupEngine([
|
||||
makeStep({ id: "step-1" }),
|
||||
makeStep({ id: "step-2", dependsOn: ["step-1"] }),
|
||||
], "wf");
|
||||
|
||||
const state = await engine.deriveState("/unused");
|
||||
const result = await engine.reconcile(state, {
|
||||
unitType: "custom-step",
|
||||
unitId: "wf/step-1",
|
||||
startedAt: Date.now() - 1000,
|
||||
finishedAt: Date.now(),
|
||||
});
|
||||
|
||||
assert.equal(result.outcome, "continue");
|
||||
|
||||
// Verify on-disk state
|
||||
const graph = readGraph(runDir);
|
||||
assert.equal(graph.steps[0].status, "complete");
|
||||
assert.ok(graph.steps[0].finishedAt, "finishedAt should be set");
|
||||
assert.equal(graph.steps[1].status, "pending");
|
||||
});
|
||||
|
||||
it("returns milestone-complete when all steps done", async () => {
|
||||
const { engine } = setupEngine([
|
||||
makeStep({ id: "only-step" }),
|
||||
], "wf");
|
||||
|
||||
const state = await engine.deriveState("/unused");
|
||||
const result = await engine.reconcile(state, {
|
||||
unitType: "custom-step",
|
||||
unitId: "wf/only-step",
|
||||
startedAt: Date.now() - 1000,
|
||||
finishedAt: Date.now(),
|
||||
});
|
||||
|
||||
assert.equal(result.outcome, "milestone-complete");
|
||||
});
|
||||
|
||||
it("handles multi-segment unitId correctly", async () => {
|
||||
const { engine, runDir } = setupEngine([
|
||||
makeStep({ id: "deep-step" }),
|
||||
], "nested/workflow");
|
||||
|
||||
const state = await engine.deriveState("/unused");
|
||||
const result = await engine.reconcile(state, {
|
||||
unitType: "custom-step",
|
||||
unitId: "nested/workflow/deep-step",
|
||||
startedAt: Date.now() - 1000,
|
||||
finishedAt: Date.now(),
|
||||
});
|
||||
|
||||
assert.equal(result.outcome, "milestone-complete");
|
||||
const graph = readGraph(runDir);
|
||||
assert.equal(graph.steps[0].status, "complete");
|
||||
});
|
||||
});
|
||||
|
||||
// ─── getDisplayMetadata ──────────────────────────────────────────────────
|
||||
|
||||
describe("CustomWorkflowEngine.getDisplayMetadata", () => {
|
||||
it("returns correct progress summary", async () => {
|
||||
const { engine } = setupEngine([
|
||||
makeStep({ id: "a", status: "complete" }),
|
||||
makeStep({ id: "b" }),
|
||||
makeStep({ id: "c" }),
|
||||
]);
|
||||
|
||||
const state = await engine.deriveState("/unused");
|
||||
const meta = engine.getDisplayMetadata(state);
|
||||
|
||||
assert.equal(meta.engineLabel, "WORKFLOW");
|
||||
assert.equal(meta.currentPhase, "running");
|
||||
assert.equal(meta.progressSummary, "Step 1/3");
|
||||
assert.deepStrictEqual(meta.stepCount, { completed: 1, total: 3 });
|
||||
});
|
||||
|
||||
it("shows 0/N when no steps complete", async () => {
|
||||
const { engine } = setupEngine([
|
||||
makeStep({ id: "a" }),
|
||||
makeStep({ id: "b" }),
|
||||
]);
|
||||
|
||||
const state = await engine.deriveState("/unused");
|
||||
const meta = engine.getDisplayMetadata(state);
|
||||
|
||||
assert.equal(meta.progressSummary, "Step 0/2");
|
||||
});
|
||||
|
||||
it("shows N/N when all steps complete", async () => {
|
||||
const { engine } = setupEngine([
|
||||
makeStep({ id: "a", status: "complete" }),
|
||||
makeStep({ id: "b", status: "complete" }),
|
||||
]);
|
||||
|
||||
const state = await engine.deriveState("/unused");
|
||||
const meta = engine.getDisplayMetadata(state);
|
||||
|
||||
assert.equal(meta.progressSummary, "Step 2/2");
|
||||
assert.equal(meta.currentPhase, "complete");
|
||||
});
|
||||
});
|
||||
|
||||
// ─── CustomExecutionPolicy ───────────────────────────────────────────────
|
||||
|
||||
describe("CustomExecutionPolicy", () => {
|
||||
it("verify returns continue", async () => {
|
||||
// verify() reads DEFINITION.yaml from runDir to find step's verify policy
|
||||
const runDir = makeTmpDir();
|
||||
writeFileSync(join(runDir, "DEFINITION.yaml"), stringify({
|
||||
version: 1, name: "wf", description: "test",
|
||||
steps: [{ id: "step-1", name: "Step 1", prompt: "do it", produces: "step-1/output.md" }],
|
||||
}));
|
||||
const policy = new CustomExecutionPolicy(runDir);
|
||||
const result = await policy.verify("custom-step", "wf/step-1", { basePath: runDir });
|
||||
assert.equal(result, "continue");
|
||||
});
|
||||
|
||||
it("selectModel returns null", async () => {
|
||||
const policy = new CustomExecutionPolicy("/tmp/run");
|
||||
const result = await policy.selectModel("custom-step", "wf/step-1", { basePath: "/tmp" });
|
||||
assert.equal(result, null);
|
||||
});
|
||||
|
||||
it("recover returns retry", async () => {
|
||||
const policy = new CustomExecutionPolicy("/tmp/run");
|
||||
const result = await policy.recover("custom-step", "wf/step-1", { basePath: "/tmp" });
|
||||
assert.deepStrictEqual(result, { outcome: "retry", reason: "Default retry" });
|
||||
});
|
||||
|
||||
it("closeout returns no artifacts", async () => {
|
||||
const policy = new CustomExecutionPolicy("/tmp/run");
|
||||
const result = await policy.closeout("custom-step", "wf/step-1", {
|
||||
basePath: "/tmp",
|
||||
startedAt: Date.now(),
|
||||
});
|
||||
assert.deepStrictEqual(result, { committed: false, artifacts: [] });
|
||||
});
|
||||
|
||||
it("prepareWorkspace resolves without error", async () => {
|
||||
const policy = new CustomExecutionPolicy("/tmp/run");
|
||||
await policy.prepareWorkspace("/tmp", "M001"); // Should not throw
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,87 @@
|
|||
/**
|
||||
* dashboard-custom-engine.test.ts — Tests that the custom engine path
|
||||
* calls updateProgressWidget and that unitLabel handles "custom-step".
|
||||
*
|
||||
* Uses source-level assertions for the non-exported unitLabel function
|
||||
* and the updateProgressWidget call placement. Tests exported helpers
|
||||
* (unitVerb, unitPhaseLabel) directly.
|
||||
*/
|
||||
|
||||
import { describe, it } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
import { unitVerb, unitPhaseLabel } from "../auto-dashboard.js";
|
||||
|
||||
// ─── Tests ───────────────────────────────────────────────────────────────
|
||||
|
||||
describe("Dashboard custom-engine: unitLabel and related helpers", () => {
|
||||
it('unitVerb("custom-step") returns "executing workflow step"', () => {
|
||||
assert.equal(unitVerb("custom-step"), "executing workflow step");
|
||||
});
|
||||
|
||||
it('unitPhaseLabel("custom-step") returns "WORKFLOW"', () => {
|
||||
assert.equal(unitPhaseLabel("custom-step"), "WORKFLOW");
|
||||
});
|
||||
|
||||
it('dashboard-overlay.ts contains a case for "custom-step" returning "Workflow Step"', () => {
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const overlayPath = resolve(__filename, "../../dashboard-overlay.ts");
|
||||
const source = readFileSync(overlayPath, "utf-8");
|
||||
assert.ok(
|
||||
source.includes('"custom-step"') && source.includes('"Workflow Step"'),
|
||||
'dashboard-overlay.ts should contain case "custom-step": return "Workflow Step"',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("Dashboard custom-engine: updateProgressWidget in custom engine path", () => {
|
||||
it("loop.ts custom engine path includes updateProgressWidget call before runGuards", () => {
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const loopPath = resolve(__filename, "../../auto/loop.ts");
|
||||
const source = readFileSync(loopPath, "utf-8");
|
||||
|
||||
// Find the custom engine block
|
||||
const customEngineStart = source.indexOf('s.activeEngineId !== "dev"');
|
||||
assert.ok(customEngineStart > -1, "Should find custom engine path in loop.ts");
|
||||
|
||||
// The updateProgressWidget call should appear after the custom engine block start
|
||||
// and before the runGuards call in that block
|
||||
const afterCustomEngine = source.slice(customEngineStart);
|
||||
const widgetCallIndex = afterCustomEngine.indexOf(
|
||||
"deps.updateProgressWidget(ctx, iterData.unitType, iterData.unitId, iterData.state)",
|
||||
);
|
||||
const guardsCallIndex = afterCustomEngine.indexOf("runGuards(ic,");
|
||||
assert.ok(widgetCallIndex > -1, "updateProgressWidget should be called in custom engine path");
|
||||
assert.ok(
|
||||
widgetCallIndex < guardsCallIndex,
|
||||
"updateProgressWidget should be called before runGuards in custom engine path",
|
||||
);
|
||||
});
|
||||
|
||||
it("updateProgressWidget call is placed after iterData is built", () => {
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const loopPath = resolve(__filename, "../../auto/loop.ts");
|
||||
const source = readFileSync(loopPath, "utf-8");
|
||||
|
||||
const customEngineStart = source.indexOf('s.activeEngineId !== "dev"');
|
||||
const afterCustomEngine = source.slice(customEngineStart);
|
||||
|
||||
// Verify custom engine path has iterData built before the widget call
|
||||
const iterDataIndex = afterCustomEngine.indexOf("iterData = {");
|
||||
const widgetIndex = afterCustomEngine.indexOf("deps.updateProgressWidget");
|
||||
assert.ok(iterDataIndex > -1 && widgetIndex > -1, "Both iterData and widget call should exist");
|
||||
assert.ok(
|
||||
iterDataIndex < widgetIndex,
|
||||
"iterData should be built before updateProgressWidget is called",
|
||||
);
|
||||
|
||||
// Verify the call uses iterData.state (which holds the derived GSD state)
|
||||
assert.ok(
|
||||
afterCustomEngine.includes("iterData.state"),
|
||||
"Custom engine updateProgressWidget should reference iterData.state",
|
||||
);
|
||||
});
|
||||
});
|
||||
778
src/resources/extensions/gsd/tests/definition-loader.test.ts
Normal file
778
src/resources/extensions/gsd/tests/definition-loader.test.ts
Normal file
|
|
@ -0,0 +1,778 @@
|
|||
/**
|
||||
* Unit tests for definition-loader.ts.
|
||||
*
|
||||
* Covers V1 YAML schema validation (valid + various rejection cases),
|
||||
* filesystem loading, snake_case → camelCase conversion, forward
|
||||
* compatibility with unknown fields, parameter substitution, and the
|
||||
* four gap validations (duplicate IDs, dangling deps, self-deps, cycles).
|
||||
*/
|
||||
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
|
||||
import {
|
||||
loadDefinition,
|
||||
validateDefinition,
|
||||
substituteParams,
|
||||
substitutePromptString,
|
||||
} from "../definition-loader.ts";
|
||||
import type { WorkflowDefinition } from "../definition-loader.ts";
|
||||
|
||||
// ─── Helpers ─────────────────────────────────────────────────────────────
|
||||
|
||||
function makeTmpDir(): string {
|
||||
return mkdtempSync(join(tmpdir(), "gsd-defloader-test-"));
|
||||
}
|
||||
|
||||
/** Write a YAML string into a temp definitions directory. Returns the dir path. */
|
||||
function writeDefYaml(yaml: string, name = "test-workflow"): string {
|
||||
const dir = makeTmpDir();
|
||||
writeFileSync(join(dir, `${name}.yaml`), yaml, "utf-8");
|
||||
return dir;
|
||||
}
|
||||
|
||||
const VALID_3STEP_YAML = `
|
||||
version: 1
|
||||
name: "test-workflow"
|
||||
description: "A test workflow"
|
||||
params:
|
||||
topic: "AI"
|
||||
steps:
|
||||
- id: research
|
||||
name: "Research the topic"
|
||||
prompt: "Research {{topic}} and write findings to research.md"
|
||||
requires: []
|
||||
produces:
|
||||
- research.md
|
||||
- id: outline
|
||||
name: "Create outline"
|
||||
prompt: "Based on research.md, create an outline in outline.md"
|
||||
requires: [research]
|
||||
produces:
|
||||
- outline.md
|
||||
- id: draft
|
||||
name: "Write draft"
|
||||
prompt: "Write a draft based on outline.md"
|
||||
requires: [outline]
|
||||
produces:
|
||||
- draft.md
|
||||
`;
|
||||
|
||||
// ─── loadDefinition: valid YAML ──────────────────────────────────────────
|
||||
|
||||
test("loadDefinition: valid 3-step YAML returns correct structure", () => {
|
||||
const dir = writeDefYaml(VALID_3STEP_YAML);
|
||||
try {
|
||||
const def = loadDefinition(dir, "test-workflow");
|
||||
|
||||
assert.equal(def.version, 1);
|
||||
assert.equal(def.name, "test-workflow");
|
||||
assert.equal(def.description, "A test workflow");
|
||||
assert.deepEqual(def.params, { topic: "AI" });
|
||||
assert.equal(def.steps.length, 3);
|
||||
|
||||
// Step 1: research
|
||||
assert.equal(def.steps[0].id, "research");
|
||||
assert.equal(def.steps[0].name, "Research the topic");
|
||||
assert.equal(def.steps[0].prompt, "Research {{topic}} and write findings to research.md");
|
||||
assert.deepEqual(def.steps[0].requires, []);
|
||||
assert.deepEqual(def.steps[0].produces, ["research.md"]);
|
||||
|
||||
// Step 2: outline — depends on research
|
||||
assert.equal(def.steps[1].id, "outline");
|
||||
assert.deepEqual(def.steps[1].requires, ["research"]);
|
||||
|
||||
// Step 3: draft — depends on outline
|
||||
assert.equal(def.steps[2].id, "draft");
|
||||
assert.deepEqual(def.steps[2].requires, ["outline"]);
|
||||
assert.deepEqual(def.steps[2].produces, ["draft.md"]);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
// ─── validateDefinition: rejection cases ─────────────────────────────────
|
||||
|
||||
test("validateDefinition: missing version → error", () => {
|
||||
const result = validateDefinition({
|
||||
name: "test",
|
||||
steps: [{ id: "a", name: "A", prompt: "do A" }],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes("version")));
|
||||
});
|
||||
|
||||
test("validateDefinition: version 2 (unsupported) → error", () => {
|
||||
const result = validateDefinition({
|
||||
version: 2,
|
||||
name: "test",
|
||||
steps: [{ id: "a", name: "A", prompt: "do A" }],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes("Unsupported version: 2")));
|
||||
});
|
||||
|
||||
test("validateDefinition: missing step id → error", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [{ name: "A", prompt: "do A" }],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes("index 0") && e.includes("id")));
|
||||
});
|
||||
|
||||
test("validateDefinition: missing step prompt → error", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [{ id: "a", name: "A" }],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes("index 0") && e.includes("prompt")));
|
||||
});
|
||||
|
||||
test("validateDefinition: produces with '..' path traversal → error", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [{ id: "a", name: "A", prompt: "do A", produces: ["../secret.txt"] }],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes("..") && e.includes("produces")));
|
||||
});
|
||||
|
||||
test("validateDefinition: unknown fields (context_from, iterate) → accepted silently", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
future_top_level_field: true,
|
||||
steps: [{
|
||||
id: "a",
|
||||
name: "A",
|
||||
prompt: "do A",
|
||||
context_from: ["other-step"],
|
||||
iterate: { source: "file.md", pattern: "^## (.+)" },
|
||||
some_future_field: 42,
|
||||
}],
|
||||
});
|
||||
assert.equal(result.valid, true);
|
||||
assert.equal(result.errors.length, 0);
|
||||
});
|
||||
|
||||
test("validateDefinition: collects multiple errors in one pass", () => {
|
||||
const result = validateDefinition({
|
||||
// missing version and name
|
||||
steps: [
|
||||
{ id: "a" }, // missing name and prompt
|
||||
{ name: "B", prompt: "do B" }, // missing id
|
||||
],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
// Should have errors for: version, name, step 0 name, step 0 prompt, step 1 id
|
||||
assert.ok(result.errors.length >= 4, `Expected ≥4 errors, got ${result.errors.length}: ${result.errors.join("; ")}`);
|
||||
});
|
||||
|
||||
test("validateDefinition: null input → error", () => {
|
||||
const result = validateDefinition(null);
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes("non-null object")));
|
||||
});
|
||||
|
||||
test("validateDefinition: empty steps array → error", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes("at least one step")));
|
||||
});
|
||||
|
||||
test("validateDefinition: missing name → error", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
steps: [{ id: "a", name: "A", prompt: "do A" }],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes("name")));
|
||||
});
|
||||
|
||||
test("validateDefinition: step is not an object → error", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: ["not-an-object"],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes("index 0") && e.includes("not an object")));
|
||||
});
|
||||
|
||||
test("validateDefinition: missing step name → error", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [{ id: "a", prompt: "do A" }],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes("index 0") && e.includes("name")));
|
||||
});
|
||||
|
||||
// ─── loadDefinition: error cases ─────────────────────────────────────────
|
||||
|
||||
test("loadDefinition: missing file → descriptive error", () => {
|
||||
const dir = makeTmpDir();
|
||||
try {
|
||||
assert.throws(
|
||||
() => loadDefinition(dir, "nonexistent"),
|
||||
(err: Error) => {
|
||||
assert.ok(err.message.includes("not found"));
|
||||
assert.ok(err.message.includes("nonexistent.yaml"));
|
||||
return true;
|
||||
},
|
||||
);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("loadDefinition: invalid YAML schema → descriptive error", () => {
|
||||
const dir = writeDefYaml(`
|
||||
version: 2
|
||||
name: "bad"
|
||||
steps:
|
||||
- id: a
|
||||
name: "A"
|
||||
prompt: "do A"
|
||||
`);
|
||||
try {
|
||||
assert.throws(
|
||||
() => loadDefinition(dir, "test-workflow"),
|
||||
(err: Error) => {
|
||||
assert.ok(err.message.includes("Invalid workflow definition"));
|
||||
assert.ok(err.message.includes("Unsupported version"));
|
||||
return true;
|
||||
},
|
||||
);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
// ─── loadDefinition: snake_case → camelCase conversion ───────────────────
|
||||
|
||||
test("loadDefinition: depends_on in YAML maps to requires in TypeScript", () => {
|
||||
const dir = writeDefYaml(`
|
||||
version: 1
|
||||
name: "dep-test"
|
||||
steps:
|
||||
- id: first
|
||||
name: "First"
|
||||
prompt: "do first"
|
||||
- id: second
|
||||
name: "Second"
|
||||
prompt: "do second"
|
||||
depends_on: [first]
|
||||
`);
|
||||
try {
|
||||
const def = loadDefinition(dir, "test-workflow");
|
||||
assert.deepEqual(def.steps[1].requires, ["first"]);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("loadDefinition: context_from in YAML maps to contextFrom in TypeScript", () => {
|
||||
const dir = writeDefYaml(`
|
||||
version: 1
|
||||
name: "ctx-test"
|
||||
steps:
|
||||
- id: first
|
||||
name: "First"
|
||||
prompt: "do first"
|
||||
- id: second
|
||||
name: "Second"
|
||||
prompt: "do second"
|
||||
context_from: [first]
|
||||
`);
|
||||
try {
|
||||
const def = loadDefinition(dir, "test-workflow");
|
||||
assert.deepEqual(def.steps[1].contextFrom, ["first"]);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
// ─── validateDefinition: iterate field validation ────────────────────────
|
||||
|
||||
test("validateDefinition: valid iterate config accepted", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [{
|
||||
id: "a",
|
||||
name: "A",
|
||||
prompt: "do A",
|
||||
iterate: { source: "outline.md", pattern: "^## (.+)" },
|
||||
}],
|
||||
});
|
||||
assert.equal(result.valid, true);
|
||||
assert.equal(result.errors.length, 0);
|
||||
});
|
||||
|
||||
test("validateDefinition: iterate missing source → error", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [{
|
||||
id: "a",
|
||||
name: "A",
|
||||
prompt: "do A",
|
||||
iterate: { pattern: "^## (.+)" },
|
||||
}],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes("source")));
|
||||
});
|
||||
|
||||
test("validateDefinition: iterate source with .. → error", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [{
|
||||
id: "a",
|
||||
name: "A",
|
||||
prompt: "do A",
|
||||
iterate: { source: "../escape.md", pattern: "(.+)" },
|
||||
}],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes("path traversal") || e.includes("..")));
|
||||
});
|
||||
|
||||
test("validateDefinition: iterate invalid regex → error", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [{
|
||||
id: "a",
|
||||
name: "A",
|
||||
prompt: "do A",
|
||||
iterate: { source: "f.md", pattern: "[invalid" },
|
||||
}],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes("regex")));
|
||||
});
|
||||
|
||||
test("validateDefinition: iterate pattern without capture group → error", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [{
|
||||
id: "a",
|
||||
name: "A",
|
||||
prompt: "do A",
|
||||
iterate: { source: "f.md", pattern: "^## .+" },
|
||||
}],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes("capture group")));
|
||||
});
|
||||
|
||||
// ─── validateDefinition: verify field validation ─────────────────────────
|
||||
|
||||
test("validateDefinition: valid content-heuristic verify → accepted", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [{
|
||||
id: "a",
|
||||
name: "A",
|
||||
prompt: "do A",
|
||||
verify: { policy: "content-heuristic", minSize: 100, pattern: "^## " },
|
||||
}],
|
||||
});
|
||||
assert.equal(result.valid, true);
|
||||
assert.equal(result.errors.length, 0);
|
||||
});
|
||||
|
||||
test("validateDefinition: valid shell-command verify → accepted", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [{
|
||||
id: "a",
|
||||
name: "A",
|
||||
prompt: "do A",
|
||||
verify: { policy: "shell-command", command: "cat output.md | grep '^## '" },
|
||||
}],
|
||||
});
|
||||
assert.equal(result.valid, true);
|
||||
assert.equal(result.errors.length, 0);
|
||||
});
|
||||
|
||||
test("validateDefinition: valid prompt-verify → accepted", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [{
|
||||
id: "a",
|
||||
name: "A",
|
||||
prompt: "do A",
|
||||
verify: { policy: "prompt-verify", prompt: "Does the output contain at least 3 sections?" },
|
||||
}],
|
||||
});
|
||||
assert.equal(result.valid, true);
|
||||
assert.equal(result.errors.length, 0);
|
||||
});
|
||||
|
||||
test("validateDefinition: valid human-review verify → accepted", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [{
|
||||
id: "a",
|
||||
name: "A",
|
||||
prompt: "do A",
|
||||
verify: { policy: "human-review" },
|
||||
}],
|
||||
});
|
||||
assert.equal(result.valid, true);
|
||||
assert.equal(result.errors.length, 0);
|
||||
});
|
||||
|
||||
test("validateDefinition: invalid verify policy name → rejected", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [{
|
||||
id: "a",
|
||||
name: "A",
|
||||
prompt: "do A",
|
||||
verify: { policy: "magic-check" },
|
||||
}],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes("verify.policy must be one of")));
|
||||
});
|
||||
|
||||
test("validateDefinition: shell-command missing command → rejected", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [{
|
||||
id: "a",
|
||||
name: "A",
|
||||
prompt: "do A",
|
||||
verify: { policy: "shell-command" },
|
||||
}],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes('requires a non-empty "command"')));
|
||||
});
|
||||
|
||||
test("validateDefinition: prompt-verify missing prompt → rejected", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [{
|
||||
id: "a",
|
||||
name: "A",
|
||||
prompt: "do A",
|
||||
verify: { policy: "prompt-verify" },
|
||||
}],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes('requires a non-empty "prompt"')));
|
||||
});
|
||||
|
||||
// ─── Gap validations: duplicate IDs ──────────────────────────────────────
|
||||
|
||||
test("validateDefinition: duplicate step IDs → error", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [
|
||||
{ id: "dup", name: "A", prompt: "do A" },
|
||||
{ id: "dup", name: "B", prompt: "do B" },
|
||||
],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes("Duplicate step id")));
|
||||
assert.ok(result.errors.some((e) => e.includes("dup")));
|
||||
});
|
||||
|
||||
// ─── Gap validations: dangling dependencies ──────────────────────────────
|
||||
|
||||
test("validateDefinition: dangling dependency → error", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [
|
||||
{ id: "a", name: "A", prompt: "do A" },
|
||||
{ id: "b", name: "B", prompt: "do B", requires: ["nonexistent"] },
|
||||
],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes("requires unknown step")));
|
||||
assert.ok(result.errors.some((e) => e.includes("nonexistent")));
|
||||
});
|
||||
|
||||
test("validateDefinition: dangling dependency via depends_on → error", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [
|
||||
{ id: "a", name: "A", prompt: "do A" },
|
||||
{ id: "b", name: "B", prompt: "do B", depends_on: ["ghost"] },
|
||||
],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes("requires unknown step")));
|
||||
assert.ok(result.errors.some((e) => e.includes("ghost")));
|
||||
});
|
||||
|
||||
// ─── Gap validations: self-referencing dependencies ──────────────────────
|
||||
|
||||
test("validateDefinition: self-referencing dependency → error", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [
|
||||
{ id: "a", name: "A", prompt: "do A", requires: ["a"] },
|
||||
],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes("depends on itself")));
|
||||
});
|
||||
|
||||
// ─── Gap validations: cycle detection ────────────────────────────────────
|
||||
|
||||
test("validateDefinition: simple cycle (A→B→A) → error", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [
|
||||
{ id: "a", name: "A", prompt: "do A", requires: ["b"] },
|
||||
{ id: "b", name: "B", prompt: "do B", requires: ["a"] },
|
||||
],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes("Cycle detected")));
|
||||
});
|
||||
|
||||
test("validateDefinition: complex cycle (A→B→C→A) → error", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [
|
||||
{ id: "a", name: "A", prompt: "do A", requires: ["c"] },
|
||||
{ id: "b", name: "B", prompt: "do B", requires: ["a"] },
|
||||
{ id: "c", name: "C", prompt: "do C", requires: ["b"] },
|
||||
],
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes("Cycle detected")));
|
||||
});
|
||||
|
||||
test("validateDefinition: diamond dependency (no cycle) → accepted", () => {
|
||||
// A→B, A→C, B→D, C→D — classic diamond, no cycle
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [
|
||||
{ id: "a", name: "A", prompt: "do A" },
|
||||
{ id: "b", name: "B", prompt: "do B", requires: ["a"] },
|
||||
{ id: "c", name: "C", prompt: "do C", requires: ["a"] },
|
||||
{ id: "d", name: "D", prompt: "do D", requires: ["b", "c"] },
|
||||
],
|
||||
});
|
||||
assert.equal(result.valid, true, `Expected valid but got errors: ${result.errors.join("; ")}`);
|
||||
assert.equal(result.errors.length, 0);
|
||||
});
|
||||
|
||||
test("validateDefinition: linear chain (no cycle) → accepted", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [
|
||||
{ id: "a", name: "A", prompt: "do A" },
|
||||
{ id: "b", name: "B", prompt: "do B", requires: ["a"] },
|
||||
{ id: "c", name: "C", prompt: "do C", requires: ["b"] },
|
||||
{ id: "d", name: "D", prompt: "do D", requires: ["c"] },
|
||||
],
|
||||
});
|
||||
assert.equal(result.valid, true);
|
||||
});
|
||||
|
||||
// ─── substituteParams ────────────────────────────────────────────────────
|
||||
|
||||
test("substituteParams: replaces placeholders with defaults", () => {
|
||||
const def: WorkflowDefinition = {
|
||||
version: 1,
|
||||
name: "test",
|
||||
params: { topic: "AI", format: "markdown" },
|
||||
steps: [
|
||||
{ id: "a", name: "A", prompt: "Write about {{topic}} in {{format}}", requires: [], produces: [] },
|
||||
],
|
||||
};
|
||||
const result = substituteParams(def);
|
||||
assert.equal(result.steps[0].prompt, "Write about AI in markdown");
|
||||
});
|
||||
|
||||
test("substituteParams: overrides win over defaults", () => {
|
||||
const def: WorkflowDefinition = {
|
||||
version: 1,
|
||||
name: "test",
|
||||
params: { topic: "AI" },
|
||||
steps: [
|
||||
{ id: "a", name: "A", prompt: "Write about {{topic}}", requires: [], produces: [] },
|
||||
],
|
||||
};
|
||||
const result = substituteParams(def, { topic: "Robotics" });
|
||||
assert.equal(result.steps[0].prompt, "Write about Robotics");
|
||||
});
|
||||
|
||||
test("substituteParams: rejects values containing '..'", () => {
|
||||
const def: WorkflowDefinition = {
|
||||
version: 1,
|
||||
name: "test",
|
||||
params: { path: "safe" },
|
||||
steps: [
|
||||
{ id: "a", name: "A", prompt: "Read {{path}}", requires: [], produces: [] },
|
||||
],
|
||||
};
|
||||
assert.throws(
|
||||
() => substituteParams(def, { path: "../etc/passwd" }),
|
||||
(err: Error) => {
|
||||
assert.ok(err.message.includes(".."));
|
||||
assert.ok(err.message.includes("path traversal"));
|
||||
return true;
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
test("substituteParams: errors on unresolved placeholders", () => {
|
||||
const def: WorkflowDefinition = {
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [
|
||||
{ id: "a", name: "A", prompt: "Write about {{topic}}", requires: [], produces: [] },
|
||||
],
|
||||
};
|
||||
assert.throws(
|
||||
() => substituteParams(def),
|
||||
(err: Error) => {
|
||||
assert.ok(err.message.includes("Unresolved"));
|
||||
assert.ok(err.message.includes("topic"));
|
||||
return true;
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
test("substituteParams: does not mutate the original definition", () => {
|
||||
const def: WorkflowDefinition = {
|
||||
version: 1,
|
||||
name: "test",
|
||||
params: { topic: "AI" },
|
||||
steps: [
|
||||
{ id: "a", name: "A", prompt: "Write about {{topic}}", requires: [], produces: [] },
|
||||
],
|
||||
};
|
||||
const original = def.steps[0].prompt;
|
||||
substituteParams(def);
|
||||
assert.equal(def.steps[0].prompt, original, "Original definition should not be mutated");
|
||||
});
|
||||
|
||||
// ─── substitutePromptString ──────────────────────────────────────────────
|
||||
|
||||
test("substitutePromptString: replaces known placeholders, leaves unknown", () => {
|
||||
const result = substitutePromptString(
|
||||
"Hello {{name}}, write about {{topic}}",
|
||||
{ name: "Agent" },
|
||||
);
|
||||
assert.equal(result, "Hello Agent, write about {{topic}}");
|
||||
});
|
||||
|
||||
test("substitutePromptString: no placeholders → unchanged", () => {
|
||||
const result = substitutePromptString("No placeholders here", {});
|
||||
assert.equal(result, "No placeholders here");
|
||||
});
|
||||
|
||||
// ─── Edge cases ──────────────────────────────────────────────────────────
|
||||
|
||||
test("validateDefinition: steps is not an array → error", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: "not-an-array",
|
||||
});
|
||||
assert.equal(result.valid, false);
|
||||
assert.ok(result.errors.some((e) => e.includes("steps") && e.includes("array")));
|
||||
});
|
||||
|
||||
test("validateDefinition: valid minimal step (no requires/produces) → accepted", () => {
|
||||
const result = validateDefinition({
|
||||
version: 1,
|
||||
name: "test",
|
||||
steps: [{ id: "a", name: "A", prompt: "do A" }],
|
||||
});
|
||||
assert.equal(result.valid, true);
|
||||
assert.equal(result.errors.length, 0);
|
||||
});
|
||||
|
||||
test("loadDefinition: loads without params field → params is undefined", () => {
|
||||
const dir = writeDefYaml(`
|
||||
version: 1
|
||||
name: "no-params"
|
||||
steps:
|
||||
- id: a
|
||||
name: "A"
|
||||
prompt: "do A"
|
||||
`);
|
||||
try {
|
||||
const def = loadDefinition(dir, "test-workflow");
|
||||
assert.equal(def.params, undefined);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("loadDefinition: loads without description → description is undefined", () => {
|
||||
const dir = writeDefYaml(`
|
||||
version: 1
|
||||
name: "no-desc"
|
||||
steps:
|
||||
- id: a
|
||||
name: "A"
|
||||
prompt: "do A"
|
||||
`);
|
||||
try {
|
||||
const def = loadDefinition(dir, "test-workflow");
|
||||
assert.equal(def.description, undefined);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("loadDefinition: step with no requires/produces defaults to empty arrays", () => {
|
||||
const dir = writeDefYaml(`
|
||||
version: 1
|
||||
name: "defaults"
|
||||
steps:
|
||||
- id: a
|
||||
name: "A"
|
||||
prompt: "do A"
|
||||
`);
|
||||
try {
|
||||
const def = loadDefinition(dir, "test-workflow");
|
||||
assert.deepEqual(def.steps[0].requires, []);
|
||||
assert.deepEqual(def.steps[0].produces, []);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
318
src/resources/extensions/gsd/tests/dev-engine-wrapper.test.ts
Normal file
318
src/resources/extensions/gsd/tests/dev-engine-wrapper.test.ts
Normal file
|
|
@ -0,0 +1,318 @@
|
|||
/**
|
||||
* dev-engine-wrapper.test.ts — Contract tests for the dev engine wrapper layer (S02).
|
||||
*
|
||||
* Tests bridgeDispatchAction mapping, DevWorkflowEngine delegation,
|
||||
* DevExecutionPolicy stubs, resolver routing, kill switch, and
|
||||
* auto.ts engine ID accessors.
|
||||
*/
|
||||
|
||||
import test, { describe, before, after } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtempSync, mkdirSync, rmSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
|
||||
// ── bridgeDispatchAction mapping ────────────────────────────────────────────
|
||||
|
||||
describe("bridgeDispatchAction", () => {
|
||||
test("maps dispatch action with step fields", async () => {
|
||||
const { bridgeDispatchAction } = await import(
|
||||
"../dev-workflow-engine.ts"
|
||||
);
|
||||
const result = bridgeDispatchAction({
|
||||
action: "dispatch",
|
||||
unitType: "execute-task",
|
||||
unitId: "T01",
|
||||
prompt: "do stuff",
|
||||
matchedRule: "foo",
|
||||
} as any);
|
||||
|
||||
assert.equal(result.action, "dispatch");
|
||||
assert.ok("step" in result);
|
||||
const step = (result as any).step;
|
||||
assert.equal(step.unitType, "execute-task");
|
||||
assert.equal(step.unitId, "T01");
|
||||
assert.equal(step.prompt, "do stuff");
|
||||
});
|
||||
|
||||
test("maps stop action with reason and level", async () => {
|
||||
const { bridgeDispatchAction } = await import(
|
||||
"../dev-workflow-engine.ts"
|
||||
);
|
||||
const result = bridgeDispatchAction({
|
||||
action: "stop",
|
||||
reason: "done",
|
||||
level: "info",
|
||||
matchedRule: "bar",
|
||||
} as any);
|
||||
|
||||
assert.equal(result.action, "stop");
|
||||
assert.equal((result as any).reason, "done");
|
||||
assert.equal((result as any).level, "info");
|
||||
});
|
||||
|
||||
test("maps skip action", async () => {
|
||||
const { bridgeDispatchAction } = await import(
|
||||
"../dev-workflow-engine.ts"
|
||||
);
|
||||
const result = bridgeDispatchAction({
|
||||
action: "skip",
|
||||
matchedRule: "baz",
|
||||
} as any);
|
||||
|
||||
assert.equal(result.action, "skip");
|
||||
});
|
||||
});
|
||||
|
||||
// ── DevWorkflowEngine ───────────────────────────────────────────────────────
|
||||
|
||||
describe("DevWorkflowEngine", () => {
|
||||
test("engineId is 'dev'", async () => {
|
||||
const { DevWorkflowEngine } = await import("../dev-workflow-engine.ts");
|
||||
const engine = new DevWorkflowEngine();
|
||||
assert.equal(engine.engineId, "dev");
|
||||
});
|
||||
|
||||
test("deriveState returns EngineState with expected fields", async () => {
|
||||
const { DevWorkflowEngine } = await import("../dev-workflow-engine.ts");
|
||||
const engine = new DevWorkflowEngine();
|
||||
|
||||
// Create a minimal temp .gsd structure for deriveState
|
||||
const tempDir = mkdtempSync(join(tmpdir(), "gsd-engine-test-"));
|
||||
mkdirSync(join(tempDir, ".gsd", "milestones"), { recursive: true });
|
||||
|
||||
try {
|
||||
const state = await engine.deriveState(tempDir);
|
||||
|
||||
assert.equal(typeof state.phase, "string", "phase should be a string");
|
||||
assert.ok(
|
||||
"currentMilestoneId" in state,
|
||||
"state should have currentMilestoneId",
|
||||
);
|
||||
assert.ok(
|
||||
"activeSliceId" in state,
|
||||
"state should have activeSliceId",
|
||||
);
|
||||
assert.ok(
|
||||
"activeTaskId" in state,
|
||||
"state should have activeTaskId",
|
||||
);
|
||||
assert.equal(
|
||||
typeof state.isComplete,
|
||||
"boolean",
|
||||
"isComplete should be boolean",
|
||||
);
|
||||
assert.ok("raw" in state, "state should have raw field");
|
||||
} finally {
|
||||
rmSync(tempDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("reconcile returns continue for non-complete state", async () => {
|
||||
const { DevWorkflowEngine } = await import("../dev-workflow-engine.ts");
|
||||
const engine = new DevWorkflowEngine();
|
||||
|
||||
const state = {
|
||||
phase: "executing",
|
||||
currentMilestoneId: "M001",
|
||||
activeSliceId: "S01",
|
||||
activeTaskId: "T01",
|
||||
isComplete: false,
|
||||
raw: {},
|
||||
};
|
||||
|
||||
const result = await engine.reconcile(state, {
|
||||
unitType: "execute-task",
|
||||
unitId: "T01",
|
||||
startedAt: Date.now() - 1000,
|
||||
finishedAt: Date.now(),
|
||||
});
|
||||
|
||||
assert.equal(result.outcome, "continue");
|
||||
});
|
||||
|
||||
test("reconcile returns milestone-complete for complete state", async () => {
|
||||
const { DevWorkflowEngine } = await import("../dev-workflow-engine.ts");
|
||||
const engine = new DevWorkflowEngine();
|
||||
|
||||
const state = {
|
||||
phase: "complete",
|
||||
currentMilestoneId: "M001",
|
||||
activeSliceId: null,
|
||||
activeTaskId: null,
|
||||
isComplete: true,
|
||||
raw: {},
|
||||
};
|
||||
|
||||
const result = await engine.reconcile(state, {
|
||||
unitType: "execute-task",
|
||||
unitId: "T01",
|
||||
startedAt: Date.now() - 1000,
|
||||
finishedAt: Date.now(),
|
||||
});
|
||||
|
||||
assert.equal(result.outcome, "milestone-complete");
|
||||
});
|
||||
|
||||
test("getDisplayMetadata returns expected fields", async () => {
|
||||
const { DevWorkflowEngine } = await import("../dev-workflow-engine.ts");
|
||||
const engine = new DevWorkflowEngine();
|
||||
|
||||
const state = {
|
||||
phase: "executing",
|
||||
currentMilestoneId: "M001",
|
||||
activeSliceId: "S01",
|
||||
activeTaskId: "T01",
|
||||
isComplete: false,
|
||||
raw: {},
|
||||
};
|
||||
|
||||
const meta = engine.getDisplayMetadata(state);
|
||||
|
||||
assert.ok("engineLabel" in meta, "should have engineLabel");
|
||||
assert.ok("currentPhase" in meta, "should have currentPhase");
|
||||
assert.ok("progressSummary" in meta, "should have progressSummary");
|
||||
assert.ok("stepCount" in meta, "should have stepCount");
|
||||
assert.equal(meta.engineLabel, "GSD Dev");
|
||||
});
|
||||
});
|
||||
|
||||
// ── DevExecutionPolicy stubs ────────────────────────────────────────────────
|
||||
|
||||
describe("DevExecutionPolicy", () => {
|
||||
test("verify returns 'continue'", async () => {
|
||||
const { DevExecutionPolicy } = await import(
|
||||
"../dev-execution-policy.ts"
|
||||
);
|
||||
const policy = new DevExecutionPolicy();
|
||||
const result = await policy.verify("execute-task", "T01", {
|
||||
basePath: "/tmp",
|
||||
});
|
||||
assert.equal(result, "continue");
|
||||
});
|
||||
|
||||
test("selectModel returns null", async () => {
|
||||
const { DevExecutionPolicy } = await import(
|
||||
"../dev-execution-policy.ts"
|
||||
);
|
||||
const policy = new DevExecutionPolicy();
|
||||
const result = await policy.selectModel("execute-task", "T01", {
|
||||
basePath: "/tmp",
|
||||
});
|
||||
assert.equal(result, null);
|
||||
});
|
||||
|
||||
test("recover returns { outcome: 'retry' }", async () => {
|
||||
const { DevExecutionPolicy } = await import(
|
||||
"../dev-execution-policy.ts"
|
||||
);
|
||||
const policy = new DevExecutionPolicy();
|
||||
const result = await policy.recover("execute-task", "T01", {
|
||||
basePath: "/tmp",
|
||||
});
|
||||
assert.deepEqual(result, { outcome: "retry" });
|
||||
});
|
||||
|
||||
test("closeout returns { committed: false, artifacts: [] }", async () => {
|
||||
const { DevExecutionPolicy } = await import(
|
||||
"../dev-execution-policy.ts"
|
||||
);
|
||||
const policy = new DevExecutionPolicy();
|
||||
const result = await policy.closeout("execute-task", "T01", {
|
||||
basePath: "/tmp",
|
||||
startedAt: Date.now(),
|
||||
});
|
||||
assert.deepEqual(result, { committed: false, artifacts: [] });
|
||||
});
|
||||
|
||||
test("prepareWorkspace resolves without error", async () => {
|
||||
const { DevExecutionPolicy } = await import(
|
||||
"../dev-execution-policy.ts"
|
||||
);
|
||||
const policy = new DevExecutionPolicy();
|
||||
await assert.doesNotReject(
|
||||
() => policy.prepareWorkspace("/tmp", "M001"),
|
||||
"prepareWorkspace should resolve without error",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ── Resolver routing ────────────────────────────────────────────────────────
|
||||
|
||||
describe("Resolver routing", () => {
|
||||
test("resolveEngine returns dev engine for null activeEngineId", async () => {
|
||||
const { resolveEngine } = await import("../engine-resolver.ts");
|
||||
const result = resolveEngine({ activeEngineId: null });
|
||||
assert.ok(result.engine, "should return engine");
|
||||
assert.ok(result.policy, "should return policy");
|
||||
assert.equal(result.engine.engineId, "dev");
|
||||
});
|
||||
|
||||
test("resolveEngine returns dev engine for 'dev' activeEngineId", async () => {
|
||||
const { resolveEngine } = await import("../engine-resolver.ts");
|
||||
const result = resolveEngine({ activeEngineId: "dev" });
|
||||
assert.ok(result.engine, "should return engine");
|
||||
assert.ok(result.policy, "should return policy");
|
||||
assert.equal(result.engine.engineId, "dev");
|
||||
});
|
||||
|
||||
test("resolveEngine throws for unknown activeEngineId without activeRunDir", async () => {
|
||||
const { resolveEngine } = await import("../engine-resolver.ts");
|
||||
assert.throws(
|
||||
() => resolveEngine({ activeEngineId: "unknown" }),
|
||||
/requires activeRunDir/,
|
||||
"should throw when activeRunDir is missing for non-dev engine",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ── Kill switch ─────────────────────────────────────────────────────────────
|
||||
|
||||
describe("Kill switch (GSD_ENGINE_BYPASS)", () => {
|
||||
const originalBypass = process.env.GSD_ENGINE_BYPASS;
|
||||
|
||||
after(() => {
|
||||
// Restore original env var state
|
||||
if (originalBypass === undefined) {
|
||||
delete process.env.GSD_ENGINE_BYPASS;
|
||||
} else {
|
||||
process.env.GSD_ENGINE_BYPASS = originalBypass;
|
||||
}
|
||||
});
|
||||
|
||||
test("GSD_ENGINE_BYPASS=1 does not affect resolveEngine (bypass checked in autoLoop)", async () => {
|
||||
const { resolveEngine } = await import("../engine-resolver.ts");
|
||||
process.env.GSD_ENGINE_BYPASS = "1";
|
||||
try {
|
||||
// resolveEngine should still resolve normally — bypass is checked in autoLoop
|
||||
const { engine } = resolveEngine({ activeEngineId: null });
|
||||
assert.ok(engine, "should return an engine even with bypass set");
|
||||
} finally {
|
||||
delete process.env.GSD_ENGINE_BYPASS;
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ── auto.ts engine ID accessors ─────────────────────────────────────────────
|
||||
|
||||
describe("auto.ts engine ID accessors", () => {
|
||||
test("setActiveEngineId / getActiveEngineId round-trip", async () => {
|
||||
const { setActiveEngineId, getActiveEngineId } = await import(
|
||||
"../auto.ts"
|
||||
);
|
||||
|
||||
setActiveEngineId("dev");
|
||||
assert.equal(
|
||||
getActiveEngineId(),
|
||||
"dev",
|
||||
"getActiveEngineId should return 'dev' after setting",
|
||||
);
|
||||
|
||||
setActiveEngineId(null);
|
||||
assert.equal(
|
||||
getActiveEngineId(),
|
||||
null,
|
||||
"getActiveEngineId should return null after setting null",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,476 @@
|
|||
/**
|
||||
* e2e-workflow-pipeline-integration.test.ts — End-to-end integration test
|
||||
* proving the assembled workflow engine pipeline works.
|
||||
*
|
||||
* Exercises every engine feature in a single multi-step workflow:
|
||||
* - Dependency-ordered dispatch
|
||||
* - Parameter substitution ({{target}})
|
||||
* - Content-heuristic verification (minSize)
|
||||
* - Shell-command verification (test -f)
|
||||
* - Context injection via context_from
|
||||
* - Iterate/fan-out expansion
|
||||
* - Dashboard metadata (step N/M)
|
||||
* - Completion detection (isComplete: true)
|
||||
*
|
||||
* Operates at the engine level (CustomWorkflowEngine + CustomExecutionPolicy
|
||||
* + real temp directories) — NOT through autoLoop() — to avoid the
|
||||
* timing-dependent resolveAgentEnd pattern that causes flakiness.
|
||||
*
|
||||
* Follows the pattern from iterate-engine-integration.test.ts:
|
||||
* real temp dirs via mkdtempSync, dispatch()/reconcile() helpers, afterEach cleanup.
|
||||
*/
|
||||
|
||||
import { describe, it, afterEach } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import {
|
||||
mkdtempSync,
|
||||
rmSync,
|
||||
writeFileSync,
|
||||
mkdirSync,
|
||||
readFileSync,
|
||||
existsSync,
|
||||
} from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import { stringify, parse } from "yaml";
|
||||
|
||||
import { CustomWorkflowEngine } from "../custom-workflow-engine.ts";
|
||||
import { CustomExecutionPolicy } from "../custom-execution-policy.ts";
|
||||
import { createRun, listRuns } from "../run-manager.ts";
|
||||
import { readGraph, writeGraph } from "../graph.ts";
|
||||
import { validateDefinition } from "../definition-loader.ts";
|
||||
|
||||
// ─── Helpers ─────────────────────────────────────────────────────────────
|
||||
|
||||
const tmpDirs: string[] = [];
|
||||
|
||||
function makeTmpDir(): string {
|
||||
const dir = mkdtempSync(join(tmpdir(), "e2e-pipeline-"));
|
||||
tmpDirs.push(dir);
|
||||
return dir;
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
for (const d of tmpDirs) {
|
||||
rmSync(d, { recursive: true, force: true });
|
||||
}
|
||||
tmpDirs.length = 0;
|
||||
});
|
||||
|
||||
/** Drive deriveState → resolveDispatch. */
|
||||
async function dispatch(engine: CustomWorkflowEngine) {
|
||||
const state = await engine.deriveState("/unused");
|
||||
return { state, result: engine.resolveDispatch(state, { basePath: "/unused" }) };
|
||||
}
|
||||
|
||||
/** Drive deriveState → reconcile for a given unitId. */
|
||||
async function reconcile(engine: CustomWorkflowEngine, unitId: string) {
|
||||
const state = await engine.deriveState("/unused");
|
||||
return engine.reconcile(state, {
|
||||
unitType: "custom-step",
|
||||
unitId,
|
||||
startedAt: Date.now() - 1000,
|
||||
finishedAt: Date.now(),
|
||||
});
|
||||
}
|
||||
|
||||
// ─── The multi-feature YAML definition (snake_case for loadDefinition) ───
|
||||
|
||||
/**
|
||||
* 4-step workflow definition exercising every engine feature:
|
||||
*
|
||||
* gather → scan (iterate) → analyze (context_from scan) → report (context_from analyze)
|
||||
*
|
||||
* Note: The scan step prompt uses a literal string instead of {{item}} in the
|
||||
* definition YAML because substituteParams() checks for unresolved {{key}}
|
||||
* placeholders. After createRun, we patch GRAPH.yaml to add the {{item}}
|
||||
* placeholder so iterate expansion produces item-specific prompts.
|
||||
*/
|
||||
const E2E_DEFINITION_YAML = `
|
||||
version: 1
|
||||
name: e2e-pipeline
|
||||
description: End-to-end integration test workflow
|
||||
params:
|
||||
target: default-target
|
||||
steps:
|
||||
- id: gather
|
||||
name: Gather Information
|
||||
prompt: "Gather information about {{target}} and produce a bullet list of findings"
|
||||
requires: []
|
||||
produces:
|
||||
- output/gather-results.md
|
||||
verify:
|
||||
policy: content-heuristic
|
||||
minSize: 10
|
||||
- id: scan
|
||||
name: Scan Items
|
||||
prompt: "Scan item: ITEM_PLACEHOLDER"
|
||||
requires:
|
||||
- gather
|
||||
produces:
|
||||
- output/scan-result.txt
|
||||
verify:
|
||||
policy: shell-command
|
||||
command: "test -f output/scan-result.txt"
|
||||
iterate:
|
||||
source: output/gather-results.md
|
||||
pattern: "^- (.+)$"
|
||||
- id: analyze
|
||||
name: Analyze Results
|
||||
prompt: "Analyze all scan results and produce a summary"
|
||||
requires:
|
||||
- scan
|
||||
produces:
|
||||
- output/analysis.md
|
||||
context_from:
|
||||
- scan
|
||||
verify:
|
||||
policy: content-heuristic
|
||||
minSize: 5
|
||||
- id: report
|
||||
name: Final Report
|
||||
prompt: "Write final report for {{target}}"
|
||||
requires:
|
||||
- analyze
|
||||
produces:
|
||||
- output/report.md
|
||||
context_from:
|
||||
- analyze
|
||||
`;
|
||||
|
||||
/**
|
||||
* Create a temp project directory with the e2e-pipeline definition YAML,
|
||||
* call createRun with param overrides, and patch GRAPH.yaml so the scan
|
||||
* step's prompt contains {{item}} for iterate expansion.
|
||||
*/
|
||||
function setupProject(overrides?: Record<string, string>): {
|
||||
basePath: string;
|
||||
runDir: string;
|
||||
} {
|
||||
const basePath = makeTmpDir();
|
||||
const defsDir = join(basePath, ".gsd", "workflow-defs");
|
||||
mkdirSync(defsDir, { recursive: true });
|
||||
writeFileSync(join(defsDir, "e2e-pipeline.yaml"), E2E_DEFINITION_YAML, "utf-8");
|
||||
|
||||
const runDir = createRun(basePath, "e2e-pipeline", overrides);
|
||||
|
||||
// Patch GRAPH.yaml: replace the scan step's placeholder with {{item}}
|
||||
// so iterate expansion produces item-specific prompts. This works around
|
||||
// substituteParams() rejecting unresolved {{item}} in the definition.
|
||||
const graph = readGraph(runDir);
|
||||
const scanStep = graph.steps.find((s) => s.id === "scan");
|
||||
if (scanStep) {
|
||||
scanStep.prompt = "Scan item: {{item}}";
|
||||
writeGraph(runDir, graph);
|
||||
}
|
||||
|
||||
return { basePath, runDir };
|
||||
}
|
||||
|
||||
// ─── Tests ───────────────────────────────────────────────────────────────
|
||||
|
||||
describe("e2e-workflow-pipeline", () => {
|
||||
it("drives the full engine pipeline: create → dispatch → verify → complete", async () => {
|
||||
// ── 1. Create run with param overrides ────────────────────────────
|
||||
const { basePath, runDir } = setupProject({ target: "my-project" });
|
||||
|
||||
// Verify run directory structure
|
||||
assert.ok(existsSync(join(runDir, "DEFINITION.yaml")), "DEFINITION.yaml should exist");
|
||||
assert.ok(existsSync(join(runDir, "GRAPH.yaml")), "GRAPH.yaml should exist");
|
||||
assert.ok(existsSync(join(runDir, "PARAMS.json")), "PARAMS.json should exist");
|
||||
|
||||
// Verify PARAMS.json has the override
|
||||
const params = JSON.parse(readFileSync(join(runDir, "PARAMS.json"), "utf-8"));
|
||||
assert.deepStrictEqual(params, { target: "my-project" });
|
||||
|
||||
// Verify the frozen DEFINITION.yaml has substituted params in non-iterate steps
|
||||
const frozenDef = readFileSync(join(runDir, "DEFINITION.yaml"), "utf-8");
|
||||
assert.ok(
|
||||
frozenDef.includes("my-project"),
|
||||
"Frozen definition should have substituted 'my-project' for {{target}}",
|
||||
);
|
||||
|
||||
// Instantiate engine and policy
|
||||
const engine = new CustomWorkflowEngine(runDir);
|
||||
const policy = new CustomExecutionPolicy(runDir);
|
||||
|
||||
// Verify initial graph has 4 steps all pending
|
||||
const initialGraph = readGraph(runDir);
|
||||
assert.equal(initialGraph.steps.length, 4, "Initial graph should have 4 steps");
|
||||
assert.ok(
|
||||
initialGraph.steps.every((s) => s.status === "pending"),
|
||||
"All steps should start as pending",
|
||||
);
|
||||
|
||||
// Verify initial state is not complete
|
||||
let state = await engine.deriveState("/unused");
|
||||
assert.equal(state.isComplete, false, "Workflow should not be complete initially");
|
||||
|
||||
// Dashboard metadata: 0/4 initially
|
||||
let meta = engine.getDisplayMetadata(state);
|
||||
assert.equal(meta.stepCount!.completed, 0);
|
||||
assert.equal(meta.stepCount!.total, 4);
|
||||
assert.equal(meta.progressSummary, "Step 0/4");
|
||||
|
||||
// ── 2. Step 1: gather ─────────────────────────────────────────────
|
||||
const { result: r1 } = await dispatch(engine);
|
||||
const d1 = await r1;
|
||||
assert.equal(d1.action, "dispatch", "Should dispatch gather step");
|
||||
if (d1.action !== "dispatch") throw new Error("unreachable");
|
||||
|
||||
assert.equal(d1.step.unitId, "e2e-pipeline/gather");
|
||||
assert.ok(
|
||||
d1.step.prompt.includes("my-project"),
|
||||
`Gather prompt should contain substituted param "my-project", got: "${d1.step.prompt}"`,
|
||||
);
|
||||
assert.ok(
|
||||
!d1.step.prompt.includes("default-target"),
|
||||
"Gather prompt should NOT contain default param value",
|
||||
);
|
||||
|
||||
// Simulate agent work: write the gather artifact with bullet items for iterate
|
||||
const outputDir = join(runDir, "output");
|
||||
mkdirSync(outputDir, { recursive: true });
|
||||
writeFileSync(
|
||||
join(runDir, "output/gather-results.md"),
|
||||
"# Findings for my-project\n\n- security-audit\n- performance-review\n- code-quality\n",
|
||||
"utf-8",
|
||||
);
|
||||
|
||||
// Reconcile gather
|
||||
await reconcile(engine, "e2e-pipeline/gather");
|
||||
|
||||
// Verify gather: content-heuristic (minSize: 10) should pass
|
||||
const gatherVerify = await policy.verify("custom-step", "e2e-pipeline/gather", {
|
||||
basePath: "/unused",
|
||||
});
|
||||
assert.equal(
|
||||
gatherVerify,
|
||||
"continue",
|
||||
"Gather verification (content-heuristic) should pass",
|
||||
);
|
||||
|
||||
// Dashboard after gather: 1 completed (gather), total still 4
|
||||
state = await engine.deriveState("/unused");
|
||||
meta = engine.getDisplayMetadata(state);
|
||||
assert.equal(meta.stepCount!.completed, 1);
|
||||
assert.equal(meta.progressSummary, "Step 1/4");
|
||||
assert.equal(state.isComplete, false);
|
||||
|
||||
// ── 3. Step 2: scan with iterate ──────────────────────────────────
|
||||
// Dispatch should trigger iterate expansion from gather-results.md
|
||||
const { result: r2 } = await dispatch(engine);
|
||||
const d2 = await r2;
|
||||
assert.equal(d2.action, "dispatch", "Should dispatch first scan instance");
|
||||
if (d2.action !== "dispatch") throw new Error("unreachable");
|
||||
|
||||
// First instance should be scan--001 for "security-audit"
|
||||
assert.equal(d2.step.unitId, "e2e-pipeline/scan--001");
|
||||
assert.ok(
|
||||
d2.step.prompt.includes("security-audit"),
|
||||
`First scan instance prompt should contain "security-audit", got: "${d2.step.prompt}"`,
|
||||
);
|
||||
|
||||
// Verify graph expanded: parent "scan" is "expanded", 3 instances exist
|
||||
let graph = readGraph(runDir);
|
||||
const scanParent = graph.steps.find((s) => s.id === "scan");
|
||||
assert.ok(scanParent, "Parent scan step should exist");
|
||||
assert.equal(scanParent.status, "expanded", "Parent scan should be expanded");
|
||||
|
||||
const scanInstances = graph.steps.filter((s) => s.parentStepId === "scan");
|
||||
assert.equal(scanInstances.length, 3, "Should have 3 scan instances");
|
||||
assert.equal(scanInstances[0].id, "scan--001");
|
||||
assert.equal(scanInstances[1].id, "scan--002");
|
||||
assert.equal(scanInstances[2].id, "scan--003");
|
||||
|
||||
// Verify iterate prompts contain item-specific content
|
||||
assert.ok(scanInstances[0].prompt.includes("security-audit"));
|
||||
assert.ok(scanInstances[1].prompt.includes("performance-review"));
|
||||
assert.ok(scanInstances[2].prompt.includes("code-quality"));
|
||||
|
||||
// Verify dependency rewriting: analyze should now depend on scan--001, scan--002, scan--003
|
||||
const analyzeStep = graph.steps.find((s) => s.id === "analyze");
|
||||
assert.ok(analyzeStep);
|
||||
assert.deepStrictEqual(
|
||||
analyzeStep.dependsOn.sort(),
|
||||
["scan--001", "scan--002", "scan--003"],
|
||||
"Analyze should depend on all scan instances after expansion",
|
||||
);
|
||||
|
||||
// Graph step count increased: 4 original + 3 instances = 7 (parent stays as "expanded")
|
||||
assert.equal(graph.steps.length, 7, "Graph should have 7 steps after expansion");
|
||||
|
||||
// Dashboard after expansion: total now includes instance steps
|
||||
state = await engine.deriveState("/unused");
|
||||
meta = engine.getDisplayMetadata(state);
|
||||
// completed: gather(1), expanded steps don't count as "complete" in getDisplayMetadata
|
||||
assert.equal(meta.stepCount!.completed, 1, "Only gather should be complete");
|
||||
|
||||
// Write scan artifact (same path for all instances since the verify command checks run-dir-relative path)
|
||||
writeFileSync(join(runDir, "output/scan-result.txt"), "scan output data", "utf-8");
|
||||
|
||||
// Complete scan--001, dispatch scan--002
|
||||
await reconcile(engine, "e2e-pipeline/scan--001");
|
||||
|
||||
// Verify analyze is still blocked (not all scan instances complete)
|
||||
const { result: r3a } = await dispatch(engine);
|
||||
const d3a = await r3a;
|
||||
assert.equal(d3a.action, "dispatch");
|
||||
if (d3a.action !== "dispatch") throw new Error("unreachable");
|
||||
assert.equal(
|
||||
d3a.step.unitId,
|
||||
"e2e-pipeline/scan--002",
|
||||
"Should dispatch scan--002 (analyze still blocked)",
|
||||
);
|
||||
assert.ok(d3a.step.prompt.includes("performance-review"));
|
||||
|
||||
// Complete scan--002, dispatch scan--003
|
||||
await reconcile(engine, "e2e-pipeline/scan--002");
|
||||
const { result: r3b } = await dispatch(engine);
|
||||
const d3b = await r3b;
|
||||
assert.equal(d3b.action, "dispatch");
|
||||
if (d3b.action !== "dispatch") throw new Error("unreachable");
|
||||
assert.equal(d3b.step.unitId, "e2e-pipeline/scan--003");
|
||||
assert.ok(d3b.step.prompt.includes("code-quality"));
|
||||
|
||||
// Complete scan--003 — now analyze should be unblocked
|
||||
await reconcile(engine, "e2e-pipeline/scan--003");
|
||||
|
||||
// Dashboard after all scan instances: 4 complete (gather + 3 instances)
|
||||
state = await engine.deriveState("/unused");
|
||||
meta = engine.getDisplayMetadata(state);
|
||||
assert.equal(meta.stepCount!.completed, 4, "gather + 3 scan instances should be complete");
|
||||
assert.equal(state.isComplete, false);
|
||||
|
||||
// ── 4. Step 3: analyze (with context_from scan) ───────────────────
|
||||
const { result: r4 } = await dispatch(engine);
|
||||
const d4 = await r4;
|
||||
assert.equal(d4.action, "dispatch", "Should dispatch analyze step");
|
||||
if (d4.action !== "dispatch") throw new Error("unreachable");
|
||||
|
||||
assert.equal(d4.step.unitId, "e2e-pipeline/analyze");
|
||||
|
||||
// Context injection: the analyze prompt should include content from scan's produces
|
||||
// scan produces output/scan-result.txt and context_from references "scan"
|
||||
assert.ok(
|
||||
d4.step.prompt.includes("scan output data"),
|
||||
`Analyze prompt should include injected context from scan artifact, got: "${d4.step.prompt.slice(0, 200)}"`,
|
||||
);
|
||||
assert.ok(
|
||||
d4.step.prompt.includes("Analyze all scan results"),
|
||||
"Analyze prompt should still contain the original prompt text",
|
||||
);
|
||||
|
||||
// Write analyze artifact
|
||||
writeFileSync(
|
||||
join(runDir, "output/analysis.md"),
|
||||
"# Analysis Summary\n\nAll scans completed successfully with findings.\n",
|
||||
"utf-8",
|
||||
);
|
||||
|
||||
await reconcile(engine, "e2e-pipeline/analyze");
|
||||
|
||||
// Verify analyze: content-heuristic (minSize: 5) should pass
|
||||
const analyzeVerify = await policy.verify("custom-step", "e2e-pipeline/analyze", {
|
||||
basePath: "/unused",
|
||||
});
|
||||
assert.equal(
|
||||
analyzeVerify,
|
||||
"continue",
|
||||
"Analyze verification (content-heuristic) should pass",
|
||||
);
|
||||
|
||||
// Dashboard after analyze: 5 complete
|
||||
state = await engine.deriveState("/unused");
|
||||
meta = engine.getDisplayMetadata(state);
|
||||
assert.equal(meta.stepCount!.completed, 5);
|
||||
assert.equal(state.isComplete, false, "Should not be complete yet (report remaining)");
|
||||
|
||||
// ── 5. Step 4: report (with context_from analyze + param) ─────────
|
||||
const { result: r5 } = await dispatch(engine);
|
||||
const d5 = await r5;
|
||||
assert.equal(d5.action, "dispatch", "Should dispatch report step");
|
||||
if (d5.action !== "dispatch") throw new Error("unreachable");
|
||||
|
||||
assert.equal(d5.step.unitId, "e2e-pipeline/report");
|
||||
|
||||
// Context injection: report prompt should include content from analyze's produces
|
||||
assert.ok(
|
||||
d5.step.prompt.includes("Analysis Summary"),
|
||||
`Report prompt should include injected context from analyze artifact, got: "${d5.step.prompt.slice(0, 200)}"`,
|
||||
);
|
||||
|
||||
// Parameter substitution: report prompt should contain "my-project"
|
||||
assert.ok(
|
||||
d5.step.prompt.includes("my-project"),
|
||||
`Report prompt should contain substituted param "my-project", got: "${d5.step.prompt}"`,
|
||||
);
|
||||
|
||||
// Write report artifact
|
||||
writeFileSync(
|
||||
join(runDir, "output/report.md"),
|
||||
"# Final Report for my-project\n\nComprehensive findings documented.\n",
|
||||
"utf-8",
|
||||
);
|
||||
|
||||
await reconcile(engine, "e2e-pipeline/report");
|
||||
|
||||
// ── 6. Completion ─────────────────────────────────────────────────
|
||||
state = await engine.deriveState("/unused");
|
||||
assert.equal(state.isComplete, true, "Workflow should be complete after all steps");
|
||||
assert.equal(state.phase, "complete");
|
||||
|
||||
// Dashboard: all steps complete
|
||||
meta = engine.getDisplayMetadata(state);
|
||||
assert.equal(meta.stepCount!.completed, 6, "All 6 dispatchable steps should be complete");
|
||||
assert.equal(meta.currentPhase, "complete");
|
||||
|
||||
// Dispatch should return stop
|
||||
const { result: rFinal } = await dispatch(engine);
|
||||
const dFinal = await rFinal;
|
||||
assert.equal(dFinal.action, "stop");
|
||||
if (dFinal.action === "stop") {
|
||||
assert.equal(dFinal.reason, "All steps complete");
|
||||
}
|
||||
|
||||
// Verify shell-command policy works on the scan step (parent, not instance)
|
||||
const shellVerify = await policy.verify("custom-step", "e2e-pipeline/scan", {
|
||||
basePath: "/unused",
|
||||
});
|
||||
assert.equal(
|
||||
shellVerify,
|
||||
"continue",
|
||||
"Shell-command verification (test -f output/scan-result.txt) should pass",
|
||||
);
|
||||
});
|
||||
|
||||
describe("createRun + listRuns integration", () => {
|
||||
it("created run appears in listRuns with correct metadata", () => {
|
||||
const { basePath, runDir } = setupProject({ target: "list-test" });
|
||||
|
||||
const runs = listRuns(basePath, "e2e-pipeline");
|
||||
assert.ok(runs.length >= 1, "Should list at least one run");
|
||||
|
||||
const thisRun = runs.find((r) => r.runDir === runDir);
|
||||
assert.ok(thisRun, "Created run should appear in listRuns");
|
||||
assert.equal(thisRun.name, "e2e-pipeline");
|
||||
assert.equal(thisRun.status, "pending", "New run should have pending status");
|
||||
assert.equal(thisRun.steps.total, 4, "Should have 4 steps");
|
||||
assert.equal(thisRun.steps.completed, 0);
|
||||
assert.equal(thisRun.steps.pending, 4);
|
||||
});
|
||||
});
|
||||
|
||||
describe("validateDefinition accepts the e2e definition", () => {
|
||||
it("validates the e2e-pipeline YAML as valid V1 schema", () => {
|
||||
const parsed = parse(E2E_DEFINITION_YAML);
|
||||
const { valid, errors } = validateDefinition(parsed);
|
||||
assert.equal(
|
||||
valid,
|
||||
true,
|
||||
`Definition should be valid but got errors: ${errors.join(", ")}`,
|
||||
);
|
||||
assert.deepStrictEqual(errors, []);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,271 @@
|
|||
/**
|
||||
* engine-interfaces-contract.test.ts — Source-level contract tests for the
|
||||
* engine abstraction layer (S01).
|
||||
*
|
||||
* TypeScript interfaces are erased by --experimental-strip-types, so these
|
||||
* tests use source-level regex assertions on the .ts files to verify shapes.
|
||||
* Runtime assertions cover AutoSession.activeEngineId and resolveEngine().
|
||||
*
|
||||
* Follows the same conventions as auto-session-encapsulation.test.ts.
|
||||
*/
|
||||
|
||||
import test, { describe } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { join, dirname } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const ENGINE_TYPES_PATH = join(__dirname, "..", "engine-types.ts");
|
||||
const WORKFLOW_ENGINE_PATH = join(__dirname, "..", "workflow-engine.ts");
|
||||
const EXECUTION_POLICY_PATH = join(__dirname, "..", "execution-policy.ts");
|
||||
const ENGINE_RESOLVER_PATH = join(__dirname, "..", "engine-resolver.ts");
|
||||
|
||||
function readSource(path: string): string {
|
||||
return readFileSync(path, "utf-8");
|
||||
}
|
||||
|
||||
// ── Import smoke tests ──────────────────────────────────────────────────────
|
||||
|
||||
describe("Import smoke tests", () => {
|
||||
test("engine-types.ts can be dynamically imported", async () => {
|
||||
const mod = await import("../engine-types.ts");
|
||||
assert.ok(mod, "engine-types.ts should import without error");
|
||||
});
|
||||
|
||||
test("workflow-engine.ts can be dynamically imported", async () => {
|
||||
const mod = await import("../workflow-engine.ts");
|
||||
assert.ok(mod, "workflow-engine.ts should import without error");
|
||||
});
|
||||
|
||||
test("execution-policy.ts can be dynamically imported", async () => {
|
||||
const mod = await import("../execution-policy.ts");
|
||||
assert.ok(mod, "execution-policy.ts should import without error");
|
||||
});
|
||||
|
||||
test("engine-resolver.ts can be dynamically imported", async () => {
|
||||
const mod = await import("../engine-resolver.ts");
|
||||
assert.ok(mod, "engine-resolver.ts should import without error");
|
||||
assert.ok(
|
||||
typeof mod.resolveEngine === "function",
|
||||
"engine-resolver.ts should export resolveEngine function",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ── Leaf-node constraint ────────────────────────────────────────────────────
|
||||
|
||||
describe("Leaf-node constraint", () => {
|
||||
test("engine-types.ts has zero imports from GSD modules (only node: allowed)", () => {
|
||||
const source = readSource(ENGINE_TYPES_PATH);
|
||||
const lines = source.split("\n");
|
||||
const violations: string[] = [];
|
||||
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
const line = lines[i]!;
|
||||
// Match import lines that reference relative paths (../ or ./)
|
||||
if (/^import\s/.test(line) && /['"]\.\.?\// .test(line)) {
|
||||
violations.push(`line ${i + 1}: ${line.trim()}`);
|
||||
}
|
||||
}
|
||||
|
||||
assert.equal(
|
||||
violations.length,
|
||||
0,
|
||||
`engine-types.ts must be a leaf node with zero GSD imports. ` +
|
||||
`Only node: imports are allowed.\nViolations:\n${violations.join("\n")}`,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ── EngineState shape ───────────────────────────────────────────────────────
|
||||
|
||||
describe("EngineState shape", () => {
|
||||
test("EngineState has all required fields with correct types", () => {
|
||||
const source = readSource(ENGINE_TYPES_PATH);
|
||||
|
||||
const requiredFields = [
|
||||
"phase",
|
||||
"currentMilestoneId",
|
||||
"activeSliceId",
|
||||
"activeTaskId",
|
||||
"isComplete",
|
||||
"raw",
|
||||
];
|
||||
|
||||
for (const field of requiredFields) {
|
||||
assert.ok(
|
||||
source.includes(field),
|
||||
`EngineState must contain field: ${field}`,
|
||||
);
|
||||
}
|
||||
|
||||
// raw must be typed unknown — not a GSD-specific type
|
||||
assert.ok(
|
||||
/raw:\s*unknown/.test(source),
|
||||
"EngineState.raw must be typed 'unknown', not a GSD-specific type",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ── EngineDispatchAction shape ──────────────────────────────────────────────
|
||||
|
||||
describe("EngineDispatchAction shape", () => {
|
||||
test("EngineDispatchAction has dispatch, stop, and skip variants", () => {
|
||||
const source = readSource(ENGINE_TYPES_PATH);
|
||||
|
||||
assert.ok(
|
||||
/action:\s*"dispatch"/.test(source),
|
||||
'EngineDispatchAction must have action: "dispatch" variant',
|
||||
);
|
||||
assert.ok(
|
||||
/action:\s*"stop"/.test(source),
|
||||
'EngineDispatchAction must have action: "stop" variant',
|
||||
);
|
||||
assert.ok(
|
||||
/action:\s*"skip"/.test(source),
|
||||
'EngineDispatchAction must have action: "skip" variant',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ── WorkflowEngine interface shape ──────────────────────────────────────────
|
||||
|
||||
describe("WorkflowEngine interface shape", () => {
|
||||
test("WorkflowEngine has engineId and all required methods", () => {
|
||||
const source = readSource(WORKFLOW_ENGINE_PATH);
|
||||
|
||||
const requiredMembers = [
|
||||
"engineId",
|
||||
"deriveState",
|
||||
"resolveDispatch",
|
||||
"reconcile",
|
||||
"getDisplayMetadata",
|
||||
];
|
||||
|
||||
for (const member of requiredMembers) {
|
||||
assert.ok(
|
||||
source.includes(member),
|
||||
`WorkflowEngine must contain member: ${member}`,
|
||||
);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ── ExecutionPolicy interface shape ─────────────────────────────────────────
|
||||
|
||||
describe("ExecutionPolicy interface shape", () => {
|
||||
test("ExecutionPolicy has all required methods", () => {
|
||||
const source = readSource(EXECUTION_POLICY_PATH);
|
||||
|
||||
const requiredMethods = [
|
||||
"prepareWorkspace",
|
||||
"selectModel",
|
||||
"verify",
|
||||
"recover",
|
||||
"closeout",
|
||||
];
|
||||
|
||||
for (const method of requiredMethods) {
|
||||
assert.ok(
|
||||
source.includes(method),
|
||||
`ExecutionPolicy must contain method: ${method}`,
|
||||
);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ── Resolver stub behavior ──────────────────────────────────────────────────
|
||||
|
||||
describe("Resolver stub behavior", () => {
|
||||
test("resolveEngine returns dev engine for null activeEngineId", async () => {
|
||||
const { resolveEngine } = await import("../engine-resolver.ts");
|
||||
const result = resolveEngine({ activeEngineId: null });
|
||||
assert.ok(result.engine, "should return engine for null");
|
||||
assert.equal(
|
||||
result.engine.engineId,
|
||||
"dev",
|
||||
"engine.engineId should be 'dev' for null activeEngineId",
|
||||
);
|
||||
});
|
||||
|
||||
test("resolveEngine returns dev engine for 'dev' activeEngineId", async () => {
|
||||
const { resolveEngine } = await import("../engine-resolver.ts");
|
||||
const result = resolveEngine({ activeEngineId: "dev" });
|
||||
assert.ok(result.engine, "should return engine for 'dev'");
|
||||
assert.equal(
|
||||
result.engine.engineId,
|
||||
"dev",
|
||||
"engine.engineId should be 'dev'",
|
||||
);
|
||||
});
|
||||
|
||||
test("resolveEngine throws for unknown activeEngineId without activeRunDir", async () => {
|
||||
const { resolveEngine } = await import("../engine-resolver.ts");
|
||||
assert.throws(
|
||||
() => resolveEngine({ activeEngineId: "custom-xyz" }),
|
||||
/activeRunDir/,
|
||||
"resolveEngine should throw when custom engine has no activeRunDir",
|
||||
);
|
||||
});
|
||||
|
||||
test("resolveEngine returns custom engine for non-dev activeEngineId with activeRunDir", async () => {
|
||||
const { resolveEngine } = await import("../engine-resolver.ts");
|
||||
const result = resolveEngine({ activeEngineId: "custom-xyz", activeRunDir: "/tmp/test-run" });
|
||||
assert.ok(result.engine, "should return engine for custom ID");
|
||||
assert.equal(
|
||||
result.engine.engineId,
|
||||
"custom",
|
||||
"engine.engineId should be 'custom' for non-dev activeEngineId",
|
||||
);
|
||||
});
|
||||
|
||||
test("ResolvedEngine type is exported (source check)", () => {
|
||||
const source = readSource(ENGINE_RESOLVER_PATH);
|
||||
assert.ok(
|
||||
/export\s+(interface|type)\s+ResolvedEngine/.test(source),
|
||||
"engine-resolver.ts must export ResolvedEngine type",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ── AutoSession.activeEngineId ──────────────────────────────────────────────
|
||||
|
||||
describe("AutoSession.activeEngineId", () => {
|
||||
test("defaults to null on a fresh AutoSession", async () => {
|
||||
const { AutoSession } = await import("../auto/session.ts");
|
||||
const session = new AutoSession();
|
||||
assert.equal(
|
||||
session.activeEngineId,
|
||||
null,
|
||||
"activeEngineId should default to null",
|
||||
);
|
||||
});
|
||||
|
||||
test("is null after reset()", async () => {
|
||||
const { AutoSession } = await import("../auto/session.ts");
|
||||
const session = new AutoSession();
|
||||
session.activeEngineId = "dev";
|
||||
session.reset();
|
||||
assert.equal(
|
||||
session.activeEngineId,
|
||||
null,
|
||||
"activeEngineId should be null after reset()",
|
||||
);
|
||||
});
|
||||
|
||||
test("appears in toJSON() output", async () => {
|
||||
const { AutoSession } = await import("../auto/session.ts");
|
||||
const session = new AutoSession();
|
||||
const json = session.toJSON();
|
||||
assert.ok(
|
||||
"activeEngineId" in json,
|
||||
"toJSON() must include activeEngineId",
|
||||
);
|
||||
assert.equal(
|
||||
json.activeEngineId,
|
||||
null,
|
||||
"toJSON().activeEngineId should be null by default",
|
||||
);
|
||||
});
|
||||
});
|
||||
599
src/resources/extensions/gsd/tests/graph-operations.test.ts
Normal file
599
src/resources/extensions/gsd/tests/graph-operations.test.ts
Normal file
|
|
@ -0,0 +1,599 @@
|
|||
/**
|
||||
* graph-operations.test.ts — Comprehensive tests for graph.ts DAG operations.
|
||||
*
|
||||
* Covers: YAML I/O round-trips, DAG queries (getNextPendingStep),
|
||||
* immutable step completion, iteration expansion with downstream dep
|
||||
* rewriting, initializeGraph conversion, and atomic write safety.
|
||||
*/
|
||||
|
||||
import { describe, it } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtempSync, rmSync, readFileSync, writeFileSync, existsSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
|
||||
import {
|
||||
readGraph,
|
||||
writeGraph,
|
||||
getNextPendingStep,
|
||||
markStepComplete,
|
||||
expandIteration,
|
||||
initializeGraph,
|
||||
graphFromDefinition,
|
||||
type WorkflowGraph,
|
||||
type GraphStep,
|
||||
} from "../graph.ts";
|
||||
import type { WorkflowDefinition } from "../definition-loader.ts";
|
||||
|
||||
// ─── Helpers ─────────────────────────────────────────────────────────────
|
||||
|
||||
function makeTmpDir(): string {
|
||||
return mkdtempSync(join(tmpdir(), "graph-test-"));
|
||||
}
|
||||
|
||||
function cleanupDir(dir: string): void {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
/** Minimal valid graph for testing. */
|
||||
function makeGraph(steps: GraphStep[], name = "test-workflow"): WorkflowGraph {
|
||||
return {
|
||||
steps,
|
||||
metadata: { name, createdAt: "2026-01-01T00:00:00.000Z" },
|
||||
};
|
||||
}
|
||||
|
||||
function makeStep(overrides: Partial<GraphStep> & { id: string }): GraphStep {
|
||||
return {
|
||||
title: overrides.id,
|
||||
status: "pending",
|
||||
prompt: `Do ${overrides.id}`,
|
||||
dependsOn: [],
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
// ─── writeGraph + readGraph round-trip ───────────────────────────────────
|
||||
|
||||
describe("writeGraph + readGraph round-trip", () => {
|
||||
it("preserves all fields including parentStepId and dependsOn", () => {
|
||||
const dir = makeTmpDir();
|
||||
try {
|
||||
const graph = makeGraph([
|
||||
makeStep({ id: "step-1", title: "First Step", dependsOn: [] }),
|
||||
makeStep({
|
||||
id: "step-2",
|
||||
title: "Second Step",
|
||||
dependsOn: ["step-1"],
|
||||
parentStepId: "parent-iter",
|
||||
}),
|
||||
]);
|
||||
|
||||
writeGraph(dir, graph);
|
||||
const loaded = readGraph(dir);
|
||||
|
||||
assert.equal(loaded.steps.length, 2);
|
||||
assert.equal(loaded.steps[0].id, "step-1");
|
||||
assert.equal(loaded.steps[0].title, "First Step");
|
||||
assert.equal(loaded.steps[0].status, "pending");
|
||||
assert.deepStrictEqual(loaded.steps[0].dependsOn, []);
|
||||
|
||||
assert.equal(loaded.steps[1].id, "step-2");
|
||||
assert.deepStrictEqual(loaded.steps[1].dependsOn, ["step-1"]);
|
||||
assert.equal(loaded.steps[1].parentStepId, "parent-iter");
|
||||
|
||||
assert.equal(loaded.metadata.name, "test-workflow");
|
||||
assert.equal(loaded.metadata.createdAt, "2026-01-01T00:00:00.000Z");
|
||||
} finally {
|
||||
cleanupDir(dir);
|
||||
}
|
||||
});
|
||||
|
||||
it("preserves startedAt and finishedAt fields", () => {
|
||||
const dir = makeTmpDir();
|
||||
try {
|
||||
const graph = makeGraph([
|
||||
makeStep({
|
||||
id: "s1",
|
||||
status: "complete",
|
||||
startedAt: "2026-01-01T01:00:00.000Z",
|
||||
finishedAt: "2026-01-01T01:05:00.000Z",
|
||||
}),
|
||||
]);
|
||||
writeGraph(dir, graph);
|
||||
const loaded = readGraph(dir);
|
||||
|
||||
assert.equal(loaded.steps[0].startedAt, "2026-01-01T01:00:00.000Z");
|
||||
assert.equal(loaded.steps[0].finishedAt, "2026-01-01T01:05:00.000Z");
|
||||
} finally {
|
||||
cleanupDir(dir);
|
||||
}
|
||||
});
|
||||
|
||||
it("creates directory if it does not exist", () => {
|
||||
const base = makeTmpDir();
|
||||
const nested = join(base, "sub", "dir");
|
||||
try {
|
||||
const graph = makeGraph([makeStep({ id: "s1" })]);
|
||||
writeGraph(nested, graph);
|
||||
assert.ok(existsSync(join(nested, "GRAPH.yaml")));
|
||||
|
||||
const loaded = readGraph(nested);
|
||||
assert.equal(loaded.steps[0].id, "s1");
|
||||
} finally {
|
||||
cleanupDir(base);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ─── readGraph error paths ───────────────────────────────────────────────
|
||||
|
||||
describe("readGraph error paths", () => {
|
||||
it("throws with descriptive error when file is missing", () => {
|
||||
const dir = makeTmpDir();
|
||||
try {
|
||||
assert.throws(
|
||||
() => readGraph(dir),
|
||||
(err: Error) => {
|
||||
assert.ok(err.message.includes("GRAPH.yaml not found"));
|
||||
assert.ok(err.message.includes(dir));
|
||||
return true;
|
||||
},
|
||||
);
|
||||
} finally {
|
||||
cleanupDir(dir);
|
||||
}
|
||||
});
|
||||
|
||||
it("throws with descriptive error when YAML is malformed (missing steps)", () => {
|
||||
const dir = makeTmpDir();
|
||||
try {
|
||||
writeFileSync(join(dir, "GRAPH.yaml"), "metadata:\n name: bad\n", "utf-8");
|
||||
assert.throws(
|
||||
() => readGraph(dir),
|
||||
(err: Error) => {
|
||||
assert.ok(err.message.includes("missing or invalid 'steps' array"));
|
||||
return true;
|
||||
},
|
||||
);
|
||||
} finally {
|
||||
cleanupDir(dir);
|
||||
}
|
||||
});
|
||||
|
||||
it("throws when steps is not an array", () => {
|
||||
const dir = makeTmpDir();
|
||||
try {
|
||||
writeFileSync(join(dir, "GRAPH.yaml"), "steps: not-an-array\nmetadata:\n name: bad\n", "utf-8");
|
||||
assert.throws(
|
||||
() => readGraph(dir),
|
||||
(err: Error) => {
|
||||
assert.ok(err.message.includes("missing or invalid 'steps' array"));
|
||||
return true;
|
||||
},
|
||||
);
|
||||
} finally {
|
||||
cleanupDir(dir);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ─── getNextPendingStep ──────────────────────────────────────────────────
|
||||
|
||||
describe("getNextPendingStep", () => {
|
||||
it("returns first step with all deps complete", () => {
|
||||
const graph = makeGraph([
|
||||
makeStep({ id: "a", status: "complete" }),
|
||||
makeStep({ id: "b", dependsOn: ["a"] }),
|
||||
makeStep({ id: "c", dependsOn: ["b"] }),
|
||||
]);
|
||||
|
||||
const next = getNextPendingStep(graph);
|
||||
assert.equal(next?.id, "b");
|
||||
});
|
||||
|
||||
it("skips steps with incomplete deps", () => {
|
||||
const graph = makeGraph([
|
||||
makeStep({ id: "a" }),
|
||||
makeStep({ id: "b", dependsOn: ["a"] }),
|
||||
]);
|
||||
|
||||
// 'a' is still pending, so 'b' is blocked, but 'a' has no deps → returns 'a'
|
||||
const next = getNextPendingStep(graph);
|
||||
assert.equal(next?.id, "a");
|
||||
});
|
||||
|
||||
it("returns null when all steps are complete", () => {
|
||||
const graph = makeGraph([
|
||||
makeStep({ id: "a", status: "complete" }),
|
||||
makeStep({ id: "b", status: "complete" }),
|
||||
]);
|
||||
|
||||
assert.equal(getNextPendingStep(graph), null);
|
||||
});
|
||||
|
||||
it("returns null when all pending steps are blocked", () => {
|
||||
const graph = makeGraph([
|
||||
makeStep({ id: "a", status: "active" }), // not complete
|
||||
makeStep({ id: "b", dependsOn: ["a"] }), // blocked
|
||||
]);
|
||||
|
||||
assert.equal(getNextPendingStep(graph), null);
|
||||
});
|
||||
|
||||
it("returns first pending step with no deps when root steps exist", () => {
|
||||
const graph = makeGraph([
|
||||
makeStep({ id: "a" }),
|
||||
makeStep({ id: "b" }),
|
||||
]);
|
||||
|
||||
const next = getNextPendingStep(graph);
|
||||
assert.equal(next?.id, "a");
|
||||
});
|
||||
|
||||
it("skips expanded steps", () => {
|
||||
const graph = makeGraph([
|
||||
makeStep({ id: "a", status: "expanded" }),
|
||||
makeStep({ id: "b" }),
|
||||
]);
|
||||
|
||||
const next = getNextPendingStep(graph);
|
||||
assert.equal(next?.id, "b");
|
||||
});
|
||||
});
|
||||
|
||||
// ─── markStepComplete ────────────────────────────────────────────────────
|
||||
|
||||
describe("markStepComplete", () => {
|
||||
it("returns new graph with step status 'complete' (original unchanged)", () => {
|
||||
const original = makeGraph([
|
||||
makeStep({ id: "a" }),
|
||||
makeStep({ id: "b" }),
|
||||
]);
|
||||
|
||||
const updated = markStepComplete(original, "a");
|
||||
|
||||
// Original is untouched
|
||||
assert.equal(original.steps[0].status, "pending");
|
||||
|
||||
// New graph has the step complete
|
||||
assert.equal(updated.steps[0].status, "complete");
|
||||
assert.equal(updated.steps[0].id, "a");
|
||||
|
||||
// Other steps unchanged
|
||||
assert.equal(updated.steps[1].status, "pending");
|
||||
});
|
||||
|
||||
it("sets finishedAt timestamp", () => {
|
||||
const graph = makeGraph([makeStep({ id: "a" })]);
|
||||
const updated = markStepComplete(graph, "a");
|
||||
assert.ok(updated.steps[0].finishedAt);
|
||||
// Should be a valid ISO string
|
||||
assert.ok(!isNaN(Date.parse(updated.steps[0].finishedAt!)));
|
||||
});
|
||||
|
||||
it("throws for unknown step ID", () => {
|
||||
const graph = makeGraph([makeStep({ id: "a" })]);
|
||||
assert.throws(
|
||||
() => markStepComplete(graph, "nonexistent"),
|
||||
(err: Error) => {
|
||||
assert.ok(err.message.includes("Step not found"));
|
||||
assert.ok(err.message.includes("nonexistent"));
|
||||
return true;
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
it("preserves metadata in returned graph", () => {
|
||||
const graph = makeGraph([makeStep({ id: "a" })], "my-workflow");
|
||||
const updated = markStepComplete(graph, "a");
|
||||
assert.equal(updated.metadata.name, "my-workflow");
|
||||
assert.equal(updated.metadata.createdAt, "2026-01-01T00:00:00.000Z");
|
||||
});
|
||||
});
|
||||
|
||||
// ─── expandIteration ─────────────────────────────────────────────────────
|
||||
|
||||
describe("expandIteration", () => {
|
||||
it("creates instance steps with correct IDs (stepId--001, stepId--002)", () => {
|
||||
const graph = makeGraph([
|
||||
makeStep({ id: "iter-step", title: "Process items" }),
|
||||
makeStep({ id: "final", dependsOn: ["iter-step"] }),
|
||||
]);
|
||||
|
||||
const expanded = expandIteration(
|
||||
graph,
|
||||
"iter-step",
|
||||
["apple", "banana", "cherry"],
|
||||
"Process {{item}}",
|
||||
);
|
||||
|
||||
// Parent + 3 instances + final = 5 steps
|
||||
assert.equal(expanded.steps.length, 5);
|
||||
|
||||
// Instances are correctly named
|
||||
assert.equal(expanded.steps[1].id, "iter-step--001");
|
||||
assert.equal(expanded.steps[2].id, "iter-step--002");
|
||||
assert.equal(expanded.steps[3].id, "iter-step--003");
|
||||
});
|
||||
|
||||
it("marks parent step as 'expanded'", () => {
|
||||
const graph = makeGraph([
|
||||
makeStep({ id: "iter", title: "Iterate" }),
|
||||
]);
|
||||
|
||||
const expanded = expandIteration(graph, "iter", ["a"], "Do {{item}}");
|
||||
assert.equal(expanded.steps[0].status, "expanded");
|
||||
});
|
||||
|
||||
it("instance steps have correct titles, prompts, parentStepId, and deps", () => {
|
||||
const graph = makeGraph([
|
||||
makeStep({ id: "pre", status: "complete" }),
|
||||
makeStep({ id: "iter", title: "Process", dependsOn: ["pre"] }),
|
||||
]);
|
||||
|
||||
const expanded = expandIteration(
|
||||
graph,
|
||||
"iter",
|
||||
["foo", "bar"],
|
||||
"Handle {{item}} carefully",
|
||||
);
|
||||
|
||||
const inst1 = expanded.steps[2]; // after pre and expanded parent
|
||||
assert.equal(inst1.title, "Process: foo");
|
||||
assert.equal(inst1.prompt, "Handle foo carefully");
|
||||
assert.equal(inst1.parentStepId, "iter");
|
||||
assert.deepStrictEqual(inst1.dependsOn, ["pre"]);
|
||||
assert.equal(inst1.status, "pending");
|
||||
|
||||
const inst2 = expanded.steps[3];
|
||||
assert.equal(inst2.title, "Process: bar");
|
||||
assert.equal(inst2.prompt, "Handle bar carefully");
|
||||
assert.equal(inst2.parentStepId, "iter");
|
||||
});
|
||||
|
||||
it("rewrites downstream deps from parent ID to all instance IDs", () => {
|
||||
const graph = makeGraph([
|
||||
makeStep({ id: "iter", title: "Iterate" }),
|
||||
makeStep({ id: "after", dependsOn: ["iter"] }),
|
||||
]);
|
||||
|
||||
const expanded = expandIteration(
|
||||
graph,
|
||||
"iter",
|
||||
["x", "y"],
|
||||
"Do {{item}}",
|
||||
);
|
||||
|
||||
// 'after' should now depend on iter--001 and iter--002
|
||||
const afterStep = expanded.steps.find((s) => s.id === "after")!;
|
||||
assert.deepStrictEqual(afterStep.dependsOn, ["iter--001", "iter--002"]);
|
||||
});
|
||||
|
||||
it("preserves steps that don't depend on the parent", () => {
|
||||
const graph = makeGraph([
|
||||
makeStep({ id: "unrelated" }),
|
||||
makeStep({ id: "iter", title: "Iterate" }),
|
||||
makeStep({ id: "after", dependsOn: ["iter"] }),
|
||||
]);
|
||||
|
||||
const expanded = expandIteration(graph, "iter", ["a"], "{{item}}");
|
||||
const unrelated = expanded.steps.find((s) => s.id === "unrelated")!;
|
||||
assert.deepStrictEqual(unrelated.dependsOn, []);
|
||||
});
|
||||
|
||||
it("throws for non-pending parent step", () => {
|
||||
const graph = makeGraph([
|
||||
makeStep({ id: "iter", status: "complete" }),
|
||||
]);
|
||||
|
||||
assert.throws(
|
||||
() => expandIteration(graph, "iter", ["a"], "{{item}}"),
|
||||
(err: Error) => {
|
||||
assert.ok(err.message.includes("complete"));
|
||||
assert.ok(err.message.includes("expected \"pending\""));
|
||||
return true;
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
it("throws for unknown step ID", () => {
|
||||
const graph = makeGraph([makeStep({ id: "a" })]);
|
||||
assert.throws(
|
||||
() => expandIteration(graph, "nonexistent", ["a"], "{{item}}"),
|
||||
(err: Error) => {
|
||||
assert.ok(err.message.includes("step not found"));
|
||||
assert.ok(err.message.includes("nonexistent"));
|
||||
return true;
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
it("does not mutate the input graph", () => {
|
||||
const graph = makeGraph([
|
||||
makeStep({ id: "iter", title: "Iterate" }),
|
||||
makeStep({ id: "after", dependsOn: ["iter"] }),
|
||||
]);
|
||||
|
||||
const originalStepsLength = graph.steps.length;
|
||||
const originalAfterDeps = [...graph.steps[1].dependsOn];
|
||||
|
||||
expandIteration(graph, "iter", ["a", "b"], "{{item}}");
|
||||
|
||||
// Original unchanged
|
||||
assert.equal(graph.steps.length, originalStepsLength);
|
||||
assert.equal(graph.steps[0].status, "pending");
|
||||
assert.deepStrictEqual(graph.steps[1].dependsOn, originalAfterDeps);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── initializeGraph ─────────────────────────────────────────────────────
|
||||
|
||||
describe("initializeGraph", () => {
|
||||
it("converts a valid 3-step definition to graph with all pending steps", () => {
|
||||
const def: WorkflowDefinition = {
|
||||
version: 1,
|
||||
name: "test-workflow",
|
||||
steps: [
|
||||
{ id: "s1", name: "Step One", prompt: "Do step one", requires: [], produces: ["out.md"] },
|
||||
{ id: "s2", name: "Step Two", prompt: "Do step two", requires: ["s1"], produces: [] },
|
||||
{ id: "s3", name: "Step Three", prompt: "Do step three", requires: ["s1", "s2"], produces: [] },
|
||||
],
|
||||
};
|
||||
|
||||
const graph = initializeGraph(def);
|
||||
|
||||
assert.equal(graph.steps.length, 3);
|
||||
assert.equal(graph.metadata.name, "test-workflow");
|
||||
assert.ok(graph.metadata.createdAt); // ISO string
|
||||
|
||||
// All pending
|
||||
for (const step of graph.steps) {
|
||||
assert.equal(step.status, "pending");
|
||||
}
|
||||
|
||||
// Correct mapping
|
||||
assert.equal(graph.steps[0].id, "s1");
|
||||
assert.equal(graph.steps[0].title, "Step One");
|
||||
assert.equal(graph.steps[0].prompt, "Do step one");
|
||||
assert.deepStrictEqual(graph.steps[0].dependsOn, []);
|
||||
|
||||
assert.equal(graph.steps[1].id, "s2");
|
||||
assert.deepStrictEqual(graph.steps[1].dependsOn, ["s1"]);
|
||||
|
||||
assert.equal(graph.steps[2].id, "s3");
|
||||
assert.deepStrictEqual(graph.steps[2].dependsOn, ["s1", "s2"]);
|
||||
});
|
||||
|
||||
it("is also exported as graphFromDefinition (backward compat)", () => {
|
||||
assert.equal(graphFromDefinition, initializeGraph);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── Atomic write safety ─────────────────────────────────────────────────
|
||||
|
||||
describe("atomic write safety", () => {
|
||||
it("final file exists and .tmp file does not exist after write", () => {
|
||||
const dir = makeTmpDir();
|
||||
try {
|
||||
const graph = makeGraph([makeStep({ id: "s1" })]);
|
||||
writeGraph(dir, graph);
|
||||
|
||||
assert.ok(existsSync(join(dir, "GRAPH.yaml")));
|
||||
assert.ok(!existsSync(join(dir, "GRAPH.yaml.tmp")));
|
||||
} finally {
|
||||
cleanupDir(dir);
|
||||
}
|
||||
});
|
||||
|
||||
it("YAML content is valid and parseable", () => {
|
||||
const dir = makeTmpDir();
|
||||
try {
|
||||
const graph = makeGraph([makeStep({ id: "s1" })]);
|
||||
writeGraph(dir, graph);
|
||||
|
||||
const content = readFileSync(join(dir, "GRAPH.yaml"), "utf-8");
|
||||
// Should contain snake_case keys
|
||||
assert.ok(content.includes("created_at"));
|
||||
// Should not contain camelCase keys
|
||||
assert.ok(!content.includes("createdAt"));
|
||||
assert.ok(!content.includes("dependsOn"));
|
||||
} finally {
|
||||
cleanupDir(dir);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ─── YAML snake_case / camelCase boundary ────────────────────────────────
|
||||
|
||||
describe("YAML snake_case / camelCase boundary", () => {
|
||||
it("writes snake_case to disk and reads back as camelCase", () => {
|
||||
const dir = makeTmpDir();
|
||||
try {
|
||||
const graph = makeGraph([
|
||||
makeStep({
|
||||
id: "s1",
|
||||
dependsOn: ["s0"],
|
||||
parentStepId: "parent",
|
||||
startedAt: "2026-01-01T00:00:00Z",
|
||||
finishedAt: "2026-01-01T00:01:00Z",
|
||||
}),
|
||||
]);
|
||||
|
||||
writeGraph(dir, graph);
|
||||
|
||||
// Verify raw YAML uses snake_case
|
||||
const raw = readFileSync(join(dir, "GRAPH.yaml"), "utf-8");
|
||||
assert.ok(raw.includes("depends_on"));
|
||||
assert.ok(raw.includes("parent_step_id"));
|
||||
assert.ok(raw.includes("started_at"));
|
||||
assert.ok(raw.includes("finished_at"));
|
||||
assert.ok(raw.includes("created_at"));
|
||||
|
||||
// Verify read returns camelCase
|
||||
const loaded = readGraph(dir);
|
||||
assert.deepStrictEqual(loaded.steps[0].dependsOn, ["s0"]);
|
||||
assert.equal(loaded.steps[0].parentStepId, "parent");
|
||||
assert.equal(loaded.steps[0].startedAt, "2026-01-01T00:00:00Z");
|
||||
assert.equal(loaded.steps[0].finishedAt, "2026-01-01T00:01:00Z");
|
||||
} finally {
|
||||
cleanupDir(dir);
|
||||
}
|
||||
});
|
||||
|
||||
it("omits optional fields from YAML when undefined", () => {
|
||||
const dir = makeTmpDir();
|
||||
try {
|
||||
const graph = makeGraph([
|
||||
makeStep({ id: "s1" }),
|
||||
]);
|
||||
|
||||
writeGraph(dir, graph);
|
||||
const raw = readFileSync(join(dir, "GRAPH.yaml"), "utf-8");
|
||||
|
||||
// No depends_on, parent_step_id, started_at, finished_at when undefined/empty
|
||||
assert.ok(!raw.includes("depends_on"));
|
||||
assert.ok(!raw.includes("parent_step_id"));
|
||||
assert.ok(!raw.includes("started_at"));
|
||||
assert.ok(!raw.includes("finished_at"));
|
||||
} finally {
|
||||
cleanupDir(dir);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ─── Edge cases ──────────────────────────────────────────────────────────
|
||||
|
||||
describe("edge cases", () => {
|
||||
it("handles empty items array in expandIteration", () => {
|
||||
const graph = makeGraph([
|
||||
makeStep({ id: "iter" }),
|
||||
]);
|
||||
|
||||
const expanded = expandIteration(graph, "iter", [], "{{item}}");
|
||||
// Parent marked expanded, no instances created
|
||||
assert.equal(expanded.steps.length, 1);
|
||||
assert.equal(expanded.steps[0].status, "expanded");
|
||||
});
|
||||
|
||||
it("handles graph with single step", () => {
|
||||
const graph = makeGraph([makeStep({ id: "only" })]);
|
||||
const next = getNextPendingStep(graph);
|
||||
assert.equal(next?.id, "only");
|
||||
|
||||
const completed = markStepComplete(graph, "only");
|
||||
assert.equal(getNextPendingStep(completed), null);
|
||||
});
|
||||
|
||||
it("initializeGraph handles steps with empty requires", () => {
|
||||
const def: WorkflowDefinition = {
|
||||
version: 1,
|
||||
name: "empty-requires",
|
||||
steps: [
|
||||
{ id: "s1", name: "Step", prompt: "Go", requires: [], produces: [] },
|
||||
],
|
||||
};
|
||||
const graph = initializeGraph(def);
|
||||
assert.deepStrictEqual(graph.steps[0].dependsOn, []);
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,429 @@
|
|||
/**
|
||||
* iterate-engine-integration.test.ts — Integration tests for iterate/fan-out
|
||||
* expansion wired into CustomWorkflowEngine.
|
||||
*
|
||||
* Proves the full expansion→dispatch→reconcile cycle: the engine reads
|
||||
* iterate config from frozen DEFINITION.yaml, reads the source artifact,
|
||||
* extracts items via regex, calls expandIteration() to rewrite the graph,
|
||||
* persists it, and dispatches instance steps sequentially.
|
||||
*
|
||||
* Uses real temp directories with actual DEFINITION.yaml, GRAPH.yaml,
|
||||
* and source artifact files — no mocks.
|
||||
*/
|
||||
|
||||
import { describe, it, afterEach } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtempSync, rmSync, writeFileSync, mkdirSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import { stringify } from "yaml";
|
||||
|
||||
import { CustomWorkflowEngine } from "../custom-workflow-engine.ts";
|
||||
import {
|
||||
writeGraph,
|
||||
readGraph,
|
||||
type WorkflowGraph,
|
||||
type GraphStep,
|
||||
} from "../graph.ts";
|
||||
import type { WorkflowDefinition } from "../definition-loader.ts";
|
||||
|
||||
// ─── Helpers ─────────────────────────────────────────────────────────────
|
||||
|
||||
const tmpDirs: string[] = [];
|
||||
|
||||
function makeTmpDir(): string {
|
||||
const dir = mkdtempSync(join(tmpdir(), "iterate-test-"));
|
||||
tmpDirs.push(dir);
|
||||
return dir;
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
for (const d of tmpDirs) {
|
||||
rmSync(d, { recursive: true, force: true });
|
||||
}
|
||||
tmpDirs.length = 0;
|
||||
});
|
||||
|
||||
/**
|
||||
* Create a temp run directory with DEFINITION.yaml, GRAPH.yaml, and optional
|
||||
* artifact files. Returns the run dir path and engine instance.
|
||||
*/
|
||||
function makeTempRun(
|
||||
def: WorkflowDefinition,
|
||||
graphSteps: GraphStep[],
|
||||
files?: Record<string, string>,
|
||||
): { runDir: string; engine: CustomWorkflowEngine } {
|
||||
const runDir = makeTmpDir();
|
||||
|
||||
// Write frozen DEFINITION.yaml (camelCase — serialized from TS object)
|
||||
writeFileSync(join(runDir, "DEFINITION.yaml"), stringify(def), "utf-8");
|
||||
|
||||
// Write GRAPH.yaml via the standard writer
|
||||
const graph: WorkflowGraph = {
|
||||
steps: graphSteps,
|
||||
metadata: { name: def.name, createdAt: "2026-01-01T00:00:00.000Z" },
|
||||
};
|
||||
writeGraph(runDir, graph);
|
||||
|
||||
// Write optional artifact files
|
||||
if (files) {
|
||||
for (const [relPath, content] of Object.entries(files)) {
|
||||
const absPath = join(runDir, relPath);
|
||||
mkdirSync(join(absPath, ".."), { recursive: true });
|
||||
writeFileSync(absPath, content, "utf-8");
|
||||
}
|
||||
}
|
||||
|
||||
return { runDir, engine: new CustomWorkflowEngine(runDir) };
|
||||
}
|
||||
|
||||
/** Shorthand to build a GraphStep. */
|
||||
function makeStep(overrides: Partial<GraphStep> & { id: string }): GraphStep {
|
||||
return {
|
||||
title: overrides.id,
|
||||
status: "pending",
|
||||
prompt: `Do ${overrides.id}`,
|
||||
dependsOn: [],
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
/** Drive a full deriveState→resolveDispatch cycle. */
|
||||
async function dispatch(engine: CustomWorkflowEngine) {
|
||||
const state = await engine.deriveState("/unused");
|
||||
return engine.resolveDispatch(state, { basePath: "/unused" });
|
||||
}
|
||||
|
||||
/** Drive a full deriveState→reconcile cycle for a given unitId. */
|
||||
async function reconcile(engine: CustomWorkflowEngine, unitId: string) {
|
||||
const state = await engine.deriveState("/unused");
|
||||
return engine.reconcile(state, {
|
||||
unitType: "custom-step",
|
||||
unitId,
|
||||
startedAt: Date.now() - 1000,
|
||||
finishedAt: Date.now(),
|
||||
});
|
||||
}
|
||||
|
||||
// ─── Tests ───────────────────────────────────────────────────────────────
|
||||
|
||||
describe("iterate expansion — basic", () => {
|
||||
it("expands an iterate step into 3 instances and dispatches the first", async () => {
|
||||
const def: WorkflowDefinition = {
|
||||
version: 1,
|
||||
name: "iter-wf",
|
||||
steps: [
|
||||
{
|
||||
id: "iter-step",
|
||||
name: "Iterate Step",
|
||||
prompt: "Process {{item}}",
|
||||
requires: [],
|
||||
produces: [],
|
||||
iterate: { source: "topics.md", pattern: "^- (.+)$" },
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
const graphSteps = [
|
||||
makeStep({ id: "iter-step", prompt: "Process {{item}}" }),
|
||||
];
|
||||
|
||||
const { runDir, engine } = makeTempRun(def, graphSteps, {
|
||||
"topics.md": "- Alpha\n- Beta\n- Gamma\n",
|
||||
});
|
||||
|
||||
const result = await dispatch(engine);
|
||||
|
||||
// Should dispatch the first instance step
|
||||
assert.equal(result.action, "dispatch");
|
||||
if (result.action === "dispatch") {
|
||||
assert.equal(result.step.unitId, "iter-wf/iter-step--001");
|
||||
assert.equal(result.step.prompt, "Process Alpha");
|
||||
}
|
||||
|
||||
// Verify on-disk graph state
|
||||
const graph = readGraph(runDir);
|
||||
const parent = graph.steps.find((s) => s.id === "iter-step");
|
||||
assert.ok(parent, "Parent step should exist");
|
||||
assert.equal(parent.status, "expanded");
|
||||
|
||||
const instances = graph.steps.filter((s) => s.parentStepId === "iter-step");
|
||||
assert.equal(instances.length, 3);
|
||||
assert.equal(instances[0].id, "iter-step--001");
|
||||
assert.equal(instances[1].id, "iter-step--002");
|
||||
assert.equal(instances[2].id, "iter-step--003");
|
||||
assert.equal(instances[0].prompt, "Process Alpha");
|
||||
assert.equal(instances[1].prompt, "Process Beta");
|
||||
assert.equal(instances[2].prompt, "Process Gamma");
|
||||
});
|
||||
});
|
||||
|
||||
describe("iterate expansion — full dispatch→reconcile sequence", () => {
|
||||
it("dispatches all 3 instances sequentially then stops", async () => {
|
||||
const def: WorkflowDefinition = {
|
||||
version: 1,
|
||||
name: "seq-wf",
|
||||
steps: [
|
||||
{
|
||||
id: "fan",
|
||||
name: "Fan Step",
|
||||
prompt: "Handle {{item}}",
|
||||
requires: [],
|
||||
produces: [],
|
||||
iterate: { source: "items.md", pattern: "^- (.+)$" },
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
const graphSteps = [makeStep({ id: "fan", prompt: "Handle {{item}}" })];
|
||||
|
||||
const { engine } = makeTempRun(def, graphSteps, {
|
||||
"items.md": "- One\n- Two\n- Three\n",
|
||||
});
|
||||
|
||||
// First dispatch triggers expansion, returns instance 1
|
||||
let result = await dispatch(engine);
|
||||
assert.equal(result.action, "dispatch");
|
||||
if (result.action === "dispatch") {
|
||||
assert.equal(result.step.unitId, "seq-wf/fan--001");
|
||||
assert.equal(result.step.prompt, "Handle One");
|
||||
}
|
||||
|
||||
// Reconcile instance 1, dispatch → instance 2
|
||||
await reconcile(engine, "seq-wf/fan--001");
|
||||
result = await dispatch(engine);
|
||||
assert.equal(result.action, "dispatch");
|
||||
if (result.action === "dispatch") {
|
||||
assert.equal(result.step.unitId, "seq-wf/fan--002");
|
||||
assert.equal(result.step.prompt, "Handle Two");
|
||||
}
|
||||
|
||||
// Reconcile instance 2, dispatch → instance 3
|
||||
await reconcile(engine, "seq-wf/fan--002");
|
||||
result = await dispatch(engine);
|
||||
assert.equal(result.action, "dispatch");
|
||||
if (result.action === "dispatch") {
|
||||
assert.equal(result.step.unitId, "seq-wf/fan--003");
|
||||
assert.equal(result.step.prompt, "Handle Three");
|
||||
}
|
||||
|
||||
// Reconcile instance 3, dispatch → should stop (all done)
|
||||
await reconcile(engine, "seq-wf/fan--003");
|
||||
result = await dispatch(engine);
|
||||
assert.equal(result.action, "stop");
|
||||
if (result.action === "stop") {
|
||||
assert.equal(result.reason, "All steps complete");
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("iterate expansion — downstream blocking", () => {
|
||||
it("blocks downstream step until all instances are complete", async () => {
|
||||
const def: WorkflowDefinition = {
|
||||
version: 1,
|
||||
name: "block-wf",
|
||||
steps: [
|
||||
{
|
||||
id: "fan",
|
||||
name: "Fan Step",
|
||||
prompt: "Process {{item}}",
|
||||
requires: [],
|
||||
produces: [],
|
||||
iterate: { source: "items.md", pattern: "^- (.+)$" },
|
||||
},
|
||||
{
|
||||
id: "merge",
|
||||
name: "Merge Step",
|
||||
prompt: "Merge all results",
|
||||
requires: ["fan"],
|
||||
produces: [],
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
const graphSteps = [
|
||||
makeStep({ id: "fan", prompt: "Process {{item}}" }),
|
||||
makeStep({ id: "merge", prompt: "Merge all results", dependsOn: ["fan"] }),
|
||||
];
|
||||
|
||||
const { runDir, engine } = makeTempRun(def, graphSteps, {
|
||||
"items.md": "- X\n- Y\n",
|
||||
});
|
||||
|
||||
// First dispatch: expands and returns instance 1
|
||||
let result = await dispatch(engine);
|
||||
assert.equal(result.action, "dispatch");
|
||||
if (result.action === "dispatch") {
|
||||
assert.equal(result.step.unitId, "block-wf/fan--001");
|
||||
}
|
||||
|
||||
// Verify downstream dep was rewritten: merge now depends on fan--001, fan--002
|
||||
let graph = readGraph(runDir);
|
||||
const mergeStep = graph.steps.find((s) => s.id === "merge");
|
||||
assert.ok(mergeStep);
|
||||
assert.deepStrictEqual(mergeStep.dependsOn.sort(), ["fan--001", "fan--002"]);
|
||||
|
||||
// Complete instance 1 only — merge should NOT be dispatchable yet
|
||||
await reconcile(engine, "block-wf/fan--001");
|
||||
result = await dispatch(engine);
|
||||
assert.equal(result.action, "dispatch");
|
||||
if (result.action === "dispatch") {
|
||||
// Should get fan--002, not merge
|
||||
assert.equal(result.step.unitId, "block-wf/fan--002");
|
||||
}
|
||||
|
||||
// Complete instance 2 — now merge should be dispatchable
|
||||
await reconcile(engine, "block-wf/fan--002");
|
||||
result = await dispatch(engine);
|
||||
assert.equal(result.action, "dispatch");
|
||||
if (result.action === "dispatch") {
|
||||
assert.equal(result.step.unitId, "block-wf/merge");
|
||||
assert.equal(result.step.prompt, "Merge all results");
|
||||
}
|
||||
|
||||
// Complete merge — all done
|
||||
await reconcile(engine, "block-wf/merge");
|
||||
result = await dispatch(engine);
|
||||
assert.equal(result.action, "stop");
|
||||
});
|
||||
});
|
||||
|
||||
describe("iterate expansion — zero matches", () => {
|
||||
it("handles zero-match expansion gracefully", async () => {
|
||||
const def: WorkflowDefinition = {
|
||||
version: 1,
|
||||
name: "zero-wf",
|
||||
steps: [
|
||||
{
|
||||
id: "fan",
|
||||
name: "Fan Step",
|
||||
prompt: "Process {{item}}",
|
||||
requires: [],
|
||||
produces: [],
|
||||
iterate: { source: "items.md", pattern: "^- (.+)$" },
|
||||
},
|
||||
{
|
||||
id: "after",
|
||||
name: "After Step",
|
||||
prompt: "Do after",
|
||||
requires: ["fan"],
|
||||
produces: [],
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
const graphSteps = [
|
||||
makeStep({ id: "fan", prompt: "Process {{item}}" }),
|
||||
makeStep({ id: "after", prompt: "Do after", dependsOn: ["fan"] }),
|
||||
];
|
||||
|
||||
// Source file exists but has no matching lines
|
||||
const { runDir, engine } = makeTempRun(def, graphSteps, {
|
||||
"items.md": "No bullet items here\nJust plain text\n",
|
||||
});
|
||||
|
||||
// Dispatch should expand with zero instances
|
||||
const result = await dispatch(engine);
|
||||
|
||||
// Verify parent is expanded
|
||||
const graph = readGraph(runDir);
|
||||
const parent = graph.steps.find((s) => s.id === "fan");
|
||||
assert.ok(parent);
|
||||
assert.equal(parent.status, "expanded");
|
||||
|
||||
// With zero instances, no instance deps exist.
|
||||
// expandIteration rewrites "fan" → [] in the downstream dep list,
|
||||
// so "after" now has empty dependsOn and becomes dispatchable.
|
||||
// But first dispatch after expansion finds no pending instance steps.
|
||||
// The engine should either dispatch "after" or return stop.
|
||||
// Let's check what actually happened:
|
||||
if (result.action === "dispatch") {
|
||||
// The re-query found "after" step (since its deps were rewritten to [])
|
||||
assert.equal(result.step.unitId, "zero-wf/after");
|
||||
} else {
|
||||
// The engine returned stop for zero instances
|
||||
assert.equal(result.action, "stop");
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("iterate expansion — missing source artifact", () => {
|
||||
it("throws an error mentioning the missing file path", async () => {
|
||||
const def: WorkflowDefinition = {
|
||||
version: 1,
|
||||
name: "missing-wf",
|
||||
steps: [
|
||||
{
|
||||
id: "fan",
|
||||
name: "Fan Step",
|
||||
prompt: "Process {{item}}",
|
||||
requires: [],
|
||||
produces: [],
|
||||
iterate: { source: "nonexistent.md", pattern: "^- (.+)$" },
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
const graphSteps = [
|
||||
makeStep({ id: "fan", prompt: "Process {{item}}" }),
|
||||
];
|
||||
|
||||
// No source file written
|
||||
const { engine } = makeTempRun(def, graphSteps);
|
||||
|
||||
await assert.rejects(
|
||||
() => dispatch(engine),
|
||||
(err: Error) => {
|
||||
assert.ok(err.message.includes("nonexistent.md"), `Error should mention the filename: ${err.message}`);
|
||||
assert.ok(err.message.includes("Iterate source artifact not found"), `Error should mention it's an iterate source: ${err.message}`);
|
||||
return true;
|
||||
},
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("iterate expansion — idempotency", () => {
|
||||
it("does not re-expand an already expanded step on subsequent dispatch", async () => {
|
||||
const def: WorkflowDefinition = {
|
||||
version: 1,
|
||||
name: "idem-wf",
|
||||
steps: [
|
||||
{
|
||||
id: "fan",
|
||||
name: "Fan Step",
|
||||
prompt: "Process {{item}}",
|
||||
requires: [],
|
||||
produces: [],
|
||||
iterate: { source: "items.md", pattern: "^- (.+)$" },
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
const graphSteps = [makeStep({ id: "fan", prompt: "Process {{item}}" })];
|
||||
|
||||
const { runDir, engine } = makeTempRun(def, graphSteps, {
|
||||
"items.md": "- Uno\n- Dos\n",
|
||||
});
|
||||
|
||||
// First dispatch: triggers expansion
|
||||
let result = await dispatch(engine);
|
||||
assert.equal(result.action, "dispatch");
|
||||
if (result.action === "dispatch") {
|
||||
assert.equal(result.step.unitId, "idem-wf/fan--001");
|
||||
}
|
||||
|
||||
// Second dispatch without reconciling: should return the same instance
|
||||
// (graph already expanded on disk, parent is "expanded" so getNextPendingStep
|
||||
// skips it and returns the first pending instance step)
|
||||
result = await dispatch(engine);
|
||||
assert.equal(result.action, "dispatch");
|
||||
if (result.action === "dispatch") {
|
||||
assert.equal(result.step.unitId, "idem-wf/fan--001");
|
||||
}
|
||||
|
||||
// Verify no double-expansion: still only 2 instances
|
||||
const graph = readGraph(runDir);
|
||||
const instances = graph.steps.filter((s) => s.parentStepId === "fan");
|
||||
assert.equal(instances.length, 2);
|
||||
});
|
||||
});
|
||||
230
src/resources/extensions/gsd/tests/run-manager.test.ts
Normal file
230
src/resources/extensions/gsd/tests/run-manager.test.ts
Normal file
|
|
@ -0,0 +1,230 @@
|
|||
/**
|
||||
* run-manager.test.ts — Tests for run directory creation and listing.
|
||||
*
|
||||
* Uses real temp directories with actual definition YAML files and
|
||||
* GRAPH.yaml persistence — no mocks.
|
||||
*/
|
||||
|
||||
import { describe, it, afterEach } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import {
|
||||
mkdtempSync,
|
||||
rmSync,
|
||||
mkdirSync,
|
||||
writeFileSync,
|
||||
readFileSync,
|
||||
existsSync,
|
||||
readdirSync,
|
||||
} from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { tmpdir } from "node:os";
|
||||
import { parse } from "yaml";
|
||||
|
||||
import { createRun, listRuns } from "../run-manager.ts";
|
||||
|
||||
// ─── Helpers ─────────────────────────────────────────────────────────────
|
||||
|
||||
const tmpDirs: string[] = [];
|
||||
|
||||
function makeTmpBase(): string {
|
||||
const dir = mkdtempSync(join(tmpdir(), "run-mgr-test-"));
|
||||
tmpDirs.push(dir);
|
||||
return dir;
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
for (const d of tmpDirs) {
|
||||
rmSync(d, { recursive: true, force: true });
|
||||
}
|
||||
tmpDirs.length = 0;
|
||||
});
|
||||
|
||||
/** Write a minimal valid workflow definition YAML to the expected location. */
|
||||
function writeDefinition(
|
||||
basePath: string,
|
||||
name: string,
|
||||
content: string,
|
||||
): void {
|
||||
const defsDir = join(basePath, ".gsd", "workflow-defs");
|
||||
mkdirSync(defsDir, { recursive: true });
|
||||
writeFileSync(join(defsDir, `${name}.yaml`), content, "utf-8");
|
||||
}
|
||||
|
||||
const SIMPLE_DEF = `
|
||||
version: 1
|
||||
name: test-workflow
|
||||
description: A test workflow
|
||||
steps:
|
||||
- id: step-1
|
||||
name: First Step
|
||||
prompt: Do step 1
|
||||
requires: []
|
||||
produces: []
|
||||
- id: step-2
|
||||
name: Second Step
|
||||
prompt: Do step 2
|
||||
requires:
|
||||
- step-1
|
||||
produces: []
|
||||
`;
|
||||
|
||||
const PARAMETERIZED_DEF = `
|
||||
version: 1
|
||||
name: param-workflow
|
||||
description: A parameterized workflow
|
||||
params:
|
||||
target: default-target
|
||||
steps:
|
||||
- id: step-1
|
||||
name: Build
|
||||
prompt: "Build {{target}}"
|
||||
requires: []
|
||||
produces: []
|
||||
`;
|
||||
|
||||
// ─── createRun ───────────────────────────────────────────────────────────
|
||||
|
||||
describe("createRun", () => {
|
||||
it("creates directory structure with DEFINITION.yaml and GRAPH.yaml", () => {
|
||||
const base = makeTmpBase();
|
||||
writeDefinition(base, "test-workflow", SIMPLE_DEF);
|
||||
|
||||
const runDir = createRun(base, "test-workflow");
|
||||
|
||||
// Run directory exists
|
||||
assert.ok(existsSync(runDir), "run directory should exist");
|
||||
|
||||
// DEFINITION.yaml exists and contains the definition
|
||||
const defPath = join(runDir, "DEFINITION.yaml");
|
||||
assert.ok(existsSync(defPath), "DEFINITION.yaml should exist");
|
||||
const defContent = parse(readFileSync(defPath, "utf-8"));
|
||||
assert.equal(defContent.name, "test-workflow");
|
||||
assert.equal(defContent.steps.length, 2);
|
||||
|
||||
// GRAPH.yaml exists with all steps pending
|
||||
const graphPath = join(runDir, "GRAPH.yaml");
|
||||
assert.ok(existsSync(graphPath), "GRAPH.yaml should exist");
|
||||
const graphContent = parse(readFileSync(graphPath, "utf-8"));
|
||||
assert.equal(graphContent.steps.length, 2);
|
||||
assert.equal(graphContent.steps[0].status, "pending");
|
||||
assert.equal(graphContent.steps[1].status, "pending");
|
||||
assert.equal(graphContent.metadata.name, "test-workflow");
|
||||
|
||||
// No PARAMS.json without overrides
|
||||
assert.ok(!existsSync(join(runDir, "PARAMS.json")), "PARAMS.json should not exist without overrides");
|
||||
|
||||
// Run directory path matches convention
|
||||
assert.ok(runDir.includes(".gsd/workflow-runs/test-workflow/"), "path should follow convention");
|
||||
});
|
||||
|
||||
it("writes PARAMS.json and substituted prompts when overrides provided", () => {
|
||||
const base = makeTmpBase();
|
||||
writeDefinition(base, "param-workflow", PARAMETERIZED_DEF);
|
||||
|
||||
const runDir = createRun(base, "param-workflow", { target: "my-app" });
|
||||
|
||||
// PARAMS.json exists with overrides
|
||||
const paramsPath = join(runDir, "PARAMS.json");
|
||||
assert.ok(existsSync(paramsPath), "PARAMS.json should exist");
|
||||
const params = JSON.parse(readFileSync(paramsPath, "utf-8"));
|
||||
assert.deepStrictEqual(params, { target: "my-app" });
|
||||
|
||||
// DEFINITION.yaml has substituted prompts
|
||||
const defPath = join(runDir, "DEFINITION.yaml");
|
||||
const defContent = parse(readFileSync(defPath, "utf-8"));
|
||||
assert.equal(defContent.steps[0].prompt, "Build my-app");
|
||||
|
||||
// GRAPH.yaml also has substituted prompts
|
||||
const graphPath = join(runDir, "GRAPH.yaml");
|
||||
const graphContent = parse(readFileSync(graphPath, "utf-8"));
|
||||
assert.equal(graphContent.steps[0].prompt, "Build my-app");
|
||||
});
|
||||
|
||||
it("throws for unknown definition", () => {
|
||||
const base = makeTmpBase();
|
||||
// Don't write any definition file
|
||||
|
||||
assert.throws(
|
||||
() => createRun(base, "nonexistent"),
|
||||
(err: Error) => err.message.includes("not found"),
|
||||
);
|
||||
});
|
||||
|
||||
it("uses filesystem-safe timestamp directory names", () => {
|
||||
const base = makeTmpBase();
|
||||
writeDefinition(base, "test-workflow", SIMPLE_DEF);
|
||||
|
||||
const runDir = createRun(base, "test-workflow");
|
||||
|
||||
// Extract the timestamp directory name
|
||||
const parts = runDir.split("/");
|
||||
const timestamp = parts[parts.length - 1];
|
||||
|
||||
// Should not contain colons (filesystem-unsafe on Windows)
|
||||
assert.ok(!timestamp.includes(":"), `timestamp should not contain colons: ${timestamp}`);
|
||||
// Should match YYYY-MM-DDTHH-MM-SS pattern
|
||||
assert.match(timestamp, /^\d{4}-\d{2}-\d{2}T\d{2}-\d{2}-\d{2}$/);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── listRuns ────────────────────────────────────────────────────────────
|
||||
|
||||
describe("listRuns", () => {
|
||||
it("returns empty array when no runs exist", () => {
|
||||
const base = makeTmpBase();
|
||||
const runs = listRuns(base);
|
||||
assert.deepStrictEqual(runs, []);
|
||||
});
|
||||
|
||||
it("returns correct metadata for existing runs", () => {
|
||||
const base = makeTmpBase();
|
||||
writeDefinition(base, "test-workflow", SIMPLE_DEF);
|
||||
|
||||
// Create a run
|
||||
const runDir = createRun(base, "test-workflow");
|
||||
|
||||
const runs = listRuns(base);
|
||||
assert.equal(runs.length, 1);
|
||||
assert.equal(runs[0].name, "test-workflow");
|
||||
assert.equal(runs[0].runDir, runDir);
|
||||
assert.equal(runs[0].steps.total, 2);
|
||||
assert.equal(runs[0].steps.completed, 0);
|
||||
assert.equal(runs[0].steps.pending, 2);
|
||||
assert.equal(runs[0].steps.active, 0);
|
||||
assert.equal(runs[0].status, "pending");
|
||||
});
|
||||
|
||||
it("filters by definition name", () => {
|
||||
const base = makeTmpBase();
|
||||
writeDefinition(base, "test-workflow", SIMPLE_DEF);
|
||||
writeDefinition(base, "param-workflow", PARAMETERIZED_DEF);
|
||||
|
||||
createRun(base, "test-workflow");
|
||||
createRun(base, "param-workflow", { target: "app" });
|
||||
|
||||
const allRuns = listRuns(base);
|
||||
assert.equal(allRuns.length, 2);
|
||||
|
||||
const filtered = listRuns(base, "test-workflow");
|
||||
assert.equal(filtered.length, 1);
|
||||
assert.equal(filtered[0].name, "test-workflow");
|
||||
});
|
||||
|
||||
it("returns newest-first within same definition", () => {
|
||||
const base = makeTmpBase();
|
||||
writeDefinition(base, "test-workflow", SIMPLE_DEF);
|
||||
|
||||
const run1 = createRun(base, "test-workflow");
|
||||
// Ensure different timestamp by creating run dir manually with earlier timestamp
|
||||
const earlyDir = join(base, ".gsd", "workflow-runs", "test-workflow", "2020-01-01T00-00-00");
|
||||
mkdirSync(earlyDir, { recursive: true });
|
||||
// Copy GRAPH.yaml to make it a valid run
|
||||
const graphContent = readFileSync(join(run1, "GRAPH.yaml"), "utf-8");
|
||||
writeFileSync(join(earlyDir, "GRAPH.yaml"), graphContent, "utf-8");
|
||||
|
||||
const runs = listRuns(base, "test-workflow");
|
||||
assert.equal(runs.length, 2);
|
||||
// First should be the newer one (the one we just created)
|
||||
assert.ok(runs[0].timestamp > runs[1].timestamp, "should be sorted newest-first");
|
||||
});
|
||||
});
|
||||
38
src/resources/extensions/gsd/workflow-engine.ts
Normal file
38
src/resources/extensions/gsd/workflow-engine.ts
Normal file
|
|
@ -0,0 +1,38 @@
|
|||
/**
|
||||
* workflow-engine.ts — WorkflowEngine interface.
|
||||
*
|
||||
* Defines the contract every engine implementation must satisfy.
|
||||
* Imports only from the leaf-node engine-types.
|
||||
*/
|
||||
|
||||
import type {
|
||||
EngineState,
|
||||
EngineDispatchAction,
|
||||
CompletedStep,
|
||||
ReconcileResult,
|
||||
DisplayMetadata,
|
||||
} from "./engine-types.js";
|
||||
|
||||
/** A pluggable workflow engine that drives the auto-loop. */
|
||||
export interface WorkflowEngine {
|
||||
/** Unique identifier for this engine (e.g. "dev", "custom"). */
|
||||
readonly engineId: string;
|
||||
|
||||
/** Derive the current engine state from the project on disk. */
|
||||
deriveState(basePath: string): Promise<EngineState>;
|
||||
|
||||
/** Decide what the loop should do next given current state. */
|
||||
resolveDispatch(
|
||||
state: EngineState,
|
||||
context: { basePath: string },
|
||||
): Promise<EngineDispatchAction>;
|
||||
|
||||
/** Reconcile state after a step has been executed. */
|
||||
reconcile(
|
||||
state: EngineState,
|
||||
completedStep: CompletedStep,
|
||||
): Promise<ReconcileResult>;
|
||||
|
||||
/** Return UI-facing metadata for progress display. */
|
||||
getDisplayMetadata(state: EngineState): DisplayMetadata;
|
||||
}
|
||||
103
src/resources/skills/create-workflow/SKILL.md
Normal file
103
src/resources/skills/create-workflow/SKILL.md
Normal file
|
|
@ -0,0 +1,103 @@
|
|||
---
|
||||
name: create-workflow
|
||||
description: Conversational guide for creating valid YAML workflow definitions. Use when asked to "create a workflow", "new workflow definition", "build a workflow", "workflow YAML", "define workflow steps", or "workflow from template".
|
||||
---
|
||||
|
||||
<essential_principles>
|
||||
You are a workflow definition author. You help users create valid V1 YAML workflow definitions that the GSD workflow engine can execute.
|
||||
|
||||
**V1 Schema Basics:**
|
||||
|
||||
- Every definition requires `version: 1`, a non-empty `name`, and at least one step in `steps[]`.
|
||||
- Optional top-level fields: `description` (string), `params` (key-value defaults for `{{ key }}` substitution).
|
||||
- Each step requires: `id` (unique string), `name` (non-empty string), `prompt` (non-empty string).
|
||||
- Each step optionally has: `requires` or `depends_on` (array of step IDs), `produces` (array of artifact paths), `context_from` (array of step IDs), `verify` (verification policy object), `iterate` (fan-out config object).
|
||||
- YAML uses **snake_case** keys: `depends_on`, `context_from`. The engine converts to camelCase internally.
|
||||
|
||||
**Validation Rules:**
|
||||
|
||||
- Step IDs must be unique across the workflow.
|
||||
- Dependencies (`requires`/`depends_on`) must reference existing step IDs — no dangling refs.
|
||||
- A step cannot depend on itself.
|
||||
- The dependency graph must be acyclic (no circular dependencies).
|
||||
- `produces` paths must not contain `..` (path traversal rejected).
|
||||
- `iterate.source` must not contain `..` (path traversal rejected).
|
||||
- `iterate.pattern` must be a valid regex with at least one capture group.
|
||||
|
||||
**Four Verification Policies:**
|
||||
|
||||
1. `content-heuristic` — Checks artifact content. Optional: `minSize` (number), `pattern` (string).
|
||||
2. `shell-command` — Runs a shell command. Required: `command` (non-empty string).
|
||||
3. `prompt-verify` — Asks an LLM to verify. Required: `prompt` (non-empty string).
|
||||
4. `human-review` — Pauses for human approval. No extra fields required.
|
||||
|
||||
**Parameter Substitution:**
|
||||
|
||||
- Define defaults in top-level `params: { key: "default_value" }`.
|
||||
- Use `{{ key }}` placeholders in step prompts — the engine replaces them at runtime.
|
||||
- CLI overrides take precedence over definition defaults.
|
||||
- Parameter values must not contain `..` (path traversal guard).
|
||||
- Any unresolved `{{ key }}` after substitution causes an error.
|
||||
|
||||
**Path Traversal Guard:**
|
||||
|
||||
- The engine rejects any `produces` path or `iterate.source` containing `..`.
|
||||
- Parameter values are also checked for `..` during substitution.
|
||||
|
||||
**Output Location:**
|
||||
|
||||
- Finished definitions go in `.gsd/workflow-defs/<name>.yaml`.
|
||||
- After writing, tell the user to validate with `/gsd workflow validate <name>`.
|
||||
</essential_principles>
|
||||
|
||||
<routing>
|
||||
Determine the user's intent and route to the appropriate workflow:
|
||||
|
||||
**"I want to create a workflow from scratch" / "new workflow" / "build a workflow":**
|
||||
→ Read `workflows/create-from-scratch.md` and follow it.
|
||||
|
||||
**"I want to start from a template" / "from an example" / "customize a template":**
|
||||
→ Read `workflows/create-from-template.md` and follow it.
|
||||
|
||||
**"Help me understand the schema" / "what fields are available?":**
|
||||
→ Read `references/yaml-schema-v1.md` and explain the relevant parts.
|
||||
|
||||
**"How does verification work?" / "verify policies":**
|
||||
→ Read `references/verification-policies.md` and explain.
|
||||
|
||||
**"How do I use context_from / iterate / params?":**
|
||||
→ Read `references/feature-patterns.md` and explain the relevant feature.
|
||||
|
||||
**If intent is unclear, ask one clarifying question:**
|
||||
- "Do you want to create a workflow from scratch, or start from an existing template?"
|
||||
- Then route based on the answer.
|
||||
</routing>
|
||||
|
||||
<reference_index>
|
||||
Read these files when you need detailed schema knowledge during workflow authoring:
|
||||
|
||||
- `references/yaml-schema-v1.md` — Complete field-by-field V1 schema reference. Read when you need to explain any field's type, constraints, or defaults.
|
||||
- `references/verification-policies.md` — All four verify policies with complete YAML examples. Read when helping the user choose or configure verification for a step.
|
||||
- `references/feature-patterns.md` — Usage patterns for `context_from`, `iterate`, and `params` with complete YAML examples. Read when the user wants context chaining, fan-out iteration, or parameterized workflows.
|
||||
</reference_index>
|
||||
|
||||
<templates_index>
|
||||
Available templates in `templates/`:
|
||||
|
||||
- `workflow-definition.yaml` — Blank scaffold with all fields shown as comments. Copy and fill for a quick start.
|
||||
- `blog-post-pipeline.yaml` — Linear chain with params and content-heuristic verification.
|
||||
- `code-audit.yaml` — Iterate-based fan-out with shell-command verification.
|
||||
- `release-checklist.yaml` — Diamond dependency graph with human-review verification.
|
||||
</templates_index>
|
||||
|
||||
<output_conventions>
|
||||
When assembling the final YAML:
|
||||
|
||||
1. Use 2-space indentation consistently.
|
||||
2. Quote string values that contain special YAML characters (`:`, `{`, `}`, `[`, `]`, `#`).
|
||||
3. Always include `version: 1` as the first field.
|
||||
4. Order top-level fields: `version`, `name`, `description`, `params`, `steps`.
|
||||
5. Order step fields: `id`, `name`, `prompt`, `requires`, `produces`, `context_from`, `verify`, `iterate`.
|
||||
6. Write the file to `.gsd/workflow-defs/<name>.yaml`.
|
||||
7. After writing, tell the user: "Run `/gsd workflow validate <name>` to check the definition."
|
||||
</output_conventions>
|
||||
|
|
@ -0,0 +1,128 @@
|
|||
<feature_patterns>
|
||||
Advanced workflow features: `context_from`, `iterate`, and `params`. Each section includes a complete YAML example.
|
||||
|
||||
**Feature 1: `context_from` — Context Chaining**
|
||||
|
||||
Injects artifacts from prior steps as context when the current step runs. The value is an array of step IDs.
|
||||
|
||||
```yaml
|
||||
version: 1
|
||||
name: research-and-synthesize
|
||||
steps:
|
||||
- id: gather
|
||||
name: Gather sources
|
||||
prompt: "Find and summarize the top 5 sources on the topic."
|
||||
produces:
|
||||
- sources.md
|
||||
|
||||
- id: analyze
|
||||
name: Analyze sources
|
||||
prompt: "Analyze the gathered sources for key themes."
|
||||
requires:
|
||||
- gather
|
||||
context_from:
|
||||
- gather
|
||||
produces:
|
||||
- analysis.md
|
||||
|
||||
- id: synthesize
|
||||
name: Write synthesis
|
||||
prompt: "Synthesize the analysis into a coherent report."
|
||||
requires:
|
||||
- analyze
|
||||
context_from:
|
||||
- gather
|
||||
- analyze
|
||||
produces:
|
||||
- report.md
|
||||
```
|
||||
|
||||
How it works:
|
||||
- `context_from: [gather]` means the engine includes artifacts from the `gather` step when executing `analyze`.
|
||||
- You can reference multiple prior steps: `context_from: [gather, analyze]`.
|
||||
- The referenced steps must exist in the workflow (they are validated as step IDs).
|
||||
- `context_from` does not imply a dependency — if you want the step to wait, also add the ID to `requires`.
|
||||
|
||||
**Feature 2: `iterate` — Fan-Out Iteration**
|
||||
|
||||
Reads an artifact, applies a regex pattern, and creates one sub-execution per match. The capture group extracts the iteration variable.
|
||||
|
||||
```yaml
|
||||
version: 1
|
||||
name: file-by-file-review
|
||||
steps:
|
||||
- id: inventory
|
||||
name: List files to review
|
||||
prompt: "List all TypeScript files in src/ that need review, one per line."
|
||||
produces:
|
||||
- file-list.txt
|
||||
|
||||
- id: review
|
||||
name: Review each file
|
||||
prompt: "Review the file for code quality issues."
|
||||
requires:
|
||||
- inventory
|
||||
iterate:
|
||||
source: file-list.txt
|
||||
pattern: "^(.+\\.ts)$"
|
||||
produces:
|
||||
- reviews/
|
||||
```
|
||||
|
||||
How it works:
|
||||
- `source`: Path to an artifact (relative to the run directory). Must not contain `..`.
|
||||
- `pattern`: A regex string applied with the global flag. Must contain at least one capture group `(...)`.
|
||||
- The engine reads the source artifact, applies the pattern, and creates one execution per match.
|
||||
- Each capture group match becomes available as the iteration variable.
|
||||
- The regex is validated at definition-load time — invalid regex or missing capture groups are rejected.
|
||||
|
||||
Pattern requirements:
|
||||
- Must be a valid JavaScript regex.
|
||||
- Must contain at least one non-lookahead capture group: `(...)` not `(?:...)`.
|
||||
- Example valid patterns: `^(.+)$`, `- (.+\.ts)`, `\[(.+?)\]`.
|
||||
|
||||
**Feature 3: `params` — Parameterized Workflows**
|
||||
|
||||
Define default parameter values at the top level. Use `{{ key }}` placeholders in step prompts. CLI overrides take precedence.
|
||||
|
||||
```yaml
|
||||
version: 1
|
||||
name: blog-post
|
||||
description: Generate a blog post on a configurable topic.
|
||||
params:
|
||||
topic: "AI in healthcare"
|
||||
audience: "technical professionals"
|
||||
word_count: "1500"
|
||||
steps:
|
||||
- id: outline
|
||||
name: Create outline
|
||||
prompt: "Create a detailed outline for a blog post about {{ topic }} targeting {{ audience }}."
|
||||
produces:
|
||||
- outline.md
|
||||
|
||||
- id: draft
|
||||
name: Write draft
|
||||
prompt: "Write a {{ word_count }}-word blog post about {{ topic }} for {{ audience }} based on the outline."
|
||||
requires:
|
||||
- outline
|
||||
context_from:
|
||||
- outline
|
||||
produces:
|
||||
- draft.md
|
||||
verify:
|
||||
policy: content-heuristic
|
||||
minSize: 500
|
||||
```
|
||||
|
||||
How it works:
|
||||
- `params` is a top-level object mapping string keys to string default values.
|
||||
- `{{ key }}` in any step prompt is replaced with the corresponding param value.
|
||||
- Merge order: definition `params` (defaults) ← CLI overrides (win).
|
||||
- After substitution, any remaining `{{ key }}` that has no value causes an error — all placeholders must resolve.
|
||||
- Parameter values must not contain `..` (path traversal guard).
|
||||
- Keys in `{{ }}` match `\w+` (letters, digits, underscore).
|
||||
|
||||
Common usage:
|
||||
- Make workflows reusable across different topics, projects, or configurations.
|
||||
- Users override defaults at run time: `/gsd workflow run blog-post topic="Rust performance"`.
|
||||
</feature_patterns>
|
||||
|
|
@ -0,0 +1,76 @@
|
|||
<verification_policies>
|
||||
The `verify` field on a step defines how the engine validates the step's output. It must be an object with a `policy` field set to one of four values.
|
||||
|
||||
**Policy 1: `content-heuristic`**
|
||||
|
||||
Checks the artifact content against size and pattern criteria. All sub-fields are optional.
|
||||
|
||||
```yaml
|
||||
verify:
|
||||
policy: content-heuristic
|
||||
minSize: 500 # optional — minimum byte size of the artifact
|
||||
pattern: "## Summary" # optional — string pattern that must appear in the artifact
|
||||
```
|
||||
|
||||
Fields:
|
||||
- `policy`: `"content-heuristic"` (required)
|
||||
- `minSize`: number (optional) — minimum artifact size in bytes
|
||||
- `pattern`: string (optional) — text pattern to match in the artifact content
|
||||
|
||||
Use when: You want a lightweight sanity check that the step produced substantive output.
|
||||
|
||||
**Policy 2: `shell-command`**
|
||||
|
||||
Runs a shell command to verify the step's output. The command's exit code determines pass/fail.
|
||||
|
||||
```yaml
|
||||
verify:
|
||||
policy: shell-command
|
||||
command: "test -f output/report.md && wc -l output/report.md | awk '{print ($1 > 10)}'"
|
||||
```
|
||||
|
||||
Fields:
|
||||
- `policy`: `"shell-command"` (required)
|
||||
- `command`: string (required, non-empty) — shell command to execute
|
||||
|
||||
Use when: You need programmatic verification — file existence, test suite execution, linting, compilation, etc.
|
||||
|
||||
**Policy 3: `prompt-verify`**
|
||||
|
||||
Sends a verification prompt to an LLM to evaluate the step's output.
|
||||
|
||||
```yaml
|
||||
verify:
|
||||
policy: prompt-verify
|
||||
prompt: "Review the generated API documentation. Does it cover all endpoints with request/response examples? Answer PASS or FAIL with reasoning."
|
||||
```
|
||||
|
||||
Fields:
|
||||
- `policy`: `"prompt-verify"` (required)
|
||||
- `prompt`: string (required, non-empty) — the verification prompt sent to the LLM
|
||||
|
||||
Use when: Verification requires judgment that can't be expressed as a shell command — quality assessment, completeness review, style conformance.
|
||||
|
||||
**Policy 4: `human-review`**
|
||||
|
||||
Pauses execution and waits for a human to approve or reject the step's output.
|
||||
|
||||
```yaml
|
||||
verify:
|
||||
policy: human-review
|
||||
```
|
||||
|
||||
Fields:
|
||||
- `policy`: `"human-review"` (required)
|
||||
- No additional fields.
|
||||
|
||||
Use when: The step produces work that requires human judgment — design decisions, public-facing content, security-sensitive changes.
|
||||
|
||||
**Validation Details:**
|
||||
|
||||
The engine validates the `verify` object at definition-load time:
|
||||
- `policy` must be one of the four strings above. Any other value is rejected.
|
||||
- `shell-command` requires a non-empty `command` field. Missing or empty `command` is rejected.
|
||||
- `prompt-verify` requires a non-empty `prompt` field. Missing or empty `prompt` is rejected.
|
||||
- `content-heuristic` and `human-review` have no required sub-fields beyond `policy`.
|
||||
</verification_policies>
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
<schema_reference>
|
||||
V1 Workflow Definition Schema — complete field-by-field reference extracted from `definition-loader.ts`.
|
||||
|
||||
**Top-Level Fields:**
|
||||
|
||||
| Field | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `version` | number | **yes** | — | Must be exactly `1`. |
|
||||
| `name` | string | **yes** | — | Non-empty workflow name. |
|
||||
| `description` | string | no | `undefined` | Optional human-readable description. |
|
||||
| `params` | object | no | `undefined` | Key-value map of parameter defaults. Values must be strings. Used for `{{ key }}` substitution in step prompts. |
|
||||
| `steps` | array | **yes** | — | Non-empty array of step objects. |
|
||||
|
||||
**Step Fields:**
|
||||
|
||||
| Field | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `id` | string | **yes** | — | Unique identifier within the workflow. Must be non-empty. No two steps can share an ID. |
|
||||
| `name` | string | **yes** | — | Human-readable step name. Must be non-empty. |
|
||||
| `prompt` | string | **yes** | — | The prompt dispatched for this step. Must be non-empty. Supports `{{ key }}` parameter placeholders. |
|
||||
| `requires` | string[] | no | `[]` | IDs of steps that must complete before this step runs. Alternative name: `depends_on`. |
|
||||
| `depends_on` | string[] | no | `[]` | Alias for `requires`. If both are present, `requires` takes precedence. |
|
||||
| `produces` | string[] | no | `[]` | Artifact paths produced by this step (relative to run directory). Paths must not contain `..`. |
|
||||
| `context_from` | string[] | no | `undefined` | Step IDs whose artifacts are injected as context when this step runs. |
|
||||
| `verify` | object | no | `undefined` | Verification policy for this step. See verification-policies.md for details. |
|
||||
| `iterate` | object | no | `undefined` | Fan-out iteration config. See feature-patterns.md for details. |
|
||||
|
||||
**Validation Rules:**
|
||||
|
||||
1. `version` must be exactly `1` (number, not string).
|
||||
2. `name` must be a non-empty string.
|
||||
3. `steps` must be a non-empty array of objects.
|
||||
4. Each step must have non-empty `id`, `name`, and `prompt`.
|
||||
5. Step IDs must be unique — duplicates are rejected.
|
||||
6. Dependencies must reference existing step IDs — dangling references are rejected.
|
||||
7. A step cannot depend on itself.
|
||||
8. The dependency graph must be acyclic — cycles are detected and rejected.
|
||||
9. `produces` paths and `iterate.source` must not contain `..` (path traversal guard).
|
||||
10. Unknown top-level or step-level fields are silently accepted for forward compatibility.
|
||||
|
||||
**Type Notes:**
|
||||
|
||||
- `requires` / `depends_on`: The engine reads `requires` first. If absent, it falls back to `depends_on`. Both must be arrays of strings if present.
|
||||
- `params` values must be strings. During substitution, each `{{ key }}` in a step prompt is replaced with the merged param value (definition defaults ← CLI overrides). Any unresolved placeholder after substitution causes an error.
|
||||
- Parameter values and `produces` paths are guarded against path traversal (`..` is rejected).
|
||||
</schema_reference>
|
||||
|
|
@ -0,0 +1,60 @@
|
|||
# Example: Blog Post Pipeline
|
||||
# Demonstrates: context chaining (context_from), parameters (params),
|
||||
# and content-heuristic verification across a 3-step linear chain.
|
||||
|
||||
version: 1
|
||||
name: blog-post-pipeline
|
||||
description: >-
|
||||
Research a topic, create an outline, and draft a blog post.
|
||||
Uses params for topic/audience, context_from for chaining,
|
||||
and content-heuristic verification at every step.
|
||||
|
||||
params:
|
||||
topic: "AI"
|
||||
audience: "developers"
|
||||
|
||||
steps:
|
||||
- id: research
|
||||
name: Research the topic
|
||||
prompt: >-
|
||||
Research the topic "{{ topic }}" for an audience of {{ audience }}.
|
||||
Write detailed findings including key trends, important facts,
|
||||
and relevant examples. Save the results to research.md.
|
||||
requires: []
|
||||
produces:
|
||||
- research.md
|
||||
verify:
|
||||
policy: content-heuristic
|
||||
minSize: 200
|
||||
|
||||
- id: outline
|
||||
name: Create an outline
|
||||
prompt: >-
|
||||
Using the research findings, create a structured blog post outline
|
||||
targeting {{ audience }}. Include section headings, key points
|
||||
for each section, and a logical flow. Save to outline.md.
|
||||
requires:
|
||||
- research
|
||||
context_from:
|
||||
- research
|
||||
produces:
|
||||
- outline.md
|
||||
verify:
|
||||
policy: content-heuristic
|
||||
|
||||
- id: draft
|
||||
name: Write the draft
|
||||
prompt: >-
|
||||
Write a complete blog post draft following the outline.
|
||||
The post should be engaging for {{ audience }}, cover all
|
||||
outlined sections, and include a compelling introduction
|
||||
and conclusion. Save to draft.md.
|
||||
requires:
|
||||
- outline
|
||||
context_from:
|
||||
- outline
|
||||
produces:
|
||||
- draft.md
|
||||
verify:
|
||||
policy: content-heuristic
|
||||
minSize: 500
|
||||
|
|
@ -0,0 +1,60 @@
|
|||
# Example: Code Audit
|
||||
# Demonstrates: iterate (fan-out over file list), shell-command verification,
|
||||
# prompt-verify, and content-heuristic across a 3-step workflow.
|
||||
|
||||
version: 1
|
||||
name: code-audit
|
||||
description: >-
|
||||
Inventory TypeScript files, audit each one for quality issues,
|
||||
and produce a consolidated report. Uses iterate to fan-out
|
||||
audits across discovered files.
|
||||
|
||||
steps:
|
||||
- id: inventory
|
||||
name: Inventory source files
|
||||
prompt: >-
|
||||
List all TypeScript source files in the project that should
|
||||
be audited. Write one file path per line as a Markdown list
|
||||
item (e.g. "- src/index.ts"). Save the list to inventory.md.
|
||||
requires: []
|
||||
produces:
|
||||
- inventory.md
|
||||
verify:
|
||||
policy: content-heuristic
|
||||
|
||||
- id: audit-file
|
||||
name: Audit individual file
|
||||
prompt: >-
|
||||
Review the file for code quality issues including unused imports,
|
||||
missing error handling, type safety gaps, and potential bugs.
|
||||
Document each finding with the line number and a recommended fix.
|
||||
Append results to audit-results.md.
|
||||
requires:
|
||||
- inventory
|
||||
context_from:
|
||||
- inventory
|
||||
produces:
|
||||
- audit-results.md
|
||||
iterate:
|
||||
source: inventory.md
|
||||
pattern: "^- (.+\\.ts)$"
|
||||
verify:
|
||||
policy: shell-command
|
||||
command: "test -f audit-results.md"
|
||||
|
||||
- id: report
|
||||
name: Compile audit report
|
||||
prompt: >-
|
||||
Compile all individual file audit results into a single
|
||||
comprehensive audit report. Group findings by severity
|
||||
(critical, warning, info), include summary statistics,
|
||||
and provide prioritized recommendations. Save to audit-report.md.
|
||||
requires:
|
||||
- audit-file
|
||||
context_from:
|
||||
- audit-file
|
||||
produces:
|
||||
- audit-report.md
|
||||
verify:
|
||||
policy: prompt-verify
|
||||
prompt: "Does the report cover all audited files and group findings by severity? Answer PASS or FAIL."
|
||||
|
|
@ -0,0 +1,66 @@
|
|||
# Example: Release Checklist
|
||||
# Demonstrates: diamond dependency pattern (version-bump and test-suite
|
||||
# both depend on changelog, publish depends on both), shell-command
|
||||
# verification, and human-review policy.
|
||||
|
||||
version: 1
|
||||
name: release-checklist
|
||||
description: >-
|
||||
Prepare a software release: generate changelog, bump version,
|
||||
run tests, and publish release notes. Uses a diamond dependency
|
||||
pattern where publish waits for both version-bump and test-suite.
|
||||
|
||||
steps:
|
||||
- id: changelog
|
||||
name: Generate changelog
|
||||
prompt: >-
|
||||
Review recent commits and generate a changelog draft.
|
||||
Group changes by category (features, fixes, breaking changes).
|
||||
Follow Keep a Changelog format. Save to CHANGELOG-draft.md.
|
||||
requires: []
|
||||
produces:
|
||||
- CHANGELOG-draft.md
|
||||
verify:
|
||||
policy: content-heuristic
|
||||
|
||||
- id: version-bump
|
||||
name: Bump version number
|
||||
prompt: >-
|
||||
Based on the changelog, determine the appropriate semver bump
|
||||
(major, minor, or patch). Write the new version number to
|
||||
version.txt as a single line (e.g. "1.2.3").
|
||||
requires:
|
||||
- changelog
|
||||
produces:
|
||||
- version.txt
|
||||
verify:
|
||||
policy: shell-command
|
||||
command: "grep -E '^[0-9]+\\.[0-9]+\\.[0-9]+$' version.txt"
|
||||
|
||||
- id: test-suite
|
||||
name: Run test suite
|
||||
prompt: >-
|
||||
Run the full test suite and capture results. Include test
|
||||
counts (passed, failed, skipped), execution time, and any
|
||||
failure details. Save results to test-results.md.
|
||||
requires:
|
||||
- changelog
|
||||
produces:
|
||||
- test-results.md
|
||||
verify:
|
||||
policy: shell-command
|
||||
command: "test -f test-results.md"
|
||||
|
||||
- id: publish
|
||||
name: Publish release
|
||||
prompt: >-
|
||||
Compile the final release notes combining the changelog,
|
||||
version number, and test results. Format for GitHub Releases
|
||||
with proper Markdown. Save to release-notes.md.
|
||||
requires:
|
||||
- version-bump
|
||||
- test-suite
|
||||
produces:
|
||||
- release-notes.md
|
||||
verify:
|
||||
policy: human-review
|
||||
|
|
@ -0,0 +1,32 @@
|
|||
version: 1
|
||||
name: my-workflow
|
||||
# description: A brief description of what this workflow accomplishes.
|
||||
|
||||
# params:
|
||||
# topic: "default value"
|
||||
# target: "another default"
|
||||
|
||||
steps:
|
||||
- id: step-one
|
||||
name: First step
|
||||
prompt: "Describe what this step should accomplish."
|
||||
# requires: []
|
||||
produces:
|
||||
- output.md
|
||||
# context_from:
|
||||
# - some-prior-step
|
||||
# verify:
|
||||
# policy: content-heuristic
|
||||
# minSize: 100
|
||||
# pattern: "## Summary"
|
||||
# verify:
|
||||
# policy: shell-command
|
||||
# command: "test -f output.md"
|
||||
# verify:
|
||||
# policy: prompt-verify
|
||||
# prompt: "Does the output meet quality standards? Answer PASS or FAIL."
|
||||
# verify:
|
||||
# policy: human-review
|
||||
# iterate:
|
||||
# source: file-list.txt
|
||||
# pattern: "^(.+)$"
|
||||
|
|
@ -0,0 +1,104 @@
|
|||
<workflow>
|
||||
Guide the user through creating a workflow definition from scratch. Follow these phases in order.
|
||||
|
||||
<required_reading>
|
||||
Before starting, read these references so you can answer schema questions accurately:
|
||||
- `../references/yaml-schema-v1.md` — all fields, types, and constraints
|
||||
- `../references/verification-policies.md` — the four verify policies
|
||||
- `../references/feature-patterns.md` — context_from, iterate, params patterns
|
||||
</required_reading>
|
||||
|
||||
<phase name="purpose">
|
||||
Ask the user:
|
||||
- "What does this workflow accomplish? Give me a one-sentence description."
|
||||
- "What should the workflow be named?" (suggest a kebab-case name based on their description)
|
||||
|
||||
Record: `name`, `description`.
|
||||
</phase>
|
||||
|
||||
<phase name="steps">
|
||||
Ask the user:
|
||||
- "What are the main steps? List them in order. For each step, give a short name and what it should do."
|
||||
|
||||
For each step the user describes:
|
||||
1. Generate an `id` (lowercase, short, descriptive — e.g., `gather`, `analyze`, `write-draft`).
|
||||
2. Confirm the `name` (human-readable).
|
||||
3. Write the `prompt` — this is the instruction the engine dispatches. It should be detailed enough for an LLM to execute independently.
|
||||
4. Ask: "Does this step depend on any previous steps?" → populate `requires`.
|
||||
5. Ask: "What files or artifacts does this step produce?" → populate `produces`.
|
||||
</phase>
|
||||
|
||||
<phase name="verification">
|
||||
For each step, ask:
|
||||
- "How should we verify this step's output?"
|
||||
- **No verification needed** → omit `verify`
|
||||
- **Check that the output exists and has content** → `content-heuristic`
|
||||
- **Run a shell command to validate** → `shell-command` (ask for the command)
|
||||
- **Have an LLM review the output** → `prompt-verify` (ask for the verification prompt)
|
||||
- **Require human approval** → `human-review`
|
||||
|
||||
Refer to `../references/verification-policies.md` for the exact YAML structure of each policy.
|
||||
</phase>
|
||||
|
||||
<phase name="context_chaining">
|
||||
Ask:
|
||||
- "Should any step receive artifacts from earlier steps as context?"
|
||||
|
||||
If yes, for each such step:
|
||||
- Ask which prior steps to pull context from → populate `context_from`.
|
||||
- Remind the user: `context_from` does not imply a dependency. If the step should wait for the context source, it must also list it in `requires`.
|
||||
</phase>
|
||||
|
||||
<phase name="parameters">
|
||||
Ask:
|
||||
- "Should any values in this workflow be configurable at run time? (e.g., a topic, a target directory, a language)"
|
||||
|
||||
If yes:
|
||||
- Define each parameter with a default value in top-level `params`.
|
||||
- Replace hardcoded values in step prompts with `{{ key }}` placeholders.
|
||||
- Explain: "Users can override these when running the workflow."
|
||||
</phase>
|
||||
|
||||
<phase name="iteration">
|
||||
Ask:
|
||||
- "Does any step need to fan out — running once per item in a list? (e.g., review each file, process each section)"
|
||||
|
||||
If yes:
|
||||
- Identify the source artifact (the list to iterate over).
|
||||
- Define the `pattern` regex with a capture group to extract each item.
|
||||
- Set `iterate.source` and `iterate.pattern` on the step.
|
||||
- Refer to `../references/feature-patterns.md` for examples.
|
||||
</phase>
|
||||
|
||||
<phase name="assemble">
|
||||
Assemble the complete YAML definition:
|
||||
|
||||
1. Start with `version: 1`.
|
||||
2. Add `name` and `description`.
|
||||
3. Add `params` if any were defined.
|
||||
4. Add `steps` in dependency order.
|
||||
5. For each step, include all configured fields in this order: `id`, `name`, `prompt`, `requires`, `produces`, `context_from`, `verify`, `iterate`.
|
||||
6. Use 2-space indentation.
|
||||
|
||||
Show the complete YAML to the user for review.
|
||||
|
||||
Ask: "Does this look correct? Any changes?"
|
||||
|
||||
Apply any requested changes.
|
||||
</phase>
|
||||
|
||||
<phase name="write">
|
||||
Write the file to `.gsd/workflow-defs/<name>.yaml`.
|
||||
|
||||
Tell the user:
|
||||
- "Definition saved to `.gsd/workflow-defs/<name>.yaml`."
|
||||
- "Run `/gsd workflow validate <name>` to check it against the schema."
|
||||
- "Run `/gsd workflow run <name>` to execute it."
|
||||
</phase>
|
||||
|
||||
<success_criteria>
|
||||
- A valid YAML file exists at `.gsd/workflow-defs/<name>.yaml`
|
||||
- The definition passes `validateDefinition()` from `definition-loader.ts`
|
||||
- The user has reviewed and approved the definition
|
||||
</success_criteria>
|
||||
</workflow>
|
||||
|
|
@ -0,0 +1,72 @@
|
|||
<workflow>
|
||||
Guide the user through creating a workflow definition by customizing an existing template.
|
||||
|
||||
<required_reading>
|
||||
Before starting, read these references for schema details:
|
||||
- `../references/yaml-schema-v1.md` — all fields, types, and constraints
|
||||
- `../references/verification-policies.md` — the four verify policies
|
||||
- `../references/feature-patterns.md` — context_from, iterate, params patterns
|
||||
</required_reading>
|
||||
|
||||
<phase name="choose_template">
|
||||
List the available templates in `templates/`:
|
||||
|
||||
1. **workflow-definition.yaml** — Blank scaffold with all fields shown as comments. Best for: starting with the full schema visible.
|
||||
2. **blog-post-pipeline.yaml** — Linear 3-step chain with `params` (topic, audience) and `content-heuristic` verification. Best for: workflows with sequential steps and configurable inputs.
|
||||
3. **code-audit.yaml** — 3 steps using `iterate` to fan out over a file list, with `shell-command` verification. Best for: workflows that process each item in a list.
|
||||
4. **release-checklist.yaml** — 4 steps with diamond dependencies and `human-review` verification. Best for: workflows with branching/merging dependency graphs.
|
||||
|
||||
Ask: "Which template would you like to start from?"
|
||||
|
||||
Read the chosen template file from `templates/`.
|
||||
</phase>
|
||||
|
||||
<phase name="understand">
|
||||
Show the user the template contents and explain:
|
||||
- What each step does
|
||||
- How the dependencies flow
|
||||
- What features it demonstrates (params, context_from, iterate, verify)
|
||||
|
||||
Ask: "What do you want this workflow to do instead? I'll help you adapt the template."
|
||||
</phase>
|
||||
|
||||
<phase name="customize">
|
||||
Based on the user's goal, walk through customization:
|
||||
|
||||
1. **Rename**: Change `name` and `description` to match the new purpose.
|
||||
2. **Adjust steps**: Add, remove, or modify steps. For each change:
|
||||
- Update `id` and `name` to reflect the new purpose.
|
||||
- Rewrite `prompt` for the new task.
|
||||
- Update `requires` to reflect new dependency order.
|
||||
- Update `produces` for new artifact paths.
|
||||
3. **Modify params**: Add or remove parameters. Update `{{ key }}` placeholders in prompts to match.
|
||||
4. **Change verification**: Switch verify policies or adjust policy-specific fields.
|
||||
5. **Add/remove features**: Add `context_from`, `iterate`, or `params` if the new workflow needs them.
|
||||
|
||||
Show the modified YAML after each round of changes. Ask: "Any more changes?"
|
||||
</phase>
|
||||
|
||||
<phase name="validate_and_write">
|
||||
Once the user approves:
|
||||
|
||||
1. Review the YAML for common issues:
|
||||
- All step IDs are unique.
|
||||
- All `requires` references point to existing step IDs.
|
||||
- No circular dependencies.
|
||||
- All `{{ key }}` placeholders have corresponding `params` entries.
|
||||
- No `..` in `produces` paths or `iterate.source`.
|
||||
|
||||
2. Write to `.gsd/workflow-defs/<name>.yaml`.
|
||||
|
||||
3. Tell the user:
|
||||
- "Definition saved to `.gsd/workflow-defs/<name>.yaml`."
|
||||
- "Run `/gsd workflow validate <name>` to check it against the schema."
|
||||
- "Run `/gsd workflow run <name>` to execute it."
|
||||
</phase>
|
||||
|
||||
<success_criteria>
|
||||
- A valid YAML file exists at `.gsd/workflow-defs/<name>.yaml`
|
||||
- The definition is a meaningful customization of the template, not a copy
|
||||
- The user has reviewed and approved the definition
|
||||
</success_criteria>
|
||||
</workflow>
|
||||
Loading…
Add table
Reference in a new issue