feat(plan-milestone): per-slice vision trace (schema v69, ADR-0000 P2)

Restoration of doctrine: plan-milestone now emits a literal milestone.vision
clause per slice (traces_vision_fragment) so validate-milestone has structured
grounds for assessment instead of re-reading the vision through the LLM every
time. Schema v69 adds the column (NULL allowed for legacy rows); the prompt and
plan_milestone tool start requiring it for new slices, rejecting fragments that
do not appear verbatim in milestone.vision. See docs/adr/0000-purpose-to-software-compiler.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Mikael Hugo 2026-05-15 18:44:25 +02:00
parent 3b83f0898b
commit fa657f2523
7 changed files with 330 additions and 10 deletions

View file

@ -81,7 +81,30 @@ Then:
2. {{skillActivation}}
3. Create the roadmap: start with the milestone purpose in the `vision` field, include a structured `productResearch` payload when applicable, then decompose into demoable vertical slices - as many as the work genuinely needs, no more. A simple feature might be 1 slice. Don't decompose for decomposition's sake.
4. Order by risk (high-risk first)
5. Call `plan_milestone` to persist the milestone planning fields, slice rows, and **horizontal checklist** in the DB-backed planning path. Every slice `goal` must state the slice purpose before implementation detail. If the milestone changes how SF is driven, observed, integrated, or automated, keep the axes separate in the roadmap: surface (TUI/CLI/web/editor/machine), protocol (ACP/RPC/stdio/HTTP/wire), output format (text/json/stream-json), run control (manual/assisted/autonomous), and permission profile (restricted/normal/trusted/unrestricted). Do **not** write `{{outputPath}}`, `ROADMAP.md`, or other planning artifacts manually - the planning tool owns roadmap rendering and persistence.
5. Call `plan_milestone` to persist the milestone planning fields, slice rows, and **horizontal checklist** in the DB-backed planning path. Every slice `goal` must state the slice purpose before implementation detail. Every slice must also carry a `tracesVisionFragment` (see **Vision Trace Per Slice** below). If the milestone changes how SF is driven, observed, integrated, or automated, keep the axes separate in the roadmap: surface (TUI/CLI/web/editor/machine), protocol (ACP/RPC/stdio/HTTP/wire), output format (text/json/stream-json), run control (manual/assisted/autonomous), and permission profile (restricted/normal/trusted/unrestricted). Do **not** write `{{outputPath}}`, `ROADMAP.md`, or other planning artifacts manually - the planning tool owns roadmap rendering and persistence.
### Vision Trace Per Slice (ADR-0000 P2)
For each slice you emit, include a `tracesVisionFragment` field whose value is the **literal sentence or clause from `milestone.vision`** that this slice serves. This is the structured "this slice traces back to that purpose" link — it lets `validate-milestone` assess vision coverage from DB rows instead of re-reading prose every time.
Rules:
- The fragment **must appear verbatim** inside `milestone.vision` (whitespace and case differences are normalized; otherwise it is rejected).
- Pick the shortest clause that uniquely identifies the purpose this slice serves — usually a sub-clause, not the whole vision sentence.
- Each slice gets its own fragment. Two slices may share a fragment when they jointly serve the same purpose, but pick the fragment honestly — do not pad every slice with the same sentence to satisfy the validator.
- The fragment is required for every slice, including sketch slices. A roadmap that omits `tracesVisionFragment` on any slice is rejected by `plan_milestone`.
Examples:
- `vision`: "Users can view correlated VM+PG health on a single dashboard, and operators can drill down into per-VM logs."
- `S01.tracesVisionFragment`: `"view correlated VM+PG health"`
- `S02.tracesVisionFragment`: `"operators can drill down into per-VM logs"`
- `vision`: "Provide a CLI that lists active milestones, lets users plan a milestone, and reports slice status."
- `S01.tracesVisionFragment`: `"lists active milestones"`
- `S02.tracesVisionFragment`: `"plan a milestone"`
- `S03.tracesVisionFragment`: `"reports slice status"`
If you cannot find a literal fragment that justifies a slice, the slice is unmoored from the vision — either rewrite the vision to admit the purpose this slice serves, cut the slice, or fold it into a slice that does trace.
6. If planning produced structural decisions (e.g. slice ordering rationale, technology choices, scope exclusions), call `save_decision` for each decision - the tool auto-assigns IDs and regenerates `.sf/DECISIONS.md` automatically.
### productResearch payload

View file

@ -754,6 +754,9 @@ export function rowToSlice(row) {
replan_triggered_at: row["replan_triggered_at"] ?? null,
sketch_scope: row["sketch_scope"] ?? "",
is_sketch: row["is_sketch"] ?? 0,
// ADR-0000 P2 (schema v69): literal milestone.vision clause this slice
// serves. NULL for legacy rows planned before v69; required for new slices.
traces_vision_fragment: row["traces_vision_fragment"] ?? null,
};
}

View file

@ -15,7 +15,7 @@ function defaultQueryTimeout(operation, fallbackValue) {
}
}
const SCHEMA_VERSION = 69;
const SCHEMA_VERSION = 72;
function indexExists(db, name) {
return !!db
.prepare(
@ -980,6 +980,7 @@ export function initSchema(db, fileBacked, options = {}) {
replan_triggered_at TEXT DEFAULT NULL,
is_sketch INTEGER NOT NULL DEFAULT 0, -- SF ADR-011: 1 = slice is a sketch awaiting refine-slice
sketch_scope TEXT NOT NULL DEFAULT '', -- SF ADR-011: 2-3 sentence scope hint from plan-milestone
traces_vision_fragment TEXT, -- ADR-0000 P2 (schema v69): literal milestone.vision clause this slice serves; structured vision trace for validate-milestone (NULL allowed for legacy rows)
PRIMARY KEY (milestone_id, id),
FOREIGN KEY (milestone_id) REFERENCES milestones(id)
)
@ -3576,6 +3577,37 @@ function migrateSchema(db, { currentPath, withQueryTimeout }) {
if (ok) appliedVersion = 69;
}
if (appliedVersion < 72) {
const ok = runMigrationStep("v72", () => {
// Schema v72: per-slice vision trace (ADR-0000 P2, restoration of doctrine).
// Renumbered from the originally-planned v69 because parallel work on
// main used v69 for memory_extraction_attempts.failure_class.
//
// plan-milestone must emit a literal fragment of milestone.vision that each
// slice serves so validate-milestone has structured grounds for assessment
// instead of re-reading the entire vision via the LLM every time. The
// fragment is the canonical "this slice traces back to that purpose" link.
//
// NULL is allowed for legacy rows (slices planned before v72); the
// planning prompt and validation begin requiring it for new slices.
//
// Idempotent ALTER: probe via columnExists because the fresh-DB CREATE
// path already adds the column.
if (!columnExists(db, "slices", "traces_vision_fragment")) {
db.exec(
"ALTER TABLE slices ADD COLUMN traces_vision_fragment TEXT",
);
}
db.prepare(
"INSERT INTO schema_version (version, applied_at) VALUES (:version, :applied_at)",
).run({
":version": 72,
":applied_at": new Date().toISOString(),
});
});
if (ok) appliedVersion = 72;
}
// Post-migration assertion: ensure critical tables created by historical
// migrations are actually present. If a prior migration claimed success but
// the table is missing (e.g., due to a rolled-back transaction that failed

View file

@ -11,12 +11,12 @@ export function insertSlice(s) {
milestone_id, id, title, status, risk, depends, demo, created_at,
goal, success_criteria, proof_level, integration_closure, observability_impact,
adversarial_partner, adversarial_combatant, adversarial_architect, planning_meeting_json, sequence,
is_sketch, sketch_scope
is_sketch, sketch_scope, traces_vision_fragment
) VALUES (
:milestone_id, :id, :title, :status, :risk, :depends, :demo, :created_at,
:goal, :success_criteria, :proof_level, :integration_closure, :observability_impact,
:adversarial_partner, :adversarial_combatant, :adversarial_architect, :planning_meeting_json, :sequence,
:is_sketch, :sketch_scope
:is_sketch, :sketch_scope, :traces_vision_fragment
)
ON CONFLICT (milestone_id, id) DO UPDATE SET
title = CASE WHEN :raw_title IS NOT NULL THEN excluded.title ELSE slices.title END,
@ -35,7 +35,8 @@ export function insertSlice(s) {
planning_meeting_json = CASE WHEN :raw_planning_meeting_json IS NOT NULL THEN excluded.planning_meeting_json ELSE slices.planning_meeting_json END,
sequence = CASE WHEN :raw_sequence IS NOT NULL THEN excluded.sequence ELSE slices.sequence END,
is_sketch = CASE WHEN :raw_is_sketch IS NOT NULL THEN excluded.is_sketch ELSE slices.is_sketch END,
sketch_scope = CASE WHEN :raw_sketch_scope IS NOT NULL THEN excluded.sketch_scope ELSE slices.sketch_scope END`)
sketch_scope = CASE WHEN :raw_sketch_scope IS NOT NULL THEN excluded.sketch_scope ELSE slices.sketch_scope END,
traces_vision_fragment = CASE WHEN :raw_traces_vision_fragment IS NOT NULL THEN excluded.traces_vision_fragment ELSE slices.traces_vision_fragment END`)
.run({
":milestone_id": s.milestoneId,
":id": s.id,
@ -59,6 +60,9 @@ export function insertSlice(s) {
":sequence": s.sequence ?? 0,
":is_sketch": s.isSketch === true ? 1 : 0,
":sketch_scope": s.sketchScope ?? "",
// ADR-0000 P2 (schema v69): NULL allowed for legacy callers that pre-date
// the vision-trace requirement; plan-milestone validates non-empty + in-vision.
":traces_vision_fragment": s.tracesVisionFragment ?? null,
// Raw sentinel params: NULL when caller omitted the field, used in ON CONFLICT guards
":raw_title": s.title ?? null,
":raw_risk": s.risk ?? null,
@ -80,6 +84,7 @@ export function insertSlice(s) {
":raw_sequence": s.sequence ?? null,
":raw_is_sketch": s.isSketch === undefined ? null : s.isSketch ? 1 : 0,
":raw_sketch_scope": s.sketchScope === undefined ? null : s.sketchScope,
":raw_traces_vision_fragment": s.tracesVisionFragment ?? null,
});
insertSliceSpecIfAbsent(s.milestoneId, s.id, s.planning ?? {});
}
@ -145,7 +150,8 @@ export function upsertSlicePlanning(milestoneId, sliceId, planning) {
adversarial_partner = COALESCE(:adversarial_partner, adversarial_partner),
adversarial_combatant = COALESCE(:adversarial_combatant, adversarial_combatant),
adversarial_architect = COALESCE(:adversarial_architect, adversarial_architect),
planning_meeting_json = COALESCE(:planning_meeting_json, planning_meeting_json)
planning_meeting_json = COALESCE(:planning_meeting_json, planning_meeting_json),
traces_vision_fragment = COALESCE(:traces_vision_fragment, traces_vision_fragment)
WHERE milestone_id = :milestone_id AND id = :id`)
.run({
":milestone_id": milestoneId,
@ -161,6 +167,25 @@ export function upsertSlicePlanning(milestoneId, sliceId, planning) {
":planning_meeting_json": planning.planningMeeting
? JSON.stringify(planning.planningMeeting)
: null,
// ADR-0000 P2 (schema v69): vision trace fragment is part of planning.
":traces_vision_fragment": planning.tracesVisionFragment ?? null,
});
}
// ADR-0000 P2 (schema v69): focused setter so callers that already have a
// validated fragment (e.g. validate-milestone backfilling legacy slices) can
// update just this column without rebuilding the full planning payload.
export function updateSliceVisionTrace(milestoneId, sliceId, fragment) {
const currentDb = _getAdapter();
if (!currentDb) throw new SFError(SF_STALE_STATE, "sf-db: No database open");
currentDb
.prepare(
"UPDATE slices SET traces_vision_fragment = :fragment WHERE milestone_id = :mid AND id = :sid",
)
.run({
":fragment": fragment ?? null,
":mid": milestoneId,
":sid": sliceId,
});
}

View file

@ -0,0 +1,176 @@
/**
* plan-milestone-vision-trace.test.mjs ADR-0000 P2 (schema v69).
*
* Purpose: prove plan_milestone requires every slice to carry a literal
* milestone.vision fragment in tracesVisionFragment, persists it to the
* slices row (schema v69), and rejects fragments that do not appear in
* milestone.vision. This gives validate-milestone structured grounds for
* assessment instead of having to re-read milestone.vision through the LLM
* on every check (restoration of doctrine, not new work).
*
* Consumer: plan_milestone validation + validate-milestone (downstream).
* Contract: rejects missing fragment, rejects fragment-not-in-vision,
* persists valid fragment to slices.traces_vision_fragment.
* Falsifier: a slice without the fragment can be inserted, or a fragment
* absent from milestone.vision is accepted.
*/
import { mkdirSync, mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterEach, describe, expect, test } from "vitest";
import { handlePlanMilestone } from "../tools/plan-milestone.js";
import {
closeDatabase,
getMilestoneSlices,
openDatabase,
} from "../sf-db.js";
const tmpDirs = [];
afterEach(() => {
closeDatabase();
while (tmpDirs.length > 0) {
rmSync(tmpDirs.pop(), { recursive: true, force: true });
}
});
function makeProject() {
const dir = mkdtempSync(join(tmpdir(), "sf-plan-milestone-vt-"));
tmpDirs.push(dir);
mkdirSync(join(dir, ".sf"), { recursive: true });
openDatabase(join(dir, ".sf", "sf.db"));
return dir;
}
function baseSlice(overrides = {}) {
return {
sliceId: "S01",
title: "First slice",
risk: "medium",
depends: [],
demo: "User can view correlated VM+PG health on the dashboard.",
goal: "Render correlated VM+PG health from the joined source.",
successCriteria:
"Dashboard renders correlated VM+PG status for any VM.",
proofLevel: "integration",
integrationClosure: "Dashboard ships behind feature flag.",
observabilityImpact: "Adds dashboard latency metric.",
...overrides,
};
}
const baseVision =
"Users can view correlated VM+PG health on a single dashboard, and operators can drill down into per-VM logs.";
describe("plan_milestone vision trace (schema v69)", () => {
test("rejects_slice_with_missing_tracesVisionFragment", async () => {
const base = makeProject();
const result = await handlePlanMilestone(
{
milestoneId: "M050",
title: "Vision Trace Required",
vision: baseVision,
slices: [
baseSlice({
// tracesVisionFragment intentionally omitted
}),
],
},
base,
);
expect(result.error).toBeDefined();
expect(result.error).toMatch(/tracesVisionFragment/);
// DB write must not have happened.
expect(getMilestoneSlices("M050")).toHaveLength(0);
});
test("rejects_fragment_not_present_in_vision", async () => {
const base = makeProject();
const result = await handlePlanMilestone(
{
milestoneId: "M051",
title: "Fragment Must Be Verbatim",
vision: baseVision,
slices: [
baseSlice({
tracesVisionFragment:
"export Grafana dashboards as PDF reports",
}),
],
},
base,
);
expect(result.error).toBeDefined();
expect(result.error).toMatch(
/tracesVisionFragment.*must appear verbatim/i,
);
expect(getMilestoneSlices("M051")).toHaveLength(0);
});
test("persists_valid_fragment_to_slice_row", async () => {
const base = makeProject();
const result = await handlePlanMilestone(
{
milestoneId: "M052",
title: "Vision Trace Persists",
vision: baseVision,
slices: [
baseSlice({
sliceId: "S01",
tracesVisionFragment: "view correlated VM+PG health",
}),
baseSlice({
sliceId: "S02",
title: "Per-VM logs drilldown",
demo: "Operators drill down into per-VM logs.",
goal: "Wire operator drilldown to per-VM logs.",
successCriteria:
"Operator can click VM and see filtered logs.",
tracesVisionFragment:
"operators can drill down into per-VM logs",
}),
],
},
base,
);
expect(result.error).toBeUndefined();
expect(result.milestoneId).toBe("M052");
const slices = getMilestoneSlices("M052");
expect(slices).toHaveLength(2);
const s01 = slices.find((s) => s.id === "S01");
const s02 = slices.find((s) => s.id === "S02");
expect(s01?.traces_vision_fragment).toBe(
"view correlated VM+PG health",
);
expect(s02?.traces_vision_fragment).toBe(
"operators can drill down into per-VM logs",
);
});
test("fragment_match_is_whitespace_and_case_insensitive", async () => {
// The validator normalizes whitespace and case before substring check,
// so trivial formatting differences do not cause spurious rejection.
const base = makeProject();
const result = await handlePlanMilestone(
{
milestoneId: "M053",
title: "Normalization Works",
vision: baseVision,
slices: [
baseSlice({
// Mixed case, extra spaces — still a literal clause of vision.
tracesVisionFragment: " View Correlated VM+PG Health ",
}),
],
},
base,
);
expect(result.error).toBeUndefined();
const slices = getMilestoneSlices("M053");
expect(slices).toHaveLength(1);
// The stored fragment is the normalized planning text (trimmed/squeezed).
expect(slices[0].traces_vision_fragment).toMatch(
/view correlated.*vm\+pg health/i,
);
});
});

View file

@ -342,6 +342,14 @@ test("openDatabase_migrates_v27_tasks_without_created_at_through_spec_backfill",
`${col} column should exist after v66 migration`,
);
}
// v69: slices gained traces_vision_fragment column (ADR-0000 P2 — per-slice
// vision trace so validate-milestone has structured grounds for assessment).
const sliceColumns = db.prepare("PRAGMA table_info(slices)").all();
const sliceColNames = sliceColumns.map((c) => c.name);
assert.ok(
sliceColNames.includes("traces_vision_fragment"),
"traces_vision_fragment column should exist after v69 migration",
);
const taskSpec = db
.prepare(
"SELECT milestone_id, slice_id, task_id, verify FROM task_specs WHERE task_id = 'T01'",
@ -383,11 +391,15 @@ test("openDatabase_v52_db_heals_routing_history_and_auto_start_path_works", () =
initRoutingHistory(dbPath);
}, "initRoutingHistory should not throw on a v52 DB");
// Schema should have migrated to the current head.
// Schema should have migrated to the current head (v72 after the
// ADR-0000 swarm: main's v69 = memory_extraction failure_class,
// then v70 = tasks.purpose_trace (P3), v71 = self_feedback.purpose_anchor
// (P5), v72 = slices.traces_vision_fragment (P2, renumbered from v69
// after the parallel-work collision).
const version = db
.prepare("SELECT MAX(version) AS version FROM schema_version")
.get();
assert.equal(version.version, 69);
assert.equal(version.version, 72);
});
test("openDatabase_when_fresh_db_supports_schedule_entries", () => {

View file

@ -132,10 +132,24 @@ function validateProductResearch(value) {
}
return normalized;
}
function validateSlices(value) {
// ADR-0000 P2 (schema v69): normalize text for fragment containment comparison.
// Collapses runs of whitespace, trims, and lowercases so trivial formatting
// differences (newlines, double spaces, capitalization) don't cause false
// rejections. The fragment must still be a literal substring of the vision
// after the same normalization is applied.
function normalizeForFragmentMatch(value) {
return String(value ?? "")
.replace(/\s+/g, " ")
.trim()
.toLowerCase();
}
function validateSlices(value, { vision } = {}) {
if (!Array.isArray(value) || value.length === 0) {
throw new Error("slices must be a non-empty array");
}
const normalizedVision =
typeof vision === "string" ? normalizeForFragmentMatch(vision) : "";
const seen = new Set();
return value.map((entry, index) => {
if (!entry || typeof entry !== "object") {
@ -154,6 +168,7 @@ function validateSlices(value) {
const observabilityImpact = obj.observabilityImpact;
const isSketchRaw = obj.isSketch;
const sketchScopeRaw = obj.sketchScope;
const tracesVisionFragmentRaw = obj.tracesVisionFragment;
// SF ADR-011: preserve 3-valued isSketch semantics (true/false/absent).
// Absent must round-trip as undefined so DB upsert ON CONFLICT preserves
// any existing is_sketch row state rather than silently overwriting.
@ -180,6 +195,27 @@ function validateSlices(value) {
throw new Error(`slices[${index}].demo must be a non-empty string`);
if (!isNonEmptyString(goal))
throw new Error(`slices[${index}].goal must be a non-empty string`);
// ADR-0000 P2 (schema v69): every new slice must carry a literal
// fragment of milestone.vision so validate-milestone has structured
// grounds for assessment without re-reading the vision via the LLM.
if (!isNonEmptyString(tracesVisionFragmentRaw)) {
throw new Error(
`slices[${index}].tracesVisionFragment must be a non-empty string — emit the literal clause from milestone.vision this slice serves (ADR-0000)`,
);
}
const fragmentNormalized = normalizeForFragmentMatch(
tracesVisionFragmentRaw,
);
if (!fragmentNormalized) {
throw new Error(
`slices[${index}].tracesVisionFragment must be a non-empty string`,
);
}
if (!normalizedVision.includes(fragmentNormalized)) {
throw new Error(
`slices[${index}].tracesVisionFragment must appear verbatim in milestone.vision (ADR-0000 P2); got ${JSON.stringify(tracesVisionFragmentRaw)}`,
);
}
// SF ADR-011: sketch slices defer the heavyweight planning fields to
// refine-slice. Non-sketch slices must populate them up front.
if (isSketch === true) {
@ -248,6 +284,11 @@ function validateSlices(value) {
`slices[${index}].sketchScope`,
)
: "",
// ADR-0000 P2 (schema v69): validated, normalized vision trace clause.
tracesVisionFragment: normalizePlanningText(
tracesVisionFragmentRaw,
`slices[${index}].tracesVisionFragment`,
),
};
});
}
@ -361,7 +402,11 @@ function validateParams(params) {
visionMeeting: hasStructuredVisionAlignmentMeeting(params.visionMeeting)
? normalizeVisionMeeting(params.visionMeeting)
: undefined,
slices: validateSlices(slicesInput),
// ADR-0000 P2 (schema v69): pass normalized vision so each slice's
// tracesVisionFragment can be checked for verbatim containment.
slices: validateSlices(slicesInput, {
vision: normalizePlanningText(params.vision, "vision"),
}),
};
}
export async function handlePlanMilestone(rawParams, basePath) {
@ -460,6 +505,8 @@ export async function handlePlanMilestone(rawParams, basePath) {
sequence: i + 1, // Preserve agent-ordered sequence (#3356)
isSketch: slice.isSketch,
sketchScope: slice.sketchScope,
// ADR-0000 P2 (schema v69): persist validated vision-trace fragment.
tracesVisionFragment: slice.tracesVisionFragment,
});
// SF ADR-011: sketches defer planning fields to refine-slice — only
// upsert when we actually have content to write.
@ -470,6 +517,8 @@ export async function handlePlanMilestone(rawParams, basePath) {
proofLevel: slice.proofLevel,
integrationClosure: slice.integrationClosure,
observabilityImpact: slice.observabilityImpact,
// ADR-0000 P2 (schema v69): keep upsert path in sync with insert.
tracesVisionFragment: slice.tracesVisionFragment,
});
}
}