feat(S05/T02): Extend migrateHierarchyToDb to populate v8 planning colu…

- src/resources/extensions/gsd/md-importer.ts
- src/resources/extensions/gsd/tests/gsd-recover.test.ts
This commit is contained in:
TÂCHES 2026-03-23 11:52:46 -06:00
parent 64908fc822
commit 4d3ccb5b08
6 changed files with 233 additions and 10 deletions

View file

@ -26,6 +26,7 @@
- `node --import ./src/resources/extensions/gsd/tests/resolve-ts.mjs --experimental-strip-types --test src/resources/extensions/gsd/tests/gsd-recover.test.ts` — extended recovery tests pass (v8 column population)
- `grep -rn 'import.*parseRoadmap\|import.*parsePlan\|import.*parseRoadmapSlices' src/resources/extensions/gsd/*.ts | grep -v '/tests/' | grep -v 'md-importer' | grep -v 'files.ts'` — returns zero module-level imports (only lazy createRequire references)
- Regression suites: doctor.test.ts, auto-recovery.test.ts, auto-dashboard.test.ts, derive-state-db.test.ts, derive-state-crossval.test.ts, planning-crossval.test.ts, markdown-renderer.test.ts all pass
- Diagnostic: `gsd-recover.test.ts` v8 column assertions include SQL-level queryability checks for vision, goal, files, verify columns — verifying inspectable state after migration failure or empty data
## Observability / Diagnostics
@ -49,7 +50,7 @@
- Verify: `node --import ./src/resources/extensions/gsd/tests/resolve-ts.mjs --experimental-strip-types --test src/resources/extensions/gsd/tests/flag-file-db.test.ts`
- Done when: deriveStateFromDb returns phase='replanning-slice' from DB-only data (no REPLAN.md or REPLAN-TRIGGER.md on disk) and returns phase='executing' when replan_history exists (loop protection). SCHEMA_VERSION=10.
- [ ] **T02: Extend migrateHierarchyToDb with v8 column population** `est:30m`
- [x] **T02: Extend migrateHierarchyToDb with v8 column population** `est:30m`
- Why: Existing projects migrating to the DB need their parsed ROADMAP/PLAN data written into the v8 planning columns so DB queries return meaningful data. The `gsd recover` test must verify this.
- Files: `src/resources/extensions/gsd/md-importer.ts`, `src/resources/extensions/gsd/tests/gsd-recover.test.ts`
- Do: (1) In `migrateHierarchyToDb()`, extend the `insertMilestone()` call to pass `planning: { vision: roadmap.vision, successCriteria: roadmap.successCriteria, boundaryMapMarkdown: boundaryMapSection }` where `boundaryMapMarkdown` is the raw "## Boundary Map" section extracted from the roadmap content. (2) Extend `insertSlice()` calls to pass `planning: { goal: plan.goal }` from the parsed plan (when plan exists). (3) Extend `insertTask()` calls to pass `planning: { files: task.files, verify: task.verify }` from TaskPlanEntry. (4) Extend `gsd-recover.test.ts` to assert: after recover, milestone has non-empty `vision`; slice has non-empty `goal`; task has populated `files` array and `verify` string.

View file

@ -0,0 +1,18 @@
{
"schemaVersion": 1,
"taskId": "T01",
"unitId": "M001/S05/T01",
"timestamp": 1774287990073,
"passed": false,
"discoverySource": "package-json",
"checks": [
{
"command": "npm run test",
"exitCode": 1,
"durationMs": 39607,
"verdict": "fail"
}
],
"retryAttempt": 1,
"maxRetries": 2
}

View file

@ -65,3 +65,9 @@ Extend `migrateHierarchyToDb()` in `md-importer.ts` to populate v8 planning colu
- `src/resources/extensions/gsd/md-importer.ts` — migrateHierarchyToDb() populates v8 planning columns
- `src/resources/extensions/gsd/tests/gsd-recover.test.ts` — extended with v8 column population assertions
## Observability Impact
- **Signals changed:** After migration, `SELECT vision, success_criteria, boundary_map_markdown FROM milestones WHERE id = :mid` returns non-empty values for pre-M002 projects (previously all empty). `SELECT goal FROM slices` and `SELECT files, verify FROM tasks` similarly populated.
- **Inspection:** `getMilestone(id).vision`, `getSlice(mid, sid).goal`, `getTask(mid, sid, tid).files/verify` return meaningful data post-recovery.
- **Failure visibility:** If `parseRoadmap()` or `parsePlan()` returns empty fields (no Vision in markdown, no Goal in plan), planning columns remain empty — detectable by `SELECT COUNT(*) FROM milestones WHERE vision = ''`.

View file

@ -0,0 +1,66 @@
---
id: T02
parent: S05
milestone: M001
key_files:
- src/resources/extensions/gsd/md-importer.ts
- src/resources/extensions/gsd/tests/gsd-recover.test.ts
key_decisions:
- v8 planning columns populated only with parser-extractable fields; tool-only fields (keyRisks, requirementCoverage, proofLevel) left empty per D004
- Boundary map extracted via inline string operations (indexOf + slice) rather than importing extractSection from files.ts — avoids coupling to unexported function
- Plan parsing moved before insertSlice to make goal available at insertion time instead of using a post-insert upsert
duration: ""
verification_result: passed
completed_at: 2026-03-23T17:52:14.780Z
blocker_discovered: false
---
# T02: Extend migrateHierarchyToDb to populate v8 planning columns (vision, successCriteria, boundaryMapMarkdown on milestones; goal on slices; files/verify on tasks)
**Extend migrateHierarchyToDb to populate v8 planning columns (vision, successCriteria, boundaryMapMarkdown on milestones; goal on slices; files/verify on tasks)**
## What Happened
Extended `migrateHierarchyToDb()` in `md-importer.ts` to populate v8 planning columns from parsed markdown during recovery/migration.
**Milestone planning columns:** Refactored to parse the roadmap once (not twice) — saved the `parseRoadmap()` result early and reused it. Added inline extraction of the raw `## Boundary Map` section from roadmap markdown (finds heading, takes content until next `##` or EOF). The `insertMilestone()` call now passes `planning: { vision, successCriteria, boundaryMapMarkdown }`. Per D004, tool-only fields (keyRisks, requirementCoverage, proofStrategy, etc.) are left empty.
**Slice planning columns:** Restructured the loop to parse the plan file *before* `insertSlice()` (previously parsed after). The `insertSlice()` call now passes `planning: { goal: plan.goal }`. When no plan file exists, goal defaults to empty string.
**Task planning columns:** The `insertTask()` call now passes `planning: { files: taskEntry.files ?? [], verify: taskEntry.verify ?? '' }` from the `TaskPlanEntry` parsed by `parsePlan()`.
**Test extensions:** Enhanced the `gsd-recover.test.ts` fixtures — added `## Success Criteria` and `## Boundary Map` sections to the ROADMAP fixture, and `- Files:` / `- Verify:` lines to all task entries in both PLAN fixtures. Added a comprehensive test block (Test a2) with 27 assertions verifying: milestone vision matches fixture, success_criteria populated with correct entries, boundary_map_markdown contains expected content, D004 tool-only fields remain empty (key_risks, requirement_coverage, proof_level), slice goals populated for both S01 and S02, task files arrays populated correctly, task verify strings populated (discovered parser preserves backtick formatting), and SQL-level queryability diagnostics for all v8 columns.
## Verification
Ran gsd-recover.test.ts — all 65 assertions pass including 27 new v8 column population assertions. Ran 7 regression suites (migrate-hierarchy.test.ts: 57 pass, derive-state-crossval.test.ts: 189 pass, integration-proof.test.ts: 3 pass, derive-state-db.test.ts: 105 pass, doctor.test.ts: 55 pass, auto-recovery.test.ts: 33 pass, auto-dashboard.test.ts: 24 pass, planning-crossval.test.ts: 65 pass, markdown-renderer.test.ts: 106 pass, flag-file-db.test.ts: 14 pass) — zero regressions.
## Verification Evidence
| # | Command | Exit Code | Verdict | Duration |
|---|---------|-----------|---------|----------|
| 1 | `node --import ./src/resources/extensions/gsd/tests/resolve-ts.mjs --experimental-strip-types --test src/resources/extensions/gsd/tests/gsd-recover.test.ts` | 0 | ✅ pass | 524ms |
| 2 | `node --import ./src/resources/extensions/gsd/tests/resolve-ts.mjs --experimental-strip-types --test src/resources/extensions/gsd/tests/migrate-hierarchy.test.ts` | 0 | ✅ pass | 686ms |
| 3 | `node --import ./src/resources/extensions/gsd/tests/resolve-ts.mjs --experimental-strip-types --test src/resources/extensions/gsd/tests/derive-state-crossval.test.ts` | 0 | ✅ pass | 692ms |
| 4 | `node --import ./src/resources/extensions/gsd/tests/resolve-ts.mjs --experimental-strip-types --test src/resources/extensions/gsd/tests/integration-proof.test.ts` | 0 | ✅ pass | 756ms |
| 5 | `node --import ./src/resources/extensions/gsd/tests/resolve-ts.mjs --experimental-strip-types --test src/resources/extensions/gsd/tests/flag-file-db.test.ts` | 0 | ✅ pass | 176ms |
| 6 | `node --import ./src/resources/extensions/gsd/tests/resolve-ts.mjs --experimental-strip-types --test src/resources/extensions/gsd/tests/doctor.test.ts` | 0 | ✅ pass | 1100ms |
| 7 | `node --import ./src/resources/extensions/gsd/tests/resolve-ts.mjs --experimental-strip-types --test src/resources/extensions/gsd/tests/auto-recovery.test.ts` | 0 | ✅ pass | 752ms |
| 8 | `node --import ./src/resources/extensions/gsd/tests/resolve-ts.mjs --experimental-strip-types --test src/resources/extensions/gsd/tests/derive-state-db.test.ts` | 0 | ✅ pass | 238ms |
| 9 | `node --import ./src/resources/extensions/gsd/tests/resolve-ts.mjs --experimental-strip-types --test src/resources/extensions/gsd/tests/auto-dashboard.test.ts` | 0 | ✅ pass | 554ms |
| 10 | `node --import ./src/resources/extensions/gsd/tests/resolve-ts.mjs --experimental-strip-types --test src/resources/extensions/gsd/tests/planning-crossval.test.ts` | 0 | ✅ pass | 208ms |
| 11 | `node --import ./src/resources/extensions/gsd/tests/resolve-ts.mjs --experimental-strip-types --test src/resources/extensions/gsd/tests/markdown-renderer.test.ts` | 0 | ✅ pass | 257ms |
## Deviations
Discovered that parsePlan() preserves backtick formatting in verify fields (e.g. `` `npm test` `` not `npm test`). Adjusted test expectations to match. Refactored roadmap parsing to avoid double parseRoadmap() call — the function was called once for title and again for slices; now parsed once with result reused. Changed the loop guard from `if (!roadmapContent) continue` to `if (!roadmap) continue` to match the refactored variable.
## Known Issues
None.
## Files Created/Modified
- `src/resources/extensions/gsd/md-importer.ts`
- `src/resources/extensions/gsd/tests/gsd-recover.test.ts`

View file

@ -536,9 +536,10 @@ export function migrateHierarchyToDb(basePath: string): {
// Determine milestone title from roadmap H1 or CONTEXT heading
let milestoneTitle = '';
let roadmapContent: string | null = null;
let roadmap: ReturnType<typeof parseRoadmap> | null = null;
if (hasRoadmap) {
roadmapContent = readFileSync(roadmapPath!, 'utf-8');
const roadmap = parseRoadmap(roadmapContent);
roadmap = parseRoadmap(roadmapContent);
milestoneTitle = roadmap.title;
}
if (!milestoneTitle && hasContext) {
@ -554,23 +555,47 @@ export function migrateHierarchyToDb(basePath: string): {
dependsOn = parseContextDependsOn(contextContent);
}
// Extract raw "## Boundary Map" section from roadmap markdown for planning column
let boundaryMapSection = '';
if (roadmapContent) {
const bmIdx = roadmapContent.indexOf('## Boundary Map');
if (bmIdx >= 0) {
const afterBm = roadmapContent.slice(bmIdx);
// Take content until next ## heading or EOF
const nextHeading = afterBm.indexOf('\n## ', 1);
boundaryMapSection = nextHeading >= 0 ? afterBm.slice(0, nextHeading).trim() : afterBm.trim();
}
}
// Insert milestone (FK parent — must come first)
insertMilestone({
id: milestoneId,
title: milestoneTitle,
status: milestoneStatus,
depends_on: dependsOn,
planning: {
vision: roadmap?.vision ?? '',
successCriteria: roadmap?.successCriteria ?? [],
boundaryMapMarkdown: boundaryMapSection,
},
});
counts.milestones++;
// Parse roadmap for slices
if (!roadmapContent) continue;
const roadmap = parseRoadmap(roadmapContent);
if (!roadmap) continue;
for (const sliceEntry of roadmap.slices) {
// Per K002: use 'complete' not 'done'
const sliceStatus = sliceEntry.done ? 'complete' : 'pending';
// Parse slice plan early so goal is available for insertSlice planning column
const planPath = resolveSliceFile(basePath, milestoneId, sliceEntry.id, 'PLAN');
let plan: ReturnType<typeof parsePlan> | null = null;
if (planPath && existsSync(planPath)) {
const planContent = readFileSync(planPath, 'utf-8');
plan = parsePlan(planContent);
}
insertSlice({
id: sliceEntry.id,
milestoneId: milestoneId,
@ -579,15 +604,14 @@ export function migrateHierarchyToDb(basePath: string): {
risk: sliceEntry.risk,
depends: sliceEntry.depends,
demo: sliceEntry.demo,
planning: {
goal: plan?.goal ?? '',
},
});
counts.slices++;
// Parse slice plan for tasks
const planPath = resolveSliceFile(basePath, milestoneId, sliceEntry.id, 'PLAN');
if (!planPath || !existsSync(planPath)) continue;
const planContent = readFileSync(planPath, 'utf-8');
const plan = parsePlan(planContent);
// Insert tasks from parsed plan
if (!plan) continue;
for (const taskEntry of plan.tasks) {
// Per K002: use 'complete' not 'done'
@ -615,6 +639,10 @@ export function migrateHierarchyToDb(basePath: string): {
milestoneId: milestoneId,
title: taskEntry.title,
status: taskStatus,
planning: {
files: taskEntry.files ?? [],
verify: taskEntry.verify ?? '',
},
});
counts.tasks++;
}

View file

@ -16,6 +16,9 @@ import {
insertMilestone,
insertSlice,
insertTask,
getMilestone,
getSlice,
getTask,
} from '../gsd-db.ts';
import { migrateHierarchyToDb } from '../md-importer.ts';
import { deriveStateFromDb, invalidateStateCache } from '../state.ts';
@ -47,6 +50,11 @@ const ROADMAP_M001 = `# M001: Recovery Test
**Vision:** Test recovery round-trip.
## Success Criteria
- All recovery tests pass
- State matches after round-trip
## Slices
- [x] **S01: Setup** \`risk:low\` \`depends:[]\`
@ -54,6 +62,12 @@ const ROADMAP_M001 = `# M001: Recovery Test
- [ ] **S02: Core** \`risk:medium\` \`depends:[S01]\`
> After this: Core done.
## Boundary Map
| From | To | Produces | Consumes |
|------|-----|----------|----------|
| S01 | S02 | setup artifacts | setup artifacts |
`;
const PLAN_S01_COMPLETE = `---
@ -71,9 +85,13 @@ skills_used: []
- [x] **T01: Init** \`est:15m\`
Initialize things.
- Files: \`init.ts\`, \`config.ts\`
- Verify: \`node test-init.ts\`
- [x] **T02: Config** \`est:10m\`
Configure things.
- Files: \`settings.ts\`
- Verify: \`node test-config.ts\`
`;
const PLAN_S02_PARTIAL = `---
@ -91,12 +109,18 @@ skills_used: []
- [x] **T01: Build** \`est:30m\`
Build it.
- Files: \`core.ts\`
- Verify: \`node test-build.ts\`
- [ ] **T02: Test** \`est:20m\`
Test it.
- Files: \`test-core.ts\`, \`helpers.ts\`
- Verify: \`npm test\`
- [ ] **T03: Polish** \`est:15m\`
Polish it.
- Files: \`polish.ts\`
- Verify: \`node test-polish.ts\`
`;
const SUMMARY_S01 = `---
@ -208,6 +232,86 @@ async function main() {
}
}
// ─── Test (a2): v8 planning columns populated after recovery ───────────
console.log('\n=== recover: v8 planning columns populated ===');
{
const base = createFixtureBase();
try {
writeFile(base, 'milestones/M001/M001-ROADMAP.md', ROADMAP_M001);
writeFile(base, 'milestones/M001/slices/S01/S01-PLAN.md', PLAN_S01_COMPLETE);
writeFile(base, 'milestones/M001/slices/S01/S01-SUMMARY.md', SUMMARY_S01);
writeFile(base, 'milestones/M001/slices/S02/S02-PLAN.md', PLAN_S02_PARTIAL);
openDatabase(':memory:');
migrateHierarchyToDb(base);
// Milestone planning columns
const milestone = getMilestone('M001');
assertTrue(milestone !== null, 'v8: milestone exists');
assertEq(milestone!.vision, 'Test recovery round-trip.', 'v8: milestone vision populated');
assertTrue(milestone!.success_criteria.length >= 2, 'v8: milestone success_criteria has entries');
assertEq(milestone!.success_criteria[0], 'All recovery tests pass', 'v8: first success criterion');
assertTrue(milestone!.boundary_map_markdown.includes('Boundary Map'), 'v8: boundary_map_markdown populated');
assertTrue(milestone!.boundary_map_markdown.includes('S01'), 'v8: boundary_map_markdown has S01');
// Tool-only fields left empty per D004
assertEq(milestone!.key_risks.length, 0, 'v8: key_risks left empty (tool-only per D004)');
assertEq(milestone!.requirement_coverage, '', 'v8: requirement_coverage left empty (tool-only per D004)');
// Slice planning columns
const sliceS01 = getSlice('M001', 'S01');
assertTrue(sliceS01 !== null, 'v8: slice S01 exists');
assertEq(sliceS01!.goal, 'Setup fixtures.', 'v8: S01 goal populated');
const sliceS02 = getSlice('M001', 'S02');
assertTrue(sliceS02 !== null, 'v8: slice S02 exists');
assertEq(sliceS02!.goal, 'Build core.', 'v8: S02 goal populated');
// Slice tool-only fields left empty per D004
assertEq(sliceS01!.proof_level, '', 'v8: S01 proof_level left empty (tool-only per D004)');
// Task planning columns — S01/T01
const taskS01T01 = getTask('M001', 'S01', 'T01');
assertTrue(taskS01T01 !== null, 'v8: task S01/T01 exists');
assertTrue(taskS01T01!.files.length >= 2, 'v8: S01/T01 files populated');
assertTrue(taskS01T01!.files.includes('init.ts'), 'v8: S01/T01 files includes init.ts');
assertTrue(taskS01T01!.files.includes('config.ts'), 'v8: S01/T01 files includes config.ts');
assertEq(taskS01T01!.verify, '`node test-init.ts`', 'v8: S01/T01 verify populated');
// Task planning columns — S02/T02
const taskS02T02 = getTask('M001', 'S02', 'T02');
assertTrue(taskS02T02 !== null, 'v8: task S02/T02 exists');
assertTrue(taskS02T02!.files.length >= 2, 'v8: S02/T02 files populated');
assertTrue(taskS02T02!.files.includes('test-core.ts'), 'v8: S02/T02 files includes test-core.ts');
assertEq(taskS02T02!.verify, '`npm test`', 'v8: S02/T02 verify populated');
// Task with no Files/Verify — not applicable since all fixtures now have them,
// but confirm a task from S02 has correct data
const taskS02T03 = getTask('M001', 'S02', 'T03');
assertTrue(taskS02T03 !== null, 'v8: task S02/T03 exists');
assertTrue(taskS02T03!.files.includes('polish.ts'), 'v8: S02/T03 files includes polish.ts');
assertEq(taskS02T03!.verify, '`node test-polish.ts`', 'v8: S02/T03 verify populated');
// Diagnostic: v8 planning columns queryable via SQL
const db = _getAdapter()!;
const milestoneRow = db.prepare("SELECT vision, success_criteria, boundary_map_markdown FROM milestones WHERE id = 'M001'").get() as any;
assertTrue(milestoneRow.vision.length > 0, 'v8-diag: vision column queryable');
assertTrue(milestoneRow.boundary_map_markdown.length > 0, 'v8-diag: boundary_map_markdown column queryable');
const sliceRow = db.prepare("SELECT goal FROM slices WHERE milestone_id = 'M001' AND id = 'S01'").get() as any;
assertTrue(sliceRow.goal.length > 0, 'v8-diag: goal column queryable');
const taskRow = db.prepare("SELECT files, verify FROM tasks WHERE milestone_id = 'M001' AND slice_id = 'S01' AND id = 'T01'").get() as any;
assertTrue(taskRow.files.length > 2, 'v8-diag: files column queryable (JSON array)');
assertTrue(taskRow.verify.length > 0, 'v8-diag: verify column queryable');
closeDatabase();
} finally {
closeDatabase();
cleanup(base);
}
}
// ─── Test (b): Idempotent recovery — double recover ────────────────────
console.log('\n=== recover: idempotent — double recovery produces same state ===');
{