docs(M004): context, requirements, and roadmap

This commit is contained in:
Lex Christopherson 2026-03-15 20:05:05 -06:00
parent c8b42ed2ae
commit 3a90165f38
5 changed files with 487 additions and 4 deletions

View file

@ -50,3 +50,6 @@
| D042 | M003/S04 | pattern | shouldUseWorktreeIsolation override parameter | Accept optional overridePrefs for testability | loadEffectiveGSDPreferences computes PROJECT_PREFERENCES_PATH at module load time from process.cwd(). chdir-based test fixtures cannot influence it. Override parameter enables reliable testing. | Yes — if preference loading becomes dynamic |
| D043 | M003/S04 | pattern | validatePreferences exported | Export from preferences.ts for direct test access | Was module-private. Tests need to call it directly without full file-loading pipeline. No downstream consumers affected. | No |
| D044 | M003/S05 | pattern | Self-heal strategy for merge failures | Detect real conflicts immediately (skip retry), retry only transient failures once | Real conflicts will fail identically on retry — wasting time. Transient failures (stale index, leftover merge state) recover after abort+reset. Fast escalation for conflicts, automatic recovery for everything else. | Yes — if retry proves useful for some conflict types |
| D045 | M004 | arch | SQLite provider strategy | Tiered chain: node:sqlite → better-sqlite3 → null | node:sqlite available on Node 22.5+ (our target), better-sqlite3 as fallback for older Node, null for graceful degradation. DbAdapter normalizes API differences. | Yes — if node:sqlite stabilizes and better-sqlite3 path can be dropped |
| D046 | M004 | arch | createWorktree sync/async for DB copy | Keep synchronous, use copyFileSync | Memory-db made createWorktree async for dynamic imports, but copyWorktreeDb is purely sync (copyFileSync). Static import + isDbAvailable() guard avoids async cascade through createAutoWorktree and auto.ts call sites. | No |
| D047 | M004 | arch | Port strategy | Adapt to current architecture, not blind merge | 145 commits divergence, auto.ts decomposed into 6 modules. Memory-db code is reference — capabilities ported into current file structure (auto-prompts.ts, auto-dispatch.ts, etc.), not cherry-picked. | No |

View file

@ -2,7 +2,7 @@
## What This Is
A pi coding agent extension (GSD — "Get Stuff Done") that provides structured planning, auto-mode execution, and project management for autonomous coding sessions. Includes proactive secret management, browser automation tools for UI verification, and worktree-isolated git architecture for zero-friction autonomous execution.
A pi coding agent extension (GSD — "Get Stuff Done") that provides structured planning, auto-mode execution, and project management for autonomous coding sessions. Includes proactive secret management, browser automation tools for UI verification, worktree-isolated git architecture for zero-friction autonomous execution, and SQLite-backed surgical context injection for token-efficient prompt assembly.
## Core Value
@ -21,11 +21,13 @@ The GSD extension is fully functional with:
- Worktree-isolated git architecture: auto-worktree per milestone, --no-ff slice merges, milestone squash to main, preference-gated isolation modes, self-healing git repair, doctor git health checks, full e2e test coverage
- Auto-worktree lifecycle: `auto-worktree.ts` module creates isolated worktrees per milestone (`milestone/<MID>` branches), wired into auto.ts startAuto/resume/stop with split-brain prevention
- Branch-per-slice git model with squash merge to main (legacy mode, supported via `git.isolation: "branch"` preference)
- Decomposed auto-mode: `auto-prompts.ts` (prompt builders), `auto-dispatch.ts` (unit→prompt routing), `auto-recovery.ts` (timeout/crash recovery), `auto-worktree.ts` (worktree lifecycle)
## Architecture / Key Patterns
- **Extension model**: pi extensions register tools, commands, hooks via `ExtensionAPI`
- **State machine**: `auto.ts` drives `dispatchNextUnit()` which reads disk state and dispatches fresh sessions
- **Dispatch pipeline**: `auto-dispatch.ts` resolves phase → unit type + prompt via `resolveDispatch()`. Prompt builders live in `auto-prompts.ts`.
- **Secrets gate**: `startAuto()` checks `getManifestStatus()` before first dispatch
- **Disk-driven state**: `.gsd/` files are the source of truth, `STATE.md` is derived cache
- **File parsing**: `files.ts` has markdown parsers for all GSD file types
@ -43,3 +45,4 @@ See `.gsd/REQUIREMENTS.md` for the explicit capability contract, requirement sta
- [x] M001: Proactive Secret Management — Front-loaded API key collection into planning so auto-mode runs uninterrupted (10 requirements validated)
- [x] M002: Browser Tools Performance & Intelligence — Module decomposition, action pipeline optimization, sharp-based screenshots, form intelligence, intent-ranked retrieval, semantic actions, 108-test suite (12 requirements validated)
- [x] M003: Worktree-Isolated Git Architecture — Auto-worktree per milestone, --no-ff slice merges, milestone squash to main, preferences + backwards compat, self-healing git repair, doctor health checks, full e2e test suite (13 requirements validated)
- [ ] M004: SQLite Context Store — Surgical context injection via SQLite-backed query layer, replacing whole-file prompt dumps with scoped DB queries for ≥30% token savings

View file

@ -4,7 +4,148 @@ This file is the explicit capability and coverage contract for the project.
## Active
(No active requirements — all M003 requirements validated.)
### R045 — SQLite DB layer with tiered provider chain
- Class: core-capability
- Status: active
- Description: A SQLite abstraction layer that tries `node:sqlite` (Node 22.5+), falls back to `better-sqlite3`, then to null. A thin `DbAdapter` interface normalizes API differences. Schema init creates decisions, requirements, artifacts tables plus filtered views. WAL mode on file-backed databases.
- Why it matters: The foundation for surgical context injection. Without a queryable store, prompts must dump entire files.
- Source: execution (memory-db port)
- Primary owning slice: M004/S01
- Supporting slices: none
- Validation: unmapped
- Notes: Port from memory-db worktree `gsd-db.ts`. Tiered provider chain proven on Node 22.20.0. `node:sqlite` returns null-prototype rows — DbAdapter normalizes via spread.
### R046 — Graceful degradation when SQLite unavailable
- Class: continuity
- Status: active
- Description: When no SQLite provider loads, all query functions return empty results and all prompt builders fall back to `inlineGsdRootFile` filesystem loading. No crash, no visible error.
- Why it matters: SQLite must be optional. Users on exotic platforms or old Node versions must not be blocked.
- Source: execution (memory-db port)
- Primary owning slice: M004/S01
- Supporting slices: M004/S03
- Validation: unmapped
- Notes: Every query function guards with `isDbAvailable()` + try/catch. Every prompt builder falls back to existing `inlineGsdRootFile`.
### R047 — Auto-migration from markdown to DB on first run
- Class: core-capability
- Status: active
- Description: When auto-mode starts on a project with `.gsd/` markdown files but no `gsd.db`, silently import all artifact types into a fresh DB. Idempotent — safe to re-run.
- Why it matters: Existing projects must transparently gain DB benefits without manual migration.
- Source: execution (memory-db port)
- Primary owning slice: M004/S02
- Supporting slices: M004/S01
- Validation: unmapped
- Notes: Port from memory-db `md-importer.ts`. Custom parsers for DECISIONS.md pipe-table format and REQUIREMENTS.md section/bullet format. Hierarchy walker for milestones → slices → tasks.
### R048 — Round-trip fidelity for all artifact types
- Class: quality-attribute
- Status: active
- Description: Importing markdown into DB and regenerating markdown produces field-identical output. No data loss, no format drift.
- Why it matters: Dual-write means DB→markdown generation must be faithful. Format drift corrupts the human-readable artifacts.
- Source: execution (memory-db port)
- Primary owning slice: M004/S02
- Supporting slices: M004/S06
- Validation: unmapped
- Notes: Port from memory-db. Custom parsers and generators must produce/consume identical formats.
### R049 — Surgical prompt injection via DB queries
- Class: core-capability
- Status: active
- Description: All prompt builders in `auto-prompts.ts` use scoped DB queries instead of whole-file `inlineGsdRootFile` for decisions, requirements, and project context. Decisions filtered by milestone, requirements filtered by slice ownership.
- Why it matters: This is the core value — smaller, more relevant prompts mean better agent reasoning and fewer wasted tokens.
- Source: user
- Primary owning slice: M004/S03
- Supporting slices: M004/S01, M004/S02
- Validation: unmapped
- Notes: Port from memory-db DB-aware helpers. Must be rewired into current `auto-prompts.ts` (not the old monolithic auto.ts). 19 `inlineGsdRootFile` calls to replace across 11 prompt builders.
### R050 — Dual-write keeping markdown and DB in sync
- Class: continuity
- Status: active
- Description: After each dispatch unit completes and auto-commits, re-import modified markdown files into the DB. Structured LLM tools write to DB first, then regenerate markdown. Both directions stay synchronized.
- Why it matters: Markdown files are the human-readable source of truth. The DB is the query index. They must agree.
- Source: execution (memory-db port)
- Primary owning slice: M004/S03
- Supporting slices: M004/S06
- Validation: unmapped
- Notes: Re-import in `handleAgentEnd` after auto-commit. DB-first write in structured tools triggers markdown generation.
### R051 — Token measurement with before/after comparison
- Class: operability
- Status: active
- Description: `promptCharCount` and `baselineCharCount` fields added to `UnitMetrics`. Measurement wired into all `snapshotUnitMetrics` call sites. Baseline = full markdown content. Prompt = DB-scoped content. Difference = token savings.
- Why it matters: Proves the ≥30% savings claim with real data. Enables ongoing monitoring of prompt efficiency.
- Source: execution (memory-db port)
- Primary owning slice: M004/S04
- Supporting slices: M004/S03
- Validation: unmapped
- Notes: Port from memory-db. Module-scoped measurement vars reset at top of `dispatchNextUnit`.
### R052 — DB-first state derivation with filesystem fallback
- Class: core-capability
- Status: active
- Description: `deriveState()` queries the artifacts table for file content when DB is available, replacing the batch file-parse step. File discovery still uses disk. Falls back to filesystem when DB unavailable.
- Why it matters: Faster state derivation on large projects. Consistent with DB-first architecture.
- Source: execution (memory-db port)
- Primary owning slice: M004/S04
- Supporting slices: M004/S01, M004/S02
- Validation: unmapped
- Notes: Port from memory-db. File discovery (which milestones/slices/tasks exist) stays on disk. Only content loading switches to DB.
### R053 — Worktree DB copy on creation
- Class: integration
- Status: active
- Description: When a worktree is created, copy `gsd.db` from the source project into the worktree's `.gsd/` directory. Skip WAL/SHM files. Non-fatal on failure.
- Why it matters: Worktrees need their own DB with the project's current state. Without a copy, the worktree starts with no DB context.
- Source: execution (memory-db port)
- Primary owning slice: M004/S05
- Supporting slices: M004/S01
- Validation: unmapped
- Notes: Port from memory-db `copyWorktreeDb`. Keep `createWorktree` synchronous — `copyFileSync` is sufficient. Guard with `isDbAvailable()`.
### R054 — Worktree DB merge reconciliation
- Class: integration
- Status: active
- Description: When a worktree merges back (slice or milestone), ATTACH the worktree's DB and reconcile rows: INSERT OR REPLACE in a transaction with conflict detection by content column comparison.
- Why it matters: The worktree may have added decisions, requirements, or artifacts that the main DB doesn't have.
- Source: execution (memory-db port)
- Primary owning slice: M004/S05
- Supporting slices: M004/S01
- Validation: unmapped
- Notes: Port from memory-db `reconcileWorktreeDb`. ATTACH/DETACH pattern with try/finally for cleanup.
### R055 — Structured LLM tools for decisions/requirements/summaries
- Class: core-capability
- Status: active
- Description: Three tools registered: `gsd_save_decision` (auto-assigns D-numbers, writes to DB + regenerates DECISIONS.md), `gsd_update_requirement` (verifies existence, updates DB + regenerates REQUIREMENTS.md), `gsd_save_summary` (writes artifact to DB + disk).
- Why it matters: Eliminates the markdown-then-parse roundtrip. LLM writes structured data directly, guaranteeing parseable output.
- Source: execution (memory-db port)
- Primary owning slice: M004/S06
- Supporting slices: M004/S03
- Validation: unmapped
- Notes: Port from memory-db. DB-first write pattern: upsert → fetch all → generate markdown → write file.
### R056 — /gsd inspect command for DB diagnostics
- Class: operability
- Status: active
- Description: A `/gsd inspect` slash command that dumps schema version, table row counts, and recent entries from each table.
- Why it matters: When things go wrong, the user needs visibility into DB state without running raw SQL.
- Source: execution (memory-db port)
- Primary owning slice: M004/S06
- Supporting slices: M004/S01
- Validation: unmapped
- Notes: Port from memory-db. Autocomplete for subcommands (decisions, requirements, artifacts, all).
### R057 — ≥30% token savings on planning/research dispatches
- Class: quality-attribute
- Status: active
- Description: Surgical prompt injection delivers ≥30% fewer prompt characters compared to whole-file loading, measured on mature projects with multiple milestones, decisions, and requirements.
- Why it matters: The primary user-visible value of the entire DB architecture. If savings aren't real, the complexity isn't justified.
- Source: user
- Primary owning slice: M004/S07
- Supporting slices: M004/S03, M004/S04
- Validation: unmapped
- Notes: Memory-db proved: 52.2% plan-slice, 66.3% decisions-only, 32.2% research composite, 42.4% lifecycle. Must re-prove against current codebase.
## Validated
@ -516,11 +657,24 @@ This file is the explicit capability and coverage contract for the project.
| R042 | core-capability | deferred | none | none | unmapped |
| R043 | quality-attribute | deferred | none | none | unmapped |
| R044 | anti-feature | out-of-scope | none | none | n/a |
| R045 | core-capability | active | M004/S01 | none | unmapped |
| R046 | continuity | active | M004/S01 | M004/S03 | unmapped |
| R047 | core-capability | active | M004/S02 | M004/S01 | unmapped |
| R048 | quality-attribute | active | M004/S02 | M004/S06 | unmapped |
| R049 | core-capability | active | M004/S03 | M004/S01, M004/S02 | unmapped |
| R050 | continuity | active | M004/S03 | M004/S06 | unmapped |
| R051 | operability | active | M004/S04 | M004/S03 | unmapped |
| R052 | core-capability | active | M004/S04 | M004/S01, M004/S02 | unmapped |
| R053 | integration | active | M004/S05 | M004/S01 | unmapped |
| R054 | integration | active | M004/S05 | M004/S01 | unmapped |
| R055 | core-capability | active | M004/S06 | M004/S03 | unmapped |
| R056 | operability | active | M004/S06 | M004/S01 | unmapped |
| R057 | quality-attribute | active | M004/S07 | M004/S03, M004/S04 | unmapped |
## Coverage Summary
- Active requirements: 0
- Mapped to slices: 0
- Active requirements: 13
- Mapped to slices: 13
- Validated: 35
- Deferred: 5
- Out of scope: 4

View file

@ -0,0 +1,126 @@
# M004: SQLite Context Store — Surgical Prompt Injection
**Gathered:** 2026-03-15
**Status:** Ready for planning
## Project Description
Port the completed memory-db worktree's SQLite-backed context store into the current GSD codebase. The memory-db work (7 slices, 21 requirements validated, 293 tests) was built against a pre-v2.12.0 codebase that has since diverged significantly — 145 commits on main including auto.ts decomposition, worktree architecture overhaul, and extensive refactoring. This is a port, not a merge.
## Why This Milestone
The current prompt assembly dumps entire files (DECISIONS.md, REQUIREMENTS.md, PROJECT.md) into every dispatch prompt regardless of relevance. On a mature project with 40+ decisions and 30+ requirements, most of that context is irrelevant to the active slice. A SQLite query layer enables surgical injection — only the decisions scoped to this milestone, only the requirements owned by this slice. The user's emphasis: "super fast context ingestion" — the DB is the mechanism for being "very, very surgically" selective about what context each task sees.
## User-Visible Outcome
### When this milestone is complete, the user can:
- Run auto-mode and see ≥30% smaller prompts with only relevant context injected
- Use `gsd_save_decision`, `gsd_update_requirement`, `gsd_save_summary` tool calls that bypass markdown parsing
- Run `/gsd inspect` to see DB state for diagnostics
- Start auto-mode on an existing project and have gsd.db appear silently with all artifacts imported
### Entry point / environment
- Entry point: `/gsd auto` CLI command, structured LLM tools during dispatch, `/gsd inspect` slash command
- Environment: local dev (Node 22.5+, runs in pi agent process)
- Live dependencies involved: none (SQLite is embedded, no external services)
## Completion Class
- Contract complete means: DB opens, queries return scoped data, prompt builders use DB queries, tests pass
- Integration complete means: full auto-mode cycle runs with DB-backed context injection, dual-write keeps markdown in sync, worktree lifecycle copies/reconciles DB
- Operational complete means: existing projects migrate transparently, graceful fallback when SQLite unavailable, token savings measured and ≥30%
## Final Integrated Acceptance
To call this milestone complete, we must prove:
- A full auto-mode dispatch cycle (research → plan → execute → complete) produces correct prompts with scoped context from the DB
- An existing project with markdown artifacts silently migrates to DB on first run with zero data loss
- Token measurement shows ≥30% savings on planning/research units
- The system works identically (via fallback) when SQLite is unavailable
- TypeScript compiles clean, all existing tests pass, new DB test suite passes
## Risks and Unknowns
- `auto-prompts.ts` has 11 prompt builders with 19 `inlineGsdRootFile` calls — rewiring must preserve existing prompt structure and fallback behavior
- `handleAgentEnd` in `auto.ts` has new post-unit-hook machinery since memory-db was built — dual-write re-import must integrate without disrupting hooks/doctor/rebuildState sequence
- `worktree-manager.ts` `createWorktree` is sync on main — DB copy must work synchronously (decision: use `copyFileSync`, keep sync)
- `node:sqlite` is experimental in Node 22 — API could change, but the DbAdapter abstraction insulates against this
- Memory-db's markdown parsers for DECISIONS.md and REQUIREMENTS.md are custom (not using `files.ts`) — must verify they handle current file formats
## Existing Codebase / Prior Art
- `src/resources/extensions/gsd/auto-prompts.ts` — 880 lines, 11 `build*Prompt()` functions, 19 `inlineGsdRootFile` calls. This is where surgical injection happens.
- `src/resources/extensions/gsd/auto-dispatch.ts``resolveDispatch()` maps units to prompt builders. Imports from `auto-prompts.ts`.
- `src/resources/extensions/gsd/auto.ts``startAuto()`, `handleAgentEnd()`, `dispatchNextUnit()`. DB init/migration goes in startup, re-import in handleAgentEnd.
- `src/resources/extensions/gsd/state.ts``deriveState()` — 587 lines. DB-first content loading replaces batch file parse.
- `src/resources/extensions/gsd/metrics.ts``UnitMetrics` interface, `snapshotUnitMetrics()`. Add `promptCharCount`/`baselineCharCount`.
- `src/resources/extensions/gsd/worktree-manager.ts``createWorktree()` (sync), `mergeWorktreeToMain()`. DB copy/reconcile hooks here.
- `src/resources/extensions/gsd/index.ts` — tool registrations. 3 new structured tools.
- `src/resources/extensions/gsd/commands.ts` — slash command registration. `/gsd inspect`.
- `src/resources/extensions/gsd/types.ts` — needs Decision/Requirement interfaces.
- `.gsd/worktrees/memory-db/` — the source worktree with all memory-db implementation. Reference code lives here.
### Memory-db source modules to port:
- `.gsd/worktrees/memory-db/src/resources/extensions/gsd/gsd-db.ts` — 750 lines, SQLite abstraction layer
- `.gsd/worktrees/memory-db/src/resources/extensions/gsd/context-store.ts` — 195 lines, query layer + formatters
- `.gsd/worktrees/memory-db/src/resources/extensions/gsd/md-importer.ts` — 526 lines, markdown parsers + migration orchestrator
- `.gsd/worktrees/memory-db/src/resources/extensions/gsd/db-writer.ts` — 337 lines, DB→markdown generators + DB-first write helpers
- `.gsd/worktrees/memory-db/src/resources/extensions/gsd/tests/` — 13 test files covering all DB capabilities
> See `.gsd/DECISIONS.md` for all architectural and pattern decisions — it is an append-only register; read it during planning, append to it during execution.
## Relevant Requirements
- R045R057 — all 13 active requirements map to this milestone's 7 slices
## Scope
### In Scope
- SQLite DB layer with tiered provider chain (node:sqlite → better-sqlite3 → null)
- Auto-migration from markdown files to DB
- Surgical prompt injection via DB queries in all prompt builders
- Dual-write keeping markdown and DB in sync (both directions)
- Token measurement with before/after comparison in UnitMetrics
- DB-first state derivation in deriveState()
- Worktree DB copy on creation and merge reconciliation
- 3 structured LLM tools (gsd_save_decision, gsd_update_requirement, gsd_save_summary)
- /gsd inspect slash command
- Full test suite for all DB capabilities
### Out of Scope / Non-Goals
- Vector/embedding search on artifacts (deferred — schema supports future extension)
- DB export/dump command
- Changing file discovery in deriveState (stays on disk)
- Making createWorktree async (keep sync, use copyFileSync for DB copy)
## Technical Constraints
- `node:sqlite` is experimental — use DbAdapter abstraction to insulate
- `node:sqlite` returns null-prototype rows — normalize via spread in DbAdapter
- Named SQL parameters must use colon-prefix (`:id`, `:scope`) for `node:sqlite` compatibility
- `createWorktree` must remain synchronous — no async cascade
- All DB operations must be wrapped in try/catch with fallback to existing behavior
- Memory-db source code is reference — adapt to current architecture, don't copy blindly
## Integration Points
- `auto-prompts.ts` — replace `inlineGsdRootFile` with DB-aware helpers (scoped queries with filesystem fallback)
- `auto.ts` `startAuto()` — DB open + auto-migration before first dispatch
- `auto.ts` `handleAgentEnd()` — re-import markdown after auto-commit (after doctor + rebuildState, before dispatch)
- `metrics.ts` — extend `UnitMetrics` with measurement fields, extend `snapshotUnitMetrics` signature
- `state.ts` `deriveState()` — DB-first content loading with filesystem fallback
- `worktree-manager.ts` `createWorktree()` — sync DB copy after worktree creation
- `worktree-command.ts` / merge paths — DB reconciliation after merge
- `index.ts` — 3 new tool registrations
- `commands.ts``/gsd inspect` command registration
- `types.ts` — Decision/Requirement interface additions
## Open Questions
- Whether memory-db's custom DECISIONS.md parser handles the current format (pipe tables with supersession chains) — needs verification during S02 implementation
- Whether current `deriveState()` batch-parse logic is structurally compatible with the DB-first replacement — needs verification during S04

View file

@ -0,0 +1,197 @@
# M004: SQLite Context Store — Surgical Prompt Injection
**Vision:** Replace GSD's whole-file prompt dumps with a SQLite-backed query layer that surgically injects only the context each dispatch unit needs — delivering ≥30% token savings, eliminating context pollution, and enabling structured LLM output that bypasses fragile markdown parsing.
## Success Criteria
- All prompt builders use DB queries for context injection (zero direct `inlineGsdRootFile` for data artifacts in prompt builders)
- Existing GSD projects migrate silently to DB on first run with zero data loss
- Planning and research dispatch units show ≥30% fewer prompt characters on mature projects
- System works identically via fallback when SQLite unavailable — no crash, transparent degradation
- Worktree creation copies gsd.db; worktree merge reconciles rows
- LLM can write decisions/requirements/summaries via structured tool calls
- `/gsd inspect` shows DB state for debugging
- Dual-write keeps markdown files in sync with DB state in both directions
- `deriveState()` reads from DB when available, falls back to filesystem
- All existing tests continue to pass, TypeScript compiles clean
## Key Risks / Unknowns
- `auto-prompts.ts` has 11 prompt builders with 19 `inlineGsdRootFile` calls — rewiring is high-surface-area
- `handleAgentEnd` has new post-unit-hook/doctor/rebuildState machinery — dual-write re-import must integrate cleanly
- Memory-db's custom markdown parsers may not handle format changes since the fork point
- `node:sqlite` is experimental — API stability risk (mitigated by DbAdapter abstraction)
## Proof Strategy
- SQLite provider risk → retire in S01 by proving tiered chain loads and queries on target platform
- Parser/format risk → retire in S02 by round-trip testing every artifact type against current file formats
- Prompt builder rewiring risk → retire in S03 by verifying all 11 builders produce correct output with DB vs markdown
- Worktree integration risk → retire in S05 by testing copy/reconcile against current worktree architecture
## Verification Classes
- Contract verification: unit tests for DB layer, importers, query layer, state derivation, writer, tools. Round-trip fidelity tests for migration.
- Integration verification: prompt builders produce equivalent output with DB vs markdown. Full auto-mode cycle completes. Worktree DB copy/merge works.
- Operational verification: graceful fallback when SQLite unavailable. Token measurement reports savings ≥30%.
- UAT / human verification: user runs auto-mode on a real project and confirms output quality equivalent or better
## Milestone Definition of Done
This milestone is complete only when all are true:
- All prompt builders in `auto-prompts.ts` use DB queries for context injection
- Silent auto-migration works on existing GSD projects with all artifact types
- Dual-write keeps markdown files in sync with DB state (both directions)
- Graceful fallback to markdown when SQLite unavailable
- Token measurement shows ≥30% reduction on planning/research units
- `deriveState()` derives from DB, producing identical GSDState output
- Worktree DB copy and merge reconciliation work with current worktree architecture
- Structured LLM tools registered and functional with DB-first write
- `/gsd inspect` command works
- All existing tests pass, new DB test suite passes, `npx tsc --noEmit` clean
- Success criteria re-checked against live behavior
## Requirement Coverage
- Covers: R045, R046, R047, R048, R049, R050, R051, R052, R053, R054, R055, R056, R057
- Partially covers: none
- Leaves for later: none
- Orphan risks: none
## Slices
- [ ] **S01: DB Foundation + Schema** `risk:high` `depends:[]`
> After this: SQLite DB opens with tiered provider chain, schema inits with decisions/requirements/artifacts tables plus filtered views, typed CRUD wrappers work, graceful fallback returns empty results when SQLite unavailable. Proven by unit tests against real DB.
- [ ] **S02: Markdown Importers + Auto-Migration** `risk:medium` `depends:[S01]`
> After this: Existing GSD project with markdown files starts up → gsd.db appears silently with all artifact types imported. Round-trip fidelity proven for every artifact type — import then regenerate produces identical output.
- [ ] **S03: Surgical Prompt Injection + Dual-Write** `risk:high` `depends:[S01,S02]`
> After this: All 11 `build*Prompt()` functions in `auto-prompts.ts` use scoped DB queries instead of `inlineGsdRootFile`. Decisions filtered by milestone, requirements filtered by slice. Dual-write re-import in `handleAgentEnd` keeps DB in sync after each dispatch unit. Falls back to filesystem when DB unavailable.
- [ ] **S04: Token Measurement + State Derivation** `risk:medium` `depends:[S03]`
> After this: `promptCharCount`/`baselineCharCount` in UnitMetrics, measurement wired into all `snapshotUnitMetrics` call sites. `deriveState()` reads content from DB when available. Savings ≥30% confirmed on fixture data.
- [ ] **S05: Worktree DB Isolation** `risk:medium` `depends:[S01,S02]`
> After this: `createWorktree` copies gsd.db to new worktrees (sync, non-fatal). Merge paths reconcile worktree DB rows back via ATTACH DATABASE with conflict detection.
- [ ] **S06: Structured LLM Tools + /gsd inspect** `risk:medium` `depends:[S03]`
> After this: LLM writes decisions/requirements/summaries via tool calls that write to DB first, then regenerate markdown. `/gsd inspect` dumps schema version, table counts, recent entries.
- [ ] **S07: Integration Verification + Polish** `risk:low` `depends:[S03,S04,S05,S06]`
> After this: Full auto-mode lifecycle test proves all subsystems compose correctly — migration → scoped queries → formatted prompts → token savings → re-import → round-trip. Edge cases (empty projects, partial migrations, fallback mode) verified. ≥30% savings confirmed on realistic fixture data.
## Boundary Map
### S01 → S02
Produces:
- `gsd-db.ts``openDatabase()`, `closeDatabase()`, `initSchema()`, `migrateSchema()`, typed insert/query wrappers for decisions, requirements, artifacts tables
- `gsd-db.ts``isDbAvailable()` boolean, `getDbProvider()` provider name
- `gsd-db.ts``insertDecision()`, `insertRequirement()`, `insertArtifact()`, `upsertDecision()`, `upsertRequirement()`
- `gsd-db.ts``transaction()` wrapper for batch operations
- `context-store.ts``queryDecisions(opts?)`, `queryRequirements(opts?)`, `queryArtifact(path)`, `queryProject()`
- `context-store.ts``formatDecisionsForPrompt()`, `formatRequirementsForPrompt()`
- `types.ts``Decision`, `Requirement` interfaces
- Fallback: all query functions return empty when DB unavailable
Consumes:
- nothing (first slice)
### S01 → S03
Produces:
- Same as S01 → S02 (DB layer + query functions + formatters)
- `isDbAvailable()` for conditional DB vs markdown loading in prompt builders
Consumes:
- nothing (first slice)
### S01 → S05
Produces:
- `gsd-db.ts``copyWorktreeDb(srcPath, destPath)` — sync file copy
- `gsd-db.ts``reconcileWorktreeDb(mainDbPath, worktreeDbPath)` — ATTACH-based merge
- `openDatabase()` for opening DB at arbitrary paths
Consumes:
- nothing (first slice)
### S02 → S03
Produces:
- `md-importer.ts``migrateFromMarkdown(basePath)` — full project import function
- `md-importer.ts` → individual parsers for all artifact types
- Auto-migration detection and execution wired into `startAuto()`
Consumes from S01:
- `gsd-db.ts``openDatabase()`, typed insert wrappers, `transaction()`
- Schema tables for all artifact types
### S02 → S05
Produces:
- `md-importer.ts``migrateFromMarkdown()` for importing markdown into a fresh worktree DB
Consumes from S01:
- `gsd-db.ts` → database layer
### S03 → S04
Produces:
- All `build*Prompt()` functions rewired to use DB queries
- DB-aware inline helpers: `inlineDecisionsFromDb()`, `inlineRequirementsFromDb()`, `inlineProjectFromDb()`
- Dual-write re-import in `handleAgentEnd`
Consumes from S01:
- `context-store.ts` → query functions and formatters
- `gsd-db.ts``isDbAvailable()`
Consumes from S02:
- `md-importer.ts``migrateFromMarkdown()` for re-import after auto-commit
### S03 → S06
Produces:
- `context-store.ts` → complete query layer that structured tools can use
- Dual-write infrastructure (re-import pattern)
Consumes from S01:
- `gsd-db.ts` → typed upsert wrappers
### S04 → S07
Produces:
- Token measurement in `UnitMetrics` (`promptCharCount`, `baselineCharCount`)
- `deriveState()` DB-first content loading
- Measurement infrastructure in `dispatchNextUnit`
Consumes from S03:
- Rewired prompt builders
### S05 → S07
Produces:
- `copyWorktreeDb` wired into `createWorktree`
- `reconcileWorktreeDb` wired into merge paths
Consumes from S01:
- `gsd-db.ts``copyWorktreeDb()`, `reconcileWorktreeDb()`, `openDatabase()`
Consumes from S02:
- `md-importer.ts``migrateFromMarkdown()` for fallback import
### S06 → S07
Produces:
- 3 structured LLM tools registered: `gsd_save_decision`, `gsd_update_requirement`, `gsd_save_summary`
- `/gsd inspect` slash command with autocomplete
Consumes from S03:
- `context-store.ts` → query layer for inspect output
- Dual-write infrastructure for tool-triggered markdown regeneration
Consumes from S01:
- `gsd-db.ts``upsertDecision()`, `upsertRequirement()`, `insertArtifact()`
- `db-writer.ts``generateDecisionsMd()`, `generateRequirementsMd()`, DB-first write helpers