Commit graph

207 commits

Author SHA1 Message Date
Mikael Hugo
53259aebf1 fix(self-feedback): 3 sf-internal defects resolved
Some checks are pending
CI / detect-changes (push) Waiting to run
CI / docs-check (push) Blocked by required conditions
CI / lint (push) Blocked by required conditions
CI / build (push) Blocked by required conditions
CI / integration-tests (push) Blocked by required conditions
CI / windows-portability (push) Blocked by required conditions
CI / rtk-portability (linux, blacksmith-4vcpu-ubuntu-2404) (push) Blocked by required conditions
CI / rtk-portability (macos, macos-15) (push) Blocked by required conditions
CI / rtk-portability (windows, blacksmith-4vcpu-windows-2025) (push) Blocked by required conditions
1. cooldown failover (sf-mp8w9cg9-arixq7, high)
   When a provider hits AUTH_COOLDOWN in unit execution, block the
   failing model with an expiry using the existing blockModel() API,
   then try a non-cooldowned provider via isProviderRequestReady.
   Only stops if every provider is unavailable, with an enumerated
   message showing which ones are down. loop.js consecutiveCooldowns
   is not touched here (it tracks the loop-level retry budget for
   provider-not-ready errors that bypass phases-unit; the cooldown
   path in loop.js is separate and handles errors thrown before
   runUnitPhase, while this fix handles cancellation returned from
   runUnitPhase due to provider error during session creation).

2. redundant reassess-roadmap on completed slices (sf-mp8wa4qr-xw8fjb, medium)
   Doctor-triggered reassess path (loop.js P4-A) now checks whether
   the target slice already has an ASSESSMENT file before queuing
   reassess-roadmap. Mirrors the guard already present in the
   normal dispatch path (checkNeedsReassessment).

3. empty structured fields in slice summary (sf-mp8w6s88-ckv4yr, low)
   Added explicit instruction in complete-slice.md prompt template
   directing the executor to derive key_files, key_decisions, and
   patterns_established from task summaries before calling
   complete_slice.
2026-05-17 00:55:56 +02:00
Mikael Hugo
a8a28bd7c0 docs(specs): add sf-prompt-modularization.md operator guide
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-15 19:47:20 +02:00
Mikael Hugo
f3571475d5 docs: DB-first planning state migration proposal
Some checks are pending
CI / detect-changes (push) Waiting to run
CI / docs-check (push) Blocked by required conditions
CI / lint (push) Blocked by required conditions
CI / build (push) Blocked by required conditions
CI / integration-tests (push) Blocked by required conditions
CI / windows-portability (push) Blocked by required conditions
CI / rtk-portability (linux, blacksmith-4vcpu-ubuntu-2404) (push) Blocked by required conditions
CI / rtk-portability (macos, macos-15) (push) Blocked by required conditions
CI / rtk-portability (windows, blacksmith-4vcpu-windows-2025) (push) Blocked by required conditions
Design doc for moving SF's milestone planning state from
markdown-as-source-of-truth to DB-as-source-of-truth, with markdown
becoming a render target.

463 lines, ~4500 words. Includes:
- Survey of all markdown artifacts under .sf/milestones/M*/ and
  who writes/reads each today (drift authoritative-ness is
  ambiguous in most cases)
- MVP picks *-VALIDATION.md as first artifact to migrate — three
  read-site fixes, no schema change, the doctor's
  db_projection_validation_drift check retires immediately
- Hybrid editing UX (option c): CONTEXT-DRAFT and in-progress PLAN
  stay LLM-writable markdown; tool-call-bounded artifacts
  (validate_milestone, complete_slice, etc.) become DB-first with
  generated <!-- generated --> headers
- 5-phase rollout plan
- Open question flagged: git atomicity for milestone-level
  syncMilestoneLevelFiles calls — needs explicit tracing before
  Phase 4/5

No source-code changes. Implementation comes later.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 06:35:02 +02:00
Mikael Hugo
1478579069 docs: AgentRuntime unification proposal
Design doc for collapsing the five parallel agent-dispatch sites
(defaultAgentRunner, runHeadlessPrompt, runSingleAgent, runUnitViaSwarm,
slice-parallel-orchestrator) onto one runtime with three orthogonal
axes — persistence, isolation, routing.

590 lines, ~5200 words. Includes:
- Problem statement with five concrete pain points from this session's
  swarm convergence rounds (spawn hangs, inbox cache, checkpoint
  synthesis, ledger isolation, etc.)
- Worked-out TypeScript interface
- Mapping of each existing site to runtime options (table)
- 8-step migration plan in blast-radius order (~4-5 days focused work)
- Open questions

No source-code changes. Implementation comes later.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 06:32:28 +02:00
Mikael Hugo
a3b68bb269 fix(env): align SF_PERMISSION_LEVEL enum with permission-profile values
Schema now accepts the same five levels used elsewhere in the codebase
(minimal/low/medium/high/bypassed) instead of the stale full/restricted/
sandbox triple. Docs and env test updated to match.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 21:11:36 +02:00
Mikael Hugo
5ce9df2e37 refactor: make bundled agents internal 2026-05-14 19:54:56 +02:00
Mikael Hugo
18aa257ede refactor: rename review gate agent 2026-05-14 19:43:01 +02:00
Mikael Hugo
62fbc5d57b refactor: align agent resource overlays 2026-05-14 19:32:41 +02:00
Mikael Hugo
1d753af6b6 docs(dev): draft model registry contract for upcoming refactor
Spec for consolidating the three alias tables (benchmark-selector,
auto-model-selection, model-router) into a single SF-extension registry
that reads from @singularity-forge/ai's MODELS and enriches it with
canonical_id, generation, and tier. Shared interface for parallel
Swarm A/B/C work.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 02:57:27 +02:00
Mikael Hugo
085beb5199 docs(sf-ace): restore parked location + keep ADR cross-references
SF's S05/T02 executor moved the doc back to docs/dev/sf-ace-patterns.md
while completing the slice (correctly: that was the task's stated
deliverable location). The doc is parked under docs/dev/drafts/ because
ACE Coder has no active consumer for it; re-park it.

Keep the ADR-019 / ADR-020 cross-references the executor added —
they are real content improvements over the previous version.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 22:24:12 +02:00
Mikael Hugo
288a2a5fd7 docs(sf-ace): park SF→ACE pattern reference under docs/dev/drafts/
Promotes the .draft stub into a fuller 183-line reference covering six
SF patterns (Preferences, PDD, UOK Gates, Notifications, Skills-as-
Contracts, Idempotency) with SF source paths and ACE adoption notes.

Filed under docs/dev/drafts/ with a STATUS: Draft header — no active
consumer yet. SF's own priorities take precedence until ACE Coder
maintainers pull on convergence.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 21:30:34 +02:00
Mikael Hugo
65e195a9fd feat: Created draft mapping of SF patterns to ACE reference draft
SF-Task: S05/T01
2026-05-13 02:01:41 +02:00
Mikael Hugo
55229f6604 fix(auto): split autonomous solver from executor per ADR-0079
- Lock solver model to kimi-k2.6 independent of unit-type router
- Executor prompt no longer requires checkpoint tool call
- Add dedicated solver pass that reads executor transcript and emits canonical checkpoint
- Classify executor refusals as blocker outcomes (already partially implemented)
- Classify no-op iterations (continue with zero work) as missing-checkpoint-retry
- Add tests for executor prompt block, solver pass prompt, no-op detection, and no-op assessment

Fixes sf-mp34nxb6-27zdx7
2026-05-12 23:55:02 +02:00
Mikael Hugo
16db710468 sf snapshot: uncommitted changes after 49m inactivity 2026-05-12 16:45:04 +02:00
Mikael Hugo
2bb9cdbeef feat(scaffold): ADR-022 scaffold profiles (all phases)
Add profile-aware scaffold system so SF does not lay down irrelevant
templates in infra/ops/docs repos.

## What ships

Phase 1 — data model
- scaffold-versioning.js: add 'disabled' to VALID_STATES; readScaffoldManifest
  returns profile field; recordScaffoldApply preserves manifest.profile (fixes
  roundtrip bug where profile was stripped on every write).
- scaffold-constants.js: PROFILES (app/library/infra/docs/minimal as Set<string>)
  and PROFILE_NAMES exports.

Phase 2 — profile-aware drift detection
- scaffold-drift.js: disabled bucket in emptyCounts, resolveActiveProfileSet
  integration, profile param on detectScaffoldDrift/migrateLegacyScaffold.
- doc-checker.js: filter to active profile, skip disabled-state files.

Phase 3 — auto-detection on first run
- scaffold-profiles.js: detectRepoProfile() heuristics (nix→infra,
  terraform→infra, react→app, node-no-ui→library, docs-only→docs, else→app).
- agentic-docs-scaffold.js: reads profile from manifest, auto-detects on first
  run, persists to manifest, filters SCAFFOLD_FILES to active profile.

Phase 4 — migrate command
- commands-scaffold-migrate.js: sf scaffold migrate --profile <name>
  Re-enables pending files entering the new profile; stamps state=disabled
  (or prunes with --prune) files leaving it; warns on editing/completed files.
- commands/handlers/ops.js, commands/catalog.js: registered and tab-completed.

Phase 5 — custom profiles + PREFERENCES.md frontmatter
- scaffold-profiles.js: readPreferencesProfile(), loadCustomProfileSet()
  (~/.sf/profiles/<name>.yaml with extends/add/remove), resolveActiveProfileSet()
  implementing full ADR-022 §6 precedence.
- All callers updated to use resolveActiveProfileSet as the single source of truth.

Tests: 28 new tests in adr-022-scaffold-profiles.test.mjs — all passing.
Pre-existing node:test stubs (3 files) unaffected.

ADR: docs/dev/ADR-022-scaffold-profiles.md

Misc: triage TODO.md dump into BACKLOG.md (phases-helpers export error T1,
/todo triage typed-handler gap T1, structured triage tiers T2, sha-track
markdown files T2, cross-repo triage T3). Reset TODO.md to empty template.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-12 15:28:03 +02:00
Mikael Hugo
605cd712be refactor: capability-tier isHeavyModelId, search provider registry, frontmatter_version field, schema docs
- preferences-models.js: replace 6-regex isHeavyModelId() with MODEL_CAPABILITY_TIER
  lookup + regex fallback for unknown models; new models in model-router.js
  are automatically reflected without touching preferences-models.js
- search-the-web/provider.js: replace ~200-line per-provider waterfall with
  PROVIDER_REGISTRY array + firstAvailable()/resolveWithFallback() helpers;
  preserves Tavily→Brave→Serper→Exa→Ollama→MiniMax auto-fallback order
- sf-db.js: bump SCHEMA_VERSION 58→60 (v59 now reachable); add
  frontmatter_version column to tasks table via v60 migration and CREATE
  TABLE definition; wire frontmatter_version into upsertTaskPlanning() SQL
  and .run() params
- task-frontmatter.js: add frontmatterVersion:1 to DEFAULT_TASK_FRONTMATTER,
  add validation block in validateTaskFrontmatter(), add frontmatterVersion
  mapping in taskFrontmatterFromRecord()
- sf-db-migration.test.mjs: update hardcoded version assertion 58→60
- docs/specs/sf-operating-model.md: add Planning Schema section documenting
  the 3-table model (milestones/slices/tasks, their PKs, spec tables, and
  ID naming conventions)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-10 23:42:29 +02:00
Mikael Hugo
20c0d74106 sf snapshot: pre-dispatch, uncommitted changes after 31m inactivity 2026-05-10 06:26:32 +02:00
Mikael Hugo
e4c951ff0c feat: improve sf runtime self-reload and safeguards 2026-05-08 23:52:35 +02:00
Mikael Hugo
7318af029a sf snapshot: uncommitted changes after 33m inactivity 2026-05-08 18:18:47 +02:00
Mikael Hugo
d7c2663ca5 sf snapshot: uncommitted changes after 113m inactivity 2026-05-08 17:44:49 +02:00
Mikael Hugo
15269f4176 sf snapshot: uncommitted changes after 202m inactivity 2026-05-08 13:31:08 +02:00
Mikael Hugo
aa46a29cdd docs(runtime): align source docs with node 26 2026-05-08 07:17:33 +02:00
Mikael Hugo
10694440e3 feat(sf): align uok task state and steering 2026-05-08 06:57:59 +02:00
Mikael Hugo
378ab702e1 feat(sf): streamline uok state and direct modes 2026-05-08 05:51:06 +02:00
Mikael Hugo
19bfc3d3f6 feat(sf): align node sqlite uok runtime 2026-05-08 03:01:20 +02:00
Mikael Hugo
b5893d1c28 Make SF direct command surface baseline 2026-05-08 01:34:07 +02:00
Mikael Hugo
6fc054e7c3 sf snapshot: uncommitted changes after 49m inactivity 2026-05-08 01:07:24 +02:00
Mikael Hugo
89677b7e9b sf snapshot: uncommitted changes after 110m inactivity 2026-05-08 00:17:47 +02:00
Mikael Hugo
d05e7164a9 feat: journal execution policy decisions 2026-05-07 22:27:29 +02:00
Mikael Hugo
e9df932234 feat: add execution policy profiles 2026-05-07 18:21:47 +02:00
Mikael Hugo
b0fce94f9e feat: record retrieval evidence across context tools 2026-05-07 18:17:41 +02:00
Mikael Hugo
05f185256c docs: record local cli survey cross-check 2026-05-07 17:22:03 +02:00
Mikael Hugo
b1a7749763 fix: harden widget and provider auth handling 2026-05-07 17:20:52 +02:00
Mikael Hugo
8088489e38 sf snapshot: uncommitted changes after 258m inactivity 2026-05-07 15:37:55 +02:00
Mikael Hugo
87362f27fc docs: remove mcp server roadmap residue 2026-05-07 06:25:59 +02:00
Mikael Hugo
5c32d91124 feat: promote schedule and self-feedback state to db 2026-05-07 05:34:42 +02:00
Mikael Hugo
fce0c4c781 Tier 1.1: Implement vault credential resolver for provider keys
- Add vault-credential-resolver.js: Async credential resolution with vault:// URI support
- Integration with vault-resolver.js (low-level Vault client)
- Update doctor-providers.js to detect and report vault URIs
- Synchronous doctor checks (no network I/O) with lazy async resolution
- Fail-open semantics: vault unavailable -> fall back to plaintext
- 28 tests for credential resolver (all passing)
- ADR-0078: Architecture and auth chain documentation

Features:
- vault://secret/path/to/secret#fieldname URI format
- Auth chain: VAULT_TOKEN -> ~/.vault-token -> AppRole (reserved)
- Helper functions: couldBeVaultUri, hasProviderCredentialEnvVar, resolveProviderCredential, getCredentialValue, formatCredentialInfo
- Full backward compatibility with plaintext keys and auth.json

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-07 04:59:07 +02:00
Mikael Hugo
87aa04cf05 Tier 1.3: Add spec/runtime/evidence schema separation (v32)
Implements the 3-table normalization model for milestone, slice, and task entities:

- 9 new tables: {milestone,slice,task}_{specs,evidence} + runtime tables
- milestone_specs: immutable record of intent (vision, goals, risks, proof strategy)
- slice_specs: immutable slice-level intent
- task_specs: immutable task verification criteria
- {entity}_evidence: append-only audit trail with timestamps and phase metadata
- Indices on evidence tables for efficient chronological queries

Key improvements:
- Spec immutability: Write-once specs preserve original intent
- Audit trail: Evidence chain enables data archaeology and decision history
- Query efficiency: Each table contains only relevant columns
- Re-planning clarity: Multiple spec versions can exist for same entity ID
- Forensic capability: Timestamp + phase metadata on evidence rows

Migration:
- Schema version bumped to 32
- Migration runs on first open of existing databases
- No data loss; existing milestone/slice/task rows preserved
- Creates spec and evidence tables from existing columns (future work)

This is Phase 1 of Tier 1.3 implementation (schema definition + basic setup).
Phases 2-5 (migration, data layer updates, tool updates, tests) follow in next PRs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-07 04:20:32 +02:00
Mikael Hugo
4f217cc88c docs: promote sf state guidance 2026-05-07 03:59:38 +02:00
Mikael Hugo
932f17b93a refactor: rename workflow tool boundary 2026-05-07 03:45:41 +02:00
Mikael Hugo
e35cc3c6b8 docs: align schedule and package state wording 2026-05-07 03:36:56 +02:00
Mikael Hugo
3e6827e7dc docs: remove stale direct db and mcp guidance 2026-05-07 03:33:14 +02:00
Mikael Hugo
9ab0b9fe63 docs: tighten legacy state fallback wording 2026-05-07 03:25:20 +02:00
Mikael Hugo
39382f7e54 docs: clarify db-backed state guidance 2026-05-07 03:20:20 +02:00
Mikael Hugo
2fae96d539 docs: align runtime state and mcp boundaries 2026-05-07 03:09:55 +02:00
Mikael Hugo
f192dbfca0 docs: add ADR-076 for UOK memory integration decisions
Document the three-phase integration of SF memory system with UOK:

Phase 1: Unit outcome recording (recordUnitOutcomeInMemory)
- Records success/failure patterns with 0.9/0.5 confidence
- Fire-and-forget async, never blocks execution

Phase 2: Dispatch ranking enhancement (enhanceUnitRankingWithMemory)
- Queries memory for similar patterns
- Boosts matching candidates by up to 15% (conservative limit)
- Deterministic embeddings ensure reproducible ranking

Phase 3: Gate context enrichment (enrichGateResultWithMemory)
- Diagnostic only; never changes gate pass/fail logic
- Helps operators understand recurring issues

All memory operations gracefully degrade if DB unavailable.
56 test cases validate integration across all phases.

Relates to ADR-0075 (UOK gates), ADR-008 (SF tools).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-07 02:05:01 +02:00
Mikael Hugo
a8634d4a3b docs: add memory system integration guide for developers
Practical quick-start guide for using SF's autonomous memory system:

- Record unit outcomes (success/failure patterns)
- Enhance dispatch ranking with learned patterns
- Add context to gate failures
- Core memory operations (create, query, relations)
- Common integration patterns
- Graceful degradation strategy
- Performance notes and best practices
- Testing with mocked memory
- Debugging helpers

Guide covers:
- Fire-and-forget async pattern
- Never blocks dispatch/execution
- Testing strategies for memory-enhanced code
- Performance characteristics
- Architecture decision: memory is SF-internal

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-07 02:03:34 +02:00
Mikael Hugo
b384c8e0df docs: clarify memory system is SF-internal, not MCP-exposed
Add architecture decision: Memory is not exposed as MCP server.

- SF is an MCP client only (consumes external MCP tools)
- Memory is internal SF infrastructure (uses SQLite, fire-and-forget async)
- Memory exposed as SF tools only (capture, query, graph)
- No external MCP exposure needed (memory is autonomous learning, not a service)

This keeps SF's learning system private and prevents:
- External memory pollution
- Uncontrolled confidence scoring
- Inconsistent learning patterns
- Loss of autonomy (memory decisions stay internal)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-07 01:41:33 +02:00
Mikael Hugo
b6ea800e2e docs: comprehensive SF memory system architecture reference
Add MEMORY-SYSTEM-ARCHITECTURE.md documenting:
- All 10 memory modules (store, embeddings, relations, etc.)
- Core functions and APIs for each module
- Storage schema (SQLite tables)
- Integration points (UOK, dispatch, gates)
- Usage examples and architecture diagram
- Performance characteristics
- Graceful degradation strategy
- Data retention and growth management

This serves as:
1. Reference guide for developers using memory system
2. Architecture overview of autonomous learning
3. Integration point documentation for extensions
4. Future enhancement roadmap

Discovered during UOK memory integration work:
- Memory system already complete (no duplication needed)
- Used for pattern learning, dispatch ranking, and diagnostics
- Node 24 native SQLite backend (no external deps)
- Fire-and-forget async operations (never blocks)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-07 01:36:08 +02:00
Mikael Hugo
3f099e240c Update test coverage plan: Phase 3 complete
- Phase 1: 48 tests (metrics + triage) ✓
- Phase 2: 31 tests (crash recovery) ✓
- Phase 3: 17 tests (property-based FSM) ✓
- Total: 96 critical path tests + 25 env schema tests = 104 new tests
- All passing, coverage targets met

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-07 01:01:47 +02:00