docs: add memory system integration guide for developers
Practical quick-start guide for using SF's autonomous memory system: - Record unit outcomes (success/failure patterns) - Enhance dispatch ranking with learned patterns - Add context to gate failures - Core memory operations (create, query, relations) - Common integration patterns - Graceful degradation strategy - Performance notes and best practices - Testing with mocked memory - Debugging helpers Guide covers: - Fire-and-forget async pattern - Never blocks dispatch/execution - Testing strategies for memory-enhanced code - Performance characteristics - Architecture decision: memory is SF-internal Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
parent
e94a0d95e9
commit
a8634d4a3b
1 changed files with 284 additions and 0 deletions
284
docs/dev/MEMORY-SYSTEM-INTEGRATION-GUIDE.md
Normal file
284
docs/dev/MEMORY-SYSTEM-INTEGRATION-GUIDE.md
Normal file
|
|
@ -0,0 +1,284 @@
|
|||
# Memory System Integration Guide
|
||||
|
||||
Quick-start guide for using SF's autonomous memory system in extensions and dispatch logic.
|
||||
|
||||
## Quick Start: Record Unit Outcomes
|
||||
|
||||
```typescript
|
||||
import { recordUnitOutcomeInMemory } from './uok/unit-runtime.js';
|
||||
|
||||
// After a unit completes execution:
|
||||
recordUnitOutcomeInMemory(unit, 'completed', {
|
||||
success: true,
|
||||
duration: 2341,
|
||||
modelUsed: 'claude-sonnet-4-6'
|
||||
});
|
||||
|
||||
// Fire-and-forget:
|
||||
// - Stores pattern in memory (never blocks)
|
||||
// - Success: 0.9 confidence (strong signal)
|
||||
// - Failure: 0.5 confidence (weaker signal)
|
||||
```
|
||||
|
||||
## Quick Start: Enhance Dispatch with Memory
|
||||
|
||||
```typescript
|
||||
import { enhanceUnitRankingWithMemory } from './auto-dispatch.js';
|
||||
|
||||
const candidates = [
|
||||
{ id: 'unit-a', type: 'research', readiness: 0.8 },
|
||||
{ id: 'unit-b', type: 'refactor', readiness: 0.6 }
|
||||
];
|
||||
|
||||
const baseScores = { 'unit-a': 0.8, 'unit-b': 0.6 };
|
||||
const enhanced = await enhanceUnitRankingWithMemory(candidates, baseScores);
|
||||
|
||||
// Returns: candidates with memory-boosted scores
|
||||
// Max boost: 15% of pattern confidence (conservative)
|
||||
// Result: unit-a boosted if similar patterns succeeded before
|
||||
```
|
||||
|
||||
## Quick Start: Add Context to Gate Results
|
||||
|
||||
```typescript
|
||||
import { enrichGateResultWithMemory } from './uok/gate-runner.js';
|
||||
|
||||
// After gate evaluation:
|
||||
const gateResult = { outcome: 'fail', reason: 'timeout' };
|
||||
const enriched = await enrichGateResultWithMemory(gateResult, 'deployment-gate');
|
||||
|
||||
// Returns:
|
||||
// {
|
||||
// outcome: 'fail', // Gate decision unchanged
|
||||
// reason: 'timeout',
|
||||
// memoryContext: {
|
||||
// hasHistoricalPattern: true,
|
||||
// similarFailures: [
|
||||
// { id: 'm1', confidence: 0.92, content: 'Network timeout...' }
|
||||
// ]
|
||||
// }
|
||||
// }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Core Memory Operations
|
||||
|
||||
### 1. Create a Memory Entry
|
||||
|
||||
```typescript
|
||||
import { createMemory } from './memory-store.js';
|
||||
|
||||
const id = createMemory({
|
||||
category: 'gotcha',
|
||||
content: 'TypeScript strict mode requires null checks on optional params',
|
||||
confidence: 0.95
|
||||
});
|
||||
```
|
||||
|
||||
**Categories:**
|
||||
- `gotcha` — Known issues, workarounds, edge cases
|
||||
- `convention` — Coding patterns, naming standards
|
||||
- `architecture` — Design decisions, responsibilities
|
||||
- `pattern` — Recurring execution patterns
|
||||
- `environment` — Config, setup, deployment
|
||||
- `preference` — Optimization decisions
|
||||
|
||||
**Confidence:** 0.0-1.0 (default 0.8)
|
||||
|
||||
### 2. Query Memory by Similarity
|
||||
|
||||
```typescript
|
||||
import { getRelevantMemoriesRanked } from './memory-store.js';
|
||||
|
||||
const embedding = computeUnitEmbedding('research-type');
|
||||
const memories = await getRelevantMemoriesRanked(
|
||||
embedding,
|
||||
'pattern', // Category filter
|
||||
5 // Top 5 results
|
||||
);
|
||||
|
||||
// Returns sorted by cosine similarity + confidence
|
||||
```
|
||||
|
||||
### 3. Create Relationships
|
||||
|
||||
```typescript
|
||||
import { createRelation } from './memory-relations.js';
|
||||
|
||||
createRelation(
|
||||
failureMemoryId,
|
||||
rootCauseMemoryId,
|
||||
'caused_by',
|
||||
0.9
|
||||
);
|
||||
```
|
||||
|
||||
**Relation Types:**
|
||||
- `caused_by` — Unit failure → root cause
|
||||
- `similar_to` — Pattern similarity
|
||||
- `workaround_for` — Known fix
|
||||
- `depends_on` — Architectural dependency
|
||||
|
||||
### 4. Query Relationships
|
||||
|
||||
```typescript
|
||||
import { traverseGraph } from './memory-relations.js';
|
||||
|
||||
const graph = traverseGraph(startMemoryId, maxDepth);
|
||||
// Returns: { nodes, edges }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common Integration Patterns
|
||||
|
||||
### Pattern 1: Learn from Success/Failure
|
||||
|
||||
```typescript
|
||||
// In unit-runtime or dispatch logic:
|
||||
if (unitCompleted) {
|
||||
const pattern = {
|
||||
unitType: unit.type,
|
||||
outcome: unit.status,
|
||||
modelUsed: selectedModel,
|
||||
duration: executionTime,
|
||||
toolsNeeded: requiredTools
|
||||
};
|
||||
|
||||
recordUnitOutcomeInMemory(unit, unit.status, pattern);
|
||||
}
|
||||
```
|
||||
|
||||
### Pattern 2: Boost Dispatch Based on History
|
||||
|
||||
```typescript
|
||||
// In dispatch decision logic:
|
||||
const baseScores = computeBaseScores(candidates);
|
||||
const enhanced = await enhanceUnitRankingWithMemory(candidates, baseScores);
|
||||
const selected = enhanced.reduce((best, curr) =>
|
||||
curr.score > best.score ? curr : best
|
||||
);
|
||||
```
|
||||
|
||||
### Pattern 3: Diagnostic Context on Failures
|
||||
|
||||
```typescript
|
||||
// In gate evaluation:
|
||||
const gateResults = await evaluateAllGates(unit);
|
||||
const enriched = await Promise.all(
|
||||
gateResults.map(result =>
|
||||
enrichGateResultWithMemory(result, result.gateId)
|
||||
)
|
||||
);
|
||||
// Enriched results now have historical context
|
||||
```
|
||||
|
||||
### Pattern 4: Manual Memory Capture
|
||||
|
||||
```typescript
|
||||
// From SF CLI or tools:
|
||||
import { executeMemoryCapture } from './tools/memory-tools.js';
|
||||
|
||||
executeMemoryCapture({
|
||||
category: 'convention',
|
||||
content: 'Always validate input before processing',
|
||||
confidence: 0.85
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Graceful Degradation
|
||||
|
||||
All memory operations degrade gracefully:
|
||||
|
||||
```typescript
|
||||
// If DB unavailable → operation fails silently
|
||||
// If memory lookup fails → continue without boost
|
||||
// If embedding computation fails → use default
|
||||
|
||||
// Example: safe dispatch enhance
|
||||
try {
|
||||
const enhanced = await enhanceUnitRankingWithMemory(units, scores);
|
||||
return enhanced; // Use memory-boosted scores
|
||||
} catch (err) {
|
||||
console.warn('Memory enhance failed, using base scores');
|
||||
return units.map((u, i) => ({ ...u, score: scores[u.id] || 0 }));
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Performance Notes
|
||||
|
||||
| Operation | Latency | Notes |
|
||||
|-----------|---------|-------|
|
||||
| `createMemory()` | <5ms | Fire-and-forget async |
|
||||
| `getRelevantMemoriesRanked()` | 10-50ms | Depends on DB size |
|
||||
| `cosineSimilarity()` | <1ms | 128D vectors |
|
||||
| `computeEmbedding()` | 5-20ms | Deterministic hash |
|
||||
| Dispatch boost overhead | <10ms | Per dispatch cycle |
|
||||
|
||||
**Best Practices:**
|
||||
- Never block on memory operations
|
||||
- Use fire-and-forget async for recording
|
||||
- Cache embeddings if used repeatedly
|
||||
- Batch memory queries when possible
|
||||
|
||||
---
|
||||
|
||||
## Testing with Memory
|
||||
|
||||
```typescript
|
||||
// Mock memory in tests:
|
||||
vi.mock('./memory-store.js', () => ({
|
||||
getRelevantMemoriesRanked: vi.fn().mockResolvedValue([
|
||||
{ id: 'm1', confidence: 0.9, content: 'test memory' }
|
||||
]),
|
||||
createMemory: vi.fn().mockReturnValue('m-new-id'),
|
||||
isDbAvailable: vi.fn().mockReturnValue(true)
|
||||
}));
|
||||
|
||||
// Test memory-enhanced dispatch:
|
||||
it('boosts matching unit scores', async () => {
|
||||
const enhanced = await enhanceUnitRankingWithMemory(candidates, scores);
|
||||
expect(enhanced[0].score).toBeGreaterThan(scores[candidates[0].id]);
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Debugging Memory
|
||||
|
||||
```typescript
|
||||
import { getActiveMemoriesRanked } from './memory-store.js';
|
||||
|
||||
// List all memories:
|
||||
const all = getActiveMemoriesRanked(1000);
|
||||
|
||||
// Filter by category:
|
||||
const gotchas = all.filter(m => m.category === 'gotcha');
|
||||
|
||||
// Check hit counts:
|
||||
all.forEach(m => console.log(`${m.id}: ${m.hit_count} hits`));
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Architecture: Memory is Internal
|
||||
|
||||
- **No MCP server** — memory stays inside SF
|
||||
- **SQLite only** — Node 24 native (no external deps)
|
||||
- **Fire-and-forget** — never blocks dispatch
|
||||
- **Private learning** — autonomous pattern extraction
|
||||
|
||||
---
|
||||
|
||||
## See Also
|
||||
|
||||
- `docs/dev/MEMORY-SYSTEM-ARCHITECTURE.md` — Full architecture reference
|
||||
- `src/resources/extensions/sf/uok/unit-runtime.js` — Unit recording
|
||||
- `src/resources/extensions/sf/auto-dispatch.js` — Dispatch ranking
|
||||
- `src/resources/extensions/sf/uok/gate-runner.js` — Gate enrichment
|
||||
- `src/resources/extensions/sf/tools/memory-tools.js` — CLI access
|
||||
Loading…
Add table
Reference in a new issue