singularity-forge/docs/dev/MEMORY-SYSTEM-INTEGRATION-GUIDE.md

# Memory System Integration Guide

Quick-start guide for using SF's autonomous memory system in extensions and dispatch logic.

## Quick Start: Record Unit Outcomes

```typescript
import { recordUnitOutcomeInMemory } from './uok/unit-runtime.js';

// After a unit completes execution:
recordUnitOutcomeInMemory(unit, 'completed', {
  success: true,
  duration: 2341,
  modelUsed: 'claude-sonnet-4-6'
});

// Fire-and-forget: 
// - Stores pattern in memory (never blocks)
// - Success: 0.9 confidence (strong signal)
// - Failure: 0.5 confidence (weaker signal)
```

## Quick Start: Enhance Dispatch with Memory

```typescript
import { enhanceUnitRankingWithMemory } from './auto-dispatch.js';

const candidates = [
  { id: 'unit-a', type: 'research', readiness: 0.8 },
  { id: 'unit-b', type: 'refactor', readiness: 0.6 }
];

const baseScores = { 'unit-a': 0.8, 'unit-b': 0.6 };
const enhanced = await enhanceUnitRankingWithMemory(candidates, baseScores);

// Returns: candidates with memory-boosted scores
// Max boost: 15% of pattern confidence (conservative)
// Result: unit-a boosted if similar patterns succeeded before
```

## Quick Start: Add Context to Gate Results

```typescript
import { enrichGateResultWithMemory } from './uok/gate-runner.js';

// After gate evaluation:
const gateResult = { outcome: 'fail', reason: 'timeout' };
const enriched = await enrichGateResultWithMemory(gateResult, 'deployment-gate');

// Returns:
// {
//   outcome: 'fail',  // Gate decision unchanged
//   reason: 'timeout',
//   memoryContext: {
//     hasHistoricalPattern: true,
//     similarFailures: [
//       { id: 'm1', confidence: 0.92, content: 'Network timeout...' }
//     ]
//   }
// }
```

---

## Core Memory Operations

### 1. Create a Memory Entry

```typescript
import { createMemory } from './memory-store.js';

const id = createMemory({
  category: 'gotcha',
  content: 'TypeScript strict mode requires null checks on optional params',
  confidence: 0.95
});
```

**Categories:**
- `gotcha` — Known issues, workarounds, edge cases
- `convention` — Coding patterns, naming standards
- `architecture` — Design decisions, responsibilities
- `pattern` — Recurring execution patterns
- `environment` — Config, setup, deployment
- `preference` — Optimization decisions

**Confidence:** 0.0-1.0 (default 0.8)

### 2. Query Memory by Similarity

```typescript
import { getRelevantMemoriesRanked } from './memory-store.js';

const embedding = computeUnitEmbedding('research-type');
const memories = await getRelevantMemoriesRanked(
  embedding,
  'pattern',  // Category filter
  5           // Top 5 results
);

// Returns sorted by cosine similarity + confidence
```

### 3. Create Relationships

```typescript
import { createRelation } from './memory-relations.js';

createRelation(
  failureMemoryId,
  rootCauseMemoryId,
  'caused_by',
  0.9
);
```

**Relation Types:**
- `caused_by` — Unit failure → root cause
- `similar_to` — Pattern similarity
- `workaround_for` — Known fix
- `depends_on` — Architectural dependency

### 4. Query Relationships

```typescript
import { traverseGraph } from './memory-relations.js';

const graph = traverseGraph(startMemoryId, maxDepth);
// Returns: { nodes, edges }
```

---

## Common Integration Patterns

### Pattern 1: Learn from Success/Failure

```typescript
// In unit-runtime or dispatch logic:
if (unitCompleted) {
  const pattern = {
    unitType: unit.type,
    outcome: unit.status,
    modelUsed: selectedModel,
    duration: executionTime,
    toolsNeeded: requiredTools
  };
  
  recordUnitOutcomeInMemory(unit, unit.status, pattern);
}
```

### Pattern 2: Boost Dispatch Based on History

```typescript
// In dispatch decision logic:
const baseScores = computeBaseScores(candidates);
const enhanced = await enhanceUnitRankingWithMemory(candidates, baseScores);
const selected = enhanced.reduce((best, curr) => 
  curr.score > best.score ? curr : best
);
```

### Pattern 3: Diagnostic Context on Failures

```typescript
// In gate evaluation:
const gateResults = await evaluateAllGates(unit);
const enriched = await Promise.all(
  gateResults.map(result => 
    enrichGateResultWithMemory(result, result.gateId)
  )
);
// Enriched results now have historical context
```

### Pattern 4: Manual Memory Capture

```typescript
// From SF CLI or tools:
import { executeMemoryCapture } from './tools/memory-tools.js';

executeMemoryCapture({
  category: 'convention',
  content: 'Always validate input before processing',
  confidence: 0.85
});
```

---

## Graceful Degradation

All memory operations degrade gracefully:

```typescript
// If DB unavailable → operation fails silently
// If memory lookup fails → continue without boost
// If embedding computation fails → use default

// Example: safe dispatch enhance
try {
  const enhanced = await enhanceUnitRankingWithMemory(units, scores);
  return enhanced; // Use memory-boosted scores
} catch (err) {
  console.warn('Memory enhance failed, using base scores');
  return units.map((u, i) => ({ ...u, score: scores[u.id] || 0 }));
}
```

---

## Performance Notes

| Operation | Latency | Notes |
|-----------|---------|-------|
| `createMemory()` | <5ms | Fire-and-forget async |
| `getRelevantMemoriesRanked()` | 10-50ms | Depends on DB size |
| `cosineSimilarity()` | <1ms | 128D vectors |
| `computeEmbedding()` | 5-20ms | Deterministic hash |
| Dispatch boost overhead | <10ms | Per dispatch cycle |

**Best Practices:**
- Never block on memory operations
- Use fire-and-forget async for recording
- Cache embeddings if used repeatedly
- Batch memory queries when possible

---

## Testing with Memory

```typescript
// Mock memory in tests:
vi.mock('./memory-store.js', () => ({
  getRelevantMemoriesRanked: vi.fn().mockResolvedValue([
    { id: 'm1', confidence: 0.9, content: 'test memory' }
  ]),
  createMemory: vi.fn().mockReturnValue('m-new-id'),
  isDbAvailable: vi.fn().mockReturnValue(true)
}));

// Test memory-enhanced dispatch:
it('boosts matching unit scores', async () => {
  const enhanced = await enhanceUnitRankingWithMemory(candidates, scores);
  expect(enhanced[0].score).toBeGreaterThan(scores[candidates[0].id]);
});
```

---

## Debugging Memory

```typescript
import { getActiveMemoriesRanked } from './memory-store.js';

// List all memories:
const all = getActiveMemoriesRanked(1000);

// Filter by category:
const gotchas = all.filter(m => m.category === 'gotcha');

// Check hit counts:
all.forEach(m => console.log(`${m.id}: ${m.hit_count} hits`));
```

---

## Architecture: Memory is Internal

- **No MCP server** — memory stays inside SF
- **SQLite only** — Node 26 native (no external deps)
- **Fire-and-forget** — never blocks dispatch
- **Private learning** — autonomous pattern extraction

---

## See Also

- `docs/dev/MEMORY-SYSTEM-ARCHITECTURE.md` — Full architecture reference
- `src/resources/extensions/sf/uok/unit-runtime.js` — Unit recording
- `src/resources/extensions/sf/auto-dispatch.js` — Dispatch ranking
- `src/resources/extensions/sf/uok/gate-runner.js` — Gate enrichment
- `src/resources/extensions/sf/tools/memory-tools.js` — CLI access