---
actionable: true
kind: design-research
date: 2026-05-02
promoted: true
promoted_to: M012
---

# PDD v2 — Research Findings

Three independent lines of research identified gaps in PDD v1 and drove the v1 → v2 upgrade.

---

## Gap 1 — No explicit domain assumptions

**Source:** Jackson, M. & Zave, P. "Four Dark Corners of Requirements Engineering." *ACM Transactions on Software Engineering and Methodology* 6(1), 1997.
https://dl.acm.org/doi/10.1145/237432.237434

**Finding.** The world-machine framework formalises that a specification S is valid only relative to domain assumptions A. The machine (software) operates in a world it cannot fully control. Failure Boundary captures *machine-side* failures; it does not capture *world-side* failures where an assumption is violated and the spec itself becomes inapplicable — even when the code is perfectly correct.

**PDD gap.** PDD v1 had no Assumptions field. World-model violations (unexpected locking protocol, changed API contract, invalid caller invariant, deployment context mismatch) fell through the Failure Boundary without a name. Code proved correct against a wrong model is the most expensive failure class because internal tests cannot detect it.

**Resolution.** Added Assumptions as the 8th PDD field: "What conditions about the world MUST be true for this spec to be valid?"

---

## Gap 2 — Invariants not distinguished by verification class

**Sources:**
- Alpern, B. & Schneider, F.B. "Defining Liveness." *Information Processing Letters* 21(4), 1985. https://doi.org/10.1016/0020-0190(85)90056-0
- Alpern, B. & Schneider, F.B. "Recognizing Safety and Liveness." *Distributed Computing* 2(3), 1987. https://doi.org/10.1007/BF01782772

**Finding.** Every temporal property is the intersection of a safety property and a liveness property. Safety: "X never happens" — verifiable by point-in-time assertion, like a DbC precondition. Liveness: "Y eventually happens" — requires temporal reasoning (traces, reachability). The two require different verification techniques; treating them as one category produces proof gaps.

**PDD gap.** PDD v1 listed a single "Invariants" field without distinguishing the two classes. For pure synchronous code this is harmless. For anything touching async operations, queues, timers, retries, or state machines, the absence of liveness invariants means eventual-progress guarantees go unstated and unverified.

**Resolution.** Invariants now carry a note: split into Safety ("X never happens") and Liveness ("Y eventually happens") when the change touches async, queues, timers, or state machines. Pure synchronous code may use safety-only invariants.

---

## Gap 3 — Prose evidence enables agent self-deception

**Sources:**
- SWE-bench Pro. arXiv:2509.16941, 2025. https://arxiv.org/abs/2509.16941
- Hong, S. et al. (MetaGPT). "MetaGPT: Meta Programming for a Multi-Agent Collaborative Framework." *ICLR 2024*. https://arxiv.org/abs/2308.00352
- Agentic Rubrics. Scale AI, 2024. (internal benchmark, cited in SWE-bench Pro §4)

**Finding.** Empirical benchmarking across agentic coding systems found two distinct failure modes correlated with evidence specification:

- *Prose evidence* ("the feature should work correctly") enabled self-deception: agents accepted their own implementation as correct without an external falsifiability check. SWE-bench Pro identified this as the primary cause of agent task failure, outweighing model capability differences.
- *Machine-executable evidence* (named test file, queryable metric, runnable command with expected exit code) enabled self-correction loops: agents could run the check, observe failure, and revise.

MetaGPT's specification protocol and Scale AI's Agentic Rubrics both independently required verifiable acceptance criteria as a structural element of agent task definitions.

**PDD gap.** PDD v1 Evidence field accepted prose descriptions. No distinction between falsifiable and unfalsifiable criteria.

**Resolution.** Evidence field now requires each criterion to be either machine-executable (named test, metric, command) or explicitly tagged `[MANUAL: reviewer + scenario]` when automation is not feasible. Prose-only evidence is forbidden.

**Example of compliant Evidence:**
```
Evidence:
- tests/scaffold-versioning.test.ts → "stampScaffoldFile preserves body" passes
- typecheck:extensions exits 0 after change
- [MANUAL: mhugo, M001 milestone close — visually inspect .sf/CODEBASE.md regenerated correctly]
```

---

## Methodologies that disagree

For completeness, three widely-used approaches take a different position:

- **XP (Extreme Programming, Beck 1999):** Evidence is "all tests pass" — no per-change evidence decomposition. Works at the test-suite level; loses granularity for agent-executed changes where individual acceptance criteria must be self-checkable.
- **Cynefin / Lean Startup (Snowden 2002, Ries 2011):** In complex/chaotic domains, upfront assumption enumeration impedes learning; probe-sense-respond is preferred. Applicable to product discovery; not applicable to implementation-level specifications where world-model violations must be detected before code ships.
- **BDD (Cucumber / Gherkin):** Assumes evidence is prose-first (Given/When/Then in natural language), with automation as optional layer. The PDD v2 requirement inverts this: automation is the default, prose is the named exception.

---

## Integration work

This record carries `actionable: true`. The record-promoter will create a milestone for:

1. Audit existing PDD packets in `.sf/active/` and `.sf/completed-units/` — identify any missing Assumptions or prose-only Evidence fields.
2. Update `requesting-code-review` skill to validate all 8 fields.
3. Consider linting the PDD output template for machine-executable Evidence markers.
