0003. EDD as the default for prompts, MCP tools, and routing

LLM tool calling fails probabilistically (wrong tool, hallucinated args, retry loops). Markdown-only standards drift. We needed a red/green/refactor loop for agent contracts with CI-gated thresholds.

Context and Problem Statement

LLM tool calling fails probabilistically (wrong tool, hallucinated args, retry loops). Markdown-only standards drift. We needed a red/green/refactor loop for agent contracts with CI-gated thresholds.

Decision Drivers

Treat prompts and tool schemas as versioned contracts
Gate merges on measurable routing accuracy
Keep always-on context thin (kit measure-context) so evals stay meaningful

Considered Options

**Option A:** Prompt review via human PR comments only
**Option B:** Eval-Driven Development (EDD) as the default for agentic contract changes
**Option C:** Optional evals, never blocking CI

Decision Outcome

Chosen option: "**Option B**", because kit CI already gates EDD routing (≥95%) and the product pitch is testing rigor for agents. Guide: docs/edd.md. SOP: SOPs/eval-driven-development.md.

Consequences

Good, because prompt/schema/routing regressions fail loudly
Bad, because authors must write eval cases before changing agent contracts
Follow-up: unit coverage reporting complements EDD; it does not replace it

Architecture sketch

sequenceDiagram
  participant Dev as Author
  participant Eval as kit eval ci
  participant CI as GitHub Actions
  Dev->>Eval: failing JSONL/YAML case
  Dev->>Eval: register tool contract / prompt
  Eval->>CI: routing threshold gate
  CI-->>Dev: merge or fail