Kit review backlog
Open findings from kit-value-and-model-agnostic-review.md (refreshed 2026-09-01). Track work here; mark items done with a PR reference.
P0: honesty / prove-what-you-claim
| ID | Finding | Proposed action | Status |
|---|
| R1 | Skill-trigger harness (kit/src/edd/run_evals.ts) does not invoke a model and ignores required_patterns / required_output_sections; banner still says “Live Trigger Evaluation” | Either (a) assert declared fields via scripted/LLM output checks, or (b) rename harness + kit check copy to “skill registration / prompt hygiene” and stop implying live routing proof | Open |
| R2 | README (and similar) imply peer multi-IDE depth while MCP install and skill discovery are Cursor-first | Narrow copy to: canonical AGENTS.md + thin entry stubs; Cursor is the reference host for skills/MCP | Partial: light README honesty in the refresh PR; fuller pass on site/llms.txt still open |
P1: eval / CI proof
| ID | Finding | Proposed action | Status |
|---|
| R3 | Merge gate is scripted-only; nightly live skips without KIT_EVAL_API_KEY | Keep scripted as default; document required secrets for live; optionally fail a scheduled job loudly when key missing in production kit repos | Open (docs clearer; ops still optional) |
| R4 | Live client is OpenAI-compatible only; ANTHROPIC_API_KEY as bearer is easy to misconfigure | Document gateways honestly; consider native Anthropic/Gemini adapters only if demand appears | Open |
P2: maintainability / adoption
| ID | Finding | Proposed action | Status |
|---|
| R5 | Role skills over ~150-line kit-review budget (agent-prune, agent-orchestrator, agent-debug, agent-copy) | Move procedure back to SOPs; keep role skills as routers | Open |
| R6 | Several lang-* / framework-* skills are short checklists (~38–48 lines) | Cut unused profiles or deepen with evals + concrete anti-patterns (see lang-typescript direction) | Open |
| R7 | Full lifecycle ceremony is heavy for day-to-day coding | Keep shortcuts (debug route, light XFN); surface “minimal path” in README onboarding | Open |
Done since Aug 30 draft (do not reopen without new evidence)
| ID | Finding | Resolution |
|---|
| D1 | Architecture stack felt non-optional / blocks adoption | Applicability & opt-out + seed ADRs (#36) |
| D2 | EDD closed-loop entirely aspirational | Suites/depth (#23), shadow/OTel path (#28–#30), skill coverage (#31), nightly live workflow |
How to use
- Prefer small PRs that close one ID.
- Link the ID in the PR body (
Closes review backlog R1).
- When closing an item, set Status to
Done (PR #N) and leave one line of evidence.
