arthai-marketplace

/qa-learn

Review QA knowledge base stats, prune stale entries, show learning metrics.

Synopsis

/qa-learn [prune]

When to use it

• Periodically, to see what the self-improving QA system has accumulated — incidents, promoted tests, bug patterns, coverage gaps
• When the knowledge base feels noisy or stale — prune clears out entries that no longer earn their keep
• Not for: adding a new incident — that’s /qa-incident; or running tests — that’s /qa

Quickstart

/qa-learn

What you’ll see: a structured table of knowledge-base metrics — incident counts (covered vs. uncovered), promoted test files, bug patterns, coverage gaps, and the adversarial challenger’s hit rate (attacks that found real issues / total attacks).

Examples

/qa-learn          # show stats — read-only
/qa-learn prune    # remove stale entries and report what was pruned

What it does

Default (stats):

1. Counts incidents in .claude/qa-knowledge/incidents/, split into covered (regression test exists) vs. uncovered.
2. Lists promoted tests from .claude/qa-knowledge/promoted-tests/ — generated scenarios that earned a permanent place in the suite.
3. Counts bug patterns (bug-patterns.md) and coverage gaps (coverage-gaps.md).
4. Computes the challenger hit rate from challenger-log.md — how often adversarial QA attacks actually found issues.
5. Presents everything as one table so you can see whether the QA system is learning.

prune mode (modifies the knowledge base — it reports exactly what was removed):

1. Archives or deletes covered incidents older than 30 days.
2. Removes provenance for promoted tests whose source test file no longer exists.
3. Removes coverage gaps that now have test coverage.
4. Truncates challenger log entries older than 90 days.

Output & artifacts

• Stats mode: a metrics table in the conversation — nothing written
• Prune mode: stale entries removed from .claude/qa-knowledge/, with a report of what was pruned

Troubleshooting

Problem	Fix
QA knowledge base not initialized	Run /qa commit or /qa full first — the knowledge base accumulates from QA runs
All counts are zero	The KB exists but nothing has been learned yet — normal for a fresh project; counts grow as QA runs catch bugs and gaps
Pruned something you wanted to keep	Prune only touches covered/stale entries by rule — re-log anything still relevant with /qa-incident

Related

• /qa — the runs that populate this knowledge base
• /qa-incident — manually add an incident to the knowledge base
• QA Guide — how the self-improving QA loop works end to end