arthai-marketplace

Automate CI Recovery

Red CI doesn’t need you to debug it by hand. /ci-fix remediates failures with a bounded retry loop, and a Monitor watcher can wake it the moment CI fails — so the pipeline often repairs itself before you’ve noticed it broke.

The three modes

/ci-fix                        # CI failures on the current branch
/ci-fix ci feature/my-branch   # CI on a specific branch
/ci-fix staging                # staging deploy failure — reads deploy logs + health endpoints
/ci-fix prod                   # production deploy failure — read-only investigation first

Staging mode patches code or env vars based on deploy logs and health endpoints. Prod mode is the most conservative: read-only first, never touches the production database, and considers a git revert on the final attempt.

The 3-attempt loop

Each attempt pulls the failed logs, classifies the failure (lint / types / tests / build / migration / dependency), applies a fix scoped to only the failing files, verifies locally before pushing, then commits, pushes, and waits for the new CI result. Each retry uses a different strategy:

Known flaky tests are checked first — a match is reported instead of debugged. Hard rules throughout: never repo-wide lint auto-fix, never # noqa / # type: ignore suppressions, never deleted or skipped tests, never direct deploy commands — fixes always go through git.

On exhaustion (3 failed attempts): a Discord alert posts to #deployments (when discord-ops is configured), a QA incident file is written with the diagnosis trail, and it hands back to you. The attempt counter persists per-branch in .claude/monitors/.ci-fix-state.json, so a Monitor-triggered run won’t loop forever on the same failure — a green run resets it.

Event-driven watching with Monitor

Polling CI wastes tokens. Monitors register background watchers that fire on external events — zero token cost while idle:

Event What happens
CI fails on any branch /ci-fix wakes, diagnoses, patches, resubmits. Discord alert after 3 failed attempts.
Deploy fails or service crashes /sre debug wakes, applies an infra fix or escalates to /ci-fix.
Staging deploy succeeds /qa staging runs automatically.

Setup is part of /calibrate: it detects your CI system and deploy platform, generates pre-adapted configs in .claude/monitors/, and prints the exact webhook URL and platform steps. Add the webhook once on your platform and it never needs to change. All of this is additive — if you never configure a monitor, /ci-fix behaves exactly as before, just manually invoked.

Prevention beats recovery

The cheapest CI fix is the one that never reaches CI: /precheck runs the relevant suites locally in ~30 seconds before you push. And after every fix, /ci-fix leaves CI improvement recommendations (caching, parallelism, timeouts, flaky-test retries) — recurring failures across branches usually mean a flaky test, not a code problem.