arthai-marketplace

Automate CI Recovery

Red CI doesn’t need you to debug it by hand. /ci-fix remediates failures with a bounded retry loop, and a Monitor watcher can wake it the moment CI fails, so the pipeline often repairs itself before you’ve noticed it broke.

The three modes

/ci-fix                        # CI failures on the current branch
/ci-fix ci feature/my-branch   # CI on a specific branch
/ci-fix staging                # staging deploy failure — reads deploy logs + health endpoints
/ci-fix prod                   # production deploy failure — read-only investigation first

Staging mode patches code or env vars based on deploy logs and health endpoints. Prod mode is the most conservative: read-only first, never touches the production database, and considers a git revert on the final attempt.

The 3-attempt loop

Each attempt pulls the failed logs, classifies the failure (lint / types / tests / build / migration / dependency), applies a fix scoped to only the failing files, verifies locally before pushing, then commits, pushes, and waits for the new CI result. Each retry uses a different strategy:

Known flaky tests are checked first, a match is reported instead of debugged. Hard rules throughout: never repo-wide lint auto-fix, never # noqa / # type: ignore suppressions, never deleted or skipped tests, never direct deploy commands, fixes always go through git.

On exhaustion (3 failed attempts): a Discord alert posts to #deployments (when discord-ops is configured), a QA incident file is written with the diagnosis trail, and it hands back to you. The attempt counter persists per-branch in .claude/monitors/.ci-fix-state.json, so a Monitor-triggered run won’t loop forever on the same failure, a green run resets it.

Event-driven watching with Monitor

Polling CI wastes tokens. Monitors register background watchers that fire on external events, zero token cost while idle:

Event What happens
CI fails on any branch /ci-fix wakes, diagnoses, patches, resubmits. Discord alert after 3 failed attempts.
Deploy fails or service crashes /sre debug wakes, applies an infra fix or escalates to /ci-fix.
Staging deploy succeeds /qa staging runs automatically.

Setup is part of /calibrate: it detects your CI system and deploy platform, generates pre-adapted configs in .claude/monitors/, and prints the exact webhook URL and platform steps. Add the webhook once on your platform and it never needs to change. All of this is additive, if you never configure a monitor, /ci-fix behaves exactly as before, just manually invoked.

Prevention beats recovery

The cheapest CI fix is the one that never reaches CI: /precheck runs the relevant suites locally in ~30 seconds before you push. And after every fix, /ci-fix leaves CI improvement recommendations (caching, parallelism, timeouts, flaky-test retries), recurring failures across branches usually mean a flaky test, not a code problem.