Decisions is the fourth optional Arth product, next to Intelligence, Memory and Router. It gives the toolkit’s workflows a fast, cheap, typed judge for the small yes/no and pick-one questions they answer dozens of times a day: did the workers actually run the tests? · does this diff need a deep review or a skim? · is this CI failure flaky or real? · which workflow does this request belong to?
Today those questions are answered by spawning a Sonnet agent (7 units, about a minute) or by keyword regexes. Decisions answers them with Jev, a small model by TypeSafe built for exactly this shape of question: it returns a probability for every option in well under a second, at $0.042 per million input tokens (a typical decision costs a few hundred-thousandths of a dollar). Where the answer is confident, the expensive spawn is skipped. Where it is not, today’s path runs unchanged.
Availability: shipped in shadow mode with the toolkit’s
/qa,/pr,/ci-fixand the triage router (Phase 1) and with the selectors in/implement,/fix,/review-pr,/autopilotand/incident(Phase 2). Enforce mode is unlocked per gate after calibration (below). The direct TypeSafe backend is the one we run; Cloudflare Workers AI and Vercel AI Gateway are offered as backups and are verified by a probe when you add one (their request formats are implemented from the vendors’ docs).
Nothing, until you opt in. Tick decisions in /arth-setup (it is the
fourth checkbox; setup asks no stack question). Setup then asks one thing about
Jev itself, How should Jev decide? (enforce, shadow or off, below; enforce
is the default), and every later session shows the mode you chose in its
bootstrap line:
decisions: jev ✓ (enforce, direct)
In enforce, the first thing you notice is the triage router: your next message is routed by Jev when it is confident (the routing trace says so), by the keyword ladder otherwise. Then workflows print one extra line where a spawn used to be:
COMPLETION CHECK: PASS (jev, n=5) # /qa Step 11, /pr — Sonnet verifier skipped
GAPS FOUND (jev): lint_ran_on_full_dir # a specific statement failed; still no spawn
PBT: no candidates (jev) # /qa Step 4b — property-test agent skipped
TEST TRIAGE (jev): flaky_timing 0.82 # after a failing test run — /ci-fix reruns once first
Setup asks How should Jev decide? when you tick decisions (and again on every re-run). The three answers, in the order they are offered:
| Mode | What happens | Who uses it |
|---|---|---|
| enforce (the default) | Jev’s answer is used wherever it is confident; today’s logic runs when it is not. The per-gate thresholds are fitted defaults and keep being calibrated on your own rows; a gate whose trailing precision drops below target is demoted to shadow automatically. | Setup users, from the day they opt in |
| shadow | Jev is asked; its answer and today’s decision are both logged; today’s decision still wins. Nothing changes for you. | Anyone who wants to compare before acting, and a demoted gate |
| off | Byte-identical to the toolkit without Decisions. No call is made. | Anyone who did not opt in, or who wants it parked |
Your answer is written to .claude/.arth-jev.env as JEV_MODE. Change it
later without going through setup again, or override it wherever is
convenient, most specific wins:
/arth-setup switch shadow decisions # this project's env file: off | shadow | enforce, asks nothing else
/qa --jev enforce # this run only (also /pr, /review-pr, /implement, /fix, /autopilot, /ci-fix, /incident)
JEV_MODE=shadow claude # this shell session
.claude/qa-config.json # this project, per gate: "jev": {"mode": "shadow", "gates": {"verifier_qa": "enforce"}}
One lock never moves: with the local runtime backend (the
Claude + one local runtime preset), enforce is recorded but always
resolves to shadow, because the local adapter is an uncalibrated
emulation. The bootstrap line shows the resolved mode.
Two people on the same repository can run different modes and compare their logs.
Unattended runs (CLAUDE_AUTOPILOT=1 or CI=true, which is how /eval and
/autopilot drive the skills) ask the same gates: a CLI flag such as
--redteam-once is the fallback when Jev is off or unsure, never a bypass,
and in enforce the recommended option is taken automatically since nobody is
there to press Enter.
If the service is slow or down, nothing waits on it. Every ask has a hard
deadline (3 seconds; hooks never retry), and any failure means the toolkit
runs exactly the path it ran before Decisions existed. After three failures
in a row the toolkit stops asking for five minutes and answers instantly from
today’s logic, then tries again; one successful answer resets it. You will see
decision=fallback in a skill’s mode line and jev: circuit open in the
log — never a stalled command.
Every decision is a row in .claude/qa-knowledge/jev-decisions.jsonl
(git-ignored): the question, Jev’s probabilities, what today’s logic decided,
and later the outcome, back-filled from whatever actually ran (the Sonnet
verifier’s verdict, whether the property-test agent found a property, whether
the rerun passed). That makes every shadow row a labelled example for free.
/qa-learn and jev.sh stats turn the rows into a per-question table:
precision and recall at the current and at the fitted thresholds, agreement
with today’s logic, escalation rate, cost and latency, and one recommendation:
keep shadow: n=41/100 · ready to enforce · reverted: precision 0.84 < target · drop: AUROC < 0.55
A gate is flipped to enforce only after at least 100 labelled rows and a precision at or above its target (0.9 for gates that skip a spawn, 0.8 for triage). If an enforced gate’s trailing-week precision drops below target, it is reverted to shadow automatically, with one log line. Question wording is version-locked: an edit starts a fresh calibration.
Phase 1 decided whether an agent runs; Phase 2 decides which option a
workflow recommends. In every prompt the options stay exactly as they are —
Jev only moves the recommended marker and appends a small jev: fragment
to that line, and any explicit flag (--redteam-once, --lite, --type
bug, …) bypasses the gate entirely.
Phase 1 — spawn-or-skip gates
| Where | Question | Enforce behaviour |
|---|---|---|
/qa Step 3 |
which test surface does each changed file belong to; does it touch a business rule | replaces the directory heuristic per file |
/qa Step 3 |
does the code review need skim, standard or deep; is a merge blocker likely | skim + no blocker → Haiku review instead of Sonnet |
/qa Step 4b |
is each changed function pure / idempotent / round-trip / order-preserving | spawns the property-test agent only with candidates, with hints |
/qa Step 11, /pr |
five statements about the QA report and the workers’ RESULT: lines |
skips the Sonnet completion verifier when all five clear |
/qa A2 |
is the verifier’s gap about domain logic | replaces the keyword list |
/qa Step 10 |
is this knowledge-base entry a duplicate of an existing one | reinforces the existing node instead of appending |
test-summary hook, /ci-fix |
is this failure flaky, environmental, or a real regression | one rerun before any investigation when flaky |
| triage router | which of 14 workflows does this message want | routes directly when confident, regex otherwise. Asked on every message, the first of a session included, and, when the Arth Router is on, ahead of its lane decision: in enforce a confident route is sent to the router as the task type together with Jev’s read of the request’s risk, complexity, blast radius, how much it depends on reliable tool use, how much someone is waiting on it, and whether it touches a protected path (one Jev request, seven questions), so the router picks the model and effort from that judgment and keeps its own keyword classification beside it; high risk, a wide blast radius or a protected path put a floor under both, whatever kind of task it is; in shadow nothing is sent and the row records what would have been; the router in turn learns effort and model quality from the labelled outcomes of its earlier decisions; the skill that then starts is written back as the row’s outcome, and reported to the linked router decision, so jev.sh stats route reports precision |
Phase 2 — selectors
| Where | Question | Enforce behaviour |
|---|---|---|
/implement mode prompt |
how risky is this plan: blast radius, reversibility, confidence, domain sensitivity (0–12); does it touch auth/payments or need a migration | the [recommended] marker moves to Guarded (low score) or Strict, with a suffix such as · jev: guarded 3/12; picking Auto follows the same recommendation. Same five options, same order |
/implement staffing |
should the orchestrator make the edits itself, hand them to one worker, or spawn the per-layer team plus QA | picks the shape before any agent is spawned; below confidence, today’s per-layer table. Measured 2026-09-19: the same plan cost 3.7× more when nothing decided this and the orchestrator coded inline for 71 turns |
/implement, /fix and /qa worker spawns |
which model tier (Haiku / Sonnet / Opus) and how much reasoning effort (low / medium / high) does THIS sub-task need; does it touch auth, payments or a migration | each spawned worker gets the lane its sub-task needs instead of one fixed lane per agent type: a rename drops to Haiku, a concurrency fix moves up to Opus or high effort. Never lowered for auth, payment or migration work; reviewers keep their pinned model; when unsure, model-policy.yml decides exactly as before. With the Arth Router configured, Jev’s judgment is sent to it as hints and the router — which knows your plan pace, credits and past results — picks the final model and effort inside the same limits, on every worker it is asked about; if it does not answer in 2 seconds Jev’s own pick is used, and if Jev cannot answer model-policy.yml decides. The effort half only takes effect where the toolkit has generated the matching agent definitions (install.sh --apply-model-policy --effort-variants) — Claude Code takes a sub-agent’s reasoning effort from its definition and nowhere else. It also does nothing on a Haiku lane, because Haiku has no reasoning-effort setting, or when you have set CLAUDE_CODE_EFFORT_LEVEL, which outranks it. In each of those cases the model tier still moves, and the effort choice is recorded with the reason it did not apply rather than counted as if it had. The label both learn from: did that worker succeed first time — and a failure your PLAN caused is excluded, so a missing requirement never teaches the system that the model was too small |
/autopilot risk classification |
the same four scores, plus migration / auth-webhook / vague-requirements checks | the 0–12 score comes from Jev instead of hand scoring; the bands and the hard escalation rules are unchanged, and the score is recorded with the item |
/fix depth |
is there a smoking-gun traceback; which depth (hotfix / lite / verified / full) fits | picks Lite automatically only when both are confident; otherwise today’s auto-detect. The rare depth prompt gains a suffix on its [auto] line only |
/fix review swarm |
did the first reviewer find a blocking issue | the second reviewer (qa-attacker) is spawned only when it might have; otherwise skipped. Shadow keeps both in parallel |
/fix root-cause pre-gate |
does the diff change code upstream of the symptom | one advisory line beside the existing PASS / FAIL verdict; never changes it |
/review-pr triage |
bug fix or feature; skim, standard or deep; blast radius; domain logic, public API, auth, migration, merge blocker | replaces the label/title regex when confident (the bug/feature prompt is unchanged when not); skim with no blocker/auth → Haiku reviewer; auth always forces Sonnet |
/review-pr checklists |
is the root cause described, is that hunk a real test, is a task implemented, does a new file have a test, is this over-engineered | drives the same ✓/⚠/✗ marks; the complexity score is advisory only |
/review-pr findings |
how severe is each finding; is it specific, a duplicate, or pre-existing | disagreements appear as a row in the summary table; the reviewer’s labels are never rewritten |
/review-pr extension rules |
does each enabled compliance rule apply to, and is it violated by, this diff | advisory hints for the code reviewer; the rules’ own blocking behaviour is unchanged |
/incident classification |
severity and type; later, does the routing match the evidence | replaces the keyword tables when confident (tables stay as the fallback); a confident routing skips the 30–120 s challenge. CRITICAL never asks questions, in any mode |
/pr review activity |
is this review comment a nit, a small local change, a design change, a question, an optional bot finding, or an echo of our own comment | sizes the comment before anything acts on it; an echo of our own comment is never acted on |
A later phase adds a browser runner where Jev picks the next action in Playwright tests.
Your key never goes through the chat. Claude Code’s shell has no terminal for hidden input, so setup does not ask you to paste anything into the conversation. Instead it prints one command for you to run in a separate terminal window:
bash ~/.claude-agents/scripts/arth-keys.sh set TYPESAFE_API_KEY # symlink install
bash "$CLAUDE_PLUGIN_ROOT/scripts/arth-keys.sh" set TYPESAFE_API_KEY # plugin install
The helper prompts there with hidden input, stores the value in
~/.arthai/.env (mode 600) and prints only stored TYPESAFE_API_KEY (value
hidden). Have several keys? arth-keys.sh wizard walks TypeSafe, the
Cloudflare pair, Vercel and Fireworks one after another (Enter skips a
name), and --help says where each one is created. Back in the chat, setup
asks Done / Skip for now, detects the key by name, and probes it once. If the name is still missing after Done it
says so once and asks again; a second miss simply leaves decisions off. A
dead key is reported and never wired in. The same helper lists what is
stored (list, names only) and removes a value (unset NAME).
Setup then asks one optional question: add a backup backend for
outages? Choose Cloudflare Workers AI (two values: CLOUDFLARE_API_TOKEN,
created at dash.cloudflare.com → My Profile → API Tokens with the “Workers
AI” template, and CLOUDFLARE_ACCOUNT_ID, the 32-character id after
dash.cloudflare.com/ in any dashboard URL; two set commands behind one
Done / Skip) or Vercel AI Gateway (AI_GATEWAY_API_KEY), or skip. A backup is probed the
same way and never wired in when the probe fails.
What a backup buys you: after three failed or timed-out calls in a row the
toolkit stops asking the primary for five minutes; with a backup it asks the
backup instead of running today’s heuristics. Hooks never take the hop (they
keep their 3-second budget), skills take at most one. If a backup key is
already in ~/.arthai/.env when setup runs, it is used without asking; re-run
/arth-setup → decisions any time to add one later.
Trimmed, redacted diffs, log tails, report text and file lists, capped at
30k tokens and hashed for replay. Never screenshots, HTML, cookies, fixture
values, secrets, or the conversation transcript. Keys live in
~/.arthai/.env and are detected by name only.
Modeled, until the PrepMe study measures it: /qa commit ~28 → ~17 units
and /pr ~39 → ~23 units once the Phase 1 gates are enforced; each Phase 2
skill’s own Cost section carries its modeled row. Jev’s own spend is cents
per week. Shadow mode adds the Jev calls (cents) and saves
nothing yet, by design.
/eval/eval treats the decisions product as a configuration of the system under test, not a model: the toolkit+jev and toolkit+memory+jev config arms run the same workflow chain with every gate enforced inside the cell, and every cell keeps its own decision log. The report then shows the decision layer’s own numbers beside the verdict (decisions, cost, escalation rate, latency, agreement with the existing logic), /eval compare gives the paired pass and cost deltas against the matching non-Jev arm, and each graded cell’s verdict is stamped onto the decisions it made as a clearly marked proxy label. Jev is never asked to grade or annotate a run it steered. See /eval.
The gates live inside the skills, so an install that predates a phase picks
them up by refreshing its skills — never by uninstalling anything. Your
.claude/.arth-jev.env, keys, decision log and locks are untouched.
Plugin installs — update the bundles you use:
/plugin update sentinel # /arth-setup + the decisions setup step, /incident
/plugin update forge # /qa, /pr, /implement, /review-pr gates
/plugin update prism # /qa gates
/plugin update scalpel # /fix gates, /ci-fix rerun rule
/plugin update cruise # /autopilot classification
/plugin update router # the routing step (triage router + decisions setup)
/plugin update prime # everything, if that is what you installed
Symlink installs (install.sh) — the decisions setup step (jev-setup)
is part of the router category, and the gated skills are in development,
quality, operations and guardrails. Re-run `install.sh –setup