arthai-marketplace

Measure What AI Costs You

Arth Intelligence records every Claude Code session — tool calls, agent spawns, tokens, dollars — on your own machine. Its Experiments page answers the question this guide is about: what does the toolkit actually change, and what does my AI usage cost? This page is the condensed workflow; the full guide has setup, dashboard tour, and privacy details.

Prerequisite: observability enabled via /otel-setup — see Getting Started Step 7. Cost and token data require Claude Code’s native OTEL (CLAUDE_CODE_ENABLE_TELEMETRY=1, which /otel-setup writes); without it the cost columns stay empty.

Run a baseline vs toolkit A/B

Every session is auto-tagged with an arth.experiment label — no setup before each launch:

Mode	Label format
No-toolkit baseline (plain `claude`, no plugin)	`auto-baseline-<git-branch>-<first-prompt-slug>-<unix>`
Toolkit-on session	`auto-toolkit-<git-branch>-<unix>`

The zero-config flow:

Run the task without the toolkit (plain claude in a project with no plugin).
Run the same task with the toolkit installed.
Open http://localhost:3100/experiments, pick one label on each side, click Compare.

Want telemetry on but toolkit side effects off for the baseline? export OTEL_OBSERVE_ONLY=true before launching. Want a human-readable label instead of the auto one? Set arth.experiment in OTEL_RESOURCE_ATTRIBUTES before launching — your value wins. Auto-tagging can be disabled with ARTH_AUTO_EXPERIMENT_DISABLED=1. Full details: Getting Started 7f.

Reading the six metrics

The comparison fills in side-by-side:

Metric	What it tells you
Cost (USD)	Total spend per run — the bottom line
Tokens	Input/output/cache volume behind that cost
API calls	How many model round-trips the run took
Cache hit rate	How much context was reused instead of re-paid
Lines edited	Output volume of actual code change
Active time	Wall-clock engagement, not idle time

Plus workflow attribution — which skills and agents the toolkit-on run used. Two caveats: cost/token/cache metrics need native OTEL (sessions without it show 0), and lines-edited only populates for toolkit-instrumented sessions.

Going deeper

Mark moments mid-run — /marker "spike here" drops an amber ◆ on the session’s DAG timeline within ~5s, so you can correlate a cost spike with what you were doing.
Cost by owner — each session’s detail view includes an owner × model cross-tab: spend by toolkit skills vs Claude Code built-ins vs your own config, per model tier (Opus 60x / Sonnet 10x / Haiku 1x).
Velocity score — the project view’s Today summary composites outcome rate, toolkit coverage, and cost efficiency, with data-backed coaching insights.
Export a slice — /arth logs export --since 1h or the sidebar’s diagnostic bundle; filter by experiment, marker, session ID, and time range.

Arth Intelligence — the full observability guide (source for everything above)
Getting Started Step 7 — install walkthrough, observer-only mode, auto-tagging controls
/otel-setup — the one-command setup skill
/marker — mid-session timeline annotations
Architecture — why model tiers make cost controllable in the first place

Beyond measuring: route by cost

Measuring tells you what you spent; the Arth Router decides what to spend before the work runs. It routes each task to a model/harness/platform with awareness of the capacity you already paid for — an under-used Max plan window routes to the best models at $0 marginal cost; a burned-down window right-sizes to cheaper ones — governed by budgets and tiers, with every decision explained.

/router-setup — one-command install: discovers your plan, generates a real pool config, wires the triage hook
/router — operate it: summary (override rate + estimate bias), pools (window burn), calibrate, outcome reporting

Pairs with this guide’s telemetry: the outcomes bridge feeds actual usage back into the router, so its cost estimates are continuously reconciled against what you really spent.