Skip to content

feat(usage): track JSONL tokens + cost, gated by api_key auth, with subagent rollup#127

Open
manzil-infinity180 wants to merge 8 commits into
devfrom
feat/usage-tracking-api-key
Open

feat(usage): track JSONL tokens + cost, gated by api_key auth, with subagent rollup#127
manzil-infinity180 wants to merge 8 commits into
devfrom
feat/usage-tracking-api-key

Conversation

@manzil-infinity180
Copy link
Copy Markdown
Contributor

@manzil-infinity180 manzil-infinity180 commented May 19, 2026

Two related fixes shipped together so the api_key-gated cost tracking and the parent/subagent spend rollup can be tested in one POC.

What's new

  • internal/usage — JSONL parser with (message.id, requestId) dedup and pricing table for opus/sonnet/haiku families (AFLOCK_PRICE_* overrides).
  • internal/auth/claudeauth — auth-mode probe (env var → ANTHROPIC_API_KEY~/.claude/.credentials.json → macOS keychain).
  • SessionState.AuthMode + richer SessionMetrics (cache 5m/1h, usageSource, costMeasured).
  • Hooks refresh usage from JSONL at PostToolUse + SessionEnd before CheckLimits.
  • Cost enforcement (maxSpendUSD) activates only under api_key; advisory under subscription/unknown via Evaluator.IsAdvisoryLimit. Token/turn limits enforce under both modes (transcript counts match Anthropic's per-turn accounting either way).
  • Subagent rollup (closes fix(state): subagent spend not rolled up to parent until SubagentStop — parent fail-fast can be exceeded mid-flight (paper §5 invariant #2) #135): parent's PostToolUse enumerates child sessions via new state.Manager.ListChildSessions(parentID), sums any whose IDs aren't already in parent.ChildSessionIDs, and feeds effective metrics into fail-fast — closes the window where concurrent children could blow past the parent ceiling between SubagentStop merges. Race narrows to one child-tool-call delta (per paper §5 invariant #2 acceptance).

Test plan

  • go test ./... clean
  • api_key session over maxSpendUSD → block
  • subscription session over maxSpendUSD → stderr advisory only
  • maxTokensIn enforces under both modes
  • Parent at $0.30 + child spending $0.30 with maxSpendUSD=\$0.50 → parent's next PostToolUse blocks, stderr shows Subagent rollup: including N in-flight child(ren)
  • After SubagentStop, the next rollup correctly skips the merged child (no double-count)

@manzil-infinity180 manzil-infinity180 force-pushed the feat/usage-tracking-api-key branch from c367e03 to 3c07fe0 Compare May 25, 2026 17:53
Paper §5 invariant #2 said sub-agent spend counts toward parent totals,
but rollup only fired at SubagentStop. A child running concurrently with
its parent would let the parent's fail-fast pass at $9 while the child
spent another $5 — parent ceiling $10 was breached the moment SubagentStop
finally merged. Parent's PostToolUse now enumerates child sessions
(state.Manager.ListChildSessions), sums any whose IDs aren't already in
parent.ChildSessionIDs, and feeds the effective metrics into CheckLimits.
Race window narrows to one child-tool-call delta, which the issue
explicitly accepts.
@manzil-infinity180 manzil-infinity180 changed the title feat(usage): track JSONL tokens + cost, gated by api_key auth feat(usage): track JSONL tokens + cost, gated by api_key auth, with subagent rollup May 25, 2026
@manzil-infinity180 manzil-infinity180 self-assigned this May 25, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds transcript-driven token/cache/cost accounting from Claude Code JSONL logs, gates cost-limit enforcement based on detected Claude authentication mode, and implements parent-side aggregation of in-flight subagent spend to narrow the fail-fast overspend window (issue #135).

Changes:

  • Introduce internal/usage JSONL scanning + model-family pricing with env overrides, and plumb the resulting metrics into hook limit checks.
  • Add internal/auth/claudeauth auth-mode detection and downgrade cost-based limits (e.g., maxSpendUSD) to advisory under non-api_key sessions across hooks/replay/verify.
  • Add parent PostToolUse subagent rollup via Manager.ListChildSessions + rollupUnmergedChildren for near-real-time accumulation.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
pkg/aflock/types.go Extends SessionState and SessionMetrics to include auth mode + cache token breakdown + cost provenance flags.
internal/verify/verifier.go Treats cost-based limit exceedance as advisory under non-api_key auth during verification.
internal/usage/testdata/sample.jsonl Sample transcript fixture for JSONL parser tests.
internal/usage/pricing.go Model-family pricing table + env overrides + cost computation.
internal/usage/pricing_test.go Unit tests for pricing lookup, overrides, and unknown-model tracking.
internal/usage/jsonl.go JSONL transcript scanner with (message.id, requestId) dedup and per-model rollups.
internal/usage/jsonl_test.go Unit tests for JSONL dedup, filtering, and missing-file behavior.
internal/state/session.go Adds Manager.ListChildSessions for parent-side rollup enumeration.
internal/state/session_test.go Tests child-session enumeration correctness.
internal/replay/simulate.go Aligns replay behavior with advisory cost limits under non-api_key auth.
internal/policy/evaluator.go Adds Evaluator.IsAdvisoryLimit(limitName, authMode) helper.
internal/policy/evaluator_authmode_test.go Tests advisory/enforced matrix for limits vs auth modes.
internal/mcp/server.go Exposes new metrics fields + authMode in session API output.
internal/hooks/usage_refresh.go Adds transcript refresh to populate token/cache/cost metrics in hook state.
internal/hooks/subagent_rollup_test.go Tests rollup correctness, no double-count, and no parent mutation.
internal/hooks/handler.go Wires auth-mode capture, transcript refresh, advisory gating, and in-flight subagent rollup into hooks.
internal/auth/claudeauth/detect.go Implements auth-mode detection (env/api-key/credentials/keychain).
internal/auth/claudeauth/detect_test.go Tests auth-mode detection precedence and file/env probes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread internal/hooks/usage_refresh.go Outdated
Comment thread internal/hooks/usage_refresh.go Outdated
Comment thread internal/usage/jsonl.go
Comment thread internal/hooks/handler.go
Comment thread internal/hooks/handler.go
Comment thread internal/hooks/handler.go
@manzil-infinity180
Copy link
Copy Markdown
Contributor Author

test it with the anthropic api key method :)

Signed-off-by: Rahul Vishwakarma <rahulvs2809@gmail.com>
Signed-off-by: Rahul Vishwakarma <rahulvs2809@gmail.com>
Signed-off-by: Rahul Vishwakarma <rahulvs2809@gmail.com>
…i-key

# Conflicts:
#	go.mod
#	internal/hooks/handler.go
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Signed-off-by: Rahul Vishwakarma <rahulvs2809@gmail.com>
Signed-off-by: Rahul Vishwakarma <rahulvs2809@gmail.com>
@manzil-infinity180 manzil-infinity180 force-pushed the feat/usage-tracking-api-key branch from 8654ff8 to 335b510 Compare May 28, 2026 10:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants