package as a agent skill

README.md

@@ -11,16 +11,38 @@ This isn't a metrics glossary. It's a troubleshooting guide built around how pro
 - **It connects the dots.** Rather than explaining metrics in isolation, it guides you across related signals so you can build a complete picture of what's happening — not just identify a single data point.
 - **It covers real impact.** Every entry is grounded in what the failure actually means for your system and your users, not just what the metric technically represents.
 - **It's practical beyond incidents.** The same knowledge applies to cluster and database sizing, capacity planning, and understanding system behavior under load testing.
-- **It works as a Claude skill.** The content is structured to be used directly with Claude for interactive troubleshooting.
+- **It works as a Claude skill.** The content is structured to be used directly with Claude for interactive troubleshooting. See [Installation](#installation) below.
 
 This is a living document. It will keep growing as I work through new patterns and scenarios.
 
 ---
 
+## Installation
+
+This repository is packaged as a Claude Code Agent Skill. Install it to give Claude access to this knowledge base during interactive troubleshooting sessions.
+
+### Option 1: Manual clone
+
+```bash
+mkdir -p ~/.claude/skills
+git clone https://github.com/tsurdilo/temporal-metrics ~/.claude/skills/temporal-metrics
+```
+
+### Option 2: npx skills
+
+```bash
+npx skills add tsurdilo/temporal-metrics
+```
+
+Once installed, Claude will automatically activate the skill when you ask about Temporal SDK metrics, rate limiting, throttling, gRPC status codes, or production incident troubleshooting.
+
+---
+
 Useful dashboards to use alongside this knowledge base:
 - https://github.com/tsurdilo/my-temporal-dockercompose
 - https://github.com/tsurdilo/my-temporal-dockercompose/tree/main/deployment/grafana/dashboards
 
 ## Table of Contents
 
-1. [Troubleshooting SDK Request Failure Metrics](./SDK_METRICS_REQUEST_FAILURES.md)
+1. [Troubleshooting SDK Request Failure Metrics](./references/sdk-request-failures.md)
+2. [Graph Pattern Reference](./references/graph-patterns.md)

SKILL.md

@@ -0,0 +1,65 @@
+---
+name: temporal-metrics
+description: >
+  This skill should be used when the user asks about "temporal_request_failure",
+  "temporal_long_request_failure", "ResourceExhausted", "SDK metrics",
+  "rate limiting", "throttling", "gRPC status codes", "namespace RPS",
+  "execution bucket", "visibility bucket", "poll throttling",
+  "schedule to start latency", "workflow task timeout", "activity timeout",
+  "heartbeat timeout", "resource exhausted cause", "server dashboard",
+  "metric troubleshooting", "Temporal production incident",
+  "request failure metrics", "graph patterns", "RPS quota",
+  "frontend.namespaceRPS", or asks to diagnose Temporal metrics or
+  troubleshoot Temporal SDK errors.
+version: 0.1.0
+---
+
+# Temporal Metrics Troubleshooting Skill
+
+You now have access to a production-focused troubleshooting knowledge base for Temporal SDK request failure metrics. This knowledge comes from 5+ years of real-world experience across hundreds of Temporal implementations — production incidents, load tests, and hard-won debugging sessions. It is not a metrics glossary. It connects dots across related signals so you can build a complete picture of what is happening.
+
+## What This Skill Covers
+
+- **`temporal_request_failure`** and **`temporal_long_request_failure`** — the two most useful SDK metrics for troubleshooting Temporal applications
+- How the SDK distinguishes the two metrics (based on `LongPollContextKey`)
+- SDK gRPC retry behavior — which status codes are retried, which are not, and what that means for business impact
+- Alert severity tiers — which failures warrant immediate alerting vs. passive monitoring
+- Diagnostic tables mapping every operation + status code combination to what it means operationally
+- Server-side rate limit buckets (Execution, Visibility, NamespaceReplicationInducing) and priority levels
+- How to correlate SDK metrics with server dashboard panels
+- Visual graph pattern interpretation for RPS failure graphs
+
+## How to Use the References
+
+Read the appropriate reference file based on what the user needs help with:
+
+### SDK Request Failure Metrics (the main guide)
+
+**Read `references/sdk-request-failures.md`** for:
+- Understanding what `temporal_request_failure` or `temporal_long_request_failure` means
+- Diagnosing a specific operation + status code combination
+- Understanding alert severity and business impact
+- Server-side throttling theory (rate limit buckets, priorities, dynamic config keys)
+- Mapping SDK metrics to server dashboard panels
+- Understanding gRPC retry behavior
+
+### Graph Pattern Reference
+
+**Read `references/graph-patterns.md`** for:
+- Interpreting visual patterns in RPS failure graphs
+- Identifying root causes from graph shapes (e.g., sustained plateau with periodic drops)
+- Distinguishing between causes using correlated signals
+
+## Key Concepts
+
+- **`temporal_request_failure`** covers all standard client and worker operations (starts, signals, completions, heartbeats, visibility queries)
+- **`temporal_long_request_failure`** covers long-poll operations (`PollWorkflowTaskQueue`, `PollActivityTaskQueue`, `PollWorkflowExecutionUpdate`, `GetWorkflowExecutionHistory` with `wait_new_event=true`)
+- The server has **three independent rate limit buckets** — Execution, Visibility, and NamespaceReplicationInducing — each with its own RPS quota and priority queue
+- Within each bucket, lower-priority operations are shed first under load (Priority 0 = highest, 5 = lowest)
+- Not all `ResourceExhausted` errors are throttling — can also be gRPC message size violations or `BusyWorkflow` throttle
+
+## Companion Dashboards
+
+These dashboards are referenced throughout the knowledge base:
+- https://github.com/tsurdilo/my-temporal-dockercompose
+- https://github.com/tsurdilo/my-temporal-dockercompose/tree/main/deployment/grafana/dashboards

GRAPH_PATTERNS.md → references/graph-patterns.md

@@ -2,7 +2,7 @@
 
 This is a living playbook for interpreting common visual patterns in `temporal_request_failure` and `temporal_long_request_failure` RPS graphs. Each entry maps a visual shape to its likely causes, business impact, and how to distinguish between root causes using correlated signals.
 
-For background on the metrics, operations, and rate limit buckets referenced here, see the main [README](https://github.com/tsurdilo/temporal-metrics/blob/main/README.md).
+For background on the metrics, operations, and rate limit buckets referenced here, see the main [SDK Request Failures guide](./sdk-request-failures.md).
 
 ---
 

SDK_METRICS_REQUEST_FAILURES.md → references/sdk-request-failures.md

@@ -772,7 +772,7 @@ All at Priority 1 within the visibility bucket.
 ---
 ## Graph Pattern Reference
 
-For a visual pattern playbook — common shapes you'll see in `temporal_request_failure` and `temporal_long_request_failure` RPS graphs, what they mean, their business impact, and how to triage them — see the separate **[Graph Pattern Reference](./GRAPH_PATTERNS.md)**.
+For a visual pattern playbook — common shapes you'll see in `temporal_request_failure` and `temporal_long_request_failure` RPS graphs, what they mean, their business impact, and how to triage them — see the separate **[Graph Pattern Reference](./graph-patterns.md)**.
 
 This is a living document that grows as new patterns are documented.