docs: add documentation for main features

README.md

@@ -1,121 +1,131 @@
-# SimRun
+<p align="center">
+  <img src="docs/assets/simrun-icon-rounded.png" alt="SimRun" width="120">
+</p>
 
-SimRun is an **Attack Simulation Platform (ASP)** for detection testing. It **detonates**
-attack simulations and verifies that the security alerts you expect fire in your SIEM
-(currently only Elastic Security is supported).
+<h1 align="center">SimRun</h1>
 
-It ships as a single Go binary serving a REST API + WebSocket interface backed by
-PostgreSQL, with an embedded SvelteKit frontend.
+<p align="center">
+  <strong>An Attack Simulation Platform for detection engineering.</strong><br>
+  Detonate attacks, verify the alerts fire, and measure exactly which detections you're testing — across multiple clouds.
+</p>
 
-## Getting Started
+<p align="center">
+  <a href="LICENSE"><img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="License"></a>
+  <a href="https://github.com/IBM/simrun/releases"><img src="https://img.shields.io/github/v/release/IBM/simrun" alt="Release"></a>
+  <a href="https://go.dev/"><img src="https://img.shields.io/badge/Go-1.25-00ADD8?logo=go&logoColor=white" alt="Go"></a>
+  <a href="https://github.com/IBM/simrun/actions/workflows/docker-publish.yml"><img src="https://github.com/IBM/simrun/actions/workflows/docker-publish.yml/badge.svg" alt="Docker"></a>
+</p>
 
-### Prerequisites
+---
 
-- [mise](https://mise.jdx.dev/) — manages the Go 1.25 and Node 22 toolchains (or install them yourself)
-- PostgreSQL
+## Why SimRun
 
-### Build
+Detection rules rot silently. A field gets renamed, a log source drifts, an index mapping changes — and the alert you were counting on simply stops firing. You don't find out until the breach.
 
-```bash
-mise build   # builds the SvelteKit frontend and the simrun binary into dist/simrun
-```
+SimRun closes that gap. It runs real attack techniques against your environment, confirms the detections you expect actually fire in your SIEM, and tracks **which of your rules are tested and which are blind spots**.
 
-### Run
+It's not a script you wire into CI and forget. It's a platform:
 
-simrun requires a PostgreSQL database; schema migrations run automatically on startup.
+- **🎯 Coverage you can see.** SimRun joins your live Elastic detection rules against your scenarios and their latest pass/fail result, then reports a coverage percentage. You learn what you *aren't* testing — not just whether one test passed. → [Rule Coverage](#rule-coverage)
+- **☁️ Multiple clouds, multiple accounts.** First-class connectors for AWS, GCP, Azure, and Kubernetes, each backed by managed, encrypted secret groups and selectable per scenario.
+- **🧪 Three ways to exercise a detection.** **Detonate** a real attack, **inject** a crafted log to confirm a rule is wired up without touching infrastructure, or **collect** the raw logs an attack produced to build the next rule.
+- **📦 A pack ecosystem.** Simulations ship as versioned, shareable bundles with a clean Go SDK. Use the first-party packs, wrap [Stratus Red Team](https://github.com/DataDog/stratus-red-team), or author your own. → [Ecosystem](docs/ecosystem.md)
+- **🖥️ Built to operate.** Web UI, REST API, and WebSocket interface; PostgreSQL persistence; scheduled runs; optional multi-user OAuth. It all ships as a single Go binary with the SvelteKit UI embedded.
 
-```bash
-export SR_DATABASE_URL="postgres://user:pass@localhost:5432/simrun?sslmode=disable"
-./dist/simrun
-```
+> SimRun is **primarily focused on Elastic Security** — that's where coverage analysis, injection, and log collection are richest. Datadog security signals are supported as a matching backend.
 
-The UI and API are then served on http://localhost:8080.
+## How it works
 
-> Authentication is optional. Without `SR_GOOGLE_CLIENT_ID`/`SR_GOOGLE_CLIENT_SECRET`,
-> login is disabled and the app runs unauthenticated
+An **Assessment** is a saved set of **scenarios**. Running it creates a **Run**, which executes every scenario in parallel. Each scenario triggers some activity and then asserts that an expected alert appears in your SIEM before a timeout.
 
-## Configuration
+```mermaid
+flowchart LR
+    A[Assessment<br/>scenarios] -->|run| B(Run)
+    B --> S1[Scenario 1]
+    B --> S2[Scenario 2]
+    B --> S3[Scenario N]
+    S1 --> D{Trigger}
+    D -->|detonate| E[Real attack<br/> from SimRun pack]
+    D -->|inject| F[Crafted log<br/>into Elasticsearch]
+    E --> M[Matcher polls SIEM<br/>until alert fires or timeout]
+    F --> M
+    M --> R[(Results + Coverage<br/>Postgres)]
+    E -.optional.-> C[Collector<br/>gathers related logs]
+    C --> R
+```
 
-Deploy-time configuration is read from environment variables — the only `SR_*` env
-surface. Everything else (connectors, secrets, packs, schedules, scenarios, app
-defaults) lives in the database and is managed through the web UI.
+Every detonation gets a UUID that SimRun reflects into the generated activity wherever possible (user-agent strings), so an alert maps unambiguously back to the exact attack that caused it.
 
-| Variable | Required | Default | Description |
-|---|---|---|---|
-| `SR_DATABASE_URL` | yes | — | PostgreSQL connection string |
-| `SR_WEB_PORT` | no | `8080` | HTTP listen port |
-| `SR_DATA_DIR` | no | `~/.simrun` | Local data dir (encryption key, SSH logs) |
-| `SR_ENCRYPTION_KEY_FILE` | no | `$SR_DATA_DIR/encryption.key` | Key file for encrypting stored secrets |
-| `SR_DEBUG` | no | off | Verbose logging when set to a non-zero value |
-| `SR_WEB_URL` | no | — | External base URL (used for OAuth redirects) |
-| `SR_GOOGLE_CLIENT_ID` / `SR_GOOGLE_CLIENT_SECRET` | no | — | Google OAuth credentials (enables login) |
-| `SR_GOOGLE_ALLOWED_DOMAIN` | no | — | Restrict OAuth login to a Google Workspace domain |
-| `SR_AUTH_SESSION_TTL_HOURS` | no | `168` | Session lifetime in hours |
+→ Full vocabulary in [Concepts](docs/concepts.md).
 
-### Run with Docker
+## A scenario, end to end
 
-```bash
-docker build -t simrun .
-docker run -p 8080:8080 \
-  -e SR_DATABASE_URL="postgres://..." \
-  -v simrun-data:/home/nonroot/.simrun \
-  simrun
+Build scenarios visually in the assessment editor, or write the YAML directly — the editor toggles between a forms-based **Builder** and raw **YAML**. This scenario detonates a pack simulation in AWS and asserts the matching Elastic Security rule fires:
+
+```yaml
+targets:
+  aws: prod-aws            # an AWS connector you configured in the UI
+
+scenarios:
+  - name: S3 public access block disabled
+    detonate:
+      simrunDetonator:
+        pack: simrun-base-pack
+        simulation: aws.s3-disable-public-access-block
+    expectations:
+      - timeout: 5m
+        elasticSecurityAlert:
+          name: "S3 Public Access Block Disabled"
 ```
 
-The image bundles the `aws`, `gcloud`, and `az` CLIs used by detonators. Persist
-`SR_DATA_DIR` (the volume above) so the secret-encryption key survives restarts.
+Swap `detonate` for `inject` to test a rule without running an attack, or add a `collect` block to capture the raw logs the attack produced. 
 
-## Architecture
+→ [Scenarios reference](docs/scenarios.md).
 
-A single Go binary handles:
-- Simulation detonation and orchestration
-- Alert matching and verification
-- Log collection from security platforms
-- Scenario parsing and execution
+## Quickstart (60 seconds)
 
-### Simulation Packs
+**Prerequisites:** [mise](https://mise.jdx.dev/) (manages Go 1.25 and Node 22), PostgreSQL.
 
-Simulations are distributed as external packs, installed and managed via the web UI:
+```bash
+# Build the frontend + binary → dist/simrun
+mise run build
 
-- simrun-base-pack — custom simulations (AWS, Azure, GCP)
-- simrun-stratus-pack — [Stratus Red Team](https://github.com/DataDog/stratus-red-team) simulations
+# Point it at Postgres and run
+export SR_DATABASE_URL="postgres://user:pass@localhost:5432/simrun?sslmode=disable"
+./dist/simrun           # serves UI + API on http://localhost:8080
+```
+
+Schema migrations run automatically on startup. Authentication is optional — without Google OAuth credentials, SimRun runs unauthenticated.
+
+→ Then follow the [Walkthrough](docs/walkthrough.md) to run your first detection test.
+
+## Rule Coverage
 
-## Concepts
+The Coverage view (`/rules/coverage`) pulls every detection rule from your Elastic deployment and answers the question CI never could: **which of these rules does a SimRun scenario actually exercise, and did it pass last time?** Rules with no scenario are flagged as blind spots; the summary reports an overall covered-rules percentage.
 
-### Detonators
-A **detonator** describes how and where an attack technique is executed.
-* Simrun detonator — runs a simulation pack (Terraform-based; packs can themselves
-  execute locally or over SSH)
-* AWS CLI detonator — runs AWS CLI commands
+## The pack ecosystem
 
-### Injectors
-An **injector** is an alternative to detonators: instead of executing the end-to-end
-attack it takes a generated log message and injects it directly into the SIEM. This
-covers cases where end-to-end simulation isn't feasible but you still want to confirm
-the detection is operational.
-* Elastic Injector
+Simulations are distributed as **packs** — versioned bundles of Terraform modules, scenario definitions, and a manifest. Two first-party packs are maintained alongside SimRun:
 
-### Alert Matchers
-An **alert matcher** is a platform-specific integration that checks whether an expected
-alert was triggered.
-* Elastic Security alerts
-* Datadog security signals
+| Pack | What it is |
+|---|---|
+| [**simrun-pack**](https://github.com/confluentinc/simrun-pack) | The reference pack and a worked example of authoring simulations in Go (AWS, Kubernetes, Okta injections). |
+| [**simrun-stratus-adapter**](https://github.com/confluentinc/simrun-stratus-adapter) | Plug-and-play wrapper that exposes the entire [Stratus Red Team](https://github.com/DataDog/stratus-red-team) technique registry as a SimRun pack. |
 
-### Collectors
-A **collector** retrieves logs from security platforms after detonation for analysis and
-rule generation.
-* Elastic Collector — collects related logs from Elasticsearch by execution ID or
-  user-agent correlation
 
-### Detonation and Alert Correlation
-Each detonation is assigned a UUID, reflected in the detonation where possible and used
-to ensure the matched alert corresponds exactly to that detonation. If the detonator
-cannot reflect the UUID, the matcher can correlate using indicators the user provides
-(static indicators) or terraform output (dynamic indicators).
+→ [Ecosystem & authoring guide](docs/ecosystem.md).
 
-### Simulations
-A **simulation** is a reusable module describing how to perform a specific attack.
-Simulations are distributed as [packs](#simulation-packs) and installed via the web UI.
+## Documentation
+
+- [Getting Started](docs/getting-started.md) — prerequisites, build, first run
+- [Concepts](docs/concepts.md) — assessments, runs, detonators, matchers, collectors
+- [Walkthrough](docs/walkthrough.md) — end-to-end tutorial
+- [Scenarios](docs/scenarios.md) — YAML schema reference
+- [Connectors & Secrets](docs/connectors-and-secrets.md) — SIEM and cloud integrations
+- [Packs](docs/packs.md) — install and configure simulation packs
+- [Ecosystem](docs/ecosystem.md) — the pack ecosystem and writing your own
+- [Configuration](docs/configuration.md) — environment variables reference
+- [Deployment](docs/deployment.md) — Docker, production notes, OAuth setup
 
 ## Development
 
@@ -127,6 +137,13 @@ go generate ./...         # regenerate mocks (mockery)
 mise run parser           # regenerate parser from JSON schemas
 ```
 
+## Acknowledgments
+
+SimRun stands on excellent prior work in the detection-engineering community:
+
+- [**Threatest**](https://github.com/DataDog/threatest) by Datadog pioneered the detonate-and-verify model for detection testing and shaped how we think about correlating attacks to alerts.
+- [**Stratus Red Team**](https://github.com/DataDog/stratus-red-team), also by Datadog, provides the MITRE ATT&CK–mapped cloud attack techniques that SimRun exposes through [simrun-stratus-adapter](https://github.com/confluentinc/simrun-stratus-adapter).
+
 ## Contributing
 
 Issues and pull requests are welcome.

docs/README.md

@@ -0,0 +1,29 @@
+# SimRun Documentation
+
+Learn how to set up, configure, and run detection tests against your SIEM.
+
+## Suggested Reading Order
+
+Start with **Getting Started** to install SimRun, then read **Concepts** to understand the vocabulary. Follow with the **Walkthrough** to execute your first test, then dive into the reference docs as needed.
+
+---
+
+## Documentation Index
+
+- **[Getting Started](getting-started.md)** — Install SimRun and reach the dashboard.
+
+- **[Concepts](concepts.md)** — Vocabulary and core components: assessments, runs, scenarios, detonators, matchers, and injectors.
+
+- **[Walkthrough](walkthrough.md)** — Step-by-step guide to your first detection test with AWS and Elastic Security.
+
+- **[Scenarios YAML Reference](scenarios.md)** — Complete reference for the YAML format that defines assessments and scenarios.
+
+- **[Connectors and Secrets](connectors-and-secrets.md)** — Configure endpoints and store credentials for your SIEM and cloud platforms.
+
+- **[Packs](packs.md)** — Install and manage external attack simulation bundles and tune their parameters.
+
+- **[Ecosystem](ecosystem.md)** — The pack ecosystem, the maintained satellite packs, and how to write your own.
+
+- **[Configuration](configuration.md)** — All deploy-time environment variables and operational defaults.
+
+- **[Deployment](deployment.md)** — Run SimRun in production with Docker and optional Google OAuth.

docs/concepts.md

@@ -0,0 +1,66 @@
+# Concepts
+
+An **Assessment** defines scenarios. Running it creates a **Run**. The run **executes each scenario** (in parallel, like jobs), and each scenario checks its **expectations** via **matchers**.
+
+## Vocabulary
+
+**Assessment**
+: A saved collection of scenarios that describes what to detonate and what alerts to expect. Managed on the Assessments page (`/assessments`).
+
+**Run**
+: A single execution of an Assessment. Persists results to the database and is viewable on the Runs page (`/runs`).
+
+**Scenario**
+: One unit of work inside an Assessment: a detonation or injection step, plus one or more expectations. Scenarios in a run execute in parallel.
+
+**Expectation**
+: A declared assertion that a specific alert or signal must appear in the target platform after detonation. Expectations are defined per-scenario in the YAML file.
+
+**Matcher**
+: The component that checks whether an expectation was satisfied by polling the target platform until the alert appears or the timeout expires.
+
+## Detonators
+
+Detonators execute the attack simulation.
+
+**SimRun detonator** — runs a pack simulation using Terraform. The pack can execute locally (on the SimRun host) or over SSH on a remote target. This is the primary detonation method and requires a pack to be installed.
+
+## Injectors
+
+Injectors skip the attack execution and push a document directly into the SIEM.
+
+**Elastic Injector** — writes a log document into Elasticsearch. Use this to confirm that a detection rule is operational without actually running an attack: if the matcher finds the expected alert after injection, the rule is wired up correctly.
+
+## Matchers
+
+Matchers verify that an expectation fired after detonation or injection.
+
+**Elastic Security alert matcher** — polls Kibana for a Detection Engine alert whose `kibana.alert.rule.name` matches the expected rule name.
+
+## Collectors
+
+Collectors retrieve related logs after detonation for post-hoc analysis and rule development.
+
+**Elastic Collector** — queries Elasticsearch for log events correlated to the detonation, either by the execution UUID (when it was reflected into the activity) or by a user-agent string. The collected logs are stored with the run result and can be used to understand what raw data a detection rule would have to match.
+
+## Correlation
+
+Every detonation is assigned a UUID (nanoid) at execution time. Where possible, SimRun reflects that UUID into the generated activity by injecting it into the user-agent string — so that the resulting alert maps unambiguously to exactly one detonation event.
+
+When the UUID cannot be reflected into the activity (for example, in a managed AWS API call), SimRun uses **indicators** for correlation:
+
+- `static` — a fixed string you provide (e.g. a known username or resource name) that will appear in the alert or log.
+- `terraformOutput` — a value extracted dynamically from a Terraform output after the simulation completes (e.g. a generated resource ARN).
+
+## Packs, Connectors, Secrets
+
+**Packs** distribute simulations as versioned bundles. A pack contains Terraform modules, scenario definitions, and a manifest. Install packs from the UI and reference them by name in your scenario YAML. See [packs.md](packs.md) for installation and configuration.
+
+**Connectors** point SimRun at an external platform — Elastic, AWS, GCP, Azure, Kubernetes, or SSH. Each connector stores the endpoint and non-secret configuration needed to reach that platform. Connectors are managed on the Connectors page (`/connectors`). See [connectors-and-secrets.md](connectors-and-secrets.md) for setup.
+
+**Secret groups** hold the credentials (API keys, passwords, certificates) that a connector uses. A connector links to at most one secret group. Secrets are managed on the Secrets page (`/secrets`) and are never returned in plaintext after saving.
+
+## See also
+
+- [walkthrough.md](walkthrough.md) — run your first detection test end to end
+- [scenarios.md](scenarios.md) — full YAML reference for scenario files

docs/configuration.md

@@ -0,0 +1,40 @@
+# Configuration
+
+Every deploy-time setting SimRun reads.
+
+## Overview
+
+Deploy-time configuration is environment-only. The `SR_*` environment variables below are the only settings SimRun reads at startup. Everything else — connectors, secrets, packs, schedules, assessments, and app-level operational defaults — lives in the database and is managed through the UI.
+
+## Environment variables
+
+| Variable | Required | Default | Description |
+|---|---|---|---|
+| `SR_DATABASE_URL` | yes | — | PostgreSQL connection string |
+| `SR_WEB_PORT` | no | `8080` | HTTP listen port |
+| `SR_DATA_DIR` | no | `~/.simrun` | Local data dir (encryption key, SSH logs) |
+| `SR_ENCRYPTION_KEY_FILE` | no | `$SR_DATA_DIR/encryption.key` | Key file for encrypting stored secrets |
+| `SR_DEBUG` | no | off | Verbose logging when set to a non-zero/non-`false` value |
+| `SR_WEB_DEV` | no | off | Dev mode when set to `1` |
+| `SR_WEB_URL` | no | — | External base URL (used for OAuth redirects) |
+| `SR_GOOGLE_CLIENT_ID` / `SR_GOOGLE_CLIENT_SECRET` | no | — | Google OAuth credentials (enables login) |
+| `SR_GOOGLE_ALLOWED_DOMAIN` | no | — | Restrict OAuth login to a Google Workspace domain |
+| `SR_AUTH_SESSION_TTL_HOURS` | no | `168` | Session lifetime in hours |
+
+## App config defaults
+
+Operational defaults that admins tune through the UI at `/config`. These are stored in the database (the `app_config` table) and take effect without a restart.
+
+Confirmed fields include:
+
+- **Parallelism** (default `5`) — number of scenarios that run concurrently within a single run.
+- **Terraform version** — pin the Terraform binary version used by pack detonations.
+- **Pack logs** — toggle capture of Terraform output in run logs (enabled by default).
+- **SSH logging** — toggle SSH session logging (disabled by default).
+- **Run log retention** — automatically purge run logs after a configurable number of days (default 7 days, enabled by default).
+- **Run retention** — automatically purge run records after a configurable number of days (default 30 days, disabled by default).
+
+## See also
+
+- [deployment.md](deployment.md) — production deploy checklist, OAuth setup, Docker
+- [getting-started.md](getting-started.md) — build SimRun and reach the dashboard for the first time