Skip to content

docs: update README.md and CLAUDE.md for all new features #23

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
copyleftdev merged 1 commit into from
Apr 10, 2026
Merged

docs: update README.md and CLAUDE.md for all new features #23

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,11 @@ vajra/
├── vajra-essence/ # concern profiles, scoring, ranking, rendering, templates
├── vajra-query/ # path expressions, analysis functions
├── vajra-cli/ # CLI commands, argument parsing, output formatting
├── vajra-cascade/ # temporal cause-effect chain detection
├── vajra-domain-med/ # optional medical/EDI plugin
├── vajra-domain-sec/ # optional security plugin (CVE, MITRE, IPs, hashes, JWT)
├── vajra-domain-devops/ # optional DevOps plugin (K8s, Docker, Terraform, ARN, semver)
├── vajra-domain-github/ # optional GitHub plugin (PRs, issues, reviews, releases, actions)
├── vajra-source/ # source code parsing via tree-sitter (9 languages)
├── vajra-domain-source/ # source code recognizers (naming conventions, paths)
└── vajra-domain-encoding/ # encoding detection (Base64, hex, URL, PEM, layers)
Expand Down Expand Up @@ -80,6 +82,23 @@ Skills activate automatically when the work matches. No need to invoke them —
| **vajra-domain-devops** | "devops plugin", "K8s recognizers", "detect container IDs", "Terraform" — infrastructure type recognizers, hints, profiles |
| **vajra-cli-command** | "add CLI command", "implement inspect", "add subcommand" — clap patterns, output modes, error style |

## CLI Commands

| Command | Purpose |
|---------|---------|
| `inspect` | Full structural analysis — paths, types, fingerprints, domain recognition |
| `stats` | Statistical summary — entropy, frequency, numeric distributions, temporal windowing |
| `anomalies` | MAD-based outliers, rarity scoring, type instability |
| `fingerprint` | BLAKE3 structural hashes, Merkle motifs |
| `essence` | Concern-oriented reduction — 7 profiles, token budgets, compact-AI |
| `drift` | Schema drift — JSD, Wasserstein, severity classification, `--group-by` |
| `cascade` | Temporal cause-effect chain detection across events |
| `cluster` | MinHash + LSH similarity clustering |
| `invariants` | Cross-field relationships — conditional entropy, PMI |
| `query` | Path expressions with analysis functions |
| `batch` | Parallel batch analysis across directories |
| `profiles` | List available profiles |

## Hooks

| Event | Trigger | Action |
Expand Down Expand Up @@ -161,6 +180,7 @@ Agent(isolation: worktree) → vajra-anomaly
| Conditional entropy | stats | functional dependency detection |
| Benford's Law | stats | leading digit forensics (Nigrini 1996) |
| RFC 8785 (JCS) | core | IETF canonical JSON |
| Linear regression | stats | temporal trend detection in windowed series |
| DFA bank | core | O(m) type inference, no backtracking |

## Dependency Stack
Expand All @@ -181,6 +201,7 @@ Agent(isolation: worktree) → vajra-anomaly
| Float formatting | `ryu` |
| Unicode | `unicode-normalization` |
| Mutation tests | `cargo-mutants` |
| ISO 8601 parsing | manual (no `chrono` — hand-rolled in `vajra-stats/src/temporal.rs`) |

---

Expand Down
46 changes: 45 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@

---

A high-performance Rust CLI and library that analyzes arbitrary structured data — JSON, YAML, CSV, NDJSON, Markdown, PDF — extracts structural and semantic signal, detects anomalies and drift, and emits compact deterministic essences optimized for humans, auditors, and AI pipelines.
A high-performance Rust CLI and library that analyzes arbitrary structured data — JSON, YAML, CSV, NDJSON, Markdown, PDF, git repos, CPU profiles, strace logs — extracts structural and semantic signal, detects anomalies and drift, discovers temporal cause-effect chains, and emits compact deterministic essences optimized for humans, auditors, and AI pipelines.

## Install

Expand Down Expand Up @@ -60,6 +60,21 @@ vajra batch data_directory/

# Cluster similar documents
vajra cluster batch/*.json

# Analyze a git repo directly
vajra stats /path/to/repo --input-format git

# Temporal cause-effect chains
vajra cascade commits.json --entity-field '$.file' --time-field '$.date' --event-field '$.intent'

# Time-series windowed stats with trend detection
vajra stats data.json --window month --time-field '$.date'

# Population-level drift comparison
vajra drift data.json --group-by '$.team'

# Semantic path labels for source code
vajra inspect src/main.rs --input-format source --lang rust --semantic-paths
```

## What It Does
Expand All @@ -77,6 +92,7 @@ Feed Vajra any structured data. It returns **shape, signal, anomalies, and truth
| `cluster` | MinHash + LSH similarity clustering |
| `invariants` | Cross-field relationships — conditional entropy, PMI |
| `query` | Path expressions with analysis functions |
| `cascade` | Temporal cause-effect chain detection across events |
| `batch` | Parallel batch analysis across directories |
| `profiles` | List available profiles |

Expand All @@ -93,6 +109,9 @@ Feed Vajra any structured data. It returns **shape, signal, anomalies, and truth
| PDF | `.pdf` | Yes |
| Gzip | `.gz`, `.json.gz` | Yes (magic bytes) |
| Zstd | `.zst`, `.zstd` | Yes (magic bytes) |
| Git | `--input-format git` | No (explicit) |
| V8 CPU Profile | `.cpuprofile` | Yes |
| strace | `--input-format strace` | No (explicit) |
| HTTP | `http://`, `https://` | Yes |
| Stdin | `-` | Yes |

Expand All @@ -105,8 +124,31 @@ Feed Vajra any structured data. It returns **shape, signal, anomalies, and truth
| `auditor` | Completeness, traceability | Compliance |
| `ai` | Entropy, coverage, compact output | LLM pipelines |
| `fraud` | Outliers, rarity, suspicious patterns | Investigation |
| `health` | Velocity, staleness, bus factor | Project health assessment |
| Custom | Your weights, your rules | TOML configuration |

## Domain Plugins

| Plugin | Recognizers | Covers |
|--------|-------------|--------|
| `vajra-domain-med` | ICD-10, CPT, NPI, NDC | Medical/EDI |
| `vajra-domain-sec` | CVE, MITRE ATT&CK, IPs, hashes, JWT | Security |
| `vajra-domain-devops` | K8s, Docker, Terraform, ARN, semver | DevOps/Infra |
| `vajra-domain-github` | PRs, issues, reviews, releases, commits, labels, milestones, deployments, check runs, actions | GitHub |
| `vajra-domain-source` | Naming conventions, file paths | Source code |
| `vajra-domain-encoding` | Base64, hex, URL, PEM, layers | Encoding |

## Global Options

| Flag | Applies To | Purpose |
|------|-----------|---------|
| `--window month/week/day` | `stats` | Temporal windowing with trend detection |
| `--time-field '$.path'` | `stats`, `cascade` | JSONPath to the timestamp field |
| `--group-by '$.path'` | `drift` | Partition records by field for population-level comparison |
| `--git-limit N` | `--input-format git` | Cap number of git commits ingested |
| `--git-branch name` | `--input-format git` | Read from a specific branch |
| `--semantic-paths` | `inspect` | Map tree-sitter AST paths to human-readable labels (9 languages) |

## The Engine

Every algorithm was chosen against three gates: **works at any scale** (O(n) or O(n log n)), **battle-tested** (published, peer-reviewed), and **deterministic** (same input = same output, always).
Expand Down Expand Up @@ -156,9 +198,11 @@ vajra/
├── vajra-drift # JSD, Wasserstein, path diff, severity
├── vajra-essence # Profiles, scoring, rendering, TOML config, chunking
├── vajra-query # Expression parser, analysis functions
├── vajra-cascade # Temporal cause-effect chain detection
├── vajra-domain-med # Medical/EDI plugin (ICD-10, CPT, NPI, NDC)
├── vajra-domain-sec # Security plugin (CVE, MITRE ATT&CK, IPs, hashes, JWT)
├── vajra-domain-devops # DevOps plugin (K8s, Docker, Terraform, ARN, semver)
├── vajra-domain-github # GitHub plugin (PRs, issues, reviews, releases, actions)
├── vajra-source # Source code parsing via tree-sitter (9 languages)
├── vajra-domain-source # Source code recognizers (naming conventions, paths)
├── vajra-domain-encoding # Encoding detection (Base64, hex, URL, PEM, layers)
Expand Down
Loading