SAF Classifier

A CLI tool that classifies a SOC product's autonomy level against the SOC Autonomy Framework (SAF).

Framework: soc-autonomy-framework
Paper: The Autonomous SOC Manifesto

---

What It Does

Answer a structured questionnaire about a SOC product's capabilities and governance. The classifier outputs:

• The product's SAF level (L0–L5)
• A per-dimension breakdown showing which dimension is the limiting factor
• Specific gaps preventing the next level

Useful for:

• Vendors wanting to understand where their product sits
• Buyers evaluating and comparing SOC AI products
• Researchers classifying systems in papers

---

Install

pip install saf-classifier

Or run directly:

git clone https://github.com/sirp-labs/saf-classifier
cd saf-classifier
pip install -r requirements.txt
python classifier.py

---

Usage

Interactive mode (recommended for first use)

python classifier.py --interactive

Walks you through each dimension with explanations.

From a capability profile JSON

python classifier.py --profile examples/crowdstrike-charlotte.json

With output to file

python classifier.py --interactive --output my-product-classification.json

---

Example Output

SAF Classification Report
─────────────────────────────────────────────────────
System:     Example SOC Product
Date:       2026-04-16
Evaluator:  self-reported

DIMENSION SCORES
  Decision Scope:          L2  ██████░░░░
  Autonomous Action Rate:  L2  ██████░░░░
  Governance:              L1  ████░░░░░░  ← limiting factor
  Human Role:              L2  ██████░░░░

DERIVED SAF LEVEL:  L1

WHY NOT L2?
  Governance is the limiting dimension.
  Missing: documented confidence thresholds, action boundary definitions.

GAPS TO REACH L2:
  □ Define and document confidence thresholds for automated decisions
  □ Document which action types are permitted without human approval
  □ Implement audit logging for automated triage actions

─────────────────────────────────────────────────────
Note: This classification is self-reported. Verified classification
requires independent measurement against the SAF Benchmark.

---

The Questionnaire

The classifier asks questions across four SAF dimensions. Key questions:

Decision Scope

• Does the system automatically triage and enrich alerts without human initiation?
• Does the system form hypotheses about attacker intent?
• Does the system execute response actions autonomously?

Autonomous Action Rate

• What % of alerts are resolved end-to-end without human touchpoints?
• Is this measured empirically or estimated?

Governance

• Are confidence thresholds defined and enforced?
• Is there a documented and enforced action boundary policy?
• Are all autonomous actions logged with full reasoning chains?
• Has the system been adversarially tested?

Human Role

• What decisions always require human approval?
• What triggers escalation to a human analyst?
• Can the system operate without a human on-call?

Full question set: data/questions.json

---

Example Profiles

File	System	Classification
examples/generic-l1.json	Generic L1 product	L1
examples/generic-l2.json	Generic L2 product	L2
examples/generic-l3.json	Generic L3 system	L3

---

Contributing

• Add example profiles for publicly documented products (based on public information only)
• Propose refinements to the questionnaire
• Submit real classification results to the community dataset

See CONTRIBUTING.md

---

Disclaimer

Classifications produced by this tool are self-reported assessments based on questionnaire responses. They are not independently validated. The SAF Benchmark provides empirical measurement.

A vendor should not use this tool's output as a marketing claim without independent validation.

---

License

Apache 2.0