Skip to content
/ snowl Public

LLM-Powered Cyber Threat Intelligence from RSS Feeds

License

Notifications You must be signed in to change notification settings

bitsalv/snowl

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Snowl

Snowl

Local-first Cyber Threat Intelligence platform for security analysts.

Status Python License

Snowl automates the collection, extraction, enrichment, and reporting of threat intelligence from RSS feeds. It combines LLM-powered structured extraction with a Zettelkasten knowledge graph for campaign detection and CTI framework analysis.


Features

Intelligence Analysis

Feature Description
Structured LLM Extraction Single-pass extraction of entities, IOCs, TTPs, and CTI categories via OpenRouter
11 CTI Categories From CVEs to Geopolitical threats, with auto-classification
IOC Extraction IPs, domains, hashes, CVEs, MITRE ATT&CK TTPs
Automatic Enrichment NIST NVD + MITRE ATT&CK mapping via MCP servers
Semantic Search TF-IDF based article similarity

Knowledge Graph (Zettelkasten)

Feature Description
Atomic Notes One note per intelligence entity, auto-generated from analysis
Auto-Backlinks Bidirectional links between related notes
Hierarchical Tags Diamond Model, Kill Chain, severity, temporal tags
Campaign Detection Cluster related articles via shared entities and relationships

CTI Frameworks

Framework Implementation
Diamond Model Automatic 4-vertex identification (Adversary, Capability, Infrastructure, Victim)
Cyber Kill Chain MITRE TTPs mapped to 7 Kill Chain phases
MITRE ATT&CK Tactic and technique extraction with enrichment

Reporting & Export

Feature Description
5 Report Types Executive, Technical, Threat Landscape, IOC, Campaign
PDF Export Professional reports via WeasyPrint + Jinja2
STIX 2.1 Export Standards-compliant threat intelligence bundles
CSV Export Filtered data export

Platform

Feature Description
CTI Assistant LLM-powered chatbot with MCP tool integration and session persistence
Interactive Dashboard Plotly charts for framework analytics and KPIs
TT-RSS Integration Full Tiny Tiny RSS integration with custom plugin, theme, and content filters

Quick Start

Prerequisites

  • Python 3.10+
  • Docker + Docker Compose (for TT-RSS)
  • OpenRouter API Key

Install & Run

git clone https://github.com/bitsalv/snowl.git
cd snowl

make init          # Create venv, install deps, generate .env
nano .env          # Add your OPENROUTER_API_KEY
make db            # Initialize database
make run           # Start Streamlit at http://localhost:8501

Architecture

                    +------------------+
                    |     TT-RSS       |
                    |  RSS aggregation |
                    |  (Docker stack)  |
                    +--------+---------+
                             |
                     snowl_bridge API
                             |
+----------------------------v----------------------------+
|                        Snowl                            |
|                                                         |
|  Ingest ─> Extract ─> Enrich ─> Classify ─> Store      |
|    │          │          │          │          │         |
|  RSS/API   LLM pass   NVD/ATT&CK  11 cats   SQLite    |
|            IOC regex   MCP servers            FTS5      |
|                                                         |
|  Pipeline output:                                       |
|  ├── Zettelkasten notes (atomic, backlinked)            |
|  ├── Tags (Diamond Model, Kill Chain, severity)         |
|  ├── Reports (PDF, 5 types)                             |
|  └── Export (STIX 2.1, CSV)                             |
|                                                         |
|  Interactive:                                           |
|  ├── Dashboard (Plotly analytics)                       |
|  ├── Assistant (LLM + MCP tools)                        |
|  └── IOC search & validation                            |
+----------------------------------------------------------+

Streamlit Pages

Page Description
Home Platform status, pending articles, pipeline trigger
Analytics KPI dashboard with timeline, category distribution, top threats
Assistant CTI chatbot with MCP tool integration
Notes Zettelkasten knowledge base with backlinks and graph
Reports Generate and preview CTI reports
Archive Browse and manage generated reports
IOC Search and validate indicators by type
Export CSV and STIX 2.1 export with filters
Settings API keys, model selection, connection tests

TT-RSS Integration

Snowl uses Tiny Tiny RSS (GPL-3.0) as its feed aggregation backend.

Component Description
snowl_bridge plugin Exposes articles to Snowl via JSON API
Custom theme Dark theme optimized for CTI triage
Content filters Auto-highlight MITRE ATT&CK, Kill Chain, Diamond Model terms
Auto-labeling trigger PostgreSQL trigger assigns "Pending Analysis" to new articles
Docker Compose Pre-configured TT-RSS + PostgreSQL stack

Setup

# Start TT-RSS
docker compose -f snowl/docker/tt-rss/docker-compose.yml up -d

# Install plugin, theme, and triggers
make ttrss-install-trigger

# Install CTI content filters
make ttrss-install-filters

TT-RSS runs at http://localhost:8280. Snowl connects via the snowl_bridge API to fetch articles for LLM analysis.

License: The snowl_bridge plugin (AGPL-3.0) runs inside TT-RSS (GPL-3.0) — these licenses are compatible. The main Snowl platform communicates with TT-RSS via HTTP API as separate programs.


Configuration

Copy env.template to .env:

# Required
OPENROUTER_API_KEY=sk-or-v1-...
OPENROUTER_MODEL=anthropic/claude-3.5-sonnet

# Database
DB_PATH=data/snowl.db

# Features
USE_LLM_CLASSIFY=true
USE_EMBEDDINGS=false
ENABLE_PDF=true

# MCP Servers (optional)
MCP_CVE_SEARCH_ENDPOINT=
MCP_NIST_ENDPOINT=
MCP_ATTACK_ENDPOINT=

# Logging
LOG_LEVEL=INFO

Feeds

53 curated CTI feeds ship in snowl/data/default_feeds.yaml across categories: CVE, Malware, Threat Intel, CERT, Cloud, ICS, and more. Add custom feeds via the Settings page or by editing the YAML.


Development

make help           # All available commands
make test           # Run test suite
make test-cov       # Tests with coverage report
make format         # Format code (black)
make lint           # Run linters
make clean          # Clean caches

Project Layout

snowl/
├── src/
│   ├── Home.py                # Streamlit entry point
│   ├── config.py              # Pydantic config + .env
│   ├── pages/                 # 8 Streamlit pages
│   ├── pipeline/              # LLM analysis + Zettelkasten
│   ├── ingest/                # RSS/TT-RSS ingestion
│   ├── extract/               # LLM + IOC extraction
│   ├── enrich/                # NVD, ATT&CK, MCP enrichment
│   ├── export/                # CSV, STIX 2.1, PDF
│   ├── reports/               # Report generation (5 types)
│   ├── chat/                  # CTI Assistant (LLM + MCP)
│   ├── search/                # TF-IDF semantic search
│   ├── zettelkasten/          # Knowledge graph operations
│   ├── integrations/          # TT-RSS client (5 modules)
│   ├── db/                    # SQLite + APSW + FTS5 + migrations
│   └── ui/                    # Streamlit components + theming
├── data/
│   ├── snowl.db               # SQLite database
│   └── default_feeds.yaml     # 53 curated CTI feeds
├── docker/
│   ├── tt-rss/                # TT-RSS Docker stack + plugin
│   └── mcp/                   # MCP server containers
├── tests/                     # Test suite
└── config/                    # Configuration files

License

GNU Affero General Public License v3.0 (AGPL-3.0) — see LICENSE.

Built for security analysts, by security analysts.

About

LLM-Powered Cyber Threat Intelligence from RSS Feeds

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 2

  •  
  •