Skip to content

USAGE.md

Latest commit

 

History

History
1187 lines (867 loc) · 30.3 KB

USAGE.md

File metadata and controls

1187 lines (867 loc) · 30.3 KB

📖 ShadowHunter Usage Guide

Comprehensive guide for using ShadowHunter's threat intelligence and OSINT capabilities.

Table of Contents

  1. Installation
  2. Quick Start
  3. Core Commands
  4. Phase 3 Multi-Agent Hunters
  5. Module Deep Dives
  6. OSINT Tools
  7. Google Dorking
  8. Web Crawling
  9. API Server
  10. Configuration
  11. Integration Examples
  12. Best Practices
  13. Troubleshooting

🔧 Installation

Prerequisites

  • Python 3.9 or higher
  • pip (Python package manager)
  • Optional: Tor (for dark web access)
  • Optional: Neo4j (for graph database)

Step 1: Clone the Repository

git clone https://github.com/fevra-dev/ShadowHunter.git
cd ShadowHunter

Step 2: Create Virtual Environment

python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

Step 3: Install Dependencies

pip install -r requirements.txt

Step 4: Verify Installation

python main.py info

You should see a status table showing all available modules.

Step 5: (Optional) Create Alias

Add to your ~/.bashrc or ~/.zshrc:

alias shadowhunter="python /path/to/ShadowHunter/main.py"

🚀 Quick Start

View All Commands

shadowhunter --help

Check System Status

shadowhunter info

Run Your First Scan

# Scan a domain for credential leaks
shadowhunter scan --domain example.com

# Parse a stealer log archive
shadowhunter parse /path/to/stealer_logs.zip

# Analyze a cryptocurrency wallet
shadowhunter blockchain 0x742d35Cc6634C0532925a3b844Bc9e7595f5d5E2

# Discover attack surface
shadowhunter surface targetcompany.com

📋 Core Commands

Command Description Example
info Show system status and modules shadowhunter info
scan Credential monitoring scan shadowhunter scan -d example.com
parse Parse stealer log archives shadowhunter parse ./logs/
enrich Enrich IOCs with threat intel shadowhunter enrich 1.2.3.4
validate Validate session cookies shadowhunter validate sessions.json
leaks Monitor for data leaks shadowhunter leaks -k "company"
surface Attack surface discovery shadowhunter surface domain.com
blockchain Cryptocurrency forensics shadowhunter blockchain 0x...
dork Google dorking recon shadowhunter dork target.com
crawl Web crawling shadowhunter crawl https://site.com
osint OSINT lookups shadowhunter osint email user@domain.com
monitor Continuous monitoring shadowhunter monitor -d company.com
hunters Multi-agent hunters (19 hunters, synthesis) shadowhunter hunters [--events-file f.json] [--enhanced]
search Dark web search (LLM refine → Tor → scrape → summary; multi-model) shadowhunter search -q "query" [-m model] [-o file.md]
api Start REST API server shadowhunter api

🤖 Phase 3 Multi-Agent Hunters

Phase 3 runs 19 specialized hunters over an event stream, processes escape hatches, de-duplicates results, writes high-confidence findings to Neo4j, and produces a synthesis report.

Run hunters

# Default: sample event (wallet + email), config Neo4j
shadowhunter hunters

# Custom event stream (JSON array of events)
shadowhunter hunters --events-file events.json

# Explicit Neo4j connection
shadowhunter hunters --neo4j-uri bolt://localhost:7687 --neo4j-user neo4j --neo4j-password <pwd>

# Enhanced orchestrator (priority routing, circuit breakers, condition-based escape hatches)
shadowhunter hunters --enhanced
shadowhunter hunters --enhanced --events-file events.json

Each event in events.json should have: source, raw_content, timestamp, and optional metadata, stix_objects.

Neo4j schema (optional)

Apply constraints and indexes once for best performance (Neo4j 5.x):

cypher-shell -u neo4j -p <password> < scripts/neo4j_schema_phase3.cypher

See RUNBOOK §8 and scripts/README.md.

NL-to-Cypher

Translate natural language to read-only Cypher (for analysts):

from shadowhunter.nl_to_cypher import translate_nl_to_cypher

# Requires schema context and optional LLM; see shadowhunter/nl_to_cypher/
translation = translate_nl_to_cypher("Find threat actors by IP")
# Validate with: validate_cypher_readonly(cypher, neo4j_driver)

Tests: pytest tests/test_nl_to_cypher.py.

Dark Web Search

LLM-refined query → Tor onion search engines → filter results → scrape pages → LLM summary. Requires Tor (proxy 127.0.0.1:9050) and at least one LLM API key (OpenAI, Anthropic, Google, xAI/Grok, or Ollama via OPENAI_BASE_URL).

# Basic run (saves summary_<timestamp>.md)
shadowhunter search -q "credential leak 2024"

# Custom model and output file
shadowhunter search -q "ransomware forum" -m gpt-4o -o report.md

# More parallel threads for search/scrape
shadowhunter search -q "marketplace" -t 8 -o findings.md

Without any LLM API key configured, the command uses the raw query and the first 20 results and produces a minimal text summary (no LLM refine/filter/summarize). Use --model to pick a provider/model:

Provider Env var Example --model
OpenAI OPENAI_API_KEY gpt-4o
Anthropic ANTHROPIC_API_KEY claude-3-5-sonnet
Google GOOGLE_API_KEY gemini-1.5-flash
xAI (Grok) XAI_API_KEY grok-2
Ollama OPENAI_BASE_URL=http://127.0.0.1:11434/v1 any local model name

🔬 Module Deep Dives

1. Stealer Log Parser

Parse malware exfiltration logs from info-stealers like RedLine, Vidar, Lumma, Raccoon, and RisePro.

Basic Usage

# Parse a single archive
shadowhunter parse /path/to/stealer_log.zip

# Parse a directory of archives
shadowhunter parse /path/to/logs_directory/

# Specify stealer family (auto-detected by default)
shadowhunter parse log.zip --family redline

# Export results to JSON
shadowhunter parse log.zip --output results.json

# Limit concurrent processing
shadowhunter parse ./logs/ --concurrent 5

Output Format

[PARSE_START] Opening archive: stealer_log.zip
[PASSWORDS] Extracted 142 credentials
[COOKIES] Extracted 89 cookies
[WALLET] Detected ethereum wallet: 0x742d...
[PARSE_COMPLETE] 142 creds, 89 cookies, 3 wallets, Corporate: True

📊 Parse Summary
├── Total Credentials: 142
├── Session Cookies: 89 (34 high-value)
├── Crypto Wallets: 3
├── Corporate Victim: Yes
└── Risk Indicators: VPN access, Domain joined

Extracted Data

  • Credentials: URL, username, password (hashed for storage)
  • Session Cookies: Domain, name, value, expiry, security flags
  • Crypto Wallets: Bitcoin, Ethereum, Monero addresses + seed phrases
  • Corporate Indicators: VPN credentials, domain-joined machines, enterprise software

Python API

from shadowhunter.parsers import StealerLogPipeline, PipelineConfig
from pathlib import Path

async def parse_logs():
    config = PipelineConfig(max_concurrent=10, timeout_seconds=300)
    pipeline = StealerLogPipeline(config)
    
    archives = [Path("log1.zip"), Path("log2.zip")]
    
    async for result in pipeline.process_archives(archives):
        print(f"Parsed: {result.source_file}")
        print(f"  Credentials: {result.total_credentials}")
        print(f"  Corporate: {result.is_corporate_victim}")
        
        for wallet in result.wallets:
            print(f"  Wallet: {wallet.wallet_type} - {wallet.address}")

2. IOC Enrichment

Enrich Indicators of Compromise with threat intelligence from VirusTotal, Shodan, and AbuseIPDB.

Basic Usage

# Enrich an IP address
shadowhunter enrich 185.220.101.1

# Enrich a domain
shadowhunter enrich malicious-domain.com

# Enrich a file hash
shadowhunter enrich 44d88612fea8a8f36de82e1278abb02f

# Extract and enrich IOCs from text
shadowhunter enrich --file suspicious_email.txt

# Use specific API keys
shadowhunter enrich 1.2.3.4 \
  --virustotal YOUR_VT_KEY \
  --shodan YOUR_SHODAN_KEY \
  --abuseipdb YOUR_ABUSE_KEY

# Export results
shadowhunter enrich 1.2.3.4 --output enriched.json

Environment Variables

Set these to avoid passing keys every time:

export VIRUSTOTAL_API_KEY="your_key"
export SHODAN_API_KEY="your_key"
export ABUSEIPDB_API_KEY="your_key"

Output Example

🔍 IOC Enrichment: 185.220.101.1

[VirusTotal]
├── Detection: 42/93 engines flagged as malicious
├── Tags: tor-exit-node, malware, scanner
├── First Seen: 2023-01-15
└── Reputation: -87

[Shodan]
├── Ports: 22, 80, 443, 9001
├── Country: Germany
├── ISP: Hetzner Online GmbH
├── Hostnames: tor-exit.example.com
└── Vulns: None detected

[AbuseIPDB]
├── Abuse Score: 100%
├── Reports: 2,847
├── Categories: SSH brute-force, Web attack, Tor exit
└── Last Reported: 2 hours ago

[Summary]
├── Threat Level: CRITICAL
├── Type: Tor Exit Node
├── Risk Score: 95/100
└── Recommended Action: Block immediately

Supported IOC Types

Type Example Sources
IPv4 192.168.1.1 VT, Shodan, AbuseIPDB
IPv6 2001:db8::1 VT, AbuseIPDB
Domain evil.com VT, Shodan
URL https://evil.com/malware.exe VT
MD5 44d88612... VT
SHA1 3395856c... VT
SHA256 e3b0c442... VT
Email hacker@evil.com VT

3. Session Validator

Validate stolen session cookies to check if they're still active (for your own organization's credentials only).

Basic Usage

# Validate sessions from parsed stealer logs
shadowhunter validate parsed_results.json

# Validate with allowed domains (safety filter)
shadowhunter validate sessions.json --allowed yourcompany.com

# Specify platforms to check
shadowhunter validate sessions.json --platforms google,microsoft

# Limit concurrent validations
shadowhunter validate sessions.json --concurrent 3

# Export results
shadowhunter validate sessions.json --output valid_sessions.json

Supported Platforms

Platform Cookie Names Checked Validation Method
Google Workspace SID, HSID, SSID, APISID accounts.google.com check
Microsoft 365 ESTSAUTH, ESTSAUTHPERSISTENT graph.microsoft.com check

Output Example

🔐 Session Validator

Target: sessions.json (156 sessions)
Allowed Domains: yourcompany.com

[Validating Google Sessions]
├── test@yourcompany.com: ✓ VALID (Admin, MFA enabled)
├── user2@yourcompany.com: ✗ EXPIRED
└── shared@yourcompany.com: ✓ VALID (Standard user)

[Validating Microsoft Sessions]
├── admin@yourcompany.onmicrosoft.com: ✓ VALID (Global Admin!)
└── service@yourcompany.com: ✗ INVALID

📊 Summary
├── Total Checked: 156
├── Valid: 3
├── Expired: 89
├── Invalid: 64
└── ⚠️ HIGH RISK: 1 Global Admin session active!

Recommendations:
1. Immediately revoke admin@yourcompany session
2. Force password reset for affected users
3. Review MFA status for all compromised accounts

⚠️ Ethical Use Warning

ONLY validate sessions for:

  • Domains you own
  • Organizations that have authorized testing
  • Incident response for your own infrastructure

NEVER validate sessions for:

  • Personal accounts you don't own
  • Third-party services without authorization
  • Any illegal or unethical purposes

4. Data Leak Monitor

Monitor GitHub, paste sites, and other sources for exposed secrets and credentials.

Basic Usage

# Search for company secrets on GitHub
shadowhunter leaks -k "acmecorp"

# Multiple keywords
shadowhunter leaks -k "acmecorp" -k "acme-internal" -k "@acmecorp.com"

# With GitHub token (higher rate limits)
shadowhunter leaks -k "mycompany" --github-token YOUR_TOKEN

# Monitor specific domains
shadowhunter leaks --domain mycompany.com --domain internal.mycompany.com

# Continuous monitoring
shadowhunter leaks -k "mycompany" --continuous --interval 30

# Export alerts
shadowhunter leaks -k "mycompany" --output leaks.json

Detected Secret Types

Secret Type Example Pattern
AWS Access Key AKIA[0-9A-Z]{16}
AWS Secret Key aws_secret_access_key = ...
GitHub Token ghp_[A-Za-z0-9]{36}
Google API Key AIza[0-9A-Za-z-_]{35}
Slack Token xox[baprs]-...
Stripe Key sk_live_[0-9a-zA-Z]{24}
Private Keys -----BEGIN RSA PRIVATE KEY-----
Database URLs postgres://user:pass@host/db
JWT Secrets jwt_secret, JWT_KEY

Output Example

🔍 Data Leak Monitor

Keywords: acmecorp, @acmecorp.com
Sources: GitHub, Pastebin

[GitHub Search]
├── Found: 23 results
├── Scanned: 18 files
└── Secrets: 4 detected

[Alert 1] CRITICAL - AWS Credentials
├── Source: github.com/dev123/old-project
├── File: config/settings.py
├── Line: 45
├── Secret: AKIA***************EXAMPLE
├── Exposed Since: 2024-06-15
└── Action: Rotate immediately

[Alert 2] HIGH - Database URL
├── Source: pastebin.com/abc123
├── Secret: postgres://admin:***@db.acmecorp.com/prod
├── Exposed Since: 2024-12-01
└── Action: Change password, audit access

📊 Summary
├── Total Leaks: 4
├── Critical: 1
├── High: 2
├── Medium: 1
└── Sources: 3 unique

5. Attack Surface Scanner

Discover and monitor an organization's external attack surface.

Basic Usage

# Basic surface scan
shadowhunter surface example.com

# Deep scan (extended port list)
shadowhunter surface example.com --deep

# Skip specific discovery methods
shadowhunter surface example.com --no-ports --no-crtsh

# Custom ports
shadowhunter surface example.com -p 80 -p 443 -p 8080 -p 8443

# Custom subdomain wordlist
shadowhunter surface example.com -w /path/to/wordlist.txt

# Export results
shadowhunter surface example.com --output surface_report.json

# Brief output
shadowhunter surface example.com --format brief

Discovery Methods

Method Description Flag to Skip
crt.sh Certificate Transparency logs --no-crtsh
DNS Brute Common subdomain wordlist Built-in
Wayback Historical subdomains --no-wayback
Port Scan Service discovery --no-ports
Tech Fingerprint Web technology detection Always on

Output Example

🌐 Attack Surface Scanner

Target: example.com
Deep Scan: Enabled
Ports: Extended (1000+ ports)

[Subdomain Discovery]
├── crt.sh: 45 subdomains
├── DNS Brute: 12 subdomains
├── Wayback: 8 subdomains
└── Total Unique: 52 subdomains

[Port Scanning]
├── www.example.com: 80, 443
├── api.example.com: 443, 8443
├── dev.example.com: 22, 80, 443, 3000
└── staging.example.com: 80, 443

[Technology Fingerprinting]
├── www.example.com: Nginx 1.21, React, Cloudflare
├── api.example.com: Express.js, Node 18
└── dev.example.com: Apache 2.4, PHP 8.1

📊 Scan Summary
├── Duration: 45.2s
├── Total Assets: 52
├── Open Services: 156
├── High Risk Assets: 3
├── SSL Issues: 2

⚠️ High Risk Findings:
1. dev.example.com - SSH exposed (port 22)
2. staging.example.com - Outdated Apache (CVE-2023-xxxx)
3. old.example.com - SSL certificate expired

Risk Scoring

Assets are scored 0-100 based on:

  • Open high-risk ports (SSH, RDP, databases)
  • SSL/TLS issues (expired, self-signed, weak ciphers)
  • Outdated software versions
  • Missing security headers
  • Exposed admin panels

6. Blockchain Forensics

Analyze cryptocurrency wallets for threat intelligence.

Basic Usage

# Analyze any wallet (auto-detect chain)
shadowhunter blockchain 0x742d35Cc6634C0532925a3b844Bc9e7595f5d5E2

# Specify chain explicitly
shadowhunter blockchain SoLaNaAdDrEsS --chain solana

# Zcash transparent address
shadowhunter blockchain t1ABC123... --chain zcash

# With Helius API for Solana (better data)
shadowhunter blockchain SOL_ADDRESS --helius YOUR_HELIUS_KEY

# With Bubblemaps for EVM tokens
shadowhunter blockchain 0x... --bubblemaps YOUR_KEY

# Trace transactions (follow the money)
shadowhunter blockchain 0x... --trace --depth 3

# Export report
shadowhunter blockchain 0x... --output wallet_report.json

Supported Chains

Chain Address Format Data Source
Bitcoin 1..., 3..., bc1... Blockchain.info
Ethereum 0x... (40 hex) Etherscan, Bubblemaps
Solana Base58 (32-44 chars) Helius, Solscan
Zcash t1..., t3... zcha.in
BSC 0x... BSCScan
Polygon 0x... Polygonscan

Output Example

🔗 Blockchain Forensics

Address: 0x742d35Cc6634C0532925a3b844Bc9e7595f5d5E2
Chain: Ethereum

🔍 Wallet Profile
├── Type: Unknown
├── Risk: HIGH (75/100)
├── Balance: 12.5 ETH
├── Transactions: 1,247
├── First Seen: 2023-03-15
├── Last Active: 2 hours ago
└── Labels: active, high-volume

⚠️ Risk Factors:
• Interacted with Tornado Cash (mixer)
• High transaction volume
• Connected to 3 flagged addresses

💰 Token Holdings:
├── USDC: 15,000.00
├── WETH: 5.25
├── UNI: 450.00
└── +8 more tokens

🔗 Recent Transactions:
├── 0x1234... → 0xABCD... (0.5 ETH) 2 hours ago
├── 0x5678... → This (1.0 ETH) 5 hours ago
└── This → 0x9999... (2.0 ETH) 1 day ago

📊 Transaction Trace (depth=2):
└── 0x742d35Cc... (this)
    ├── 0xABCD1234... (3 connections)
    │   ├── 0x1111...
    │   └── 0x2222... ⚠️ Tornado Cash
    └── 0xDEF567... (1 connection)
        └── 0x3333... (Binance Hot Wallet)

Explorer: https://etherscan.io/address/0x742d35Cc...

Known Entities Detection

ShadowHunter flags:

  • Ransomware wallets (WannaCry, Ryuk, REvil, Conti, etc.)
  • Mixers (Tornado Cash, Wasabi, etc.)
  • Exchanges (Binance, Coinbase, Kraken)
  • OFAC sanctioned addresses
  • Known scam/exploit wallets

🔎 OSINT Tools

Perform open-source intelligence lookups.

Email Lookup

# Full email investigation
shadowhunter osint email target@example.com

# Check for breaches only
shadowhunter osint email target@example.com --breaches-only

Output includes:

  • Email validation (deliverable, disposable, role account)
  • Breach history (sources, dates, data types)
  • Associated domains
  • Gravatar/profile pictures
  • Social media presence

Phone Lookup

# Analyze phone number
shadowhunter osint phone +14155551234
shadowhunter osint phone 4155551234 --country US

Output includes:

  • Country and carrier
  • Line type (mobile, landline, VoIP)
  • Validity check
  • Associated services (WhatsApp, Telegram)

Username Enumeration

# Search across 30+ platforms
shadowhunter osint username johndoe

# Specific platforms only
shadowhunter osint username johndoe --platforms twitter,github,linkedin

Platforms checked: Twitter, GitHub, LinkedIn, Instagram, Reddit, TikTok, YouTube, Facebook, Discord, Telegram, Steam, Twitch, Pinterest, Medium, and 20+ more.

Domain Reconnaissance

# Full domain recon
shadowhunter osint domain example.com

Output includes:

  • WHOIS information (registrar, dates, contacts)
  • DNS records (A, AAAA, MX, TXT, NS)
  • Subdomains discovered
  • Technologies detected
  • SSL certificate details

🔍 Google Dorking

Automated Google dork-based reconnaissance.

Basic Usage

# Quick scan with all categories
shadowhunter dork example.com

# Specific category
shadowhunter dork example.com --category credentials
shadowhunter dork example.com --category sensitive_files
shadowhunter dork example.com --category admin_panels

# Filter by severity
shadowhunter dork example.com --severity CRITICAL
shadowhunter dork example.com --severity HIGH,CRITICAL

# Custom Google API (for more results)
shadowhunter dork example.com --google-api YOUR_KEY --cse-id YOUR_CSE

# Export results
shadowhunter dork example.com --output dork_report.json

Dork Categories

Category Description Example Dorks
credentials Exposed passwords/keys "password" filetype:env
sensitive_files Config, backup files filetype:sql, filetype:bak
admin_panels Login pages inurl:admin, inurl:login
exposed_docs Internal documents filetype:pdf "confidential"
cloud_storage S3, Azure, GCP buckets site:s3.amazonaws.com
git_exposure Exposed .git folders inurl:.git
api_endpoints API documentation inurl:api, inurl:swagger
error_messages Stack traces, errors "fatal error", "sql syntax"
technology Version disclosure "powered by", "running on"
subdomains Subdomain discovery site:*.example.com

🕷️ Web Crawling

Asynchronous web crawler with data extraction.

Basic Usage

# Basic crawl
shadowhunter crawl https://example.com

# Set depth and page limits
shadowhunter crawl https://example.com --depth 3 --max-pages 100

# Extract specific data
shadowhunter crawl https://example.com --extract emails,phones,secrets

# Follow subdomains
shadowhunter crawl https://example.com --include-subdomains

# Respect robots.txt
shadowhunter crawl https://example.com --respect-robots

# Export results
shadowhunter crawl https://example.com --output crawl_report.json

Extracted Data

  • Emails: All email addresses found
  • Phone Numbers: Detected phone formats
  • URLs: Internal and external links
  • Forms: Form fields and actions
  • API Endpoints: REST/GraphQL patterns
  • Secrets: API keys, tokens, credentials
  • Subdomains: Discovered subdomains
  • Technologies: Detected frameworks/libraries

🌐 API Server

ShadowHunter includes a REST API for programmatic access.

Starting the Server

# Default (port 8000)
shadowhunter api

# Custom port
shadowhunter api --port 3000

# Development mode with auto-reload
shadowhunter api --reload

# Production mode
shadowhunter api --host 0.0.0.0 --workers 4

API Endpoints

Endpoint Method Description
/api/health GET Health check
/api/overview GET Dashboard statistics
/api/alerts GET List alerts (supports limit, offset; returns { items, total, limit, offset })
/api/alerts POST Create alert
/api/domains GET List monitored domains
/api/domains POST Add domain
/api/scan/start POST Trigger scan
/api/scan/status GET Scan status
/api/export/report/{domain} GET Export report

API Documentation

Access Swagger UI at: http://localhost:8000/docs

Example Requests

# Health check
curl http://localhost:8000/api/health

# Get overview stats
curl http://localhost:8000/api/overview

# List alerts
curl http://localhost:8000/api/alerts?severity=CRITICAL

# Add domain to monitoring
curl -X POST http://localhost:8000/api/domains \
  -H "Content-Type: application/json" \
  -d '{"domain": "example.com", "organization": "ACME Corp"}'

⚙️ Configuration

Environment Variables

Create a .env file in the project root:

# API Keys
VIRUSTOTAL_API_KEY=your_key
SHODAN_API_KEY=your_key
ABUSEIPDB_API_KEY=your_key
HELIUS_API_KEY=your_key
BUBBLEMAPS_API_KEY=your_key
GITHUB_TOKEN=your_token

# HaveIBeenPwned
HIBP_API_KEY=your_key

# Telegram (for monitoring)
TELEGRAM_API_ID=your_id
TELEGRAM_API_HASH=your_hash
TELEGRAM_PHONE=+1234567890

# Neo4j (optional)
NEO4J_URI=bolt://localhost:7687
NEO4J_USER=neo4j
NEO4J_PASSWORD=password

# Email Alerts
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_USER=your_email@gmail.com
SMTP_PASSWORD=your_app_password
ALERT_RECIPIENT=security@yourcompany.com

# API Server
SHADOWHUNTER_API_HOST=0.0.0.0
SHADOWHUNTER_API_PORT=8000
SHADOWHUNTER_SECRET_KEY=your_jwt_secret

Log Levels

Control verbosity with --log-level:

shadowhunter scan -d example.com --log-level DEBUG
shadowhunter scan -d example.com --log-level INFO   # default
shadowhunter scan -d example.com --log-level WARNING
shadowhunter scan -d example.com --log-level ERROR

🔗 Integration Examples

Python Script Integration

import asyncio
from shadowhunter.parsers import StealerLogPipeline, PipelineConfig
from shadowhunter.blockchain import BlockchainForensicsEngine
from shadowhunter.intelligence import EnrichmentEngine

async def full_investigation(stealer_log_path: str):
    """Complete threat investigation workflow."""
    
    # Step 1: Parse stealer logs
    pipeline = StealerLogPipeline(PipelineConfig())
    results = []
    async for result in pipeline.process_archives([stealer_log_path]):
        results.append(result)
        print(f"Found {result.total_credentials} credentials")
    
    # Step 2: Enrich extracted IOCs
    enrichment = EnrichmentEngine(
        virustotal_key="YOUR_KEY",
        shodan_key="YOUR_KEY"
    )
    
    for result in results:
        for cred in result.credentials:
            # Enrich the domain
            ioc_results = await enrichment.enrich_text(cred.domain)
            for ioc, enrichment_data in ioc_results:
                if enrichment_data.threat_level == "CRITICAL":
                    print(f"CRITICAL: {ioc.value} - {enrichment_data.summary}")
    
    # Step 3: Analyze crypto wallets
    blockchain = BlockchainForensicsEngine(helius_api_key="YOUR_KEY")
    
    for result in results:
        if result.wallets:
            profiles = await blockchain.analyze_stealer_wallets(result.wallets)
            for profile in profiles:
                if profile.is_high_risk:
                    print(f"HIGH RISK WALLET: {profile.address}")
                    print(f"  Risk Factors: {profile.risk_factors}")

asyncio.run(full_investigation("stealer_logs.zip"))

Bash Automation

#!/bin/bash
# Daily threat scan script

DOMAINS="company.com subsidiary.com"
OUTPUT_DIR="/var/log/shadowhunter"
DATE=$(date +%Y-%m-%d)

# Create output directory
mkdir -p $OUTPUT_DIR/$DATE

# Scan each domain
for domain in $DOMAINS; do
    echo "Scanning $domain..."
    
    # Attack surface scan
    shadowhunter surface $domain --output $OUTPUT_DIR/$DATE/${domain}_surface.json
    
    # Data leak check
    shadowhunter leaks -k $domain --output $OUTPUT_DIR/$DATE/${domain}_leaks.json
    
    # Google dorking
    shadowhunter dork $domain --output $OUTPUT_DIR/$DATE/${domain}_dorks.json
done

# Combine results
echo "Scans complete. Results in $OUTPUT_DIR/$DATE/"

CI/CD Integration

# .github/workflows/security-scan.yml
name: Security Scan

on:
  schedule:
    - cron: '0 6 * * *'  # Daily at 6 AM
  workflow_dispatch:

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Setup Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      
      - name: Install ShadowHunter
        run: |
          git clone https://github.com/fevra-dev/ShadowHunter.git
          pip install -r ShadowHunter/requirements.txt
      
      - name: Run Attack Surface Scan
        run: |
          cd ShadowHunter
          python main.py surface ${{ secrets.TARGET_DOMAIN }} \
            --output ../surface_report.json
      
      - name: Run Leak Detection
        run: |
          cd ShadowHunter
          python main.py leaks \
            -k ${{ secrets.COMPANY_KEYWORD }} \
            --github-token ${{ secrets.GITHUB_TOKEN }} \
            --output ../leaks_report.json
      
      - name: Upload Reports
        uses: actions/upload-artifact@v4
        with:
          name: security-reports
          path: |
            surface_report.json
            leaks_report.json

✅ Best Practices

Operational Security

  1. Use a dedicated VM for dark web research
  2. Enable Tor for anonymous scanning
  3. Rotate API keys regularly
  4. Never store credentials in plaintext
  5. Encrypt exports containing sensitive data
  6. Log all activities for audit trails

Ethical Guidelines

  1. Only scan assets you own or have permission to test
  2. Follow responsible disclosure for vulnerabilities
  3. Respect rate limits on APIs
  4. Don't abuse session validation - only for your organization
  5. Report illegal content to appropriate authorities
  6. Document your authorization before testing

Performance Tips

  1. Use API keys for higher rate limits
  2. Limit concurrent operations based on your system
  3. Cache results where appropriate
  4. Use --output to save results for later analysis
  5. Run intensive scans during off-hours

🔧 Troubleshooting

Common Issues

"Module not found" error

# Ensure virtual environment is activated
source venv/bin/activate

# Reinstall dependencies
pip install -r requirements.txt

API rate limits

# Use your own API keys
export VIRUSTOTAL_API_KEY="your_key"

# Reduce concurrency
shadowhunter parse logs/ --concurrent 2

Tor connection failed

# Ensure Tor is running
sudo systemctl start tor

# Check Tor status
sudo systemctl status tor

Permission denied

# Fix permissions
chmod +x main.py

# Or run with Python directly
python main.py --help

Neo4j connection failed

# Check Neo4j status
sudo systemctl status neo4j

# Verify credentials in .env
NEO4J_URI=bolt://localhost:7687
NEO4J_USER=neo4j
NEO4J_PASSWORD=your_password

Getting Help

  • GitHub Issues: Report bugs
  • Logs: Check logs/shadowhunter.log for detailed errors
  • Debug mode: Run with --log-level DEBUG

ShadowHunter — Illuminating the Shadows

Built for cybersecurity professionals, threat hunters, and security researchers.