🔥 HEARTH: Hunting Exchange and Research Threat Hub 🔥

A community-driven, AI-powered exchange for threat hunting ideas and methodologies.
Explore the Live Database »

Submit a Hunt · Report a Bug · Request a Feature

---

📖 About The Project

HEARTH (Hunting Exchange and Research Threat Hub) is a centralized, open-source platform for security professionals to share, discover, and collaborate on threat hunting hypotheses. Generating effective and timely hunts is a major challenge, and HEARTH aims to solve it by building a comprehensive, community-curated knowledge base.

Our goal is to create a vibrant ecosystem where hunters can:

• Discover new and creative hunting ideas.
• Contribute their own research and CTI.
• Collaborate with others to refine and improve detection strategies.
• Automate the mundane parts of hunt creation and focus on what matters.

This project uses the PEAK Threat Hunting Framework to categorize hunts into three types:

• 🔥 Flames: Hypothesis-driven hunts with clear, testable objectives.
• 🪵 Embers: Baselining and exploratory analysis to understand an environment.
• 🔮 Alchemy: Model-assisted and algorithmic approaches to threat detection.

---

✨ Key Features

HEARTH is more than just a list of hunts; it's a fully-featured platform with a sophisticated automation backend.

Feature	Description
🔍 Interactive UI	A searchable, filterable, and sortable database of all hunts, making it easy to find exactly what you're looking for.
🤖 AI-Powered CTI Analysis	Submit a link to a CTI report, and our system uses Claude Sonnet 4.5 to automatically read, analyze, and draft a complete hunt hypothesis for you.
🌐 Advanced Web Scraping	Intelligent content extraction supporting Brotli/Zstandard compression, JavaScript-rendered content, and multiple formats (HTML, PDF, DOCX).
🎯 MITRE ATT&CK Integration	Validates technique IDs and provides accurate tactic mappings using official MITRE ATT&CK Enterprise framework data (691 techniques, 99% accuracy).
🛡️ Duplicate Detection	AI-powered system analyzes new submissions against the existing database to flag potential duplicates and ensure content quality. 30-60x faster with SQLite indexing.
⚡ Performance Optimized	SQLite database index provides lightning-fast queries while keeping markdown files as the source of truth.
⚙️ Automated Workflows	GitHub Actions manage the entire lifecycle of a submission, from initial draft to final approval, including creating branches and PRs.
🏆 Contributor Leaderboard	We recognize and celebrate our contributors! An automated system tracks submissions and maintains a public leaderboard.
✅ Review & Regeneration Loop	Maintainers can request a new version of an AI-generated hunt by simply adding a regenerate label to the submission issue.

---

🚀 How to Contribute

Contributing to HEARTH is designed to be as easy as possible. We use GitHub Issues as a streamlined submission hub.

Option 1: Automated CTI Submission (Recommended)

Have a link to a great threat intelligence report, blog post, or whitepaper? Let our AI do the heavy lifting.

1. Click here to open a CTI Submission issue.
2. Paste the URL to the CTI source and provide your name/handle for attribution.
3. Submit the issue. Our bot will:
   • Read and analyze the content.
   • Generate a complete hunt draft.
   • Check for duplicates.
   • Post the draft in a new branch and comment on your issue with a link for review.

Option 2: Manual Hunt Submission

If you have a fully-formed hunt idea of your own, you can submit it manually.

1. Click here to open a Manual Hunt Submission issue.
2. Fill out the template with your hypothesis, tactic, references, and other details.
3. Submit the issue for review by the maintainers.

Important

All approved submissions are integrated into the HEARTH database and credited to the submitter on our Contributors Leaderboard.

---

🛠️ Built With

HEARTH combines a simple frontend with a powerful, serverless backend built on GitHub Actions.

• Frontend:
  • HTML5
  • CSS3
  • Vanilla JavaScript
• Backend & Automation:
  • GitHub Actions
  • Python
  • Claude (Anthropic) API and OpenAI API
  • SQLite (for fast indexing and queries)
• Hosting:
  • GitHub Pages

---

🏗️ Architecture

HEARTH uses a hybrid approach that balances simplicity with performance:

Hunt Storage

• Markdown files in Flames/, Embers/, and Alchemy/ are the source of truth
• Hunt files remain human-readable and version-controlled
• Easy to browse, edit, and contribute via standard Git workflows

Database Index

• SQLite database (database/hunts.db) provides fast querying for duplicate detection
• Automatically updated when hunt files are added, modified, or deleted
• Provides 30-60x faster duplicate detection in GitHub Actions
• See database/README.md for technical details

CTI Extraction

Our content extraction system handles diverse web sources:

• Compression Support: Brotli, Zstandard, and Gzip decompression
• JavaScript Rendering: Falls back to readability-lxml for JS-heavy sites
• Multiple Formats: HTML, PDF, and DOCX file support
• Smart Parsing: Extracts article content from common blog/report structures

MITRE ATT&CK Integration

Accurate technique and tactic validation using official data:

• 691 Techniques: Complete Enterprise ATT&CK framework indexed
• Real-time Validation: Technique IDs validated against MITRE data
• Accurate Tactic Mapping: 99% accuracy vs 85% with keywords
• Confidence Scoring: Multi-tier fallback (MITRE → table → keywords)

Automation Workflows

• Duplicate Detection: Fast similarity analysis using vector embeddings
• Hunt Generation: AI-powered draft creation from CTI sources
• Database Updates: Automatic index maintenance on file changes
• Quality Checks: TTP diversity analysis and content validation

For more details on the technical implementation, see:

• Database Architecture
• Optimization Guide - Performance improvements and cost reduction strategies
• Testing Guide - Local development and testing instructions
• Scripts Documentation
• Workflow Configurations

---

⚙️ Configuration

For maintainers and self-hosted instances, HEARTH can be configured using environment variables.

Environment Variables

Variable	Description	Default	Required
AI_PROVIDER	AI provider to use for hunt generation (claude or openai)	claude	No
ANTHROPIC_API_KEY	API key for Claude (required if using Claude)	-	Yes (for Claude)
OPENAI_API_KEY	API key for OpenAI (required if using OpenAI)	-	Yes (for OpenAI)
CLAUDE_MODEL	Specific Claude model version to use	claude-sonnet-4-5-20250929	No

GitHub Actions Configuration

When running in GitHub Actions, these variables should be set as:

• Repository Secrets: ANTHROPIC_API_KEY, OPENAI_API_KEY, HEARTH_TOKEN
• Repository Variables: AI_PROVIDER, CLAUDE_MODEL (optional)

To update the Claude model version, you can either:

1. Set the CLAUDE_MODEL repository variable in GitHub Settings
2. Update the default value in scripts/generate_from_cti.py

---

🔧 Troubleshooting

CTI Submission Issues

Problem: "Failed to retrieve or process content from the URL"

• Solution: Verify the URL is correct and publicly accessible
• Check if the article requires authentication or is behind a paywall
• Try submitting the content manually instead of via URL

Problem: Content appears garbled or incomplete

• Solution: This has been fixed! Ensure you're using the latest version of HEARTH
• The system now supports Brotli and Zstandard compression
• If issues persist, try the manual submission workflow

Database Issues

Problem: Duplicate detection is slow or failing

• Solution: The database is automatically rebuilt on file changes
• Maintainers can manually rebuild: python scripts/build_hunt_database.py --rebuild
• Check database/README.md for more troubleshooting

Problem: Database appears out of sync

• Solution: Database auto-updates via GitHub Actions on every merge to main
• For local testing, run: python scripts/build_hunt_database.py

General Issues

For other issues or questions:

1. Check existing GitHub Issues
2. Search the database documentation
3. Open a new issue with details

---

License

Distributed under the MIT License. See LICENSE for more information.

---

❤️ Acknowledgements

This project is made possible by the security community and our amazing contributors.

Project Maintainers:

• Lauren Proehl (@jotunvillur)
• Sydney Marrone (@letswastetime)
• John Grageda (@AngryInfoSecGuy)

🔥 **Keep the HEARTH burning!** 🔥