Feature/domainator integration

.github/SECURITY.md

@@ -0,0 +1,223 @@
+# Security Policy
+
+## Purpose
+
+This document defines how security issues affecting the python release of `Hamstring` must be reported, handled, remediated, and disclosed. The goal is to ensure that security-relevant findings are managed confidentially, triaged consistently, and resolved within appropriate operational timelines.
+
+## Supported Versions
+
+Security fixes are provided for the latest released major version.
+
+| Version | Supported |
+| ------- | --------- |
+| `main` / latest release | Yes |
+| Previous minor release | Yes |
+| Older releases | No |
+
+## Confidential Reporting
+
+Security issues must be reported through confidential channels only. Public
+GitHub issues, public pull requests, or other public forums must not be used
+for reporting vulnerabilities.
+
+Approved reporting channels:
+
+- GitHub Private Vulnerability Reporting / Security Advisories
+- Email: `pm300@uni-heidelberg.de`
+
+Reports should include, where available:
+
+- A concise description of the issue
+- Affected component, endpoint, container, or workflow
+- Reproduction steps or validation details
+- Expected impact and potential exploitation scenario
+- Affected version, branch, commit, or image tag
+- Relevant logs, screenshots, request samples, or proof-of-concept material
+- Whether the issue appears actively exploited or time-sensitive
+
+## Confidentiality Requirements
+
+All reported security issues are handled as confidential until review is
+complete and a coordinated disclosure decision has been made.
+
+During this period:
+
+- Issue details must not be disclosed publicly
+- Sensitive technical details must be shared only on a need-to-know basis
+- Access to internal discussion, remediation branches, and advisory drafts
+  should be restricted
+- If there is evidence of active exploitation, internal escalation should occur
+  immediately
+
+## Intake and Triage
+
+Each report is reviewed to determine:
+
+- Whether the issue is reproducible
+- Whether it affects a supported version
+- Whether there is a meaningful confidentiality, integrity, or availability impact
+- Whether exploitation requires special conditions or privileged access
+- Whether the issue represents active exploitation, a misconfiguration, or a
+  theoretical weakness without practical impact
+
+Severity should be classified as `Critical / High / Medium / Low`.
+
+## Response Targets
+
+The following target times apply to supported versions and valid security
+reports. These targets are operational goals, not contractual guarantees.
+
+| Stage | Target |
+| ----- | ------ |
+| Initial acknowledgement | Within 2 business days |
+| Initial triage decision | Within 5 business days |
+| First remediation update | Within 7 calendar days |
+| Ongoing status updates | At least every 7 calendar days |
+| Critical issue remediation plan | Within 7 calendar days |
+| High severity remediation plan | Within 14 calendar days |
+| Medium severity remediation plan | Within 30 calendar days |
+| Low severity remediation plan | Best effort |
+
+If a report indicates active exploitation, credential exposure, remote code
+execution, or broad unauthorized access, the issue should be escalated as an
+incident and handled with priority outside normal backlog processes.
+
+## Remediation Process
+
+When a security issue is confirmed, maintainers should:
+
+- Reproduce and validate the issue
+- Define affected versions and deployment scenarios
+- Prepare a remediation plan proportional to the severity
+- Implement and review the fix
+- Backport the fix to supported versions where feasible
+- Validate the fix before release
+- Prepare customer-facing or operator-facing guidance if configuration or
+  operational action is required
+
+Where immediate remediation is not possible, temporary mitigations should be
+documented and communicated clearly.
+
+## Disclosure and Publication
+
+Confirmed vulnerabilities are disclosed in a coordinated manner after one of
+the following conditions is met:
+
+- A fix has been released
+- A mitigation has been published and the residual risk is understood
+- A disclosure deadline has been reached and leadership approves publication
+
+The default disclosure target is `90 days`, but the actual window may be
+shortened or extended based on:
+
+- Evidence of exploitation
+- Fix availability and deployment risk
+- Customer exposure
+- Dependency or vendor coordination needs
+
+Public disclosures may include:
+
+- A security advisory
+- Release notes
+- Upgrade or mitigation instructions
+- Severity and affected-version information
+
+## Operational Communication
+
+Where a confirmed issue affects deployed environments, communication should be
+proportionate to impact. This may include:
+
+- Internal security or operations escalation
+- Notification to administrators, customers, or service owners
+- Temporary mitigation guidance
+- Required upgrade or rotation steps
+- Post-remediation confirmation and closure
+
+Security communications should avoid unnecessary disclosure of exploit details
+before mitigations are available.
+
+## Scope
+
+The following areas are considered in scope for security handling:
+
+- Authentication and authorization controls
+- Password handling and account lifecycle
+- File upload, parsing, and processing pipelines
+- Secrets handling and environment configuration
+- Data access controls and audit logging
+- Container, service, and network configuration
+- Dependency vulnerabilities with validated product impact
+- Sensitive data exposure, privilege escalation, SSRF, RCE, injection, and
+  broken access control
+
+## Out of Scope
+
+The following are generally not treated as security vulnerabilities unless
+clear and demonstrated security impact exists:
+
+- Cosmetic misconfigurations without exploitability
+- Missing hardening headers without a practical attack path
+- Issues affecting unsupported or end-of-life releases only
+- Hypothetical findings without reproducible impact
+- Third-party platform issues outside the control of this project
+- Reports based only on scanner output without technical validation
+
+## Safe Handling Expectations
+
+Anyone validating a suspected issue is expected to act in a controlled and
+minimal manner.
+
+Expected behavior:
+
+- Limit activity to what is necessary to confirm the issue
+- Avoid unauthorized access to non-public or third-party data
+- Avoid disruption of production systems
+- Avoid persistence, data modification, or data destruction
+- Stop testing and report promptly once the issue is confirmed
+
+This policy does not authorize:
+
+- Access to data belonging to other users or organizations
+- Service disruption or denial-of-service activity
+- Data exfiltration or retention of sensitive information
+- Any activity that violates applicable law or contractual obligations
+
+## Security Updates
+
+Security fixes may be distributed through one or more of the following:
+
+- Normal release process
+- Out-of-band patch release
+- Security advisory
+- Operational mitigation notice
+
+Where appropriate, the published update should include:
+
+- Affected versions
+- Fixed versions
+- Severity
+- Upgrade path
+- Required operational actions
+
+## Escalation
+
+If no acknowledgement is received within the response target above, the report
+should be resent to:
+
+- `pm300@uni-heidelberg.de`
+
+Urgent reports involving active exploitation or high-confidence compromise
+should use the subject line:
+
+`[URGENT SECURITY REPORT]`
+
+## Policy Maintenance
+
+This policy should be reviewed whenever:
+
+- Reporting channels change
+- Supported versions change
+- Incident response expectations change
+- Disclosure commitments change
+
+Last reviewed: `27-03-2026`

.readthedocs.yml

@@ -1,9 +1,9 @@
 version: "2"
 
 build:
-  os: "ubuntu-22.04"
+  os: "ubuntu-24.04"
   tools:
-    python: "3.10"
+    python: "3.11"
   jobs:
     pre_build:
       - sphinx-apidoc -T -M -o docs/api src/ "*/tests"
@@ -20,4 +20,4 @@ python:
     - requirements: requirements/requirements.logserver.txt
 
 sphinx:
-  configuration: docs/conf.py
+  configuration: docs/conf.py

README.md

@@ -16,9 +16,7 @@
 <!-- PROJECT LOGO -->
 <br />
 <div align="center">
-  <a href="https://github.com/hamstring-ndr/hamstring">
-    <img src="" alt="Logo">
-  </a>
+    <img src="./assets/hamstring.svg" alt="Logo">
 
 <h3 align="center">HAMSTRING</h3>
 
@@ -34,8 +32,6 @@
   </p>
 </div>
 
-> [!CAUTION]
-> This project has been moved to https://github.com/Hamstring-NDR/hamstring. Future development, issues, and releases will be maintained there.
 
 <table>
 <tr>
@@ -56,7 +52,7 @@
 
 ## About the Project
 
-![Pipeline overview](https://raw.githubusercontent.com/hamstring-ndr/hamstring/main/docs/media/hamstring_overview_detailed.drawio.png?raw=true)
+![Pipeline overview](./assets/heidgaf_architecture.svg)
 
 ## Getting Started
 
@@ -68,27 +64,11 @@ HOST_IP=127.0.0.1 docker compose -f docker/docker-compose.yml --profile prod up
   <img src="https://raw.githubusercontent.com/hamstring-ndr/hamstring/main/assets/terminal_example.gif?raw=true" alt="Terminal example"/>
 </p>
 
-#### Use the dev profile for testing out changes in docke containers:
+#### Use the dev profile for testing out changes in docker containers:
 ```sh
 HOST_IP=127.0.0.1 docker compose -f docker/docker-compose.yml --profile dev up
 ```
 
-
-#### Or run the modules locally on your machine:
-```sh
-python -m venv .venv
-source .venv/bin/activate
-
-sh install_requirements.sh
-```
-Alternatively, you can use `pip install` and enter all needed requirements individually with `-r requirements.*.txt`.
-
-Now, you can start each stage, e.g. the inspector:
-
-```sh
-python src/inspector/inspector.py
-```
-
 <p align="right">(<a href="#readme-top">back to top</a>)</p>
 
 
@@ -126,6 +106,11 @@ For more in-depth information on your options, have a look at our
 [official documentation](https://hamstring.readthedocs.io/en/latest/usage.html), where we provide tables explaining all
 values in detail.
 
+
+### Testing Your Own Data
+
+If you want to ingest data to the pipeline, you can do so via the zeek container. Either select the interface in the `config.yaml` zeek should be listening on and set `static_analysis: false` or provide PCAPs to Zeek by adding them in the `data/test_pcaps` directory, which is mounted per default for Zeek to ingest static data. 
+
 ### Monitoring
 To monitor the system and observe its real-time behavior, multiple Grafana dashboards have been set up.
 
@@ -209,22 +194,17 @@ Have a look at the following pictures showing examples of how these dashboards m
 
 To train and test our and possibly your own models, we currently rely on the following datasets:
 
-- [CICBellDNS2021](https://www.unb.ca/cic/datasets/dns-2021.html)
 - [DGTA Benchmark](https://data.mendeley.com/datasets/2wzf9bz7xr/1)
 - [DNS Tunneling Queries for Binary Classification](https://data.mendeley.com/datasets/mzn9hvdcxg/1)
 - [UMUDGA - University of Murcia Domain Generation Algorithm Dataset](https://data.mendeley.com/datasets/y8ph45msv8/1)
 - [DGArchive](https://dgarchive.caad.fkie.fraunhofer.de/)
+- [DNS Exfiltration](https://data.mendeley.com/datasets/c4n7fckkz3/3)
 
 We compute all features separately and only rely on the `domain` and `class` for binary classification.
 
 ### Inserting Data for Testing
 
-For testing purposes, we provide multiple scripts in the `scripts` directory. Use `real_logs.dev.py` to send data from
-the datasets into the pipeline. After downloading the dataset and storing it under `<project-root>/data`, run
-```sh
-python scripts/real_logs.dev.py
-```
-to start continuously inserting dataset traffic.
+For testing purposes, you can ingest PCAPs or tap on network interfaces using the zeek-based sensor in its `1.0.0` release. For more information on it, please refer to [the documentation](https://github.com/Hamstring-NDR/hamstring-zeek).
 
 ### Training Your Own Models
 
@@ -270,7 +250,7 @@ The results will be saved per default to `./results`, if not configured otherwis
 #### Model Tests
 
 ```sh
-> python src/train/train.py test  --dataset <dataset_type> --dataset_path <path/to/your/datasets> --model <model_name> --model_path <path_to_model_version>
+> python src/train/train.py test  --dataset <dataset_type> --dataset_path <path/to/your/datasets> --model <model_name> --model_output_path <path_to_model_version>
 ```
 
 #### Model Explain