diff --git a/docs/migration/3.0_breaking_changes.md b/docs/migration/3.0_breaking_changes.md index a7fae3a761..3aede01c46 100644 --- a/docs/migration/3.0_breaking_changes.md +++ b/docs/migration/3.0_breaking_changes.md @@ -164,8 +164,9 @@ truth. `output.webhook` in 3.0. Old scans that ran `bbot -om http ...` were emitting events to a webhook; - in 3.0 the equivalent is `bbot -om webhook ...`. The new `bbot -m http ...` - is a scan module that probes URLs, which is a completely different thing. + in 3.0 the equivalent is `bbot -om webhook ...`. By contrast, `bbot -m http + ...` enables the scan module, which probes URLs and is unrelated to the + former output module. ### New modules worth knowing about @@ -286,12 +287,6 @@ rest of the scan and the scan continues without it. This is separate from the ASN-as-target expansion above, which must abort because there would be nothing to scan. -### SCAN event - -The `SCAN` event now includes a `network` key containing the scanner's hostname, -primary outbound IP, network interfaces (IPv4/IPv6 with netmasks), and default -routes. Useful for correlating scan activity back to a specific agent. - ### JSON output: before and after If you have downstream tooling that parses BBOT's NDJSON output, the schema @@ -513,7 +508,8 @@ architecture was deleted. Rust client. The `self.helpers.dns.resolver` `dnspython` resolver is gone. - **HTTP**: `bbot/core/helpers/web/client.py` and `web/engine.py` were removed. `WebHelper` no longer inherits from `EngineClient`. All HTTP goes - through the shared `self.helpers.blasthttp` client. The httpx-based + through the shared [blasthttp](https://github.com/blacklanternsecurity/blasthttp) + client (`self.helpers.blasthttp`). The httpx-based `request_batch` / `request_custom_batch` / `curl` methods were replaced by `request()`, `request_batch_stream(urls, threads=10, **kwargs)`, and `download()`. @@ -596,10 +592,11 @@ If your downstream tooling relied on seeing `HTTP_RESPONSE` or ## Dependencies and tooling - **Build system**: migrated from Poetry to [uv](https://docs.astral.sh/uv/) - + hatchling. `pyproject.toml` is now PEP-621. `poetry.lock` is replaced by - `uv.lock`. Dev install: `uv sync --group dev`. The - `poetry-dynamic-versioning` integration is gone; version is now plain - `3.0.0`. + with a hatchling build backend. `pyproject.toml` now follows PEP 621, and + `uv.lock` replaces `poetry.lock`; install dev dependencies with + `uv sync --group dev`. Versioning is static (`version = "3.0.0"` in + `pyproject.toml`) rather than derived from git tags by + `poetry-dynamic-versioning`. - **License**: changed from `GPL-3.0` to `AGPL-3.0`. - **Python**: minimum bumped from `3.9` to `3.10`. Upper bound is now `<3.15`. diff --git a/docs/scanning/output.md b/docs/scanning/output.md index 8715698ed3..f2c2646618 100644 --- a/docs/scanning/output.md +++ b/docs/scanning/output.md @@ -413,22 +413,79 @@ The `web_report` output module generates a markdown report of web assets, includ The `nmap_xml` output module exports open ports, DNS names, IP addresses, and protocols in Nmap XML format for compatibility with tools that consume Nmap output. -### Message Queues +### Kafka -BBOT supports streaming events as JSON to several message queue systems: +The `kafka` output module publishes events as JSON to a Kafka topic. -| Module | Description | Key Config | -|--------|-------------|------------| -| `kafka` | Publish to a Kafka topic | `bootstrap_servers`, `topic` | -| `rabbitmq` | Publish to a RabbitMQ queue | `url`, `queue` | -| `nats` | Publish to a NATS subject | `servers`, `subject` | -| `zeromq` | Publish to a ZeroMQ PUB socket | `zmq_address` | -| `websocket` | Stream to a WebSocket endpoint | `url`, `token` | +```yaml title="kafka_preset.yml" +output_modules: + - kafka -These can be enabled like any other output module: +config: + modules: + kafka: + bootstrap_servers: localhost:9092 + topic: bbot_events +``` -```bash -bbot -t evilcorp.com -om kafka -c modules.kafka.bootstrap_servers=localhost:9092 modules.kafka.topic=bbot_events +### RabbitMQ + +The `rabbitmq` output module publishes events as JSON to a RabbitMQ queue. + +```yaml title="rabbitmq_preset.yml" +output_modules: + - rabbitmq + +config: + modules: + rabbitmq: + url: amqp://guest:guest@localhost/ + queue: bbot_events +``` + +### NATS + +The `nats` output module publishes events as JSON to a NATS subject. + +```yaml title="nats_preset.yml" +output_modules: + - nats + +config: + modules: + nats: + servers: + - nats://localhost:4222 + subject: bbot_events +``` + +### ZeroMQ + +The `zeromq` output module publishes events as JSON to a ZeroMQ PUB socket. + +```yaml title="zeromq_preset.yml" +output_modules: + - zeromq + +config: + modules: + zeromq: + zmq_address: tcp://localhost:5555 +``` + +### WebSocket + +The `websocket` output module streams events as JSON to a WebSocket endpoint. Set `token` to send an `Authorization: Bearer` header. + +```yaml title="websocket_preset.yml" +output_modules: + - websocket + +config: + modules: + websocket: + url: ws://localhost:8080 + token: my-auth-token ``` ### MongoDB