diff --git a/README.md b/README.md index cd11023..2453840 100644 --- a/README.md +++ b/README.md @@ -17,6 +17,8 @@ The Double Extortion connector ingests ransomware and data-leak announcements pu - Supports querying different Double Extortion Platform datasets via `DEP_DSET`. - Maintains connector state with a configurable overlap window to capture late DEP updates. - Uses stable identifiers (based on DEP `hashid`) for both reports and incidents so refreshed DEP records update existing objects. +- Filters low-quality actor values such as `unknown`, `anonymous`, or `ransomware group` before creating intrusion sets. +- Normalizes DEP values before STIX generation, including victim domains, sector/actor/country strings, and URL-decoded announcement descriptions. image (1) image (2) @@ -35,7 +37,7 @@ The Double Extortion connector ingests ransomware and data-leak announcements pu ## Configuration -All configuration values can be supplied via the `config.yml` file or through environment variables. Environment variables take precedence and follow the naming convention described below. +All configuration values can be supplied via the `config.yml` file or through environment variables. Environment variables take precedence over YAML values. The connector loads `config.yml` from the project root by default, or from `OPENCTI_CONFIG_FILE` when that variable is set. ### Required values @@ -46,7 +48,7 @@ All configuration values can be supplied via the `config.yml` file or through en | `dep.username` | `DEP_USERNAME` | Username for the Double Extortion Platform portal. | | `dep.password` | `DEP_PASSWORD` | Password for the portal. | | `dep.api_key` | `DEP_API_KEY` | API key issued by the Double Extortion Platform. | -| `dep.client_id` | `DEP_CLIENT_ID` | AWS Cognito App Client ID. | +| `dep.client_id` | `DEP_CLIENT_ID` | AWS Cognito App Client ID. Required at startup. | ### Optional values @@ -58,7 +60,7 @@ All configuration values can be supplied via the `config.yml` file or through en | `dep.api_endpoint` | `DEP_API_ENDPOINT` | `https://api.eu-ep1.doubleextortion.com/v1/dbtr/privlist` | REST endpoint for announcements. | | `dep.lookback_days` | `DEP_LOOKBACK_DAYS` | `7` | Days to look back on the first run. | | `dep.overlap_hours` | `DEP_OVERLAP_HOURS` | `72` | Hours to overlap from the previous `last_run` when fetching, to catch late updates. | -| `dep.extended_results` | `DEP_EXTENDED_RESULTS` | `true` | Request extended leak information. | +| `dep.extended_results` | `DEP_EXTENDED_RESULTS` | `true` | Request extended leak information by adding `extended=true` to DEP API requests. | | `dep.dset` | `DEP_DSET` | `ext` | Dataset to query (for example `ext`, `sanctions`). | | `dep.enable_site_indicator` | `DEP_ENABLE_SITE_INDICATOR` | `true` | Create a domain indicator per victim. | | `dep.enable_hash_indicator` | `DEP_ENABLE_HASH_INDICATOR` | `true` | Create a hash indicator when a hash is provided. | @@ -68,6 +70,17 @@ All configuration values can be supplied via the `config.yml` file or through en | `dep.primary_object` | `DEP_PRIMARY_OBJECT` | `report` | Primary object: `report` wraps all objects in a STIX Report container; `incident` creates a standalone Incident object. | | `dep.create_country_locations` | `DEP_CREATE_COUNTRY_LOCATIONS` | `true` | Create country locations and link victim identities with `located-at`. | +### DEP request behavior + +Each DEP fetch sends: + +- `ts` +- `te` +- `dset` +- `full=true` + +The connector adds `extended=true` only when `DEP_EXTENDED_RESULTS=true`. + ## Why `IntrusionSet` for DEP actor values DEP `actor` values are modeled as STIX `IntrusionSet` objects instead of `ThreatActor` by default. @@ -96,6 +109,7 @@ docker run --rm \ -e DEP_USERNAME=username \ -e DEP_PASSWORD=password \ -e DEP_API_KEY=apikey \ + -e DEP_CLIENT_ID=aws-cognito-client-id \ opencti-connector-dep ``` @@ -109,6 +123,7 @@ docker run --rm \ - In `incident` mode the announcement is modeled as a STIX `Incident` with explicit `targets`, `attributed-to`, and `indicates` relationships. - Sector names are normalized before sector-identity generation to reduce duplicates caused by inconsistent casing or whitespace in DEP data. - The API occasionally URL-encodes announcement descriptions. The connector automatically decodes the description before sending it to OpenCTI. +- DEP `annLink` values are repaired for a known scrape bug (`https//...` or `http//...`) before they are used as external references. - DEP actor and country values can be materialized as entities using `DEP_CREATE_INTRUSION_SETS` and `DEP_CREATE_COUNTRY_LOCATIONS`. - DEP actor and country values are also stored in the primary object custom properties (`dep_actor`, `dep_country`) for source traceability. - Generated indicators are also linked to the victim with `related-to` so those indicator nodes are connected in the Knowledge Graph. diff --git a/config.yml.sample b/config.yml.sample index 7fb1478..ad381f0 100644 --- a/config.yml.sample +++ b/config.yml.sample @@ -14,18 +14,19 @@ connector: # All values can also be provided with environment variables prefixed with # DEP_ (for example DEP_USERNAME) # and the generic connector values with CONNECTOR_ and OPENCTI_ prefixes. +# Environment variables override YAML values. dep: username: your_dep_username password: your_dep_password api_key: your_dep_api_key - client_id: "" + client_id: "" # Required at startup. confidence: 70 login_endpoint: https://cognito-idp.eu-west-1.amazonaws.com/ api_endpoint: https://api.eu-ep1.doubleextortion.com/v1/dbtr/privlist dset: ext lookback_days: 7 overlap_hours: 72 - extended_results: true + extended_results: true # Adds extended=true to DEP API requests. enable_site_indicator: true enable_hash_indicator: true skip_empty_victim: true