|
4 | 4 |
|
5 | 5 | ### Added Elasticsearch Sink. |
6 | 6 | Supports batch writing parsed records to the Elasticsearch `/_bulk` API over HTTP/HTTPS, with timeout, retry, and password authentication support. |
| 7 | +Example: |
| 8 | +```toml |
| 9 | +[connectors.params] |
| 10 | +protocol = "http" # Transport protocol; switch to https if needed |
| 11 | +host = "localhost" # Elasticsearch server host |
| 12 | +port = "9200" # Elasticsearch service port |
| 13 | +index = "wp_nginx" # Target index name |
| 14 | +username = "elastic" # Username |
| 15 | +password = "" # Password |
| 16 | +batch_size = 1_0000 # Number of records per batch |
| 17 | +``` |
7 | 18 | - Usage doc: [Elasticsearch Sink](../10-user/05-connectors/02-sinks/20-elasticsearch_sink.md) |
8 | 19 | - Example index: [wp-examples/extensions/elasticsearch/README.md](https://github.com/wp-labs/wp-examples/blob/main/extensions/elasticsearch/README.md) |
9 | 20 |
|
10 | 21 | ### Added PostgreSQL Sink. |
11 | | -Supports writing Record data to PostgreSQL tables in batches based on the configured `columns`, which fits relational storage and idempotent retry scenarios. |
12 | 22 | - Usage doc: [PostgreSQL Sink](../10-user/05-connectors/02-sinks/23-postgresql_sink.md) |
13 | 23 | - Example index: [wp-examples/extensions/tcp_postgres/README.md](https://github.com/wp-labs/wp-examples/blob/main/extensions/tcp_postgres/README.md) |
14 | 24 |
|
15 | 25 | ### Added Doris Sink. |
16 | 26 | Supports batch loading data into Apache Doris through the Stream Load API, with custom Stream Load headers, timeout, retry, and password authentication support. |
| 27 | +Example: |
| 28 | +```toml |
| 29 | +[connectors.params] |
| 30 | +endpoint = "http://localhost:8040" # Use the BE HTTP port |
| 31 | +user = "root" # Username |
| 32 | +password = "" # Password |
| 33 | +database = "" # Database name |
| 34 | +table = "" # Table name |
| 35 | +batch_size = 1_0000 # Request batch size |
| 36 | + |
| 37 | +# Optional: custom Stream Load parameters |
| 38 | +[connectors.params.headers] |
| 39 | +strip_outer_array = "false" |
| 40 | +max_filter_ratio = "0.1" |
| 41 | +strict_mode = "false" |
| 42 | +``` |
17 | 43 | - Usage doc: [Doris Sink](../10-user/05-connectors/02-sinks/17-doris.md) |
18 | 44 | - Example index: [wp-examples/extensions/doris/README.md](https://github.com/wp-labs/wp-examples/blob/main/extensions/doris/README.md) |
19 | 45 |
|
20 | 46 | ### Added ClickHouse Sink. |
21 | 47 | Supports writing to ClickHouse with the RowBinary format over HTTP, with timeout, retry, and password authentication support. |
| 48 | +Example: |
| 49 | +```toml |
| 50 | +[connectors.params] |
| 51 | +endpoint = "http://localhost:8123" |
| 52 | +username = "" # Username |
| 53 | +password = "" # Password |
| 54 | +database = "" # Database name |
| 55 | +table = "" # Table name |
| 56 | +batch_size = 1_0000 # Request batch size |
| 57 | +``` |
22 | 58 | - Usage doc: [ClickHouse Sink](../10-user/05-connectors/02-sinks/21-clickhouse.md) |
23 | 59 | - Example index: [wp-examples/extensions/clickhouse](https://github.com/wp-labs/wp-examples/tree/main/extensions/clickhouse) |
24 | 60 |
|
25 | 61 | ### Added HTTP Sink. |
26 | 62 | Supports sending parsed records to HTTP/HTTPS endpoints for webhooks, API integrations, and custom pipelines, with `json`, `ndjson`, `csv`, `kv`, `raw`, and `proto-text` output formats, plus Basic Auth, custom headers, gzip compression, timeout, and retry support. |
| 63 | +Example: |
| 64 | +```toml |
| 65 | +[sink_group.sinks.params] |
| 66 | +endpoint = "http://localhost:8080/webhook" # Target HTTP endpoint |
| 67 | +method = "POST" # HTTP method |
| 68 | +username = "" # Use HTTP Basic authentication |
| 69 | +password = "" # Use HTTP Basic authentication |
| 70 | +fmt = "json" # Output format: `json`, `ndjson`, `csv`, `kv`, `raw`, `proto-text` (default `json`) |
| 71 | +batch_size = 1_0000 # Batch send size |
| 72 | +compression = "none" # Whether to enable gzip compression |
| 73 | + |
| 74 | +# Optional: custom header parameters |
| 75 | +[connectors.params.headers] |
| 76 | +# content-type = "application/json" |
| 77 | +``` |
27 | 78 | - Usage doc: [HTTP Sink](../10-user/05-connectors/02-sinks/22-http.md) |
28 | 79 | - Example index: [wp-examples/extensions/http/README.md](https://github.com/wp-labs/wp-examples/blob/main/extensions/http/README.md) |
29 | 80 |
|
|
0 commit comments