diff --git a/docs/cn/guides/20-self-hosted/00-editions/community.md b/docs/cn/guides/20-self-hosted/00-editions/community.md index 8bccd97c4c..54a778f672 100644 --- a/docs/cn/guides/20-self-hosted/00-editions/community.md +++ b/docs/cn/guides/20-self-hosted/00-editions/community.md @@ -27,5 +27,5 @@ Databend 社区版（Community）是 Databend 的开源版本，基于 **Apache ## 快速入门 - **[快速入门指南](/guides/self-hosted/quickstart)**：5 分钟快速上手 -- **[下载与安装](/guides/self-hosted/deployment/download)**：私有化部署选项 +- **[Docker 部署](/guides/self-hosted/deployment/deploying-local)**：私有化部署选项 - **[GitHub 仓库](https://github.com/databendlabs/databend)**：源代码与社区 diff --git a/docs/cn/guides/20-self-hosted/01-quickstart/index.md b/docs/cn/guides/20-self-hosted/01-quickstart/index.md index 3257cbd554..948b816c68 100644 --- a/docs/cn/guides/20-self-hosted/01-quickstart/index.md +++ b/docs/cn/guides/20-self-hosted/01-quickstart/index.md @@ -2,13 +2,11 @@ title: 快速入门 --- -Databend 快速入门：5 分钟体验 Databend -本指南将帮助您快速设置 Databend，连接到它，并执行基本的数据导入。 +5 分钟内在本地启动 Databend。 -## 1. 使用 Docker 启动 Databend -运行以下命令在容器中启动 Databend： +## 1. 启动 Databend -``` +```shell docker run -d \ --name databend \ --network host \ @@ -19,124 +17,48 @@ docker run -d \ --restart unless-stopped \ datafuselabs/databend ``` -检查 Databend 是否成功运行： +:::tip 国内用户 +如果拉取镜像较慢，可替换为国内镜像： +```shell +registry.databend.cn/public/databend:latest ``` -docker logs -f databend -``` -等待直到您看到日志显示 Databend 和 MinIO 已准备就绪。 +::: -## 2. 连接到 Databend -安装 bendsql (Databend CLI)： +等待 Databend 就绪： -``` -curl -fsSL https://repo.databend.com/install/bendsql.sh | bash -echo "export PATH=$PATH:~/.bendsql/bin" >>~/.bash_profile -source ~/.bash_profile +```shell +docker logs -f databend ``` -连接到 Databend： -``` -bendsql -udatabend -pdatabend -``` +看到 `Databend Query started` 即表示启动成功。 -## 3. 执行基本数据导入 -### 步骤 1：创建外部存储桶 (myupload) -安装 mc (MinIO 客户端) 并创建存储桶： +## 2. 连接 -``` -wget https://dl.min.io/client/mc/release/linux-amd64/mc -sudo cp mc /usr/local/bin/ && sudo chmod +x /usr/local/bin/mc -mc alias set myminio http://localhost:9000 minioadmin minioadmin -mc mb myminio/myupload -mc ls myminio -``` -预期输出： -``` -[0001-01-01 08:05:43 LMT] 0B databend/ -[2025-04-12 08:43:59 CST] 0B myupload/ -``` +安装 BendSQL： -### 步骤 2：生成 CSV 文件并上传到 myupload -``` -echo -e "id,name,age,city\n1,John,32,New York\n2,Emma,28,London\n3,Liam,35,Paris\n4,Olivia,40,Berlin\n5,Noah,29,Tokyo" > data.csv -mc cp data.csv myminio/myupload/ -mc ls myminio/myupload/ -``` -### 步骤 3：创建外部 Stage 并检查数据 -在 bendsql 中运行： -``` -CREATE STAGE mystage 's3://myupload' -CONNECTION=( - endpoint_url='http://127.0.0.1:9000', - access_key_id='minioadmin', - secret_access_key='minioadmin', - region='us-east-1' -); -``` -显示外部 Stage @mystage 中的文件： -``` -LIST @mystage; -``` -| name | size | md5 | last_modified | creator | -|----------|--------|-------------------|-----------------------|-------------| -| String | UInt64 | Nullable(String) | String | Nullable(String) | -| data.csv | 104 | "a27fa15258911f534fb795a8c64e05d4" | 2025-04-12 00:51:11.015 +0000 | NULL | - -预览 CSV 数据： -``` -SELECT $1, $2, $3, $4 FROM @mystage/data.csv (FILE_FORMAT=>'CSV') LIMIT 10; -``` -| \$1 | \$2 | \$3 | \$4 | -|-------------------|-------------------|-------------------|-------------------| -| Nullable(String) | Nullable(String) | Nullable(String) | Nullable(String) | -| id | name | age | city | -| 1 | John | 32 | New York | -| 2 | Emma | 28 | London | -| 3 | Liam | 35 | Paris | -| 4 | Olivia | 40 | Berlin | -| 5 | Noah | 29 | Tokyo | - - -### 步骤 4：创建表并加载数据 +```shell +curl -fsSL https://repo.databend.com/install/bendsql.sh | bash +echo "export PATH=$PATH:~/.bendsql/bin" >> ~/.bash_profile +source ~/.bash_profile ``` -CREATE DATABASE wubx; -USE wubx; -CREATE TABLE t_person ( - id INT, - name VARCHAR, - age INT UNSIGNED, - city VARCHAR -); - -COPY INTO t_person FROM @mystage PATTERN='.*[.]csv' FILE_FORMAT=(TYPE=CSV, SKIP_HEADER=1); +连接到 Databend： +```shell +bendsql -u databend -p databend ``` -| File | Rows_loaded | Errors_seen | First_error | First_error_line | -|-----------|-------------|-------------|------------------|------------------| -| String | Int32 | Int32 | Nullable(String) | Nullable(Int32) | -| data.csv | 5 | 0 | NULL | NULL | +## 3. 验证 -### 步骤 5：查询数据 -``` -SELECT * FROM t_person; +```sql +CREATE TABLE t (id INT, name VARCHAR); +INSERT INTO t VALUES (1, 'Databend'); +SELECT * FROM t; ``` -| id | name | age | city | -|----------|----------|----------|----------| -| Nullable(Int32) | Nullable(String) | Nullable(UInt32) | Nullable(String) | -| 1 | John | 32 | New York | -| 2 | Emma | 28 | London | -| 3 | Liam | 35 | Paris | -| 4 | Olivia | 40 | Berlin | -| 5 | Noah | 29 | Tokyo | -🚀 现在您已经成功将数据导入到 Databend 中了！ +完成！如需了解分组件部署的方式，请参考[在 Docker 上部署](../02-deployment/01-deploying-local.md)。 ## 替代方案：Databend Cloud -如果设置本地环境比较麻烦，您可以尝试 [Databend Cloud](https://www.databend.com) 获得完全托管的体验。 -> 💬 **社区支持** -> 有疑问？联系我们的团队： -> 💬 [Slack 讨论](https://link.databend.com/join-slack) \ No newline at end of file +如果不想搭建本地环境，可以直接使用 [Databend Cloud](https://www.databend.com) 的全托管服务。 diff --git a/docs/cn/guides/20-self-hosted/02-deployment/01-deploying-local.md b/docs/cn/guides/20-self-hosted/02-deployment/01-deploying-local.md new file mode 100644 index 0000000000..44ff028bf3 --- /dev/null +++ b/docs/cn/guides/20-self-hosted/02-deployment/01-deploying-local.md @@ -0,0 +1,195 @@ +--- +title: 在 Docker 上部署 +--- + +本指南使用 Docker Compose 分别启动 **databend-meta**、**databend-query** 和 **MinIO** 三个容器，帮助你理解 Databend 各组件的关系。 + +:::note 仅用于非生产环境 +本方案仅适用于开发和本地测试，不适合生产环境或性能测试。 +::: + +### 开始之前 + +- 已安装 [Docker](https://www.docker.com/)（推荐 v26 及以上版本）。 +- 已安装 [BendSQL](https://docs.databend.com/guides/connect/sql-clients/bendsql/#installing-bendsql)。 + +### 第一步：创建 docker-compose.yml + +创建文件 `docker-compose.yml`，内容如下： + +:::tip 国内用户 +如果拉取 Docker Hub 镜像较慢，可将以下镜像替换为国内镜像源： +- `datafuselabs/databend-meta:latest` → `registry.databend.cn/public/databend-meta:latest` +- `datafuselabs/databend-query:latest` → `registry.databend.cn/public/databend-query:latest` +::: + +```yaml +services: + minio: + image: minio/minio:latest + network_mode: "host" + environment: + MINIO_ACCESS_KEY: minioadmin + MINIO_SECRET_KEY: minioadmin + volumes: + - minio-data:/data + entrypoint: | + sh -c " + minio server --address :9000 /data & + until mc alias set myminio http://localhost:9000 minioadmin minioadmin; do + echo 'Waiting for MinIO...'; + sleep 1; + done; + if ! mc ls myminio/databend > /dev/null 2>&1; then + mc mb myminio/databend; + fi; + wait; + " + + databend-meta: + image: datafuselabs/databend-meta:latest + network_mode: "host" + depends_on: + - minio + volumes: + - databend-meta-data:/data/ + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:28101/v1/health"] + interval: 5s + timeout: 3s + retries: 10 + entrypoint: sh -c "/databend-meta \ + --log-file-level='warn' \ + --log-file-format='text' \ + --log-file-dir='/data/logs/' \ + --log-file-limit=24 \ + --admin-api-address='0.0.0.0:28101' \ + --grpc-api-address='0.0.0.0:9191' \ + --grpc-api-advertise-host='0.0.0.0' \ + --raft-listen-host='0.0.0.0' \ + --raft-api-port='28103' \ + --raft-dir='/data/metadata' \ + --id=1 \ + --single" + + databend-query: + image: datafuselabs/databend-query:latest + network_mode: "host" + depends_on: + databend-meta: + condition: service_healthy + volumes: + - query-logs:/var/log/databend/ + - query-config:/etc/databend/ + environment: + RUST_BACKTRACE: 1 + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:7070/metrics"] + interval: 10s + timeout: 5s + retries: 10 + entrypoint: | + sh -c " + cat <<'EOF' > /etc/databend/databend-query.toml + [query] + max_active_sessions = 256 + shutdown_wait_timeout_ms = 5000 + flight_api_address = '0.0.0.0:9090' + metric_api_address = '0.0.0.0:7070' + tenant_id = 'default' + cluster_id = 'c01' + + [[query.users]] + name = 'databend' + auth_type = 'double_sha1_password' + # password: databend + auth_string = '3081f32caef285c232d066033c89a96d542d09d7' + + [log] + [log.file] + level = 'WARN' + format = 'text' + dir = '/var/log/databend' + + [meta] + endpoints = ['0.0.0.0:9191'] + username = 'root' + password = 'root' + client_timeout_in_second = 30 + auto_sync_interval = 30 + + [storage] + type = 's3' + [storage.s3] + endpoint_url = 'http://127.0.0.1:9000' + region = 'us-east-1' + access_key_id = 'minioadmin' + secret_access_key = 'minioadmin' + bucket = 'databend' + EOF + exec /usr/bin/databend-query --config-file=/etc/databend/databend-query.toml + " + +volumes: + minio-data: + databend-meta-data: + query-logs: + query-config: +``` + +### 第二步：启动 Databend + +```shell +docker compose up -d +``` + +检查三个容器是否正常运行： + +```shell +docker compose ps +``` + +查看 query 日志： + +```shell +docker compose logs -f databend-query +``` + +看到 `Databend Query started` 即表示启动成功。 + +### 第三步：连接到 Databend + +```shell +bendsql -u databend -p databend +``` + +执行一条简单查询验证： + +```sql +SELECT NOW(); +``` + +### 停止 Databend + +```shell +docker compose stop +``` + +彻底删除容器和数据卷： + +```shell +docker compose down -v +``` + +### 进阶配置 + +如需自定义配置，可以在外部准备好配置文件再 mount 到容器中，参考官方配置模板： + +- databend-meta 配置：https://github.com/databendlabs/databend/blob/main/scripts/ci/deploy/config/databend-meta-node-1.toml +- databend-query 配置：https://github.com/databendlabs/databend/blob/main/scripts/ci/deploy/config/databend-query-node-1.toml + +### 下一步 + +- [使用 COPY INTO 加载数据](/guides/load-data/load/s3) +- [通过 MySQL 客户端或 JDBC 连接](/guides/connect/) +- 生产环境部署请参考 [在 Kubernetes 上部署](/guides/self-hosted/deployment/production/deploying-databend-on-kubernetes) diff --git a/docs/cn/guides/20-self-hosted/02-deployment/01-download.md b/docs/cn/guides/20-self-hosted/02-deployment/01-download.md deleted file mode 100644 index 8e8243b31e..0000000000 --- a/docs/cn/guides/20-self-hosted/02-deployment/01-download.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -title: 下载 Databend -sidebar_label: 下载 Databend ---- - -Databend 为您提供以下选项来下载安装包： - -- [手动下载](#manual-download)：您可以直接从 Databend 网站下载适用于您平台的安装包。 -- [APT 包管理器](#apt-package-manager)：您可以使用 APT 包管理器在 Ubuntu 或 Debian 上下载并安装 Databend。 -- [Docker](#docker)：您可以使用 Docker 在容器化环境中下载并运行 Databend。 - -## 手动下载 - -Databend 的主要分发包是 `.tar.gz` 压缩包，其中包含单个可执行文件，您可以从 [下载](https://www.databend.com/download) 页面下载这些文件，并将它们解压到系统上的任何位置。 - -:::note -**Linux Generic (Arm, 64-bit)** 适用于使用 musl 作为标准 C 库的 Linux 发行版；**Linux Generic (x86, 64-bit)** 适用于使用 GNU C 且 GLIBC 最低版本为 2.29 的 Linux 发行版。 -::: - -## APT 包管理器 - -Databend 为 Debian 和 Ubuntu 系统提供软件包存储库，允许您使用 apt install 命令或任何其他 APT 前端安装 Databend。 - -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - - - - -```shell -sudo curl -L -o /usr/share/keyrings/databend-keyring.gpg https://repo.databend.com/deb/databend.gpg -sudo curl -L -o /etc/apt/sources.list.d/databend.list https://repo.databend.com/deb/databend.list - -sudo apt update - -sudo apt install databend -``` - - - - - -:::note -可用平台： - -- Ubuntu Jammy (22.04) 及更高版本 -- Debian Bookworm(12) 及更高版本 - -::: - -```shell -sudo curl -L -o /etc/apt/sources.list.d/databend.sources https://repo.databend.com/deb/databend.sources - -sudo apt update - -sudo apt install databend -``` - - - - -:::tip -要在安装后启动 Databend，请运行以下命令： - -```shell -sudo systemctl start databend-meta -sudo systemctl start databend-query -``` - -::: - -## Docker - -Databend 在 Docker Hub 上提供以下类型的安装镜像： - -- [Databend All-in-One Docker Image](https://hub.docker.com/r/datafuselabs/databend)：专为本地测试、CI 等构建。 -- 分离镜像：专为生产环境、Kubernetes 和 [Helm Charts](https://github.com/databendlabs/helm-charts) 构建。 - - [databend-meta](https://hub.docker.com/r/datafuselabs/databend-meta) - - [databend-query](https://hub.docker.com/r/datafuselabs/databend-query) - -单击上面的链接获取其详细说明。 \ No newline at end of file diff --git a/docs/cn/guides/20-self-hosted/02-deployment/01-non-production/00-deploying-local.md b/docs/cn/guides/20-self-hosted/02-deployment/01-non-production/00-deploying-local.md deleted file mode 100644 index a67cd200f1..0000000000 --- a/docs/cn/guides/20-self-hosted/02-deployment/01-non-production/00-deploying-local.md +++ /dev/null @@ -1,244 +0,0 @@ ---- -title: 在 Docker 上部署 ---- - - - -本指南将引导你使用 [Docker](https://www.docker.com/) 和 [MinIO](https://min.io/) 部署 Databend，以便在本地机器上进行完全容器化的设置。 - -:::note non-production use only -本指南中介绍的 MinIO 部署仅适用于开发和演示。由于单机环境中的资源有限，因此不建议将其用于生产环境或性能测试。 -::: - -### 开始之前 - -在开始之前，请确保你已具备以下先决条件： - -- 确保你的本地机器上已安装 [Docker](https://www.docker.com/)。 -- 确保你的机器上已安装 BendSQL。有关如何使用各种包管理器安装 BendSQL 的说明，请参阅 [安装 BendSQL](/guides/connect/sql-clients/bendsql/#installing-bendsql)。 - -### 部署 MinIO - -1. 使用以下命令拉取并运行 MinIO 镜像作为容器： - -:::note - -- 我们在这里将控制台地址更改为 `9001`，以避免端口冲突。 -- 该命令还设置了 root 用户凭据 (`ROOTUSER`/`CHANGEME123`)，你需要在后续步骤中提供这些凭据进行身份验证。如果你此时更改 root 用户凭据，请确保在整个过程中保持一致。 - ::: - -```shell -docker run -d --name minio \ - -e "MINIO_ACCESS_KEY=ROOTUSER" \ - -e "MINIO_SECRET_KEY=CHANGEME123" \ - -p 9000:9000 \ - -p 9001:9001 \ - minio/minio server /data \ - --address :9000 \ - --console-address :9001 -``` - -2. 运行命令 `docker logs minio` 以在日志消息中查找 MinIO API 和控制台 (WebUI) 地址： - -```shell -docker logs minio -``` - -``` -INFO: WARNING: MINIO_ACCESS_KEY and MINIO_SECRET_KEY are deprecated. - Please use MINIO_ROOT_USER and MINIO_ROOT_PASSWORD -INFO: Formatting 1st pool, 1 set(s), 1 drives per set. -INFO: WARNING: Host local has more than 0 drives of set. A host failure will result in data becoming unavailable. -MinIO Object Storage Server -Copyright: 2015-2025 MinIO, Inc. -License: GNU AGPLv3 - https://www.gnu.org/licenses/agpl-3.0.html -Version: RELEASE.2025-04-03T14-56-28Z (go1.24.2 linux/arm64) - -API: http://172.17.0.2:9000 http://127.0.0.1:9000 -WebUI: http://172.17.0.2:9001 http://127.0.0.1:9001 - -Docs: https://docs.min.io -INFO: - You are running an older version of MinIO released 5 days before the latest release - Update: Run `mc admin update ALIAS` -``` - -3. 在本地机器上打开 Web 浏览器，然后使用日志中显示的 WebUI 地址 (`http://127.0.0.1:9001`) 访问 MinIO 控制台。 - -![Alt text](/img/deploy/docker-minio.png) - -4. 使用凭据 `ROOTUSER`/`CHANGEME123` 登录到 MinIO 控制台，然后创建一个名为 `databend` 的 bucket。 - -![Alt text](/img/deploy/docker-bucket.png) - -### 部署 Databend - -1. 使用以下命令拉取并运行 Databend 镜像作为容器： - -:::note - -- 启动 Databend Docker 容器时，可以使用环境变量 `QUERY_DEFAULT_USER` 和 `QUERY_DEFAULT_PASSWORD` 指定用户名和密码。如果未提供这些变量，将创建一个没有密码的默认 root 用户。 -- 以下命令还会创建一个 SQL 用户 (`databend`/`databend`)，你将需要使用该用户稍后连接到 Databend。如果你此时更改 SQL 用户，请确保在整个过程中保持一致。 - ::: - -```shell -docker run -d \ - --name databend \ - -p 3307:3307 \ - -p 8000:8000 \ - -p 8124:8124 \ - -p 8900:8900 \ - -e QUERY_DEFAULT_USER=databend \ - -e QUERY_DEFAULT_PASSWORD=databend \ - -e QUERY_STORAGE_TYPE=s3 \ - -e AWS_S3_ENDPOINT=http://host.docker.internal:9000 \ - -e AWS_S3_BUCKET=databend \ - -e AWS_ACCESS_KEY_ID=ROOTUSER \ - -e AWS_SECRET_ACCESS_KEY=CHANGEME123 \ - datafuselabs/databend -``` - -2. 运行命令 `docker logs databend` 以检查 Databend 日志消息，并确保 Databend 容器已成功启动： - -```shell -docker logs databend -``` - -```shell -==> QUERY_CONFIG_FILE is not set, using default: /etc/databend/query.toml -==> /tmp/std-meta.log <== -Databend Metasrv - -Version: v1.2.697-d40f88cc51-simd(1.85.0-nightly-2025-02-14T11:57:01.874747910Z) -Working DataVersion: V004(2024-11-11: WAL based raft-log) - -Raft Feature set: - Server Provide: { append:v0, install_snapshot:v1, install_snapshot:v3, vote:v0 } - Client Require: { append:v0, install_snapshot:v3, vote:v0 } - -Disk Data: V002(2023-07-22: Store snapshot in a file); Upgrading: None - Dir: /var/lib/databend/meta - -Log File: enabled=true, level='Warn,databend_=Info,openraft=Info', dir=/var/log/databend, format=json, limit=48 - Stderr: enabled=false(To enable: LOG_STDERR_ON=true or RUST_LOG=info), level=WARN, format=text -Raft Id: 0; Cluster: foo_cluster - Dir: /var/lib/databend/meta - Status: single - -HTTP API listen at: 127.0.0.1:28002 -gRPC API listen at: 127.0.0.1:9191 advertise: - -Raft API listen at: 127.0.0.1:28004 advertise: 055b9e5d09a9:28004 - -Upgrade ondisk data if out of date: V002 - Find and clean previous unfinished upgrading -Upgrade on-disk data - From: V002(2023-07-22: Store snapshot in a file) - To: V003(2024-06-27: Store snapshot in rotbl) - No V002 snapshot, skip upgrade - Finished upgrading: V003 -Upgrade on-disk data - From: V003(2024-06-27: Store snapshot in rotbl) - To: V004(2024-11-11: WAL based raft-log) - Upgrade V003 raft log in sled db to V004 - Clean upgrading: V003 -> V004 (cleaning) - Remove V003 log from sled db - Removing sled tree: header - Removing sled tree: raft_log - Removing sled tree: raft_state - Done: Remove V003 log from sled db - Removing: /var/lib/databend/meta/heap - Removing: /var/lib/databend/meta/conf - Removing: /var/lib/databend/meta/db - Removing: /var/lib/databend/meta/DO_NOT_USE_THIS_DIRECTORY_FOR_ANYTHING - Finished upgrading: V004 -Upgrade ondisk data finished: V004 -Wait for 180s for active leader... -Leader Id: 0 - Metrics: id=0, Leader, term=1, last_log=Some(3), last_applied=Some(T1-N0.3), membership={log_id:Some(T1-N0.3), {voters:[{0:EmptyNode}], learners:[]}} - -Register this node: {id=0 raft=055b9e5d09a9:28004 grpc=} - - Register-node: Ok - -Databend Metasrv started - -==> /tmp/std-query.log <== -Databend Query - -Version: v1.2.697-d40f88cc51(rust-1.85.0-nightly-2025-02-14T11:30:59.842308760Z) - -Logging: - file: enabled=true, level='INFO', dir=/var/log/databend, format=json, limit=48 - stderr: enabled=false(To enable: LOG_STDERR_ON=true or RUST_LOG=info), level=WARN, format=text - -Meta: connected to endpoints [ - "0.0.0.0:9191", -] - -Memory: - limit: unlimited - allocator: jemalloc - config: percpu_arena:percpu,oversize_threshold:0,background_thread:true,dirty_decay_ms:5000,muzzy_decay_ms:5000 - -Cluster: standalone - -Storage: s3 | bucket=databend,root=,endpoint=http://host.docker.internal:9000 -Disk cache: - storage: none - path: DiskCacheConfig { max_bytes: 21474836480, path: "./.databend/_cache", sync_data: true } - reload policy: reset - -Builtin users: databend - -Builtin UDFs: - -Admin - listened at 0.0.0.0:8080 -MySQL - listened at 0.0.0.0:3307 - connect via: mysql -u${USER} -p${PASSWORD} -h0.0.0.0 -P3307 -Clickhouse(http) - listened at 0.0.0.0:8124 - usage: echo 'create table test(foo string)' | curl -u${USER} -p${PASSWORD}: '0.0.0.0:8124' --data-binary @- -echo '{"foo": "bar"}' | curl -u${USER} -p${PASSWORD}: '0.0.0.0:8124/?query=INSERT%20INTO%20test%20FORMAT%20JSONEachRow' --data-binary @- -Databend HTTP - listened at 0.0.0.0:8000 - usage: curl -u${USER} -p${PASSWORD}: --request POST '0.0.0.0:8000/v1/query/' --header 'Content-Type: application/json' --data-raw '{"sql": "SELECT avg(number) FROM numbers(100000000)"}' -``` - -### 连接到 Databend - -在本地机器上启动一个终端，然后运行以下命令以连接到 Databend： - -```shell -bendsql -u databend -p databend -``` - -```shell -Welcome to BendSQL 0.24.1-f1f7de0(2024-12-04T12:31:18.526234000Z). -Connecting to localhost:8000 as user databend. -Connected to Databend Query v1.2.697-d40f88cc51(rust-1.85.0-nightly-2025-02-14T11:30:59.842308760Z) -Loaded 1411 auto complete keywords from server. -Started web server at 127.0.0.1:8080 -``` - -一切就绪！现在，你可以执行一个简单的查询来验证部署： - -```sql -databend@localhost:8000/default> SELECT NOW(); - -SELECT NOW() - -┌────────────────────────────┐ -│ now() │ -│ Timestamp │ -├────────────────────────────┤ -│ 2025-04-10 03:14:06.778815 │ -└────────────────────────────┘ -1 row read in 0.003 sec. Processed 1 row, 1 B (333.33 rows/s, 333 B/s) -``` - - \ No newline at end of file diff --git a/docs/cn/guides/20-self-hosted/02-deployment/01-non-production/_category_.json b/docs/cn/guides/20-self-hosted/02-deployment/01-non-production/_category_.json deleted file mode 100644 index bb3495169e..0000000000 --- a/docs/cn/guides/20-self-hosted/02-deployment/01-non-production/_category_.json +++ /dev/null @@ -1,3 +0,0 @@ -{ - "label": "非生产环境" -} \ No newline at end of file diff --git a/docs/cn/guides/20-self-hosted/02-deployment/01-non-production/index.md b/docs/cn/guides/20-self-hosted/02-deployment/01-non-production/index.md deleted file mode 100644 index 6f6ae50f54..0000000000 --- a/docs/cn/guides/20-self-hosted/02-deployment/01-non-production/index.md +++ /dev/null @@ -1,7 +0,0 @@ ---- -title: 非生产环境部署 ---- - -import IndexOverviewList from '@site/src/components/IndexOverviewList'; - - \ No newline at end of file diff --git a/docs/cn/guides/20-self-hosted/02-deployment/02-deploying-offline.md b/docs/cn/guides/20-self-hosted/02-deployment/02-deploying-offline.md new file mode 100644 index 0000000000..67e090d8d1 --- /dev/null +++ b/docs/cn/guides/20-self-hosted/02-deployment/02-deploying-offline.md @@ -0,0 +1,79 @@ +--- +title: 离线环境 Docker 部署 +--- + +本指南适用于无法直接访问 Docker Hub 的离线或受限网络环境。 + +## 第一步：在有网络的机器上下载镜像 + +在可以联网的机器上拉取并导出所需镜像： + +```shell +docker pull datafuselabs/databend-meta:latest +docker pull datafuselabs/databend-query:latest +docker pull minio/minio:latest + +docker save datafuselabs/databend-meta:latest | gzip > databend-meta.tar.gz +docker save datafuselabs/databend-query:latest | gzip > databend-query.tar.gz +docker save minio/minio:latest | gzip > minio.tar.gz +``` + +:::tip 国内用户 +也可以从国内镜像源拉取，速度更快： +```shell +VERSION=latest +docker pull registry.databend.cn/public/databend-meta:${VERSION} +docker pull registry.databend.cn/public/databend-query:${VERSION} +``` +::: + +如需固定版本（生产环境推荐）： + +```shell +VERSION=latest +docker pull datafuselabs/databend-meta:${VERSION} +docker pull datafuselabs/databend-query:${VERSION} +docker save datafuselabs/databend-meta:${VERSION} | gzip > databend-meta-${VERSION}.tar.gz +docker save datafuselabs/databend-query:${VERSION} | gzip > databend-query-${VERSION}.tar.gz +``` + +## 第二步：传输并加载镜像 + +将 `.tar.gz` 文件上传到目标机器后，执行加载： + +```shell +docker load -i databend-meta.tar.gz +docker load -i databend-query.tar.gz +docker load -i minio.tar.gz +``` + +验证镜像已加载： + +```shell +docker images | grep -E "databend|minio" +``` + +## 第三步：启动 + +使用[在 Docker 上部署](./01-deploying-local.md)中的 `docker-compose.yml`，无需修改，镜像已在本地可用。 + +```shell +docker compose up -d +``` + +## 可选：推送到私有镜像仓库 + +如果公司内部有私有镜像仓库（例如 `registry.example.com`）： + +```shell +VERSION=latest +REGISTRY=registry.example.com + +docker tag datafuselabs/databend-meta:${VERSION} ${REGISTRY}/databend-meta:${VERSION} +docker tag datafuselabs/databend-query:${VERSION} ${REGISTRY}/databend-query:${VERSION} + +docker push ${REGISTRY}/databend-meta:${VERSION} +docker push ${REGISTRY}/databend-query:${VERSION} +``` + +然后将 `docker-compose.yml` 中的镜像地址替换为私有仓库地址即可。 diff --git a/docs/cn/guides/20-self-hosted/02-deployment/02-production/10-metasrv-deploy.md b/docs/cn/guides/20-self-hosted/02-deployment/02-production/10-metasrv-deploy.md index 6d236acc0e..83401663ad 100644 --- a/docs/cn/guides/20-self-hosted/02-deployment/02-production/10-metasrv-deploy.md +++ b/docs/cn/guides/20-self-hosted/02-deployment/02-production/10-metasrv-deploy.md @@ -6,7 +6,7 @@ sidebar_label: 部署 Databend 集群 import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -Databend 建议在生产环境中部署至少包含三个元节点（Meta Node）和一个查询节点（Query Node）的集群。要深入了解 Databend 集群部署，请参阅[理解 Databend 部署模式](../00-understanding-deployment-modes.md)，该文档将帮助您掌握相关概念。本文旨在提供部署 Databend 集群的实践指南。 +Databend 建议在生产环境中部署至少包含三个元节点（Meta Node）和一个查询节点（Query Node）的集群。要深入了解 Databend 集群部署，请参阅[理解 Databend 部署模式](../05-understanding-deployment-modes.md)，该文档将帮助您掌握相关概念。本文旨在提供部署 Databend 集群的实践指南。 ## 准备工作 @@ -203,7 +203,7 @@ endpoints = ["172.16.125.128:9191","172.16.125.129:9191","172.16.125.130:9191"] -2. 每个查询节点还需在 [databend-query.toml](https://github.com/databendlabs/databend/blob/main/scripts/distribution/configs/databend-query.toml) 中配置对象存储和管理员用户，详见[此文档](../01-non-production/01-deploying-databend.md#部署查询节点)。 +2. 每个查询节点还需在 [databend-query.toml](https://github.com/databendlabs/databend/blob/main/scripts/distribution/configs/databend-query.toml) 中配置对象存储和管理员用户，详见[此文档](../03-deploying-databend.md#部署查询节点)。 3. 在每个查询节点执行启动脚本： diff --git a/docs/cn/guides/20-self-hosted/02-deployment/03-benddeploy/01-installing-benddeploy.md b/docs/cn/guides/20-self-hosted/02-deployment/02-production/90-benddeploy/01-installing-benddeploy.md similarity index 100% rename from docs/cn/guides/20-self-hosted/02-deployment/03-benddeploy/01-installing-benddeploy.md rename to docs/cn/guides/20-self-hosted/02-deployment/02-production/90-benddeploy/01-installing-benddeploy.md diff --git a/docs/cn/guides/20-self-hosted/02-deployment/03-benddeploy/02-getting-started.md b/docs/cn/guides/20-self-hosted/02-deployment/02-production/90-benddeploy/02-getting-started.md similarity index 100% rename from docs/cn/guides/20-self-hosted/02-deployment/03-benddeploy/02-getting-started.md rename to docs/cn/guides/20-self-hosted/02-deployment/02-production/90-benddeploy/02-getting-started.md diff --git a/docs/cn/guides/20-self-hosted/02-deployment/03-benddeploy/_category_.json b/docs/cn/guides/20-self-hosted/02-deployment/02-production/90-benddeploy/_category_.json similarity index 100% rename from docs/cn/guides/20-self-hosted/02-deployment/03-benddeploy/_category_.json rename to docs/cn/guides/20-self-hosted/02-deployment/02-production/90-benddeploy/_category_.json diff --git a/docs/cn/guides/20-self-hosted/02-deployment/03-benddeploy/index.md b/docs/cn/guides/20-self-hosted/02-deployment/02-production/90-benddeploy/index.md similarity index 100% rename from docs/cn/guides/20-self-hosted/02-deployment/03-benddeploy/index.md rename to docs/cn/guides/20-self-hosted/02-deployment/02-production/90-benddeploy/index.md diff --git a/docs/cn/guides/20-self-hosted/02-deployment/02-production/index.md b/docs/cn/guides/20-self-hosted/02-deployment/02-production/index.md index e5c8f1a287..d4ad7e3f23 100644 --- a/docs/cn/guides/20-self-hosted/02-deployment/02-production/index.md +++ b/docs/cn/guides/20-self-hosted/02-deployment/02-production/index.md @@ -15,4 +15,5 @@ title: 生产环境部署 - [Meta Service 中的备份和恢复](30-metasrv-backup-restore.md) - [Meta Service 状态](60-metasrv-status.md) - [Meta Service CLI API](70-metasrv-cli-api.md) -- [Meta Service gRPC 指南](16-metasrv-grpc.md) \ No newline at end of file +- [Meta Service gRPC 指南](16-metasrv-grpc.md) +- [BendDeploy](90-benddeploy/index.md) \ No newline at end of file diff --git a/docs/cn/guides/20-self-hosted/02-deployment/01-non-production/01-deploying-databend.md b/docs/cn/guides/20-self-hosted/02-deployment/03-deploying-databend.md similarity index 100% rename from docs/cn/guides/20-self-hosted/02-deployment/01-non-production/01-deploying-databend.md rename to docs/cn/guides/20-self-hosted/02-deployment/03-deploying-databend.md diff --git a/docs/cn/guides/20-self-hosted/02-deployment/04-manually/01-prepare.md b/docs/cn/guides/20-self-hosted/02-deployment/04-manually/01-prepare.md deleted file mode 100644 index e7596757dc..0000000000 --- a/docs/cn/guides/20-self-hosted/02-deployment/04-manually/01-prepare.md +++ /dev/null @@ -1,114 +0,0 @@ ---- -title: 准备软件包环境 ---- - -## 前提条件 - -- 基于 Linux 的操作系统 -- 用于下载文件的 `wget` 或 `curl` -- 用于解压软件包的 `tar` -- 用于系统级安装的 `sudo` 权限 - -## 检查系统架构 - -1. 检查系统架构： - - ```bash - uname -m - ``` - - 输出将帮助您确定要下载的软件包： - - - 若输出为 `x86_64`，请下载 x86_64 软件包 - - 若输出为 `aarch64`，请下载 aarch64 软件包 - -## 下载软件包 - -1. 访问 [Databend GitHub Releases](https://github.com/databendlabs/databend/releases) 页面。 - -2. 选择最新的稳定版本。例如，若要安装 v1.2.755-nightly，需下载： - - - `databend-v1.2.755-nightly-x86_64-unknown-linux-gnu.tar.gz`（适用于 x86_64 Linux 系统） - - `databend-v1.2.755-nightly-aarch64-unknown-linux-gnu.tar.gz`（适用于 aarch64 Linux 系统） - -3. 使用 wget 下载软件包（将 `v1.2.755-nightly` 替换为目标版本）： - - ```bash - wget https://github.com/databendlabs/databend/releases/download/v1.2.755-nightly/databend-v1.2.755-nightly-x86_64-unknown-linux-gnu.tar.gz - ``` - - 或使用 curl（将 `v1.2.755-nightly` 替换为目标版本）： - - ```bash - curl -L -O https://github.com/databendlabs/databend/releases/download/v1.2.755-nightly/databend-v1.2.755-nightly-x86_64-unknown-linux-gnu.tar.gz - ``` - - 注意：请务必将 URL 和文件名中的 `v1.2.755-nightly` 替换为目标版本号。 - -## 解压软件包 - -1. 在当前目录解压软件包： - ```bash - tar xzf databend-v1.2.755-nightly-x86_64-unknown-linux-gnu.tar.gz - ``` - -## 验证安装 - -1. 检查解压后的文件： - - ```bash - ls --tree - ``` - - 应看到以下目录结构： - - ``` - . - ├── bin - │ ├── bendsql - │ ├── databend-bendsave - │ ├── databend-meta - │ ├── databend-metactl - │ └── databend-query - ├── configs - │ ├── databend-meta.toml - │ └── databend-query.toml - ├── readme.txt - ├── scripts - │ ├── postinstall.sh - │ └── preinstall.sh - └── systemd - ├── databend-meta.default - ├── databend-meta.service - ├── databend-query.default - └── databend-query.service - ``` - -2. 验证二进制文件是否可执行： - ```bash - ./bin/databend-meta --version - ./bin/databend-metactl --version - ./bin/databend-query --version - ./bin/bendsql --version - ``` - -## 创建 Databend 用户 - -1. 运行预安装脚本以创建 databend 用户和组： - - ```bash - sudo ./scripts/preinstall.sh - ``` - - 该脚本将： - - - 若不存在，则创建 databend 用户和组 - - 设置必要的系统配置 - - 创建所需目录并赋予适当权限 - -## 下一步 - -环境准备就绪后，可继续： - -- [部署 Meta 服务](02-deploy-metasrv.md) -- [部署 Query 服务](03-deploy-query.md) \ No newline at end of file diff --git a/docs/cn/guides/20-self-hosted/02-deployment/04-manually/02-deploy-metasrv.md b/docs/cn/guides/20-self-hosted/02-deployment/04-manually/02-deploy-metasrv.md deleted file mode 100644 index df034059cd..0000000000 --- a/docs/cn/guides/20-self-hosted/02-deployment/04-manually/02-deploy-metasrv.md +++ /dev/null @@ -1,169 +0,0 @@ ---- -title: 部署 Meta Service ---- - -## 概述 - -Meta Service 是 Databend 的关键组件，负责管理元数据和集群协调。本指南将引导你完成部署 Meta Service 节点的过程。 - -## 前置要求 - -- 已完成 [准备软件包环境](01-prepare.md) 中的步骤 -- 已准备好解压后的 Databend 软件包 -- 拥有 sudo 权限 - -## 安装二进制文件 - -1. 将 Meta Service 二进制文件复制到系统二进制目录： - ```bash - sudo cp bin/databend-meta /usr/bin/ - sudo chmod +x /usr/bin/databend-meta - ``` - -2. 复制 Meta Service 控制工具的二进制文件： - ```bash - sudo cp bin/databend-metactl /usr/bin/ - sudo chmod +x /usr/bin/databend-metactl - ``` - -## 配置 Meta Service - -1. 导航到解压后的软件包目录并复制默认配置： - ```bash - sudo mkdir -p /etc/databend - sudo cp configs/databend-meta.toml /etc/databend/databend-meta.toml - ``` - -2. 编辑配置文件： - ```bash - sudo vim /etc/databend/databend-meta.toml - ``` - - 默认配置如下： - ```toml - admin_api_address = "0.0.0.0:28002" - grpc_api_address = "0.0.0.0:9191" - grpc_api_advertise_host = "localhost" # 修改此项 - - [log] - [log.file] - level = "WARN" - format = "text" - dir = "/var/log/databend" - - [raft_config] - id = 0 # 对于单节点部署或集群中的第一个节点，请保持为 0 - raft_dir = "/var/lib/databend/raft" - raft_api_port = 28004 - raft_listen_host = "0.0.0.0" - raft_advertise_host = "localhost" # 修改此项 - single = true # 对于单节点部署或集群中的第一个节点，请保持为 true - ``` - - 根据你的环境修改以下设置： - - `grpc_api_advertise_host`：用于 gRPC 通信的主机名或 IP 地址，应与机器的主机名或 IP 地址相同 - - `raft_advertise_host`：其他节点用来连接的主机名或 IP 地址，应与机器的主机名或 IP 地址相同 - -## 设置 Systemd 服务 - -1. 复制 systemd 服务文件： - ```bash - sudo cp systemd/databend-meta.service /etc/systemd/system/ - ``` - -2. 复制默认环境文件： - ```bash - sudo cp systemd/databend-meta.default /etc/default/databend-meta - ``` - -3. 编辑环境文件（可选）： - ```bash - sudo vim /etc/default/databend-meta - ``` - - 在需要时设置以下变量（可选）： - ```bash - RUST_BACKTRACE=1 # 启用回溯以进行调试 - SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt # 如果使用自定义 CA 证书，请设置 CA 证书文件的路径 - ``` - -4. 重新加载 systemd 以识别新服务： - ```bash - sudo systemctl daemon-reload - ``` - -5. 启用服务以在启动时自动运行： - ```bash - sudo systemctl enable databend-meta - ``` - -## 启动 Meta Service - -1. 启动 Meta Service： - ```bash - sudo systemctl start databend-meta - ``` - -2. 检查服务状态： - ```bash - sudo systemctl status databend-meta - ``` - -3. 查看日志： - ```bash - sudo journalctl -u databend-meta -f - ``` - -## 验证 Meta Service - -1. 检查 Meta Service 是否在配置的端口上监听： - ```bash - sudo netstat -tulpn | grep databend-meta - ``` - -2. 测试管理 API 端点： - ```bash - curl http://127.0.0.1:28002/v1/health - ``` - - 你应该会收到一个表示服务健康的响应。 - -3. 使用 metactl 检查 Meta Service 状态： - ```bash - databend-metactl status - ``` - - 你应该能看到 Meta Service 的当前状态，包括： - - 节点 ID - - Raft 状态 - - Leader 信息 - - 集群配置 - -## 故障排查 - -如果遇到问题： - -1. 检查服务状态： - ```bash - sudo systemctl status databend-meta - ``` - -2. 查看日志以获取详细的错误信息： - ```bash - # 查看 systemd 日志 - sudo journalctl -u databend-meta -f - - # 查看 /var/log/databend 中的日志文件 - sudo tail -f /var/log/databend/databend-meta-*.log - ``` - -3. 常见问题及解决方案： - - 权限被拒绝：确保 databend 用户在之前的步骤中具有适当权限 - - 端口已被占用：检查是否有其他服务正在使用配置的端口 - - 配置错误：验证配置文件的语法和路径 - -## 下一步 - -现在你已经部署了 Meta Service，可以继续进行以下操作： -- [部署 Query Service](03-deploy-query.md) -- [扩展 Meta Service 节点](04-scale-metasrv.md)（用于多节点部署） \ No newline at end of file diff --git a/docs/cn/guides/20-self-hosted/02-deployment/04-manually/03-deploy-query.md b/docs/cn/guides/20-self-hosted/02-deployment/04-manually/03-deploy-query.md deleted file mode 100644 index 4b410fb6dd..0000000000 --- a/docs/cn/guides/20-self-hosted/02-deployment/04-manually/03-deploy-query.md +++ /dev/null @@ -1,208 +0,0 @@ ---- -title: 部署查询服务 ---- - -## 概述 - -查询服务（Query Service）是 Databend 中处理 SQL 查询和数据处理的主要组件。本指南将引导您完成部署查询服务（Query Service）节点的过程。 - -## 前置准备 - -- 已完成 [准备包环境](01-prepare.md) 中的步骤 -- 已准备好解压后的 Databend 包 -- 拥有 sudo 权限 -- 拥有一个正在运行的元数据服务（Meta Service）节点（已完成 [部署元数据服务](02-deploy-metasrv.md)） - -## 安装二进制文件 - -1. 将查询服务（Query Service）的二进制文件复制到系统二进制目录： - ```bash - sudo cp bin/databend-query /usr/bin/ - sudo chmod +x /usr/bin/databend-query - ``` - -2. 复制 BendSQL 客户端二进制文件： - ```bash - sudo cp bin/bendsql /usr/bin/ - sudo chmod +x /usr/bin/bendsql - ``` - -## 配置查询服务 - -1. 导航到解压后的包目录并复制默认配置： - ```bash - sudo mkdir -p /etc/databend - sudo cp configs/databend-query.toml /etc/databend/databend-query.toml - ``` - -2. 编辑配置文件： - ```bash - sudo vim /etc/databend/databend-query.toml - ``` - - 默认配置如下： - ```toml - [query] - max_active_sessions = 256 - shutdown_wait_timeout_ms = 5000 - flight_api_address = "0.0.0.0:9091" - admin_api_address = "0.0.0.0:8080" - metric_api_address = "0.0.0.0:7070" - mysql_handler_host = "0.0.0.0" - mysql_handler_port = 3307 - clickhouse_http_handler_host = "0.0.0.0" - clickhouse_http_handler_port = 8124 - http_handler_host = "0.0.0.0" - http_handler_port = 8000 - flight_sql_handler_host = "0.0.0.0" - flight_sql_handler_port = 8900 - tenant_id = "default" # 根据需要进行更改（可选） - cluster_id = "default" # 根据需要进行更改（可选） - - [[query.users]] # 取消注释此块以启用默认内置用户 "root" - name = "root" - auth_type = "no_password" - - [log] - [log.file] - level = "WARN" - format = "text" - dir = "/var/log/databend" - - [meta] - endpoints = ["0.0.0.0:9191"] # 将此更改为所有 Meta Service 节点的地址 - username = "root" - password = "root" - client_timeout_in_second = 10 - auto_sync_interval = 60 - - # (重要) 将此更改为您自己的存储配置 - [storage] - # fs | s3 | azblob | gcs | oss | cos | hdfs | webhdfs - type = "fs" - - [storage.fs] - data_path = "/var/lib/databend/data" - - # 使用类似 Amazon S3 的存储服务 - [storage.s3] - bucket = "" - endpoint_url = "" - access_key_id = "" - secret_access_key = "" - enable_virtual_host_style = false - - [cache] - data_cache_storage = "none" # 如果要启用本地磁盘缓存，请将其更改为 "disk" - - [cache.disk] - path = "/var/lib/databend/cache" - max_bytes = 21474836480 - ``` - - 根据您的环境修改以下设置： - - `[meta].endpoints`: 您的元数据服务（Meta Service）节点的地址（格式：["host:port"]） - - `[storage]`: 用于存储数据的存储类型和配置。每个租户都应有自己的存储配置（重要） - - `[query].users`: 用于查询服务（Query Service）身份验证的用户，在创建自己的用户后，您可以稍后注释掉此块（可选） - - `[query].tenant_id`: 用于多租户部署的租户 ID（可选） - - `[query].cluster_id`: 用于多集群部署的集群 ID（可选） - - `[cache]`: 要使用的缓存配置（可选） - -## 设置 Systemd 服务 - -1. 复制 systemd 服务文件： - ```bash - sudo cp systemd/databend-query.service /etc/systemd/system/ - ``` - -2. 复制默认环境文件： - ```bash - sudo cp systemd/databend-query.default /etc/default/databend-query - ``` - -3. 编辑环境文件（可选）： - ```bash - sudo vim /etc/default/databend-query - ``` - - 在需要时设置以下变量（可选）： - ```bash - RUST_BACKTRACE=1 # 启用回溯以进行调试 - SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt # 如果使用自定义 CA 证书，请设置 CA 证书文件的路径 - ``` - -4. 重新加载 systemd 以识别新服务： - ```bash - sudo systemctl daemon-reload - ``` - -5. 启用服务以在启动时自启： - ```bash - sudo systemctl enable databend-query - ``` - -## 启动查询服务 - -1. 启动查询服务（Query Service）： - ```bash - sudo systemctl start databend-query - ``` - -2. 检查服务状态： - ```bash - sudo systemctl status databend-query - ``` - -3. 查看日志： - ```bash - sudo journalctl -u databend-query -f - ``` - -## 验证查询服务 - -1. 检查查询服务（Query Service）是否正在监听配置的端口： - ```bash - sudo netstat -tulpn | grep databend-query - ``` - -2. 测试 HTTP 端点： - ```bash - curl http://127.0.0.1:8000/health - ``` - - 您应该会收到一个表示服务健康的回应。 - -3. 使用 BendSQL 进行测试： - ```bash - bendsql -h 127.0.0.1 -u root - ``` - -## 故障排查 - -如果遇到问题： - -1. 检查服务状态： - ```bash - sudo systemctl status databend-query - ``` - -2. 查看日志以获取详细的错误信息： - ```bash - # 查看 systemd 日志 - sudo journalctl -u databend-query -f - - # 查看 /var/log/databend 中的日志文件 - sudo tail -f /var/log/databend/databend-query-*.log - ``` - -3. 常见问题及解决方案： - - 权限被拒绝：确保 databend 用户在前序步骤中具有适当的权限 - - 端口已被占用：检查是否有其他服务正在使用配置的端口 - - 配置错误：验证配置文件的语法和路径 - - 元数据服务（Meta Service）连接问题：确保元数据服务（Meta Service）正在运行且可访问 - -## 后续步骤 - -现在您已经部署了元数据服务（Meta Service）和查询服务（Query Service），可以继续进行以下操作： -- [扩展查询服务节点](05-scale-query.md)（用于多节点部署） -- [升级查询服务](07-upgrade-query.md) \ No newline at end of file diff --git a/docs/cn/guides/20-self-hosted/02-deployment/04-manually/04-scale-metasrv.md b/docs/cn/guides/20-self-hosted/02-deployment/04-manually/04-scale-metasrv.md deleted file mode 100644 index 5184495d77..0000000000 --- a/docs/cn/guides/20-self-hosted/02-deployment/04-manually/04-scale-metasrv.md +++ /dev/null @@ -1,171 +0,0 @@ ---- -title: 伸缩 Meta 服务节点 ---- - -## 概述 - -本指南将引导你完成 Databend Meta 服务集群的伸缩操作。你可以向集群中添加新节点（扩容）或移除现有节点（缩容）。 - -## 前提条件 - -- 已完成 [部署 Meta 服务](02-deploy-metasrv.md) 并拥有运行中的 Meta 服务节点 -- 已在新节点上完成 [准备软件包环境](01-prepare.md)（用于扩容） -- 在节点上拥有 sudo 权限 - -## 扩容：添加新的 Meta 服务节点 - -要添加新的 Meta 服务节点，请在新节点上按照 [部署 Meta 服务](02-deploy-metasrv.md) 的步骤操作。请确保： - -1. 从现有节点复制配置： - ```bash - sudo mkdir -p /etc/databend - sudo scp root@:/etc/databend/databend-meta.toml /etc/databend/ - ``` - -2. 修改配置文件： - ```bash - sudo vim /etc/databend/databend-meta.toml - ``` - - 更新以下设置： - ```toml - [raft_config] - id = 1 # 为每个节点更改为唯一的 ID（例如 1, 2, 3 等） - single = false # 对于多节点部署，请将其更改为 false - join = ["127.0.0.1:28004"] # 在此处添加所有现有的 Meta 服务节点 - ``` - -3. 按照 [部署 Meta 服务](02-deploy-metasrv.md) 的步骤，在新节点上安装并启动服务。 - -4. 验证新节点是否已加入集群： - ```bash - databend-metactl status - ``` - -5. 确认新节点加入集群后，更新所有现有节点的配置： - ```bash - sudo vim /etc/databend/databend-meta.toml - ``` - - 在每个现有节点上更新以下设置： - ```toml - [raft_config] - single = false # 将此项更改为 false - join = ["127.0.0.1:28004", "127.0.0.2:28004", "127.0.0.3:28004"] # 添加所有 Meta 服务节点（含新节点） - ``` - -6. 更新所有 Query 节点配置中的 Meta 服务端点： - ```bash - sudo vim /etc/databend/databend-query.toml - ``` - - 在每个 Query 节点中更新以下设置： - ```toml - [meta] - endpoints = ["127.0.0.1:9191", "127.0.0.2:9191", "127.0.0.3:9191"] # 添加所有 Meta 服务节点（含新节点） - ``` - -## 缩容：移除 Meta 服务节点 - -要从集群中移除 Meta 服务节点，请按以下步骤操作： - -1. 首先检查待移除节点是否为当前领导者（Leader）： - ```bash - databend-metactl status - ``` - 在输出中查找 “leader” 信息。若待移除节点是领导者，需先转移领导权。 - -2. 若该节点是领导者，将领导权转移至其他节点： - ```bash - databend-metactl transfer-leader - ``` - -3. 验证领导权转移： - ```bash - databend-metactl status - ``` - 确认领导权已转移至目标节点。 - -4. 使用 `databend-meta` 命令将节点优雅移出集群： - ```bash - databend-meta --leave-id --leave-via ... - ``` - - 例如移除 ID 为 1 的节点： - ```bash - databend-meta --leave-id 1 --leave-via 127.0.0.1:28004 - ``` - - 注意： - - `--leave-id` 指定待移除节点的 ID - - `--leave-via` 指定用于发送离开请求的节点广播地址列表 - - 此命令可在任意安装 `databend-meta` 的节点执行 - - 节点在离开请求完成前将被禁止与集群交互 - -5. 检查节点是否成功移出集群： - ```bash - databend-metactl status - ``` - 确认该节点 ID 已从集群成员列表中消失。 - -6. 确认节点移出集群后，停止服务： - ```bash - sudo systemctl stop databend-meta - ``` - -7. 从任意剩余节点验证集群状态： - ```bash - databend-metactl status - ``` - -8. 确认集群稳定后，更新所有剩余 Meta 节点的配置： - ```bash - sudo vim /etc/databend/databend-meta.toml - ``` - - 在每个剩余节点上更新以下设置： - ```toml - [raft_config] - join = ["127.0.0.2:28004", "127.0.0.3:28004"] # 从列表中移除待离开节点 - ``` - -9. 更新所有 Query 节点配置中的 Meta 服务端点： - ```bash - sudo vim /etc/databend/databend-query.toml - ``` - - 在每个 Query 节点中更新以下设置： - ```toml - [meta] - endpoints = ["127.0.0.2:9191", "127.0.0.3:9191"] # 从列表中移除待离开节点 - ``` - -## 故障排除 - -若遇到问题： - -1. 检查服务状态： - ```bash - sudo systemctl status databend-meta - ``` - -2. 查看日志获取详细错误信息： - ```bash - # 查看 systemd 日志 - sudo journalctl -u databend-meta -f - - # 查看 /var/log/databend 中的日志文件 - sudo tail -f /var/log/databend/databend-meta-*.log - ``` - -3. 常见问题及解决方案： - - 权限被拒绝：确保 databend 用户具备适当权限 - - 端口已被占用：检查其他服务是否占用配置端口 - - 配置错误：验证配置文件语法与路径 - - Raft 连接问题：确保所有 Meta 服务节点可互通 - -## 后续步骤 - -成功伸缩 Meta 服务集群后，你可以： -- [伸缩 Query 服务节点](05-scale-query.md)（按需） -- [升级 Meta 服务](06-upgrade-metasrv.md)（按需） \ No newline at end of file diff --git a/docs/cn/guides/20-self-hosted/02-deployment/04-manually/05-scale-query.md b/docs/cn/guides/20-self-hosted/02-deployment/04-manually/05-scale-query.md deleted file mode 100644 index 858e96e362..0000000000 --- a/docs/cn/guides/20-self-hosted/02-deployment/04-manually/05-scale-query.md +++ /dev/null @@ -1,78 +0,0 @@ ---- -title: 伸缩查询（Query）服务节点 ---- - -## 概述 - -本指南将引导您完成 Databend 查询（Query）服务集群的伸缩操作。您可以选择向集群添加新节点（扩容）或移除现有节点（缩容）。 - -## 前提条件 - -- 已完成 [部署查询服务](03-deploy-query.md) 且存在运行中的查询（Query）服务节点 -- 已在新节点完成 [准备软件包环境](01-prepare.md)（扩容场景） -- 在节点上拥有 sudo 权限 - -## 扩容：添加新查询（Query）服务节点 - -添加新查询（Query）服务节点时，请在新节点执行 [部署查询服务](03-deploy-query.md) 的步骤，并确保： - -1. 从现有节点复制配置： - ```bash - sudo mkdir -p /etc/databend - sudo scp root@:/etc/databend/databend-query.toml /etc/databend/ - ``` - -2. 按照 [部署查询服务](03-deploy-query.md) 的步骤安装并启动服务 - -3. 部署完成后验证新节点是否加入集群： - ```bash - bendsql -h 127.0.0.1 -u root - ``` - - ```sql - SELECT * FROM system.clusters; - ``` - -## 缩容：移除查询（Query）服务节点 - -移除查询（Query）服务节点时，只需停止目标节点的服务： - -```bash -sudo systemctl stop databend-query -``` - -节点将自动从集群移除。可通过任意存留节点验证集群状态： - -```bash -bendsql -h 127.0.0.1 -u root -``` - -```sql -SELECT * FROM system.clusters; -``` - -## 故障排除 - -遇到问题时： - -1. 检查服务状态： - ```bash - sudo systemctl status databend-query - ``` - -2. 查看日志获取详细错误信息： - ```bash - sudo journalctl -u databend-query -f - ``` - -3. 常见问题及解决方案： - - 权限被拒绝：确保 databend 用户具备适当权限 - - 端口已被占用：检查配置端口是否被其他服务占用 - - 配置错误：验证配置文件语法和路径正确性 - - Meta 服务 (Meta Service) 连接问题：确保新节点可访问 Meta 服务 - -## 后续步骤 - -成功伸缩查询（Query）服务集群后，您可： -- [伸缩 Meta 服务节点](04-scale-metasrv.md)（按需） -- [升级查询服务](07-upgrade-query.md)（按需） \ No newline at end of file diff --git a/docs/cn/guides/20-self-hosted/02-deployment/04-manually/06-upgrade-metasrv.md b/docs/cn/guides/20-self-hosted/02-deployment/04-manually/06-upgrade-metasrv.md deleted file mode 100644 index 87bbe7bb10..0000000000 --- a/docs/cn/guides/20-self-hosted/02-deployment/04-manually/06-upgrade-metasrv.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -title: 升级 Meta Service ---- - -## 概述 - -本指南将引导你完成将 Databend Meta Service 升级至新版本的过程。升级流程包括下载新版发布包、替换二进制文件及重启服务。 - -## 下载新版本 - -1. 按照[准备包环境](01-prepare.md)中的下载说明获取新版发布包。 - -2. 解压发布包： - ```bash - tar xzf databend-v1.2.755-nightly-x86_64-unknown-linux-gnu.tar.gz - ``` - -## 升级 Meta Service - -1. 检查当前集群状态： - ```bash - databend-metactl status - ``` - 验证所有节点是否健康，并记录当前领导者（leader）节点。 - -2. 替换二进制文件： - ```bash - sudo cp bin/databend-meta /usr/bin/ - sudo cp bin/databend-metactl /usr/bin/ - sudo chmod +x /usr/bin/databend-meta - sudo chmod +x /usr/bin/databend-metactl - ``` - -3. 重启 Meta Service： - ```bash - sudo systemctl restart databend-meta - ``` - -4. 检查服务状态： - ```bash - # 检查服务运行状态 - sudo systemctl status databend-meta - ``` - -5. 验证升级结果： - ```bash - # 检查集群状态 - databend-metactl status - ``` - -## 问题排查 - -升级过程中如遇问题： - -1. 检查服务状态： - ```bash - sudo systemctl status databend-meta - ``` - -2. 查看日志获取详细错误信息： - ```bash - sudo journalctl -u databend-meta -f - ``` - -3. 升级失败时可回滚操作： - - 还原旧版二进制文件 - - 重启服务 - -## 后续步骤 - -成功升级 Meta Service 后，可执行以下操作： -- [扩展 Meta Service 节点](04-scale-metasrv.md)（适用于多节点部署） -- [升级 Query Service](07-upgrade-query.md)（按需执行） \ No newline at end of file diff --git a/docs/cn/guides/20-self-hosted/02-deployment/04-manually/07-upgrade-query.md b/docs/cn/guides/20-self-hosted/02-deployment/04-manually/07-upgrade-query.md deleted file mode 100644 index 98a72bd16b..0000000000 --- a/docs/cn/guides/20-self-hosted/02-deployment/04-manually/07-upgrade-query.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -title: 升级查询服务（Query Service） ---- - -## 概述 - -本指南将引导你完成将 Databend 查询服务（Query Service）升级到新版本的过程。升级过程包括下载新版本发布包、替换二进制文件以及重启服务。 - -## 下载新版本 - -1. 按照 [准备软件包环境](01-prepare.md) 中的说明下载新版本发布包。 - -2. 解压软件包： - ```bash - tar xzf databend-v1.2.755-nightly-x86_64-unknown-linux-gnu.tar.gz - ``` - -## 升级查询服务（Query Service） - -1. 替换二进制文件： - ```bash - sudo cp bin/databend-query /usr/bin/ - sudo cp bin/bendsql /usr/bin/ - sudo chmod +x /usr/bin/databend-query - sudo chmod +x /usr/bin/bendsql - ``` - -2. 重启查询服务（Query Service）： - ```bash - sudo systemctl restart databend-query - ``` - -3. 检查服务状态： - ```bash - sudo systemctl status databend-query - ``` - -4. 验证升级： - ```bash - bendsql -h 127.0.0.1 -u root - ``` - - 连接后即可检查版本： - ```sql - SELECT version(); - ``` - -## 问题排查 - -升级过程中若遇到问题： - -1. 检查服务状态： - ```bash - sudo systemctl status databend-query - ``` - -2. 查看日志获取详细错误信息： - ```bash - sudo journalctl -u databend-query -f - ``` - -3. 升级失败时可通过以下方式回滚： - - 恢复原有二进制文件 - - 重启服务 - -## 后续步骤 - -成功升级查询服务（Query Service）后，可执行以下操作： -- [扩展查询服务节点](05-scale-query.md)（适用于多节点部署） -- [升级元数据服务（Meta Service）](06-upgrade-metasrv.md)（如需要） \ No newline at end of file diff --git a/docs/cn/guides/20-self-hosted/02-deployment/04-manually/_category_.json b/docs/cn/guides/20-self-hosted/02-deployment/04-manually/_category_.json deleted file mode 100644 index 33b77b20e3..0000000000 --- a/docs/cn/guides/20-self-hosted/02-deployment/04-manually/_category_.json +++ /dev/null @@ -1,3 +0,0 @@ -{ - "label": "手动部署" -} \ No newline at end of file diff --git a/docs/cn/guides/20-self-hosted/02-deployment/04-manually/index.md b/docs/cn/guides/20-self-hosted/02-deployment/04-manually/index.md deleted file mode 100644 index db1b2fd321..0000000000 --- a/docs/cn/guides/20-self-hosted/02-deployment/04-manually/index.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -title: 手动部署与维护 - ---- - -## 概述 - -在本指南中，你将学习如何使用从 GitHub releases 获取的预构建软件包手动部署 Databend。本分步指南将引导你完成使用 systemd 进行服务管理来建立和管理 Databend 集群的全过程。部署过程包括： - -1. 从 GitHub releases 下载并解压 Databend 软件包 -2. 使用 systemd 配置并启动元数据服务（Meta Service）节点 -3. 使用 systemd 配置并启动查询服务（Query Service）节点 -4. 管理集群扩缩容操作： - - 添加或移除元数据服务（Meta Service）节点 - - 添加或移除查询服务（Query Service）节点 -5. 执行版本升级： - - 升级元数据服务（Meta Service）节点 - - 升级查询服务（Query Service）节点 - -这种手动部署方法让你能够完全控制 Databend 集群的配置，适用于不方便或不倾向于使用自动化部署工具的环境。每个步骤都附有详细的说明和配置示例，以确保部署过程顺利进行。 - -## 目录 - -- [准备软件包环境](01-prepare.md) -- [部署元数据服务（Meta Service）](02-deploy-metasrv.md) -- [部署查询服务（Query Service）](03-deploy-query.md) -- [扩缩容元数据服务（Meta Service）节点](04-scale-metasrv.md) -- [扩缩容查询服务（Query Service）节点](05-scale-query.md) -- [升级元数据服务（Meta Service）](06-upgrade-metasrv.md) -- [升级查询服务（Query Service）](07-upgrade-query.md) \ No newline at end of file diff --git a/docs/cn/guides/20-self-hosted/02-deployment/00-understanding-deployment-modes.md b/docs/cn/guides/20-self-hosted/02-deployment/05-understanding-deployment-modes.md similarity index 97% rename from docs/cn/guides/20-self-hosted/02-deployment/00-understanding-deployment-modes.md rename to docs/cn/guides/20-self-hosted/02-deployment/05-understanding-deployment-modes.md index 75a0291fec..ee34205483 100644 --- a/docs/cn/guides/20-self-hosted/02-deployment/00-understanding-deployment-modes.md +++ b/docs/cn/guides/20-self-hosted/02-deployment/05-understanding-deployment-modes.md @@ -28,8 +28,8 @@ Databend 部署提供两种模式：单机（Standalone）和集群（Cluster） 在 Databend 单机部署中，可以将 Meta 节点和 Query 节点都托管在同一台服务器上。文档中的以下主题可帮助您设置和部署单机版 Databend： -- [部署单机版 Databend](01-non-production/01-deploying-databend.md) -- [本地和 Docker 部署](01-non-production/00-deploying-local.md) +- [部署单机版 Databend](03-deploying-databend.md) +- [本地和 Docker 部署](01-deploying-local.md) ### 集群部署 diff --git a/docs/cn/guides/35-connect/02-visualization/tableau.md b/docs/cn/guides/35-connect/02-visualization/tableau.md index 002c87cf38..913bd6ab2e 100644 --- a/docs/cn/guides/35-connect/02-visualization/tableau.md +++ b/docs/cn/guides/35-connect/02-visualization/tableau.md @@ -15,7 +15,7 @@ Databend 目前提供两种与 Tableau 的集成方式。第一种方法通过 T ### 步骤 1. 部署 Databend -1. 参照[本地与 Docker 部署](../../20-self-hosted/02-deployment/01-non-production/00-deploying-local.md)指南部署本地 Databend。 +1. 参照[本地与 Docker 部署](../../20-self-hosted/02-deployment/01-deploying-local.md)指南部署本地 Databend。 2. 在 Databend 中创建 SQL 用户，该账户将用于在 Tableau Desktop 中连接 Databend。 ```sql @@ -63,7 +63,7 @@ GRANT ROLE tableau_role TO tableau; ### 步骤 1. 部署 Databend -1. 参照[本地与 Docker 部署](../../20-self-hosted/02-deployment/01-non-production/00-deploying-local.md)指南部署本地 Databend。 +1. 参照[本地与 Docker 部署](../../20-self-hosted/02-deployment/01-deploying-local.md)指南部署本地 Databend。 2. 在 Databend 中创建 SQL 用户，该账户将用于在 Tableau Desktop 中连接 Databend。 ```sql diff --git a/docs/cn/guides/40-load-data/02-load-db/airbyte.md b/docs/cn/guides/40-load-data/02-load-db/airbyte.md index 119bbdeb23..e6de0eec5b 100644 --- a/docs/cn/guides/40-load-data/02-load-db/airbyte.md +++ b/docs/cn/guides/40-load-data/02-load-db/airbyte.md @@ -26,7 +26,7 @@ PRESIGN UPLOAD @airbyte_stage/test.csv; ``` 如果你收到类似 `Code: 501, Text = Presign is not supported` 的错误，则表示你无法使用此集成。 -请阅读[此文档](../../20-self-hosted/02-deployment/01-non-production/00-deploying-local.md)了解如何使用 S3 作为存储后端。 +请阅读[此文档](../../20-self-hosted/02-deployment/01-deploying-local.md)了解如何使用 S3 作为存储后端。 ## 创建 Databend 用户 diff --git a/docs/cn/guides/40-load-data/02-load-db/vector.md b/docs/cn/guides/40-load-data/02-load-db/vector.md index f6b998beb0..9af043926c 100644 --- a/docs/cn/guides/40-load-data/02-load-db/vector.md +++ b/docs/cn/guides/40-load-data/02-load-db/vector.md @@ -61,7 +61,7 @@ password = "abc123" # Databend 密码 #### 1.1 安装 Databend -按照 [Docker 和本地部署](../../20-self-hosted/02-deployment/01-non-production/00-deploying-local.md) 指南部署本地 Databend，或在 Databend Cloud 中部署一个计算集群（Warehouse）。 +按照 [Docker 和本地部署](../../20-self-hosted/02-deployment/01-deploying-local.md) 指南部署本地 Databend，或在 Databend Cloud 中部署一个计算集群（Warehouse）。 #### 1.2 创建数据库和表 diff --git a/docs/cn/release-notes/databend.md b/docs/cn/release-notes/databend.md index f82082d9e9..1ac0d54b01 100644 --- a/docs/cn/release-notes/databend.md +++ b/docs/cn/release-notes/databend.md @@ -12,7 +12,45 @@ This page provides information about recent features, enhancements, and bug fixe - + + +## Mar 31, 2026 (v1.2.888-patch-3) + +**Full Changelog**: https://github.com/databendlabs/databend/releases/tag/v1.2.888-patch-3 + + + + + +## Mar 30, 2026 (v1.2.891-nightly) + +## What's Changed +### Exciting New Features ✨ +* feat(stage): add TEXT file format params by **@youngsofun** in [#19588](https://github.com/databendlabs/databend/pull/19588) +* feat(http): add server-side parameter binding to /v1/query by **@cliftonc** in [#19601](https://github.com/databendlabs/databend/pull/19601) +* feat: enable TCP_NODELAY on gRPC listener sockets by **@drmingdrmer** in [#19619](https://github.com/databendlabs/databend/pull/19619) +* feat(query): support experimental table tags for FUSE table snapshots by **@zhyass** in [#19549](https://github.com/databendlabs/databend/pull/19549) +### Thoughtful Bug Fix 🔧 +* fix(storage): split oversized compact blocks during recluster by **@zhyass** in [#19577](https://github.com/databendlabs/databend/pull/19577) +* fix(query): preserve parentheses in UNION queries by **@sundy-li** in [#19587](https://github.com/databendlabs/databend/pull/19587) +* fix(query): avoid create or alter recursive views by **@TCeason** in [#19584](https://github.com/databendlabs/databend/pull/19584) +* fix(query): escape LIKE ESCAPE literals in display by **@sundy-li** in [#19596](https://github.com/databendlabs/databend/pull/19596) +* fix: clarify AGENTS.md guidance for conflicts and patterns by **@forsaken628** in [#19617](https://github.com/databendlabs/databend/pull/19617) +* fix(query): enforce row access policy for Direct UPDATE and split predicate fields by **@TCeason** in [#19625](https://github.com/databendlabs/databend/pull/19625) +* fix: rename PanicError and fix executor OOM mapping by **@sundy-li** in [#19614](https://github.com/databendlabs/databend/pull/19614) +* fix: restore enable_merge_into_row_fetch by **@dqhl76** in [#19624](https://github.com/databendlabs/databend/pull/19624) +### Code Refactor 🎉 +* refactor(storage): extract fuse block format abstraction by **@SkyFan2002** in [#19576](https://github.com/databendlabs/databend/pull/19576) +* refactor(sql): separate aggregate registration and reuse in binder by **@forsaken628** in [#19579](https://github.com/databendlabs/databend/pull/19579) + +## New Contributors +* **@cliftonc** made their first contribution in [#19601](https://github.com/databendlabs/databend/pull/19601) + +**Full Changelog**: https://github.com/databendlabs/databend/releases/tag/v1.2.891-nightly + + + + ## Mar 23, 2026 (v1.2.890-nightly) @@ -43,7 +81,7 @@ This page provides information about recent features, enhancements, and bug fixe - + ## Mar 16, 2026 (v1.2.889-nightly) @@ -673,53 +711,4 @@ This page provides information about recent features, enhancements, and bug fixe - - -## Dec 29, 2025 (v1.2.862-nightly) - -## What's Changed -### Exciting New Features ✨ -* feat: clustering_statistics support specify snapshot by **@zhyass** in [#19148](https://github.com/databendlabs/databend/pull/19148) -* feat(query): Enhanced Inverted Index for VARIANT Type to precise matching Object within Arrays by **@b41sh** in [#19096](https://github.com/databendlabs/databend/pull/19096) -### Thoughtful Bug Fix 🔧 -* fix: allow credential chain for system_history tables by **@dqhl76** in [#19167](https://github.com/databendlabs/databend/pull/19167) -### Code Refactor 🎉 -* refactor(query): use unchecked utf8 for payload flush by **@zhang2014** in [#19159](https://github.com/databendlabs/databend/pull/19159) -* refactor: prioritize table options for retention policy by **@dantengsky** in [#19162](https://github.com/databendlabs/databend/pull/19162) -* refactor(query): send runtime filter packets as data blocks by **@zhang2014** in [#19170](https://github.com/databendlabs/databend/pull/19170) -* refactor(query): streamline flight exchange coordination by **@zhang2014** in [#19175](https://github.com/databendlabs/databend/pull/19175) -### Build/Testing/CI Infra Changes 🔌 -* ci: upgrade llvm in build-tool image by **@everpcpc** in [#19166](https://github.com/databendlabs/databend/pull/19166) -* ci: install missing packages by **@everpcpc** in [#19171](https://github.com/databendlabs/databend/pull/19171) -* ci: enable unit test by **@everpcpc** in [#19172](https://github.com/databendlabs/databend/pull/19172) -* ci: update unit test. by **@youngsofun** in [#19168](https://github.com/databendlabs/databend/pull/19168) -### Others 📒 -* chore: ndv upper cutoff by **@forsaken628** in [#19133](https://github.com/databendlabs/databend/pull/19133) -* chore(query): add stats for new final aggregators by **@dqhl76** in [#19156](https://github.com/databendlabs/databend/pull/19156) -* chore(query): add probe rows log for hash join by **@zhang2014** in [#19165](https://github.com/databendlabs/databend/pull/19165) -* chore: Date/Time elements are used in AggregateDistinct in numerical form, reducing the memory usage of the Set. by **@KKould** in [#19157](https://github.com/databendlabs/databend/pull/19157) - - -**Full Changelog**: https://github.com/databendlabs/databend/releases/tag/v1.2.862-nightly - - - - - -## Dec 25, 2025 (v1.2.861-nightly) - -## What's Changed -### Exciting New Features ✨ -* feat(storage): organize storage credential configs for security by **@BohuTANG** in [#19147](https://github.com/databendlabs/databend/pull/19147) -### Thoughtful Bug Fix 🔧 -* fix(query): fix fragment not found in warehouse level table by **@zhang2014** in [#19152](https://github.com/databendlabs/databend/pull/19152) -* fix: recluster final infinite loop by **@zhyass** in [#19151](https://github.com/databendlabs/databend/pull/19151) -### Others 📒 -* chore(query): revert "fix(query): update opendal (#19110)" by **@zhang2014** in [#19146](https://github.com/databendlabs/databend/pull/19146) - - -**Full Changelog**: https://github.com/databendlabs/databend/releases/tag/v1.2.861-nightly - - - diff --git a/docs/cn/tutorials/develop/python/integrating-with-self-hosted-databend.md b/docs/cn/tutorials/develop/python/integrating-with-self-hosted-databend.md index 894f7a122d..8ff7591e54 100644 --- a/docs/cn/tutorials/develop/python/integrating-with-self-hosted-databend.md +++ b/docs/cn/tutorials/develop/python/integrating-with-self-hosted-databend.md @@ -7,7 +7,7 @@ sidebar_label: "Python Driver 开发 (自建)" ## 开始之前 -请确认已成功安装本地 Databend，详见 [本地与 Docker 部署](/guides/self-hosted/deployment/non-production/deploying-local)。 +请确认已成功安装本地 Databend，详见 [本地与 Docker 部署](/guides/self-hosted/deployment/deploying-local)。 ## 步骤 1：准备 SQL 账号 diff --git a/docs/en/guides/20-self-hosted/00-editions/community.md b/docs/en/guides/20-self-hosted/00-editions/community.md index 88e845bde5..d6274aafd9 100644 --- a/docs/en/guides/20-self-hosted/00-editions/community.md +++ b/docs/en/guides/20-self-hosted/00-editions/community.md @@ -27,5 +27,5 @@ See the complete list in [Enterprise Features](/guides/self-hosted/editions/ente ## Getting Started - **[Quick Start Guide](/guides/self-hosted/quickstart)**: Get up and running in 5 minutes -- **[Download & Install](/guides/self-hosted/deployment/download)**: Self-hosted deployment options +- **[Deploy with Docker](/guides/self-hosted/deployment/deploying-local)**: Self-hosted deployment options - **[GitHub Repository](https://github.com/databendlabs/databend)**: Source code and community diff --git a/docs/en/guides/20-self-hosted/01-quickstart/index.md b/docs/en/guides/20-self-hosted/01-quickstart/index.md index 1ac5514977..34243fac06 100644 --- a/docs/en/guides/20-self-hosted/01-quickstart/index.md +++ b/docs/en/guides/20-self-hosted/01-quickstart/index.md @@ -2,13 +2,11 @@ title: QuickStart --- -Databend Quick Start: Experience Databend in 5 Minutes -This guide will help you quickly set up Databend, connect to it, and perform a basic data import. +Get Databend running locally in under 5 minutes. -## 1. Start Databend with Docker -Run the following command to launch Databend in a container: +## 1. Start Databend -``` +```shell docker run -d \ --name databend \ --network host \ @@ -19,125 +17,41 @@ docker run -d \ --restart unless-stopped \ datafuselabs/databend ``` -Check if Databend is running successfully: -``` +Wait for Databend to be ready: + +```shell docker logs -f databend ``` -Wait until you see logs indicating that Databend and MinIO are ready. -## 2. Connect to Databend -Install bendsql (Databend CLI): +Look for `Databend Query started` in the output. -``` +## 2. Connect + +Install BendSQL: + +```shell curl -fsSL https://repo.databend.com/install/bendsql.sh | bash -echo "export PATH=$PATH:~/.bendsql/bin" >>~/.bash_profile +echo "export PATH=$PATH:~/.bendsql/bin" >> ~/.bash_profile source ~/.bash_profile ``` Connect to Databend: -``` -bendsql -udatabend -pdatabend -``` - -## 3. Perform a Basic Data Import -### Step 1: Create an External Bucket (myupload) -Install mc (MinIO client) and create a bucket: - -``` -wget https://dl.min.io/client/mc/release/linux-amd64/mc -sudo cp mc /usr/local/bin/ && sudo chmod +x /usr/local/bin/mc -mc alias set myminio http://localhost:9000 minioadmin minioadmin -mc mb myminio/myupload -mc ls myminio -``` -Expected output: -``` -[0001-01-01 08:05:43 LMT] 0B databend/ -[2025-04-12 08:43:59 CST] 0B myupload/ -``` -### Step 2: Generate a CSV and Upload to myupload +```shell +bendsql -u databend -p databend ``` -echo -e "id,name,age,city\n1,John,32,New York\n2,Emma,28,London\n3,Liam,35,Paris\n4,Olivia,40,Berlin\n5,Noah,29,Tokyo" > data.csv -mc cp data.csv myminio/myupload/ -mc ls myminio/myupload/ -``` -### Step 3: Create an External Stage and Check Data -Run in bendsql: -``` -CREATE STAGE mystage 's3://myupload' -CONNECTION=( - endpoint_url='http://127.0.0.1:9000', - access_key_id='minioadmin', - secret_access_key='minioadmin', - region='us-east-1' -); -``` -Show files in the external stage @mystage: -``` -LIST @mystage; -``` -| name | size | md5 | last_modified | creator | -|----------|--------|-------------------|-----------------------|-------------| -| String | UInt64 | Nullable(String) | String | Nullable(String) | -| data.csv | 104 | "a27fa15258911f534fb795a8c64e05d4" | 2025-04-12 00:51:11.015 +0000 | NULL | - -Preview the CSV data: -``` -SELECT $1, $2, $3, $4 FROM @mystage/data.csv (FILE_FORMAT=>'CSV') LIMIT 10; -``` -| \$1 | \$2 | \$3 | \$4 | -|-------------------|-------------------|-------------------|-------------------| -| Nullable(String) | Nullable(String) | Nullable(String) | Nullable(String) | -| id | name | age | city | -| 1 | John | 32 | New York | -| 2 | Emma | 28 | London | -| 3 | Liam | 35 | Paris | -| 4 | Olivia | 40 | Berlin | -| 5 | Noah | 29 | Tokyo | - - -### Step 4: Create a Table and Load Data -``` -CREATE DATABASE wubx; -USE wubx; - -CREATE TABLE t_person ( - id INT, - name VARCHAR, - age INT UNSIGNED, - city VARCHAR -); -COPY INTO t_person FROM @mystage PATTERN='.*[.]csv' FILE_FORMAT=(TYPE=CSV, SKIP_HEADER=1); +## 3. Verify +```sql +CREATE TABLE t (id INT, name VARCHAR); +INSERT INTO t VALUES (1, 'Databend'); +SELECT * FROM t; ``` -| File | Rows_loaded | Errors_seen | First_error | First_error_line | -|-----------|-------------|-------------|------------------|------------------| -| String | Int32 | Int32 | Nullable(String) | Nullable(Int32) | -| data.csv | 5 | 0 | NULL | NULL | - -### Step 5: Query the Data -``` -SELECT * FROM t_person; -``` -| id | name | age | city | -|----------|----------|----------|----------| -| Nullable(Int32) | Nullable(String) | Nullable(UInt32) | Nullable(String) | -| 1 | John | 32 | New York | -| 2 | Emma | 28 | London | -| 3 | Liam | 35 | Paris | -| 4 | Olivia | 40 | Berlin | -| 5 | Noah | 29 | Tokyo | - -🚀 Now you’ve successfully imported data into Databend! +You're all set. For a more realistic setup with separate components, see [Deploying on Docker](../02-deployment/01-deploying-local.md). ## Alternative: Databend Cloud -If setting up a local environment is troublesome, you can try [Databend Cloud](https://www.databend.com) for a fully managed experience. - -> 💬 **Community Support** -> Have questions? Connect with our team: -> 💬 [Slack Discussion](https://link.databend.com/join-slack) +Skip the local setup and try [Databend Cloud](https://www.databend.com) for a fully managed experience. diff --git a/docs/en/guides/20-self-hosted/02-deployment/01-deploying-local.md b/docs/en/guides/20-self-hosted/02-deployment/01-deploying-local.md new file mode 100644 index 0000000000..25139e5a83 --- /dev/null +++ b/docs/en/guides/20-self-hosted/02-deployment/01-deploying-local.md @@ -0,0 +1,189 @@ +--- +title: Deploying on Docker +--- + +This guide walks you through deploying Databend using Docker Compose with separate **databend-meta**, **databend-query**, and **MinIO** containers — giving you a clear picture of how the components fit together. + +:::note Non-production use only +This setup is intended for development and local testing only. It is not suitable for production environments or performance benchmarking. +::: + +### Before You Start + +- [Docker](https://www.docker.com/) (v26 or later recommended) installed on your machine. +- [BendSQL](https://docs.databend.com/guides/connect/sql-clients/bendsql/#installing-bendsql) installed on your machine. + +### Step 1: Create docker-compose.yml + +Create a file named `docker-compose.yml` with the following content: + +```yaml +services: + minio: + image: minio/minio:latest + network_mode: "host" + environment: + MINIO_ACCESS_KEY: minioadmin + MINIO_SECRET_KEY: minioadmin + volumes: + - minio-data:/data + entrypoint: | + sh -c " + minio server --address :9000 /data & + until mc alias set myminio http://localhost:9000 minioadmin minioadmin; do + echo 'Waiting for MinIO...'; + sleep 1; + done; + if ! mc ls myminio/databend > /dev/null 2>&1; then + mc mb myminio/databend; + fi; + wait; + " + + databend-meta: + image: datafuselabs/databend-meta:latest + network_mode: "host" + depends_on: + - minio + volumes: + - databend-meta-data:/data/ + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:28101/v1/health"] + interval: 5s + timeout: 3s + retries: 10 + entrypoint: sh -c "/databend-meta \ + --log-file-level='warn' \ + --log-file-format='text' \ + --log-file-dir='/data/logs/' \ + --log-file-limit=24 \ + --admin-api-address='0.0.0.0:28101' \ + --grpc-api-address='0.0.0.0:9191' \ + --grpc-api-advertise-host='0.0.0.0' \ + --raft-listen-host='0.0.0.0' \ + --raft-api-port='28103' \ + --raft-dir='/data/metadata' \ + --id=1 \ + --single" + + databend-query: + image: datafuselabs/databend-query:latest + network_mode: "host" + depends_on: + databend-meta: + condition: service_healthy + volumes: + - query-logs:/var/log/databend/ + - query-config:/etc/databend/ + environment: + RUST_BACKTRACE: 1 + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:7070/metrics"] + interval: 10s + timeout: 5s + retries: 10 + entrypoint: | + sh -c " + cat <<'EOF' > /etc/databend/databend-query.toml + [query] + max_active_sessions = 256 + shutdown_wait_timeout_ms = 5000 + flight_api_address = '0.0.0.0:9090' + metric_api_address = '0.0.0.0:7070' + tenant_id = 'default' + cluster_id = 'c01' + + [[query.users]] + name = 'databend' + auth_type = 'double_sha1_password' + # password: databend + auth_string = '3081f32caef285c232d066033c89a96d542d09d7' + + [log] + [log.file] + level = 'WARN' + format = 'text' + dir = '/var/log/databend' + + [meta] + endpoints = ['0.0.0.0:9191'] + username = 'root' + password = 'root' + client_timeout_in_second = 30 + auto_sync_interval = 30 + + [storage] + type = 's3' + [storage.s3] + endpoint_url = 'http://127.0.0.1:9000' + region = 'us-east-1' + access_key_id = 'minioadmin' + secret_access_key = 'minioadmin' + bucket = 'databend' + EOF + exec /usr/bin/databend-query --config-file=/etc/databend/databend-query.toml + " + +volumes: + minio-data: + databend-meta-data: + query-logs: + query-config: +``` + +### Step 2: Start Databend + +```shell +docker compose up -d +``` + +Check that all three containers are running: + +```shell +docker compose ps +``` + +To follow the query logs: + +```shell +docker compose logs -f databend-query +``` + +Wait until you see `Databend Query started`. + +### Step 3: Connect to Databend + +```shell +bendsql -u databend -p databend +``` + +Run a quick check: + +```sql +SELECT NOW(); +``` + +### Stop Databend + +```shell +docker compose stop +``` + +To remove containers and volumes entirely: + +```shell +docker compose down -v +``` + +### Advanced Configuration + +To customize the configuration, you can mount your own config files instead of generating them in the entrypoint. Reference configs: + +- databend-meta: https://github.com/databendlabs/databend/blob/main/scripts/ci/deploy/config/databend-meta-node-1.toml +- databend-query: https://github.com/databendlabs/databend/blob/main/scripts/ci/deploy/config/databend-query-node-1.toml + +### Next Steps + +- [Load data with COPY INTO](/guides/load-data/load/s3) +- [Connect with MySQL client or JDBC](/guides/connect/) +- For production deployment, see [Deploying on Kubernetes](/guides/self-hosted/deployment/production/deploying-databend-on-kubernetes) diff --git a/docs/en/guides/20-self-hosted/02-deployment/01-download.md b/docs/en/guides/20-self-hosted/02-deployment/01-download.md deleted file mode 100644 index 8d2b982913..0000000000 --- a/docs/en/guides/20-self-hosted/02-deployment/01-download.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -title: Downloading Databend -sidebar_label: Downloading Databend ---- - -Databend offers you these options for downloading the installation packages: - -- [Manual download](#manual-download): You can download the installation package for your platform directly from the Databend website. -- [APT Package Manager](#apt-package-manager): You can use the APT package manager to download and install Databend on Ubuntu or Debian. -- [Docker](#docker): You can use Docker to download and run Databend in a containerized environment. - -## Manual Download - -The primary distribution packages for Databend are `.tar.gz` archives containing single executable files that you can download from the [Download](https://www.databend.com/download) page and extract them anywhere on your system. - -:::note -**Linux Generic (Arm, 64-bit)** is suitable for Linux distributions that use musl as the standard C library; **Linux Generic (x86, 64-bit)** is suitable for Linux distributions that use GNU C with a minimum version 2.29 of GLIBC. -::: - -## APT Package Manager - -Databend offers package repositories for Debian and Ubuntu systems, allowing you to install Databend using the apt install command or any other APT frontend. - -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - - - - -```shell -sudo curl -L -o /usr/share/keyrings/databend-keyring.gpg https://repo.databend.com/deb/databend.gpg -sudo curl -L -o /etc/apt/sources.list.d/databend.list https://repo.databend.com/deb/databend.list - -sudo apt update - -sudo apt install databend -``` - - - - - -:::note -Available platforms: - -- Ubuntu Jammy (22.04) and later -- Debian Bookworm(12) and later - -::: - -```shell -sudo curl -L -o /etc/apt/sources.list.d/databend.sources https://repo.databend.com/deb/databend.sources - -sudo apt update - -sudo apt install databend -``` - - - - -:::tip -To start Databend after the installation, run the following commands: - -```shell -sudo systemctl start databend-meta -sudo systemctl start databend-query -``` - -::: - -## Docker - -Databend provides these types of installation images on the Docker Hub: - -- [Databend All-in-One Docker Image](https://hub.docker.com/r/datafuselabs/databend): Built for local tests, CI, and so on. -- Separated images: Built for production environments, Kubernetes, and [Helm Charts](https://github.com/databendlabs/helm-charts). - - [databend-meta](https://hub.docker.com/r/datafuselabs/databend-meta) - - [databend-query](https://hub.docker.com/r/datafuselabs/databend-query) - -Click the links above for their detailed instructions. diff --git a/docs/en/guides/20-self-hosted/02-deployment/01-non-production/00-deploying-local.md b/docs/en/guides/20-self-hosted/02-deployment/01-non-production/00-deploying-local.md deleted file mode 100644 index acacb53345..0000000000 --- a/docs/en/guides/20-self-hosted/02-deployment/01-non-production/00-deploying-local.md +++ /dev/null @@ -1,118 +0,0 @@ ---- -title: Deploying on Docker ---- - - - -This guide walks you through deploying Databend with [MinIO](https://min.io/) using [Docker](https://www.docker.com/) for a fully containerized setup on your local machine. - -:::note non-production use only -The MinIO deployment covered in this guide is only suitable for development and demonstration. Due to the limited resources in a single-machine environment, it is not recommended for production environments or performance testing. -::: - -### Before You Start - -Before you start, ensure you have the following prerequisites in place: - -- Ensure that [Docker](https://www.docker.com/) is installed on your local machine. -- Ensure that BendSQL is installed on your machine. See [Installing BendSQL](/guides/connect/sql-clients/bendsql/#installing-bendsql) for instructions on how to install BendSQL using various package managers. - -### Deploy Databend - -1. Pull and run the Databend image as a container with the following command: - -:::note - -- When starting the Databend Docker container, you can specify the username and password using the environment variables `QUERY_DEFAULT_USER` and `QUERY_DEFAULT_PASSWORD`. If these variables are not provided, a default root user will be created without a password. -- The command below also creates a SQL user (`databend`/`databend`) which you will need to use to connect to Databend later. If you make changes to the SQL user at this point, ensure that you maintain consistency throughout the entire process. - ::: - -```shell -vim docker-compose.yml - -services: - minio: - image: docker.io/minio/minio - command: server /data - ports: - - "9000:9000" - environment: - - MINIO_ACCESS_KEY=MINIO_ADMIN - - MINIO_SECRET_KEY=MINIO_SECRET - volumes: - - ./data:/data - healthcheck: - test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"] - interval: 30s - timeout: 5s - retries: 3 - databend: - image: datafuselabs/databend - environment: - - QUERY_DEFAULT_USER=databend - - QUERY_DEFAULT_PASSWORD=databend - - QUERY_STORAGE_TYPE=s3 - - AWS_S3_ENDPOINT=http://minio:9000 - - AWS_S3_BUCKET=databend - - AWS_ACCESS_KEY_ID=MINIO_ADMIN - - AWS_SECRET_ACCESS_KEY=MINIO_SECRET - ports: - - "3307:3307" - - "8000:8000" - - "8080:8080" - depends_on: - minio: - condition: service_started -``` - -2. Start Databend - -```shell -docker compose up -``` - -### Connect to Databend - -Launch a terminal on your local machine, then run the following command to connect to Databend: - -```shell -bendsql -u databend -p databend -``` - -```shell -Welcome to BendSQL 0.24.1-f1f7de0(2024-12-04T12:31:18.526234000Z). -Connecting to localhost:8000 as user databend. -Connected to Databend Query v1.2.697-d40f88cc51(rust-1.85.0-nightly-2025-02-14T11:30:59.842308760Z) -Loaded 1411 auto complete keywords from server. -Started web server at 127.0.0.1:8080 -``` - -You're all set! Now, you can execute a simple query to verify the deployment: - -```sql -🐳 databend@default:) CREATE OR REPLACE TABLE students (uid Int16, name String, age Int16); -🐳 databend@default:) INSERT INTO students VALUES (8888, 'Alice', 50); - -╭─────────────────────────╮ -│ number of rows inserted │ -│ UInt64 │ -├─────────────────────────┤ -│ 1 │ -╰─────────────────────────╯ -1 row written in 0.059 sec. Processed 1 row, 19 B (16.95 rows/s, 322 B/s) - -🐳 databend@default:) SELECT * FROM students; - -╭──────────────────────────────────────────────────────╮ -│ uid │ name │ age │ -│ Nullable(Int16) │ Nullable(String) │ Nullable(Int16) │ -├─────────────────┼──────────────────┼─────────────────┤ -│ 8888 │ Alice │ 50 │ -╰──────────────────────────────────────────────────────╯ -1 row read in 0.008 sec. Processed 1 row, 28 B (125 rows/s, 3.42 KiB/s) -``` - - \ No newline at end of file diff --git a/docs/en/guides/20-self-hosted/02-deployment/01-non-production/_category_.json b/docs/en/guides/20-self-hosted/02-deployment/01-non-production/_category_.json deleted file mode 100644 index beb9659649..0000000000 --- a/docs/en/guides/20-self-hosted/02-deployment/01-non-production/_category_.json +++ /dev/null @@ -1,3 +0,0 @@ -{ - "label": "Non-Production Deployments" -} \ No newline at end of file diff --git a/docs/en/guides/20-self-hosted/02-deployment/01-non-production/index.md b/docs/en/guides/20-self-hosted/02-deployment/01-non-production/index.md deleted file mode 100644 index c73a6ca204..0000000000 --- a/docs/en/guides/20-self-hosted/02-deployment/01-non-production/index.md +++ /dev/null @@ -1,7 +0,0 @@ ---- -title: Non-Production Deployments ---- - -import IndexOverviewList from '@site/src/components/IndexOverviewList'; - - \ No newline at end of file diff --git a/docs/en/guides/20-self-hosted/02-deployment/02-deploying-offline.md b/docs/en/guides/20-self-hosted/02-deployment/02-deploying-offline.md new file mode 100644 index 0000000000..d9381fc4a1 --- /dev/null +++ b/docs/en/guides/20-self-hosted/02-deployment/02-deploying-offline.md @@ -0,0 +1,70 @@ +--- +title: Offline Deployment with Docker +--- + +This guide covers deploying Databend in air-gapped or restricted network environments where direct access to Docker Hub is unavailable. + +## Step 1: Download Images on a Connected Machine + +On a machine with internet access, pull and export the required images: + +```shell +docker pull datafuselabs/databend-meta:latest +docker pull datafuselabs/databend-query:latest +docker pull minio/minio:latest + +docker save datafuselabs/databend-meta:latest | gzip > databend-meta.tar.gz +docker save datafuselabs/databend-query:latest | gzip > databend-query.tar.gz +docker save minio/minio:latest | gzip > minio.tar.gz +``` + +To pin a specific version (recommended for production): + +```shell +VERSION=latest +docker pull datafuselabs/databend-meta:${VERSION} +docker pull datafuselabs/databend-query:${VERSION} +docker save datafuselabs/databend-meta:${VERSION} | gzip > databend-meta-${VERSION}.tar.gz +docker save datafuselabs/databend-query:${VERSION} | gzip > databend-query-${VERSION}.tar.gz +``` + +## Step 2: Transfer and Load Images + +Copy the `.tar.gz` files to the target machine, then load them: + +```shell +docker load -i databend-meta.tar.gz +docker load -i databend-query.tar.gz +docker load -i minio.tar.gz +``` + +Verify the images are available: + +```shell +docker images | grep -E "databend|minio" +``` + +## Step 3: Deploy + +Use the same `docker-compose.yml` from [Deploying on Docker](./01-deploying-local.md) — no changes needed since the images are now available locally. + +```shell +docker compose up -d +``` + +## Optional: Push to a Private Registry + +If your organization uses a private Docker registry (e.g., `registry.example.com`): + +```shell +VERSION=latest +REGISTRY=registry.example.com + +docker tag datafuselabs/databend-meta:${VERSION} ${REGISTRY}/databend-meta:${VERSION} +docker tag datafuselabs/databend-query:${VERSION} ${REGISTRY}/databend-query:${VERSION} + +docker push ${REGISTRY}/databend-meta:${VERSION} +docker push ${REGISTRY}/databend-query:${VERSION} +``` + +Then update the image references in your `docker-compose.yml` accordingly. diff --git a/docs/en/guides/20-self-hosted/02-deployment/02-production/10-metasrv-deploy.md b/docs/en/guides/20-self-hosted/02-deployment/02-production/10-metasrv-deploy.md index 884a385171..878bdb763b 100644 --- a/docs/en/guides/20-self-hosted/02-deployment/02-production/10-metasrv-deploy.md +++ b/docs/en/guides/20-self-hosted/02-deployment/02-production/10-metasrv-deploy.md @@ -9,7 +9,7 @@ import TabItem from '@theme/TabItem'; -Databend recommends deploying a cluster with a minimum of three meta nodes and one query node for production environments. To gain a better understanding of Databend cluster deployment, see [Understanding Databend Deployments](../00-understanding-deployment-modes.md), which will familiarize you with the concept. This topic aims to provide a practical guide for deploying a Databend cluster. +Databend recommends deploying a cluster with a minimum of three meta nodes and one query node for production environments. To gain a better understanding of Databend cluster deployment, see [Understanding Databend Deployments](../05-understanding-deployment-modes.md), which will familiarize you with the concept. This topic aims to provide a practical guide for deploying a Databend cluster. ## Before You Begin @@ -206,7 +206,7 @@ endpoints = ["172.16.125.128:9191","172.16.125.129:9191","172.16.125.130:9191"] -2. For each query node, you also need to configure the object storage and admin users in the file [databend-query.toml](https://github.com/databendlabs/databend/blob/main/scripts/distribution/configs/databend-query.toml). For detailed instructions, see [here](../01-non-production/01-deploying-databend.md#deploying-a-query-node). +2. For each query node, you also need to configure the object storage and admin users in the file [databend-query.toml](https://github.com/databendlabs/databend/blob/main/scripts/distribution/configs/databend-query.toml). For detailed instructions, see [here](../03-deploying-databend.md#deploying-a-query-node). 3. Run the following script on each query node to start them: diff --git a/docs/en/guides/20-self-hosted/02-deployment/03-benddeploy/01-installing-benddeploy.md b/docs/en/guides/20-self-hosted/02-deployment/02-production/90-benddeploy/01-installing-benddeploy.md similarity index 100% rename from docs/en/guides/20-self-hosted/02-deployment/03-benddeploy/01-installing-benddeploy.md rename to docs/en/guides/20-self-hosted/02-deployment/02-production/90-benddeploy/01-installing-benddeploy.md diff --git a/docs/en/guides/20-self-hosted/02-deployment/03-benddeploy/02-getting-started.md b/docs/en/guides/20-self-hosted/02-deployment/02-production/90-benddeploy/02-getting-started.md similarity index 100% rename from docs/en/guides/20-self-hosted/02-deployment/03-benddeploy/02-getting-started.md rename to docs/en/guides/20-self-hosted/02-deployment/02-production/90-benddeploy/02-getting-started.md diff --git a/docs/en/guides/20-self-hosted/02-deployment/03-benddeploy/_category_.json b/docs/en/guides/20-self-hosted/02-deployment/02-production/90-benddeploy/_category_.json similarity index 100% rename from docs/en/guides/20-self-hosted/02-deployment/03-benddeploy/_category_.json rename to docs/en/guides/20-self-hosted/02-deployment/02-production/90-benddeploy/_category_.json diff --git a/docs/en/guides/20-self-hosted/02-deployment/03-benddeploy/index.md b/docs/en/guides/20-self-hosted/02-deployment/02-production/90-benddeploy/index.md similarity index 100% rename from docs/en/guides/20-self-hosted/02-deployment/03-benddeploy/index.md rename to docs/en/guides/20-self-hosted/02-deployment/02-production/90-benddeploy/index.md diff --git a/docs/en/guides/20-self-hosted/02-deployment/02-production/index.md b/docs/en/guides/20-self-hosted/02-deployment/02-production/index.md index a2f3375c57..2ffa8797e3 100644 --- a/docs/en/guides/20-self-hosted/02-deployment/02-production/index.md +++ b/docs/en/guides/20-self-hosted/02-deployment/02-production/index.md @@ -16,3 +16,4 @@ This section provides comprehensive details on managing and configuring Databend - [Meta Service Status](60-metasrv-status.md) - [Meta Service CLI API](70-metasrv-cli-api.md) - [Meta Service gRPC Guide](16-metasrv-grpc.md) +- [BendDeploy](90-benddeploy/index.md) diff --git a/docs/en/guides/20-self-hosted/02-deployment/01-non-production/01-deploying-databend.md b/docs/en/guides/20-self-hosted/02-deployment/03-deploying-databend.md similarity index 100% rename from docs/en/guides/20-self-hosted/02-deployment/01-non-production/01-deploying-databend.md rename to docs/en/guides/20-self-hosted/02-deployment/03-deploying-databend.md diff --git a/docs/en/guides/20-self-hosted/02-deployment/04-manually/01-prepare.md b/docs/en/guides/20-self-hosted/02-deployment/04-manually/01-prepare.md deleted file mode 100644 index c7fb441495..0000000000 --- a/docs/en/guides/20-self-hosted/02-deployment/04-manually/01-prepare.md +++ /dev/null @@ -1,114 +0,0 @@ ---- -title: Prepare Package Environment ---- - -## Prerequisites - -- A Linux-based operating system -- `wget` or `curl` for downloading files -- `tar` for extracting the package -- `sudo` privileges for system-wide installation - -## Check System Architecture - -1. Check your system architecture: - - ```bash - uname -m - ``` - - The output will help you determine which package to download: - - - If the output is `x86_64`, download the x86_64 package - - If the output is `aarch64`, download the aarch64 package - -## Download the Package - -1. Visit the [Databend GitHub Releases](https://github.com/databendlabs/databend/releases) page. - -2. Choose the latest stable release version. For example, if you want to install version v1.2.755-nightly, you'll need to download: - - - `databend-v1.2.755-nightly-x86_64-unknown-linux-gnu.tar.gz` for x86_64 Linux systems - - `databend-v1.2.755-nightly-aarch64-unknown-linux-gnu.tar.gz` for aarch64 Linux systems - -3. Download the package using wget (replace `v1.2.755-nightly` with your desired version): - - ```bash - wget https://github.com/databendlabs/databend/releases/download/v1.2.755-nightly/databend-v1.2.755-nightly-x86_64-unknown-linux-gnu.tar.gz - ``` - - Or using curl (replace `v1.2.755-nightly` with your desired version): - - ```bash - curl -L -O https://github.com/databendlabs/databend/releases/download/v1.2.755-nightly/databend-v1.2.755-nightly-x86_64-unknown-linux-gnu.tar.gz - ``` - - Note: Make sure to replace `v1.2.755-nightly` in both the URL and filename with your desired version number. - -## Extract the Package - -1. Extract the package in the current directory: - ```bash - tar xzf databend-v1.2.755-nightly-x86_64-unknown-linux-gnu.tar.gz - ``` - -## Verify the Installation - -1. Check the extracted files: - - ```bash - ls --tree - ``` - - You should see the following directory structure: - - ``` - . - ├── bin - │ ├── bendsql - │ ├── databend-bendsave - │ ├── databend-meta - │ ├── databend-metactl - │ └── databend-query - ├── configs - │ ├── databend-meta.toml - │ └── databend-query.toml - ├── readme.txt - ├── scripts - │ ├── postinstall.sh - │ └── preinstall.sh - └── systemd - ├── databend-meta.default - ├── databend-meta.service - ├── databend-query.default - └── databend-query.service - ``` - -2. Verify the binaries are executable: - ```bash - ./bin/databend-meta --version - ./bin/databend-metactl --version - ./bin/databend-query --version - ./bin/bendsql --version - ``` - -## Create Databend User - -1. Run the preinstall script to create the databend user and group: - - ```bash - sudo ./scripts/preinstall.sh - ``` - - This script will: - - - Create a `databend` user and group if they don't exist - - Set up necessary system configurations - - Create required directories with proper permissions - -## Next Steps - -Now that you have prepared the environment, you can proceed to: - -- [Deploy Meta Service](02-deploy-metasrv.md) -- [Deploy Query Service](03-deploy-query.md) diff --git a/docs/en/guides/20-self-hosted/02-deployment/04-manually/02-deploy-metasrv.md b/docs/en/guides/20-self-hosted/02-deployment/04-manually/02-deploy-metasrv.md deleted file mode 100644 index a2cfc51c77..0000000000 --- a/docs/en/guides/20-self-hosted/02-deployment/04-manually/02-deploy-metasrv.md +++ /dev/null @@ -1,169 +0,0 @@ ---- -title: Deploy Meta Service ---- - -## Overview - -The Meta Service is a critical component of Databend that manages metadata and cluster coordination. This guide will walk you through the process of deploying a Meta Service node. - -## Prerequisites - -- Completed the [Prepare Package Environment](01-prepare.md) steps -- Have the extracted Databend package ready -- Have sudo privileges - -## Install Binary Files - -1. Copy the Meta Service binary to the system binary directory: - ```bash - sudo cp bin/databend-meta /usr/bin/ - sudo chmod +x /usr/bin/databend-meta - ``` - -2. Copy the Meta Service control tool binary: - ```bash - sudo cp bin/databend-metactl /usr/bin/ - sudo chmod +x /usr/bin/databend-metactl - ``` - -## Configure Meta Service - -1. Navigate to the extracted package directory and copy the default configuration: - ```bash - sudo mkdir -p /etc/databend - sudo cp configs/databend-meta.toml /etc/databend/databend-meta.toml - ``` - -2. Edit the configuration file: - ```bash - sudo vim /etc/databend/databend-meta.toml - ``` - - The default configuration looks like this: - ```toml - admin_api_address = "0.0.0.0:28002" - grpc_api_address = "0.0.0.0:9191" - grpc_api_advertise_host = "localhost" # change this - - [log] - [log.file] - level = "WARN" - format = "text" - dir = "/var/log/databend" - - [raft_config] - id = 0 # keep this as 0 for single-node deployment or first node in cluster - raft_dir = "/var/lib/databend/raft" - raft_api_port = 28004 - raft_listen_host = "0.0.0.0" - raft_advertise_host = "localhost" # change this - single = true # keep this as true for single-node deployment or first node in cluster - ``` - - Modify the following settings based on your environment: - - `grpc_api_advertise_host`: The hostname or IP address for gRPC communication, should be the same as the host name or IP address of the machine. - - `raft_advertise_host`: The hostname or IP address that other nodes will use to connect, should be the same as the host name or IP address of the machine. - -## Set Up Systemd Service - -1. Copy the systemd service file: - ```bash - sudo cp systemd/databend-meta.service /etc/systemd/system/ - ``` - -2. Copy the default environment file: - ```bash - sudo cp systemd/databend-meta.default /etc/default/databend-meta - ``` - -3. Edit the environment file (Optional): - ```bash - sudo vim /etc/default/databend-meta - ``` - - Set the following variables when needed (Optional): - ```bash - RUST_BACKTRACE=1 # enable backtrace for debugging - SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt # set the path to the CA certificate file if you are using custom CA certificate - ``` - -4. Reload systemd to recognize the new service: - ```bash - sudo systemctl daemon-reload - ``` - -5. Enable the service to start on boot: - ```bash - sudo systemctl enable databend-meta - ``` - -## Start Meta Service - -1. Start the Meta Service: - ```bash - sudo systemctl start databend-meta - ``` - -2. Check the service status: - ```bash - sudo systemctl status databend-meta - ``` - -3. View the logs: - ```bash - sudo journalctl -u databend-meta -f - ``` - -## Verify Meta Service - -1. Check if the Meta Service is listening on the configured ports: - ```bash - sudo netstat -tulpn | grep databend-meta - ``` - -2. Test the admin API endpoint: - ```bash - curl http://127.0.0.1:28002/v1/health - ``` - - You should receive a response indicating the service is healthy. - -3. Check the Meta Service status using metactl: - ```bash - databend-metactl status - ``` - - You should see the current status of the Meta Service, including: - - Node ID - - Raft status - - Leader information - - Cluster configuration - -## Troubleshooting - -If you encounter issues: - -1. Check the service status: - ```bash - sudo systemctl status databend-meta - ``` - -2. View the logs for detailed error messages: - ```bash - # View systemd logs - sudo journalctl -u databend-meta -f - - # View log files in /var/log/databend - sudo tail -f /var/log/databend/databend-meta-*.log - ``` - -3. Common issues and solutions: - - Permission denied: Ensure the databend user has proper permissions in previous steps - - Port already in use: Check if another service is using the configured ports - - Configuration errors: Verify the configuration file syntax and paths - -## Next Steps - -Now that you have deployed the Meta Service, you can proceed to: -- [Deploy Query Service](03-deploy-query.md) -- [Scale Meta Service Nodes](04-scale-metasrv.md) (for multi-node deployment) diff --git a/docs/en/guides/20-self-hosted/02-deployment/04-manually/03-deploy-query.md b/docs/en/guides/20-self-hosted/02-deployment/04-manually/03-deploy-query.md deleted file mode 100644 index 4d0ca4cee8..0000000000 --- a/docs/en/guides/20-self-hosted/02-deployment/04-manually/03-deploy-query.md +++ /dev/null @@ -1,208 +0,0 @@ ---- -title: Deploy Query Service ---- - -## Overview - -The Query Service is the main component of Databend that handles SQL queries and data processing. This guide will walk you through the process of deploying a Query Service node. - -## Prerequisites - -- Completed the [Prepare Package Environment](01-prepare.md) steps -- Have the extracted Databend package ready -- Have sudo privileges -- Have a running Meta Service node (completed [Deploy Meta Service](02-deploy-metasrv.md)) - -## Install Binary Files - -1. Copy the Query Service binary to the system binary directory: - ```bash - sudo cp bin/databend-query /usr/bin/ - sudo chmod +x /usr/bin/databend-query - ``` - -2. Copy the BendSQL client binary: - ```bash - sudo cp bin/bendsql /usr/bin/ - sudo chmod +x /usr/bin/bendsql - ``` - -## Configure Query Service - -1. Navigate to the extracted package directory and copy the default configuration: - ```bash - sudo mkdir -p /etc/databend - sudo cp configs/databend-query.toml /etc/databend/databend-query.toml - ``` - -2. Edit the configuration file: - ```bash - sudo vim /etc/databend/databend-query.toml - ``` - - The default configuration looks like this: - ```toml - [query] - max_active_sessions = 256 - shutdown_wait_timeout_ms = 5000 - flight_api_address = "0.0.0.0:9091" - admin_api_address = "0.0.0.0:8080" - metric_api_address = "0.0.0.0:7070" - mysql_handler_host = "0.0.0.0" - mysql_handler_port = 3307 - clickhouse_http_handler_host = "0.0.0.0" - clickhouse_http_handler_port = 8124 - http_handler_host = "0.0.0.0" - http_handler_port = 8000 - flight_sql_handler_host = "0.0.0.0" - flight_sql_handler_port = 8900 - tenant_id = "default" # change as needed (Optional) - cluster_id = "default" # change as needed (Optional) - - [[query.users]] # uncomment this block to enable default built-in user "root" - name = "root" - auth_type = "no_password" - - [log] - [log.file] - level = "WARN" - format = "text" - dir = "/var/log/databend" - - [meta] - endpoints = ["0.0.0.0:9191"] # change this to the address of all your Meta Service nodes - username = "root" - password = "root" - client_timeout_in_second = 10 - auto_sync_interval = 60 - - # (Important) Change this to your own storage configs - [storage] - # fs | s3 | azblob | gcs | oss | cos | hdfs | webhdfs - type = "fs" - - [storage.fs] - data_path = "/var/lib/databend/data" - - # To use an Amazon S3-like storage service - [storage.s3] - bucket = "" - endpoint_url = "" - access_key_id = "" - secret_access_key = "" - enable_virtual_host_style = false - - [cache] - data_cache_storage = "none" # change this to "disk" if you want to enable local disk cache - - [cache.disk] - path = "/var/lib/databend/cache" - max_bytes = 21474836480 - ``` - - Modify the following settings based on your environment: - - `[meta].endpoints`: The addresses of your Meta Service nodes (format: ["host:port"]) - - `[storage]`: The storage type and configs to store your data. Each tenant should have its own storage configs (Important) - - `[query].users`: The users for Query Service authentication, you may comment this block later after you have created your own users (Optional) - - `[query].tenant_id`: The tenant ID for multi-tenant deployment (Optional) - - `[query].cluster_id`: The cluster ID for multi-cluster deployment (Optional) - - `[cache]`: The cache configs to use (Optional) - -## Set Up Systemd Service - -1. Copy the systemd service file: - ```bash - sudo cp systemd/databend-query.service /etc/systemd/system/ - ``` - -2. Copy the default environment file: - ```bash - sudo cp systemd/databend-query.default /etc/default/databend-query - ``` - -3. Edit the environment file (Optional): - ```bash - sudo vim /etc/default/databend-query - ``` - - Set the following variables when needed (Optional): - ```bash - RUST_BACKTRACE=1 # enable backtrace for debugging - SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt # set the path to the CA certificate file if you are using custom CA certificate - ``` - -4. Reload systemd to recognize the new service: - ```bash - sudo systemctl daemon-reload - ``` - -5. Enable the service to start on boot: - ```bash - sudo systemctl enable databend-query - ``` - -## Start Query Service - -1. Start the Query Service: - ```bash - sudo systemctl start databend-query - ``` - -2. Check the service status: - ```bash - sudo systemctl status databend-query - ``` - -3. View the logs: - ```bash - sudo journalctl -u databend-query -f - ``` - -## Verify Query Service - -1. Check if the Query Service is listening on the configured ports: - ```bash - sudo netstat -tulpn | grep databend-query - ``` - -2. Test the HTTP endpoint: - ```bash - curl http://127.0.0.1:8000/health - ``` - - You should receive a response indicating the service is healthy. - -3. Test with BendSQL: - ```bash - bendsql -h 127.0.0.1 -u root - ``` - -## Troubleshooting - -If you encounter issues: - -1. Check the service status: - ```bash - sudo systemctl status databend-query - ``` - -2. View the logs for detailed error messages: - ```bash - # View systemd logs - sudo journalctl -u databend-query -f - - # View log files in /var/log/databend - sudo tail -f /var/log/databend/databend-query-*.log - ``` - -3. Common issues and solutions: - - Permission denied: Ensure the databend user has proper permissions in previous steps - - Port already in use: Check if another service is using the configured ports - - Configuration errors: Verify the configuration file syntax and paths - - Meta Service connection issues: Ensure the Meta Service is running and accessible - -## Next Steps - -Now that you have deployed both Meta Service and Query Service, you can proceed to: -- [Scale Query Service Nodes](05-scale-query.md) (for multi-node deployment) -- [Upgrade Query Service](07-upgrade-query.md) diff --git a/docs/en/guides/20-self-hosted/02-deployment/04-manually/04-scale-metasrv.md b/docs/en/guides/20-self-hosted/02-deployment/04-manually/04-scale-metasrv.md deleted file mode 100644 index 0b46de9eb9..0000000000 --- a/docs/en/guides/20-self-hosted/02-deployment/04-manually/04-scale-metasrv.md +++ /dev/null @@ -1,171 +0,0 @@ ---- -title: Scale Meta Service Nodes ---- - -## Overview - -This guide will walk you through the process of scaling your Databend Meta Service cluster. You can either add new nodes (scale up) or remove existing nodes (scale down) from your cluster. - -## Prerequisites - -- Have completed [Deploy Meta Service](02-deploy-metasrv.md) and have a running Meta Service node -- Have completed [Prepare Package Environment](01-prepare.md) on the new node (for scale up) -- Have sudo privileges on the nodes - -## Scale Up: Add New Meta Service Node - -To add a new Meta Service node, follow the steps in [Deploy Meta Service](02-deploy-metasrv.md) on the new node. Make sure to: - -1. Copy the configuration from an existing node: - ```bash - sudo mkdir -p /etc/databend - sudo scp root@:/etc/databend/databend-meta.toml /etc/databend/ - ``` - -2. Modify the configuration file: - ```bash - sudo vim /etc/databend/databend-meta.toml - ``` - - Update the following settings: - ```toml - [raft_config] - id = 1 # Change this to a unique ID for each node (1, 2, 3, etc.) - single = false # Change this to false for multi-node deployment - join = ["127.0.0.1:28004"] # Add all existing Meta Service nodes here - ``` - -3. Follow the deployment steps in [Deploy Meta Service](02-deploy-metasrv.md) to install and start the service on the new node. - -4. Verify the new node is added to the cluster: - ```bash - databend-metactl status - ``` - -5. After confirming the new node is in the cluster, update the configuration on all existing nodes: - ```bash - sudo vim /etc/databend/databend-meta.toml - ``` - - On each existing node, update the following settings: - ```toml - [raft_config] - single = false # Change this to false - join = ["127.0.0.1:28004", "127.0.0.2:28004", "127.0.0.3:28004"] # Add all Meta Service nodes including the new one - ``` - -6. Update the Meta Service endpoints in all Query nodes' configuration: - ```bash - sudo vim /etc/databend/databend-query.toml - ``` - - Update the following settings in each Query node: - ```toml - [meta] - endpoints = ["127.0.0.1:9191", "127.0.0.2:9191", "127.0.0.3:9191"] # Add all Meta Service nodes including the new one - ``` - -## Scale Down: Remove Meta Service Node - -To remove a Meta Service node from the cluster, follow these steps: - -1. First, check if the node to be removed is the current leader: - ```bash - databend-metactl status - ``` - Look for the "leader" information in the output. If the node to be removed is the leader, you need to transfer leadership first. - -2. If the node is the leader, transfer leadership to another node: - ```bash - databend-metactl transfer-leader - ``` - -3. Verify the leadership transfer: - ```bash - databend-metactl status - ``` - Confirm that the leadership has been transferred to the target node. - -4. Now, gracefully remove the node from the cluster using the `databend-meta` command: - ```bash - databend-meta --leave-id --leave-via ... - ``` - - For example, to remove node with ID 1: - ```bash - databend-meta --leave-id 1 --leave-via 127.0.0.1:28004 - ``` - - Note: - - `--leave-id` specifies the ID of the node to remove - - `--leave-via` specifies the list of node advertise addresses to send the leave request to - - The command can be run from any node with `databend-meta` installed - - The node will be blocked from cluster interaction until the leave request is completed - -5. Check if the node has been successfully removed from the cluster: - ```bash - databend-metactl status - ``` - Verify that the node ID is no longer listed in the cluster members. - -6. After confirming the node is removed from the cluster, stop the service: - ```bash - sudo systemctl stop databend-meta - ``` - -7. Verify the cluster status from any remaining node: - ```bash - databend-metactl status - ``` - -8. After confirming the cluster is stable, update the configuration on all remaining Meta nodes: - ```bash - sudo vim /etc/databend/databend-meta.toml - ``` - - On each remaining node, update the following settings: - ```toml - [raft_config] - join = ["127.0.0.2:28004", "127.0.0.3:28004"] # Remove the leaving node from the list - ``` - -9. Update the Meta Service endpoints in all Query nodes' configuration: - ```bash - sudo vim /etc/databend/databend-query.toml - ``` - - Update the following settings in each Query node: - ```toml - [meta] - endpoints = ["127.0.0.2:9191", "127.0.0.3:9191"] # Remove the leaving node from the list - ``` - -## Troubleshooting - -If you encounter issues: - -1. Check the service status: - ```bash - sudo systemctl status databend-meta - ``` - -2. View the logs for detailed error messages: - ```bash - # View systemd logs - sudo journalctl -u databend-meta -f - - # View log files in /var/log/databend - sudo tail -f /var/log/databend/databend-meta-*.log - ``` - -3. Common issues and solutions: - - Permission denied: Ensure the databend user has proper permissions - - Port already in use: Check if another service is using the configured ports - - Configuration errors: Verify the configuration file syntax and paths - - Raft connection issues: Ensure all Meta Service nodes can communicate with each other - -## Next Steps - -After successfully scaling your Meta Service cluster, you can: -- [Scale Query Service Nodes](05-scale-query.md) (if needed) -- [Upgrade Meta Service](06-upgrade-metasrv.md) (if needed) diff --git a/docs/en/guides/20-self-hosted/02-deployment/04-manually/05-scale-query.md b/docs/en/guides/20-self-hosted/02-deployment/04-manually/05-scale-query.md deleted file mode 100644 index 7e1bde5278..0000000000 --- a/docs/en/guides/20-self-hosted/02-deployment/04-manually/05-scale-query.md +++ /dev/null @@ -1,78 +0,0 @@ ---- -title: Scale Query Service Nodes ---- - -## Overview - -This guide will walk you through the process of scaling your Databend Query Service cluster. You can either add new nodes (scale up) or remove existing nodes (scale down) from your cluster. - -## Prerequisites - -- Have completed [Deploy Query Service](03-deploy-query.md) and have a running Query Service node -- Have completed [Prepare Package Environment](01-prepare.md) on the new node (for scale up) -- Have sudo privileges on the nodes - -## Scale Up: Add New Query Service Node - -To add a new Query Service node, follow the steps in [Deploy Query Service](03-deploy-query.md) on the new node. Make sure to: - -1. Copy the configuration from an existing node: - ```bash - sudo mkdir -p /etc/databend - sudo scp root@:/etc/databend/databend-query.toml /etc/databend/ - ``` - -2. Follow the deployment steps in [Deploy Query Service](03-deploy-query.md) to install and start the service. - -3. After deployment, verify the new node is added to the cluster: - ```bash - bendsql -h 127.0.0.1 -u root - ``` - - ```sql - SELECT * FROM system.clusters; - ``` - -## Scale Down: Remove Query Service Node - -To remove a Query Service node from the cluster, simply stop the service on that node: - -```bash -sudo systemctl stop databend-query -``` - -The node will be automatically removed from the cluster. You can verify the cluster status from any remaining node: - -```bash -bendsql -h 127.0.0.1 -u root -``` - -```sql -SELECT * FROM system.clusters; -``` - -## Troubleshooting - -If you encounter issues: - -1. Check the service status: - ```bash - sudo systemctl status databend-query - ``` - -2. View the logs for detailed error messages: - ```bash - sudo journalctl -u databend-query -f - ``` - -3. Common issues and solutions: - - Permission denied: Ensure the databend user has proper permissions - - Port already in use: Check if another service is using the configured ports - - Configuration errors: Verify the configuration file syntax and paths - - Meta Service connection issues: Ensure the Meta Service is accessible from the new node - -## Next Steps - -After successfully scaling your Query Service cluster, you can: -- [Scale Meta Service Nodes](04-scale-metasrv.md) (if needed) -- [Upgrade Query Service](07-upgrade-query.md) (if needed) diff --git a/docs/en/guides/20-self-hosted/02-deployment/04-manually/06-upgrade-metasrv.md b/docs/en/guides/20-self-hosted/02-deployment/04-manually/06-upgrade-metasrv.md deleted file mode 100644 index 7952554549..0000000000 --- a/docs/en/guides/20-self-hosted/02-deployment/04-manually/06-upgrade-metasrv.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -title: Upgrade Meta Service ---- - -## Overview - -This guide will walk you through the process of upgrading your Databend Meta Service to a newer version. The upgrade process involves downloading the new release package, replacing the binary, and restarting the service. - -## Download New Release - -1. Follow the download instructions in [Prepare Package Environment](01-prepare.md) to download the new release package. - -2. Extract the package: - ```bash - tar xzf databend-v1.2.755-nightly-x86_64-unknown-linux-gnu.tar.gz - ``` - -## Upgrade Meta Service - -1. Check the current cluster status: - ```bash - databend-metactl status - ``` - Verify that all nodes are healthy and note the current leader node. - -2. Replace the binary files: - ```bash - sudo cp bin/databend-meta /usr/bin/ - sudo cp bin/databend-metactl /usr/bin/ - sudo chmod +x /usr/bin/databend-meta - sudo chmod +x /usr/bin/databend-metactl - ``` - -3. Restart the Meta Service: - ```bash - sudo systemctl restart databend-meta - ``` - -4. Check the service status: - ```bash - # Check the service is running - sudo systemctl status databend-meta - ``` - -5. Verify the upgrade: - ```bash - # Check the cluster status - databend-metactl status - ``` - -## Troubleshooting - -If you encounter issues during the upgrade: - -1. Check the service status: - ```bash - sudo systemctl status databend-meta - ``` - -2. View the logs for detailed error messages: - ```bash - sudo journalctl -u databend-meta -f - ``` - -3. If the upgrade fails, you can rollback by: - - Restoring the previous binary - - Restarting the service - -## Next Steps - -After successfully upgrading the Meta Service, you can: -- [Scale Meta Service Nodes](04-scale-metasrv.md) (for multi-node deployment) -- [Upgrade Query Service](07-upgrade-query.md) (if needed) diff --git a/docs/en/guides/20-self-hosted/02-deployment/04-manually/07-upgrade-query.md b/docs/en/guides/20-self-hosted/02-deployment/04-manually/07-upgrade-query.md deleted file mode 100644 index 13c738818a..0000000000 --- a/docs/en/guides/20-self-hosted/02-deployment/04-manually/07-upgrade-query.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -title: Upgrade Query Service ---- - -## Overview - -This guide will walk you through the process of upgrading your Databend Query Service to a newer version. The upgrade process involves downloading the new release package, replacing the binary, and restarting the service. - -## Download New Release - -1. Follow the download instructions in [Prepare Package Environment](01-prepare.md) to download the new release package. - -2. Extract the package: - ```bash - tar xzf databend-v1.2.755-nightly-x86_64-unknown-linux-gnu.tar.gz - ``` - -## Upgrade Query Service - -1. Replace the binary files: - ```bash - sudo cp bin/databend-query /usr/bin/ - sudo cp bin/bendsql /usr/bin/ - sudo chmod +x /usr/bin/databend-query - sudo chmod +x /usr/bin/bendsql - ``` - -2. Restart the Query Service: - ```bash - sudo systemctl restart databend-query - ``` - -3. Check the service status: - ```bash - sudo systemctl status databend-query - ``` - -4. Verify the upgrade: - ```bash - bendsql -h 127.0.0.1 -u root - ``` - - After connecting, you can check the version: - ```sql - SELECT version(); - ``` - -## Troubleshooting - -If you encounter issues during the upgrade: - -1. Check the service status: - ```bash - sudo systemctl status databend-query - ``` - -2. View the logs for detailed error messages: - ```bash - sudo journalctl -u databend-query -f - ``` - -3. If the upgrade fails, you can rollback by: - - Restoring the previous binary - - Restarting the service - -## Next Steps - -After successfully upgrading the Query Service, you can: -- [Scale Query Service Nodes](05-scale-query.md) (for multi-node deployment) -- [Upgrade Meta Service](06-upgrade-metasrv.md) (if needed) diff --git a/docs/en/guides/20-self-hosted/02-deployment/04-manually/_category_.json b/docs/en/guides/20-self-hosted/02-deployment/04-manually/_category_.json deleted file mode 100644 index 05e212306a..0000000000 --- a/docs/en/guides/20-self-hosted/02-deployment/04-manually/_category_.json +++ /dev/null @@ -1,3 +0,0 @@ -{ - "label": "Deploy and Maintain Manually" -} diff --git a/docs/en/guides/20-self-hosted/02-deployment/04-manually/index.md b/docs/en/guides/20-self-hosted/02-deployment/04-manually/index.md deleted file mode 100644 index a7a4d88e2b..0000000000 --- a/docs/en/guides/20-self-hosted/02-deployment/04-manually/index.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -title: Deploy and Maintain Manually - ---- - -## Overview - -In this guide, you will learn how to deploy Databend manually using the pre-built packages from GitHub releases. This step-by-step guide will walk you through the entire process of setting up and managing your Databend cluster using systemd for service management. The deployment process involves: - -1. Downloading and extracting the Databend package from GitHub releases -2. Configuring and starting the Meta Service nodes using systemd -3. Configuring and starting the Query Service nodes using systemd -4. Managing cluster scaling operations: - - Adding or removing Meta Service nodes - - Adding or removing Query Service nodes -5. Performing version upgrades: - - Upgrading Meta Service nodes - - Upgrading Query Service nodes - -This manual deployment approach gives you full control over your Databend cluster configuration and is suitable for environments where automated deployment tools are not available or preferred. Each step is carefully documented with detailed instructions and configuration examples to ensure a smooth deployment process. - -## Table of Contents - -- [Prepare Package Environment](01-prepare.md) -- [Deploy Meta Service](02-deploy-metasrv.md) -- [Deploy Query Service](03-deploy-query.md) -- [Scale Meta Service Nodes](04-scale-metasrv.md) -- [Scale Query Service Nodes](05-scale-query.md) -- [Upgrade Meta Service](06-upgrade-metasrv.md) -- [Upgrade Query Service](07-upgrade-query.md) diff --git a/docs/en/guides/20-self-hosted/02-deployment/00-understanding-deployment-modes.md b/docs/en/guides/20-self-hosted/02-deployment/05-understanding-deployment-modes.md similarity index 97% rename from docs/en/guides/20-self-hosted/02-deployment/00-understanding-deployment-modes.md rename to docs/en/guides/20-self-hosted/02-deployment/05-understanding-deployment-modes.md index 347db1aa09..bb3448192e 100644 --- a/docs/en/guides/20-self-hosted/02-deployment/00-understanding-deployment-modes.md +++ b/docs/en/guides/20-self-hosted/02-deployment/05-understanding-deployment-modes.md @@ -28,8 +28,8 @@ In standalone mode, a standard configuration consists of a single Meta node and In a Standalone Databend Deployment, it is possible to host both the Meta and Query nodes on a single server. The following topics in the documentation assist you in setting up and deploying a standalone Databend: -- [Deploying a Standalone Databend](01-non-production/01-deploying-databend.md) -- [Local and Docker Deployments](01-non-production/00-deploying-local.md) +- [Deploying a Standalone Databend](03-deploying-databend.md) +- [Local and Docker Deployments](01-deploying-local.md) ### Cluster Deployment diff --git a/docs/en/guides/35-connect/02-visualization/tableau.md b/docs/en/guides/35-connect/02-visualization/tableau.md index 95908d95cc..69d9d642ea 100644 --- a/docs/en/guides/35-connect/02-visualization/tableau.md +++ b/docs/en/guides/35-connect/02-visualization/tableau.md @@ -15,7 +15,7 @@ In this tutorial, you'll deploy and integrate a local Databend with [Tableau Des ### Step 1. Deploy Databend -1. Follow the [Local and Docker Deployments](../../20-self-hosted/02-deployment/01-non-production/00-deploying-local.md) guide to deploy a local Databend. +1. Follow the [Local and Docker Deployments](../../20-self-hosted/02-deployment/01-deploying-local.md) guide to deploy a local Databend. 2. Create a SQL user in Databend. You will use this account to connect to Databend in Tableau Desktop. ```sql @@ -63,7 +63,7 @@ In this tutorial, you'll deploy and integrate a local Databend with [Tableau Des ### Step 1. Deploy Databend -1. Follow the [Local and Docker Deployments](../../20-self-hosted/02-deployment/01-non-production/00-deploying-local.md) guide to deploy a local Databend. +1. Follow the [Local and Docker Deployments](../../20-self-hosted/02-deployment/01-deploying-local.md) guide to deploy a local Databend. 2. Create a SQL user in Databend. You will use this account to connect to Databend in Tableau Desktop. ```sql diff --git a/docs/en/guides/40-load-data/02-load-db/airbyte.md b/docs/en/guides/40-load-data/02-load-db/airbyte.md index 49ed70cfce..98678d163f 100644 --- a/docs/en/guides/40-load-data/02-load-db/airbyte.md +++ b/docs/en/guides/40-load-data/02-load-db/airbyte.md @@ -26,7 +26,7 @@ PRESIGN UPLOAD @airbyte_stage/test.csv; ``` If you got an error like `Code: 501, Text = Presign is not supported`, then you could not use the integration. -Please read [this](../../20-self-hosted/02-deployment/01-non-production/00-deploying-local.md) for how to use S3 as a storage backend. +Please read [this](../../20-self-hosted/02-deployment/01-deploying-local.md) for how to use S3 as a storage backend. ## Create a Databend User diff --git a/docs/en/guides/40-load-data/02-load-db/vector.md b/docs/en/guides/40-load-data/02-load-db/vector.md index b5a941781a..866db8cbcc 100644 --- a/docs/en/guides/40-load-data/02-load-db/vector.md +++ b/docs/en/guides/40-load-data/02-load-db/vector.md @@ -61,7 +61,7 @@ password = "abc123" #Databend password #### 1.1 Install Databend -Follow the [Docker and Local Deployments](../../20-self-hosted/02-deployment/01-non-production/00-deploying-local.md) guide to deploy a local Databend, or deploy a warehouse in the Databend Cloud. +Follow the [Docker and Local Deployments](../../20-self-hosted/02-deployment/01-deploying-local.md) guide to deploy a local Databend, or deploy a warehouse in the Databend Cloud. #### 1.2 Create a Database and a Table diff --git a/docs/en/release-notes/databend.md b/docs/en/release-notes/databend.md index f82082d9e9..1ac0d54b01 100644 --- a/docs/en/release-notes/databend.md +++ b/docs/en/release-notes/databend.md @@ -12,7 +12,45 @@ This page provides information about recent features, enhancements, and bug fixe - + + +## Mar 31, 2026 (v1.2.888-patch-3) + +**Full Changelog**: https://github.com/databendlabs/databend/releases/tag/v1.2.888-patch-3 + + + + + +## Mar 30, 2026 (v1.2.891-nightly) + +## What's Changed +### Exciting New Features ✨ +* feat(stage): add TEXT file format params by **@youngsofun** in [#19588](https://github.com/databendlabs/databend/pull/19588) +* feat(http): add server-side parameter binding to /v1/query by **@cliftonc** in [#19601](https://github.com/databendlabs/databend/pull/19601) +* feat: enable TCP_NODELAY on gRPC listener sockets by **@drmingdrmer** in [#19619](https://github.com/databendlabs/databend/pull/19619) +* feat(query): support experimental table tags for FUSE table snapshots by **@zhyass** in [#19549](https://github.com/databendlabs/databend/pull/19549) +### Thoughtful Bug Fix 🔧 +* fix(storage): split oversized compact blocks during recluster by **@zhyass** in [#19577](https://github.com/databendlabs/databend/pull/19577) +* fix(query): preserve parentheses in UNION queries by **@sundy-li** in [#19587](https://github.com/databendlabs/databend/pull/19587) +* fix(query): avoid create or alter recursive views by **@TCeason** in [#19584](https://github.com/databendlabs/databend/pull/19584) +* fix(query): escape LIKE ESCAPE literals in display by **@sundy-li** in [#19596](https://github.com/databendlabs/databend/pull/19596) +* fix: clarify AGENTS.md guidance for conflicts and patterns by **@forsaken628** in [#19617](https://github.com/databendlabs/databend/pull/19617) +* fix(query): enforce row access policy for Direct UPDATE and split predicate fields by **@TCeason** in [#19625](https://github.com/databendlabs/databend/pull/19625) +* fix: rename PanicError and fix executor OOM mapping by **@sundy-li** in [#19614](https://github.com/databendlabs/databend/pull/19614) +* fix: restore enable_merge_into_row_fetch by **@dqhl76** in [#19624](https://github.com/databendlabs/databend/pull/19624) +### Code Refactor 🎉 +* refactor(storage): extract fuse block format abstraction by **@SkyFan2002** in [#19576](https://github.com/databendlabs/databend/pull/19576) +* refactor(sql): separate aggregate registration and reuse in binder by **@forsaken628** in [#19579](https://github.com/databendlabs/databend/pull/19579) + +## New Contributors +* **@cliftonc** made their first contribution in [#19601](https://github.com/databendlabs/databend/pull/19601) + +**Full Changelog**: https://github.com/databendlabs/databend/releases/tag/v1.2.891-nightly + + + + ## Mar 23, 2026 (v1.2.890-nightly) @@ -43,7 +81,7 @@ This page provides information about recent features, enhancements, and bug fixe - + ## Mar 16, 2026 (v1.2.889-nightly) @@ -673,53 +711,4 @@ This page provides information about recent features, enhancements, and bug fixe - - -## Dec 29, 2025 (v1.2.862-nightly) - -## What's Changed -### Exciting New Features ✨ -* feat: clustering_statistics support specify snapshot by **@zhyass** in [#19148](https://github.com/databendlabs/databend/pull/19148) -* feat(query): Enhanced Inverted Index for VARIANT Type to precise matching Object within Arrays by **@b41sh** in [#19096](https://github.com/databendlabs/databend/pull/19096) -### Thoughtful Bug Fix 🔧 -* fix: allow credential chain for system_history tables by **@dqhl76** in [#19167](https://github.com/databendlabs/databend/pull/19167) -### Code Refactor 🎉 -* refactor(query): use unchecked utf8 for payload flush by **@zhang2014** in [#19159](https://github.com/databendlabs/databend/pull/19159) -* refactor: prioritize table options for retention policy by **@dantengsky** in [#19162](https://github.com/databendlabs/databend/pull/19162) -* refactor(query): send runtime filter packets as data blocks by **@zhang2014** in [#19170](https://github.com/databendlabs/databend/pull/19170) -* refactor(query): streamline flight exchange coordination by **@zhang2014** in [#19175](https://github.com/databendlabs/databend/pull/19175) -### Build/Testing/CI Infra Changes 🔌 -* ci: upgrade llvm in build-tool image by **@everpcpc** in [#19166](https://github.com/databendlabs/databend/pull/19166) -* ci: install missing packages by **@everpcpc** in [#19171](https://github.com/databendlabs/databend/pull/19171) -* ci: enable unit test by **@everpcpc** in [#19172](https://github.com/databendlabs/databend/pull/19172) -* ci: update unit test. by **@youngsofun** in [#19168](https://github.com/databendlabs/databend/pull/19168) -### Others 📒 -* chore: ndv upper cutoff by **@forsaken628** in [#19133](https://github.com/databendlabs/databend/pull/19133) -* chore(query): add stats for new final aggregators by **@dqhl76** in [#19156](https://github.com/databendlabs/databend/pull/19156) -* chore(query): add probe rows log for hash join by **@zhang2014** in [#19165](https://github.com/databendlabs/databend/pull/19165) -* chore: Date/Time elements are used in AggregateDistinct in numerical form, reducing the memory usage of the Set. by **@KKould** in [#19157](https://github.com/databendlabs/databend/pull/19157) - - -**Full Changelog**: https://github.com/databendlabs/databend/releases/tag/v1.2.862-nightly - - - - - -## Dec 25, 2025 (v1.2.861-nightly) - -## What's Changed -### Exciting New Features ✨ -* feat(storage): organize storage credential configs for security by **@BohuTANG** in [#19147](https://github.com/databendlabs/databend/pull/19147) -### Thoughtful Bug Fix 🔧 -* fix(query): fix fragment not found in warehouse level table by **@zhang2014** in [#19152](https://github.com/databendlabs/databend/pull/19152) -* fix: recluster final infinite loop by **@zhyass** in [#19151](https://github.com/databendlabs/databend/pull/19151) -### Others 📒 -* chore(query): revert "fix(query): update opendal (#19110)" by **@zhang2014** in [#19146](https://github.com/databendlabs/databend/pull/19146) - - -**Full Changelog**: https://github.com/databendlabs/databend/releases/tag/v1.2.861-nightly - - - diff --git a/docs/en/tutorials/develop/python/integrating-with-self-hosted-databend.md b/docs/en/tutorials/develop/python/integrating-with-self-hosted-databend.md index 0601a77f0d..ab7f9a2ae9 100644 --- a/docs/en/tutorials/develop/python/integrating-with-self-hosted-databend.md +++ b/docs/en/tutorials/develop/python/integrating-with-self-hosted-databend.md @@ -6,7 +6,7 @@ This tutorial demonstrates how to connect to a locally deployed Databend instanc ## Before You Start -Before you start, make sure you have successfully installed a local Databend. For detailed instructions, see [Local and Docker Deployments](/guides/self-hosted/deployment/non-production/deploying-local). +Before you start, make sure you have successfully installed a local Databend. For detailed instructions, see [Local and Docker Deployments](/guides/self-hosted/deployment/deploying-local). ## Step 1: Prepare a SQL User Account diff --git a/site-redirects.ts b/site-redirects.ts index 8fcbdeb71c..9736262227 100644 --- a/site-redirects.ts +++ b/site-redirects.ts @@ -1508,11 +1508,19 @@ const siteRedirects = [ }, { from: '/guides/deploy/deploy/non-production/deploying-databend', - to: '/guides/self-hosted/deployment/non-production/deploying-databend' + to: '/guides/self-hosted/deployment/deploying-databend' + }, + { + from: '/guides/self-hosted/deployment/non-production/deploying-databend', + to: '/guides/self-hosted/deployment/deploying-databend' }, { from: '/guides/deploy/deploy/non-production/deploying-local', - to: '/guides/self-hosted/deployment/non-production/deploying-local' + to: '/guides/self-hosted/deployment/deploying-local' + }, + { + from: '/guides/self-hosted/deployment/non-production/deploying-local', + to: '/guides/self-hosted/deployment/deploying-local' }, { from: '/guides/deploy/deploy/understanding-deployment-modes', @@ -1520,7 +1528,11 @@ const siteRedirects = [ }, { from: '/guides/deploy/deploy/download', - to: '/guides/self-hosted/deployment/download' + to: '/guides/self-hosted/deployment' + }, + { + from: '/guides/self-hosted/deployment/download', + to: '/guides/self-hosted/deployment' }, { from: '/guides/deploy/deploy', @@ -1532,23 +1544,43 @@ const siteRedirects = [ }, { from: '/guides/deploy/deploy/non-production', - to: '/guides/self-hosted/deployment/non-production' + to: '/guides/self-hosted/deployment' + }, + { + from: '/guides/self-hosted/deployment/non-production', + to: '/guides/self-hosted/deployment' }, { from: '/guides/deploy/deploy/benddeploy', - to: '/guides/self-hosted/deployment/benddeploy' + to: '/guides/self-hosted/deployment/production/benddeploy' + }, + { + from: '/guides/self-hosted/deployment/benddeploy', + to: '/guides/self-hosted/deployment/production/benddeploy' }, { from: '/guides/deploy/deploy/benddeploy/getting-started', - to: '/guides/self-hosted/deployment/benddeploy/getting-started' + to: '/guides/self-hosted/deployment/production/benddeploy/getting-started' + }, + { + from: '/guides/self-hosted/deployment/benddeploy/getting-started', + to: '/guides/self-hosted/deployment/production/benddeploy/getting-started' }, { from: '/guides/deploy/deploy/benddeploy/installing-benddeploy', - to: '/guides/self-hosted/deployment/benddeploy/installing-benddeploy' + to: '/guides/self-hosted/deployment/production/benddeploy/installing-benddeploy' + }, + { + from: '/guides/self-hosted/deployment/benddeploy/installing-benddeploy', + to: '/guides/self-hosted/deployment/production/benddeploy/installing-benddeploy' }, { from: '/guides/deploy/deploy/manually', - to: '/guides/self-hosted/deployment/manually' + to: '/guides/self-hosted/deployment' + }, + { + from: '/guides/self-hosted/deployment/manually', + to: '/guides/self-hosted/deployment' }, // Upgrade redirects {