diff --git a/documentation/configuration/cairo-engine.md b/documentation/configuration/cairo-engine.md index 7112f46b6..ce4d48b9e 100644 --- a/documentation/configuration/cairo-engine.md +++ b/documentation/configuration/cairo-engine.md @@ -844,6 +844,78 @@ every window function execution. Prevents stack overflow errors when evaluating complex nested SQL. The value is the approximate number of nested SELECT clauses allowed. +## Memory limits + +These limits cap how much native memory a single workload may allocate, so a +runaway query, materialized view refresh, or WAL apply batch is stopped at its +source before it can drive the whole server toward out-of-memory. Each workload +has its own independent limit: a runaway in one does not draw against another's +budget, nor against the process-wide RSS limit (`ram.usage.limit.bytes` / +`ram.usage.limit.percent`). + +All three default to `0`, which means unlimited, so behavior matches a server +without limits until you opt in. Set each limit as a byte count or a size with a +`K`, `M`, or `G` suffix, for example `512M` or `2G`. The limits are reloadable: +edit `server.conf` and call +[`reload_config()`](/docs/query/functions/meta/#reload_config), and the new value +applies to every workload that starts afterward, with no restart. A workload +already running keeps the limit it started with. + +When a workload exceeds its limit, QuestDB raises an out-of-memory error at the +allocation that crossed the line and aborts that workload, while unrelated +workloads keep running. A user query fails with the error, a materialized view +refresh is invalidated, and a WAL apply suspends the affected table. The message +names the workload so you can tell it apart from a process-wide breach: + +``` +query memory limit exceeded [workload=QUERY, queryId=..., limit=..., used=..., size=..., memoryTag=...] +``` + +:::note + +Coverage is best-effort. A limit constrains the allocation sites that grow with +data volume: hash and GROUP BY tables, sorts, joins, window partition buffers, +set operations, `LATEST BY`, SAMPLE BY fill, and Parquet decode buffers. Memory +that is structurally bounded, or that lives for the life of a process or session +(page-frame buffers, JIT buffers, table readers and writers, symbol tables, +connection buffers, memory-mapped pages), is accounted only against the global +RSS limit. Treat a workload limit as a guard against common runaway patterns, +not a hard ceiling on every allocation. + +::: + +Live usage and the effective limit for each running query are exposed by the +[`query_activity`](/docs/query/functions/meta/#query_activity) view through its +`memory_used` and `memory_limit` columns. QuestDB Enterprise can additionally set +a memory limit per user, group, or service account, which overrides this workload +limit for a principal's queries. See +[role-based access control](/docs/security/rbac/#memory-limits). + +### cairo.mat.view.refresh.memory.limit.bytes + +- **Default**: `0` +- **Reloadable**: yes + +Maximum native memory a single materialized view refresh may allocate. `0` +disables the limit. + +### cairo.query.memory.limit.bytes + +- **Default**: `0` +- **Reloadable**: yes + +Maximum native memory a single user SQL query may allocate. `0` disables the +limit. Subqueries and other nested work share the top-level query's budget +rather than each acquiring their own. + +### cairo.wal.apply.memory.limit.bytes + +- **Default**: `0` +- **Reloadable**: yes + +Maximum native memory a single WAL apply batch may allocate. `0` disables the +limit. + ## Batch operations ### cairo.create.as.select.retry.count diff --git a/documentation/configuration/materialized-views.md b/documentation/configuration/materialized-views.md index 49f73b12c..bf786894f 100644 --- a/documentation/configuration/materialized-views.md +++ b/documentation/configuration/materialized-views.md @@ -7,6 +7,9 @@ These settings control materialized view SQL support and the background refresh job. Materialized views can use dedicated worker threads or share the server's common pool. +To cap the native memory a single refresh may allocate, see +[`cairo.mat.view.refresh.memory.limit.bytes`](/docs/configuration/cairo-engine/#memory-limits). + ## cairo.mat.view.enabled - **Default**: `true` diff --git a/documentation/configuration/wal.md b/documentation/configuration/wal.md index 61c8d0749..4110797b1 100644 --- a/documentation/configuration/wal.md +++ b/documentation/configuration/wal.md @@ -7,6 +7,9 @@ These settings control the Write-Ahead Log (WAL) subsystem, including parallel apply threads, segment rollover, commit squashing, and cleanup of applied WAL files. +To cap the native memory a single WAL apply batch may allocate, see +[`cairo.wal.apply.memory.limit.bytes`](/docs/configuration/cairo-engine/#memory-limits). + ## cairo.wal.apply.parallel.sql.enabled - **Default**: `true` diff --git a/documentation/getting-started/capacity-planning.md b/documentation/getting-started/capacity-planning.md index 91a162c3c..856d47c77 100644 --- a/documentation/getting-started/capacity-planning.md +++ b/documentation/getting-started/capacity-planning.md @@ -175,6 +175,10 @@ For relatively small datasets i.e 4-40GB, and a read-heavy workload, performance can be improved by maximising use of the OS page cache. Users should consider increasing available RAM to improve the speed of read operations. +To bound how much native memory a single query, materialized view refresh, or +WAL apply may allocate, so one runaway workload cannot exhaust the server's RAM, +see [memory limits](/docs/configuration/cairo-engine/#memory-limits). + ### Memory page size configuration With frequent out-of-order (O3) writes over a large number of columns/tables, diff --git a/documentation/query/functions/meta.md b/documentation/query/functions/meta.md index 77ebe6ac8..1cf963a97 100644 --- a/documentation/query/functions/meta.md +++ b/documentation/query/functions/meta.md @@ -224,18 +224,27 @@ Returns metadata on running SQL queries, including columns such as: - query_start - timestamp of when query started - state_change - timestamp of latest query state change, such as a cancellation - state - state of running query, can be `active` or `cancelled` +- memory_used - native memory currently allocated by the query, in bytes. Only + the allocation sites covered by the + [per-query memory limit](/docs/configuration/cairo-engine/#memory-limits) are + counted +- memory_limit - effective native memory limit for the query, in bytes, or + `null` when the query runs unlimited. On QuestDB Enterprise a set principal + [memory limit](/docs/security/rbac/#memory-limits) overrides the configured + workload limit; the workload limit applies only when the principal has none - query - text of sql query **Examples:** ```questdb-sql -SELECT * FROM query_activity(); +SELECT query_id, username, state, memory_used, memory_limit, query +FROM query_activity(); ``` -| query_id | worker_id | worker_pool | username | query_start | state_change | state | query | -| -------- | --------- | ----------- | -------- | --------------------------- | --------------------------- | ------ | --------------------------------------------------------- | -| 62179 | 5 | shared | bob | 2024-01-09T10:03:05.557397Z | 2024-01-09T10:03:05.557397 | active | select \* from query_activity() | -| 57777 | 6 | shared | bob | 2024-01-09T08:58:55.988017Z | 2024-01-09T08:58:55.988017Z | active | SELECT symbol,approx_percentile(price, 50, 2) from trades | +| query_id | username | state | memory_used | memory_limit | query | +| -------- | -------- | ------ | ----------- | ------------ | ---------------------------------------------------------- | +| 62179 | bob | active | 262144 | null | SELECT \* FROM query_activity() | +| 57777 | bob | active | 8388608 | 536870912 | SELECT symbol, approx_percentile(price, 50, 2) FROM trades | ## reader_pool diff --git a/documentation/query/sql/acl/alter-group.md b/documentation/query/sql/acl/alter-group.md new file mode 100644 index 000000000..9e51d7c30 --- /dev/null +++ b/documentation/query/sql/acl/alter-group.md @@ -0,0 +1,76 @@ +--- +title: ALTER GROUP reference +sidebar_label: ALTER GROUP +description: + "ALTER GROUP SQL keywords reference documentation. Applies to RBAC in QuestDB + Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + RBAC provides fine-grained database permissions management. + + +`ALTER GROUP` modifies group settings. + +For full documentation of the Access Control List and Role-based Access Control, +see the [RBAC operations](/docs/security/rbac) page. + +--- + +## Syntax + +```questdb-sql title="Set or clear memory limit" +ALTER GROUP groupName SET MEMORY LIMIT { size | UNLIMITED }; +``` + +```questdb-sql title="Add or remove external alias" +ALTER GROUP groupName { WITH | DROP } EXTERNAL ALIAS externalAlias; +``` + +## Description + +- `ALTER GROUP groupName SET MEMORY LIMIT size` - caps the native memory that + queries of the group's members may allocate. `size` is a byte count or a size + with a `K`, `M`, or `G` suffix, such as `512M` or `2G`. +- `ALTER GROUP groupName SET MEMORY LIMIT UNLIMITED` - removes the limit. `SET + MEMORY LIMIT 0` does the same. +- `ALTER GROUP groupName WITH EXTERNAL ALIAS externalAlias` - maps an external + OIDC or LDAP group to this group. +- `ALTER GROUP groupName DROP EXTERNAL ALIAS externalAlias` - removes an external + group mapping. + +A group limit applies to a member only when that member has no limit of its own; +a user's own limit always takes priority. When several of a user's groups set a +limit, the most restrictive (smallest positive) one applies. A set limit +overrides the configured +[`cairo.query.memory.limit.bytes`](/docs/configuration/cairo-engine/#memory-limits) +workload limit and binds even when larger. Setting a group limit requires the +`SET MEMORY LIMIT` permission. See +[memory limits](/docs/security/rbac/#memory-limits) for how per-principal and +workload limits resolve. + +For external group mapping with OIDC or LDAP, see the +[OpenID Connect (OIDC) integration](/docs/security/oidc/) guide. + +## Examples + +### Set memory limit + +```questdb-sql +-- cap queries of the group's members at 2 GiB of native memory +ALTER GROUP analysts SET MEMORY LIMIT 2G; +-- remove the limit +ALTER GROUP analysts SET MEMORY LIMIT UNLIMITED; +``` + +The configured value can be verified with `SHOW GROUPS`, which reports it in the +`memory_limit` column. + +### Map an external group + +```questdb-sql +ALTER GROUP analysts WITH EXTERNAL ALIAS 'CN=Analysts,OU=Users,DC=example,DC=com'; +ALTER GROUP analysts DROP EXTERNAL ALIAS 'CN=Analysts,OU=Users,DC=example,DC=com'; +``` diff --git a/documentation/query/sql/acl/alter-service-account.md b/documentation/query/sql/acl/alter-service-account.md index a3de16f56..33c19a39a 100644 --- a/documentation/query/sql/acl/alter-service-account.md +++ b/documentation/query/sql/acl/alter-service-account.md @@ -39,6 +39,10 @@ ALTER SERVICE ACCOUNT serviceAccountName DROP TOKEN TYPE { JWK | REST [token] }; ``` +```questdb-sql title="Set or clear memory limit" +ALTER SERVICE ACCOUNT serviceAccountName SET MEMORY LIMIT { size | UNLIMITED }; +``` + ## Description - `ALTER SERVICE ACCOUNT serviceAccountName ENABLE` - enables service account. @@ -56,6 +60,20 @@ ALTER SERVICE ACCOUNT serviceAccountName DROP TOKEN TYPE adds REST token to the service account. - `ALTER USER serviceAccountName DROP TOKEN TYPE REST token` - removes REST token from the service account. +- `ALTER SERVICE ACCOUNT serviceAccountName SET MEMORY LIMIT size` - caps the + native memory the service account's queries may allocate. `size` is a byte + count or a size with a `K`, `M`, or `G` suffix, such as `512M` or `2G`. +- `ALTER SERVICE ACCOUNT serviceAccountName SET MEMORY LIMIT UNLIMITED` - removes + the limit. `SET MEMORY LIMIT 0` does the same. + +A user who assumes the service account runs under its memory limit. A set limit +overrides the configured +[`cairo.query.memory.limit.bytes`](/docs/configuration/cairo-engine/#memory-limits) +workload limit — binding even when larger — and the workload limit applies only +when the service account has none. Group limits are never merged into a service +account. Setting it requires the `SET MEMORY LIMIT` permission. See +[memory limits](/docs/security/rbac/#memory-limits) for how per-principal and +workload limits resolve. ## Examples @@ -165,3 +183,15 @@ SHOW SERVICE ACCOUNT client_app; | Password | true | | JWK Token | false | | REST Token | false | + +### Set memory limit + +```questdb-sql +-- cap the service account's queries at 1 GiB of native memory +ALTER SERVICE ACCOUNT ingest SET MEMORY LIMIT 1G; +-- remove the limit +ALTER SERVICE ACCOUNT ingest SET MEMORY LIMIT UNLIMITED; +``` + +The configured value can be verified with `SHOW SERVICE ACCOUNTS`, which reports +it in the `memory_limit` column. diff --git a/documentation/query/sql/acl/alter-user.md b/documentation/query/sql/acl/alter-user.md index b0823db42..73adab4be 100644 --- a/documentation/query/sql/acl/alter-user.md +++ b/documentation/query/sql/acl/alter-user.md @@ -17,6 +17,8 @@ import { EnterpriseNote } from "@site/src/components/EnterpriseNote" For full documentation of the Access Control List and Role-based Access Control, see the [RBAC operations](/docs/security/rbac) page. +--- + ## Syntax ```questdb-sql title="Enable / disable" @@ -37,6 +39,10 @@ ALTER USER userName DROP TOKEN TYPE { JWK | REST [token] }; ``` +```questdb-sql title="Set or clear memory limit" +ALTER USER userName SET MEMORY LIMIT { size | UNLIMITED }; +``` + ## Description - `ALTER USER username ENABLE` - enables user account. @@ -54,6 +60,19 @@ ALTER USER userName DROP TOKEN TYPE REST token to user account. - `ALTER USER username DROP TOKEN TYPE REST token` - removes REST token from user account. +- `ALTER USER username SET MEMORY LIMIT size` - caps the native memory the + user's queries may allocate. `size` is a byte count or a size with a `K`, `M`, + or `G` suffix, such as `512M` or `2G`. +- `ALTER USER username SET MEMORY LIMIT UNLIMITED` - removes the limit. `SET + MEMORY LIMIT 0` does the same. + +The limit applies to the user's queries on both the primary and replicas. A set +limit takes priority over the user's groups and overrides the configured +[`cairo.query.memory.limit.bytes`](/docs/configuration/cairo-engine/#memory-limits) +workload limit, binding even when larger. Setting it requires the +`SET MEMORY LIMIT` permission. See +[memory limits](/docs/security/rbac/#memory-limits) for how per-principal and +workload limits resolve. ## Examples @@ -160,3 +179,15 @@ SHOW USER john; | Password | true | | JWK Token | false | | REST Token | false | + +### Set memory limit + +```questdb-sql +-- cap the user's queries at 512 MiB of native memory +ALTER USER john SET MEMORY LIMIT 512M; +-- remove the limit +ALTER USER john SET MEMORY LIMIT UNLIMITED; +``` + +The configured value can be verified with `SHOW USERS`, which reports the +effective limit in the `memory_limit` column. diff --git a/documentation/query/sql/show.md b/documentation/query/sql/show.md index 241caecc5..1a91f4bff 100644 --- a/documentation/query/sql/show.md +++ b/documentation/query/sql/show.md @@ -284,10 +284,15 @@ SHOW USER john; SHOW USERS; ``` -| name | -| ----- | -| admin | -| john | +| name | enabled | memory_limit | +| ----- | ------- | ------------ | +| admin | true | null | +| john | true | 536870912 | + +The `memory_limit` column is reported in bytes (`536870912` is 512 MiB) and is +`null` when no limit applies. For a user it is the effective limit — its own +limit, or, when it has none, the most restrictive of its groups'. See +[memory limits](/docs/security/rbac/#memory-limits). ### SHOW GROUPS @@ -295,15 +300,20 @@ SHOW USERS; SHOW GROUPS; ``` -or +| name | external_alias | memory_limit | +| ---------- | -------------- | ------------ | +| management | null | null | + +Filtering by a user lists the groups that user belongs to, without the +`memory_limit` column: ```questdb-sql SHOW GROUPS john; ``` -| name | -| ---------- | -| management | +| name | external_alias | +| ---------- | -------------- | +| management | null | ### SHOW SERVICE ACCOUNT @@ -329,26 +339,30 @@ SHOW SERVICE ACCOUNT ilp_ingestion; SHOW SERVICE ACCOUNTS; ``` -| name | -| ---------- | -| management | -| svc1_admin | +| name | enabled | memory_limit | +| ---------- | ------- | ------------ | +| management | true | null | +| svc1_admin | true | null | + +Filtering by a user or group instead lists the service accounts that principal +can assume, with a `grant_option` column showing whether they may grant the +assumption to others: ```questdb-sql SHOW SERVICE ACCOUNTS john; ``` -| name | -| ---------- | -| svc1_admin | +| name | grant_option | +| ---------- | ------------ | +| svc1_admin | false | ```questdb-sql SHOW SERVICE ACCOUNTS admin_group; ``` -| name | -| ---------- | -| svc1_admin | +| name | grant_option | +| ---------- | ------------ | +| svc1_admin | false | ### SHOW PERMISSIONS FOR CURRENT USER diff --git a/documentation/security/rbac.md b/documentation/security/rbac.md index b82e3eff7..3b472bf5d 100644 --- a/documentation/security/rbac.md +++ b/documentation/security/rbac.md @@ -544,6 +544,75 @@ information without these permissions. ::: +## Memory limits {#memory-limits} + +QuestDB Enterprise can set the native memory a single principal's queries may +allocate, overriding the server-wide query memory limit for a specific user, +group, or service account. Use it to stop one tenant's runaway query from +exhausting memory shared with everyone else, or to grant a trusted principal more +headroom than the default. + +Set a limit with [`ALTER USER`](/docs/query/sql/acl/alter-user/), +[`ALTER GROUP`](/docs/query/sql/acl/alter-group/), or +[`ALTER SERVICE ACCOUNT`](/docs/query/sql/acl/alter-service-account/): + +```questdb-sql +ALTER USER johndoe SET MEMORY LIMIT 512M; +ALTER GROUP analysts SET MEMORY LIMIT 2G; +ALTER SERVICE ACCOUNT ingest SET MEMORY LIMIT 1G; +ALTER USER johndoe SET MEMORY LIMIT UNLIMITED; -- clear the override +``` + +The value is a byte count or a size with a `K`, `M`, or `G` suffix. Setting a +limit requires the `SET MEMORY LIMIT` permission, which is included in +`GRANT ALL` and held implicitly by database admins. + +### How limits resolve + +QuestDB resolves the limit for a query by strict precedence — it takes the first +level that is set, rather than the smallest across levels: + +1. **The principal's own limit.** For a user this is the user's own limit; for a + query that assumes a service account it is the service account's limit. +2. **A group limit** (users only). When a user has no own limit, it inherits the + most restrictive (smallest positive) limit among the groups it belongs to. + Service accounts never inherit group limits. +3. **The workload limit.** Otherwise the server-wide + [`cairo.query.memory.limit.bytes`](/docs/configuration/cairo-engine/#memory-limits) + applies. + +A value of `0` (or `UNLIMITED`) means "not set" at that level, so resolution +falls through to the next one. A more specific level, when set, fully overrides +the broader one and binds even when it is larger — so a per-user or per-group +override can raise a principal's ceiling above the workload limit, not only lower +it. Use a smaller override to tighten the cap for a noisy tenant, or a larger one +to lift it for a trusted principal. A user who assumes a service account takes on +the service account's limit. The cap applies to the principal's queries on both +the primary and replicas. + +Materialized view refresh and WAL apply run under internal contexts rather than a +principal, so they stay bounded only by their own +[workload limits](/docs/configuration/cairo-engine/#memory-limits) and never pick +up a per-principal limit. + +:::note + +A limit bounds a single query. Two concurrent queries by the same principal each +run under the full limit, so the principal's aggregate usage can exceed it. The +cap guards against one runaway workload, not total concurrent usage. + +::: + +### Inspecting limits + +- `SHOW USERS`, `SHOW GROUPS`, and `SHOW SERVICE ACCOUNTS` report the configured + limit in a `memory_limit` column. For a user this is the effective limit — its + own limit, or, when it has none, the most restrictive of its groups' — and is + `null` when no limit applies. +- [`query_activity`](/docs/query/functions/meta/#query_activity) exposes the + effective limit and live usage of each running query through its `memory_limit` + and `memory_used` columns. + ## Permissions reference {#permissions} Use `all_permissions()` to see all available permissions: @@ -620,6 +689,7 @@ SELECT * FROM all_permissions(); | REMOVE EXTERNAL ALIAS | Remove external group mappings | | REMOVE PASSWORD | Remove passwords | | REMOVE USER | Remove users from groups | +| SET MEMORY LIMIT | Set memory limits on users, groups, and service accounts | | USER DETAILS | View user/group/service account details | ### Special permissions @@ -634,8 +704,9 @@ SELECT * FROM all_permissions(); ## SQL commands reference - [ADD USER](/docs/query/sql/acl/add-user/) -- [ALTER USER](/docs/query/sql/acl/alter-user/) +- [ALTER GROUP](/docs/query/sql/acl/alter-group/) - [ALTER SERVICE ACCOUNT](/docs/query/sql/acl/alter-service-account/) +- [ALTER USER](/docs/query/sql/acl/alter-user/) - [ASSUME SERVICE ACCOUNT](/docs/query/sql/acl/assume-service-account/) - [CREATE GROUP](/docs/query/sql/acl/create-group/) - [CREATE SERVICE ACCOUNT](/docs/query/sql/acl/create-service-account/) diff --git a/documentation/sidebars.js b/documentation/sidebars.js index 995512908..5f8f1cf18 100644 --- a/documentation/sidebars.js +++ b/documentation/sidebars.js @@ -257,6 +257,10 @@ module.exports = { type: "category", label: "ALTER", items: [ + { + id: "query/sql/acl/alter-group", + type: "doc", + }, { id: "query/sql/acl/alter-service-account", type: "doc",