Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 37 additions & 31 deletions src/current/v26.1/cockroachdb-operator-performance.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,10 +177,12 @@ Once you've picked out an amount of CPU and memory to reserve for Cockroach, con
~~~ yaml
cockroachdb:
crdbCluster:
resources:
requests:
cpu: 3500m
memory: 12300Mi
podTemplate:
spec:
resources:
requests:
cpu: 3500m
memory: 12300Mi
~~~

When you initialize the cluster, check that all the CockroachDB pods are scheduled successfully. If you see any get stuck in the pending state, run `kubectl describe pod {podname}` and check the `Events` for information about why they're still pending. You may need to manually preempt pods on one or more nodes by running `kubectl delete pod` on them to make room for the CockroachDB pods. As long as the pods you delete were created by a higher-level Kubernetes object such as a `Deployment`, they'll be safely recreated on another node.
Expand All @@ -194,12 +196,14 @@ To set resource limits, in addition to the [resource requests](#resource-request
~~~ yaml
cockroachdb:
crdbCluster:
resources:
requests:
cpu: 3500m
memory: 12300Mi
limits:
memory: 12300Mi
podTemplate:
spec:
resources:
requests:
cpu: 3500m
memory: 12300Mi
limits:
memory: 12300Mi
~~~

Pods will be limited to their reserved resources and are unlikely to be preempted, except in rare cases. This will not improve performance on an underutilized Kubernetes cluster, but provides more predictable performance as other workloads run.
Expand Down Expand Up @@ -266,27 +270,29 @@ Client applications such as benchmarking applications running on the same machin
~~~ yaml
cockroachdb:
crdbCluster:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- loadgen
topologyKey: kubernetes.io/hostname
- weight: 99
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- cockroachdb
topologyKey: kubernetes.io/hostname
podTemplate:
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- loadgen
topologyKey: kubernetes.io/hostname
- weight: 99
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- cockroachdb
topologyKey: kubernetes.io/hostname
~~~

The preceding configuration will first prefer to put the `loadgen` pods on different nodes from each other, which is important for the fault tolerance of the `loadgen` pods themselves. As a secondary priority, it will attempt to put the pods on nodes that do not already have a running `CockroachDB` pod. This will ensure the best possible balance of fault tolerance and performance for the load generator and CockroachDB cluster.
Expand Down
18 changes: 10 additions & 8 deletions src/current/v26.1/configure-cockroachdb-operator.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,18 +27,20 @@ You can set the CPU and memory resources allocated to the CockroachDB container
1 CPU in Kubernetes is equivalent to 1 vCPU or 1 hyperthread. For best practices on provisioning CPU and memory for CockroachDB, refer to the [Production Checklist]({% link {{ page.version.version }}/recommended-production-settings.md %}#hardware).
{{site.data.alerts.end}}

Specify CPU and memory values in `cockroachdb.crdbCluster.resources.limits` and `cockroachdb.crdbCluster.resources.requests` in the values file used to [deploy the cluster]({% link {{ page.version.version }}/deploy-cockroachdb-with-cockroachdb-operator.md %}#initialize-the-cluster):
Specify CPU and memory values in `cockroachdb.crdbCluster.podTemplate.spec.resources.limits` and `cockroachdb.crdbCluster.podTemplate.spec.resources.requests` in the values file used to [deploy the cluster]({% link {{ page.version.version }}/deploy-cockroachdb-with-cockroachdb-operator.md %}#initialize-the-cluster):

~~~ yaml
cockroachdb:
crdbCluster:
resources:
limits:
cpu: 4000m
memory: 16Gi
requests:
cpu: 4000m
memory: 16Gi
podTemplate:
spec:
resources:
limits:
cpu: 4000m
memory: 16Gi
requests:
cpu: 4000m
memory: 16gi
~~~

Apply the new settings to the cluster:
Expand Down
27 changes: 23 additions & 4 deletions src/current/v26.1/deploy-cockroachdb-with-cockroachdb-operator.md
Original file line number Diff line number Diff line change
Expand Up @@ -252,7 +252,21 @@ For bare metal deployments, the specific Kubernetes infrastructure deployment st
If you intend to deploy CockroachDB nodes across multiple different regions, follow the additional steps described in [Deploy across multiple regions](#deploy-across-multiple-regions).
{{site.data.alerts.callout_end}}

1. Uncomment and modify `cockroachdb.crdbCluster.resources` in the values file with the CPU and memory requests and limits for each node to use. The default values are 4vCPU and 16GiB of memory:
1. Modify `cockroachdb.crdbCluster.podTemplate.spec.resources` in the values file with the CPU and memory limits and requests for each node to use. For example, to define default values of 4vCPU and 16GiB of memory:

~~~ yaml
cockroachdb:
crdbCluster:
podTemplate:
spec:
resources:
limits:
cpu: 4000m
memory: 16Gi
requests:
cpu: 4000m
memory: 16gi
~~~

For more information on configuring node resource allocation, refer to [Resource management]({% link {{ page.version.version }}/configure-cockroachdb-operator.md %})

Expand Down Expand Up @@ -302,13 +316,18 @@ For bare metal deployments, the specific Kubernetes infrastructure deployment st
cockroach start --locality region=us-central1,zone=us-central1-c,dc=dc2
~~~
Optionally, review the `cockroachdb.crdbCluster.topologySpreadConstraints` configuration and set `topologyKey` to the `nodeLabel` value of a locality level that has distinct values for each node. By default the lowest locality level is `zone`, so the following configuration sets that value as the `topologyKey`:
Optionally, review the `cockroachdb.crdbCluster.podTemplate.spec.topologySpreadConstraints` configuration and set `topologyKey` to the `nodeLabel` value of a locality level that has distinct values for each node. By default the lowest locality level is `zone`, so the following configuration sets that value as the `topologyKey`:
~~~ yaml
cockroachdb:
crdbCluster:
topologySpreadConstraints:
topologyKey: topology.kubernetes.io/zone
podTemplate:
spec:
topologySpreadConstraints:
# maxSkew defines the degree to which the pods can be unevenly distributed.
- maxSkew: 1
# topologyKey defines the key for topology spread.
topologyKey: topology.kubernetes.io/zone
~~~
For more information on localities and topology planning, see the [topology patterns documentation]({% link {{ page.version.version }}/topology-patterns.md %}).
Expand Down
36 changes: 4 additions & 32 deletions src/current/v26.1/override-templates-cockroachdb-operator.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ The {{ site.data.products.cockroachdb-operator }} provides abstractions that sim
- A default pod specification is used for the CockroachDB Kubernetes pod.
- The `values.yaml` configuration maps to a subset of `cockroach start` flags when CockroachDB is initialized.

This page describes configuration options that allow advanced users to manually override the pod template and `cockroach start` flags as needed for deployment.
This page describes configuration options that allow advanced users to manually override pod template metadata and `cockroach start` flags as needed for deployment.

{{site.data.alerts.callout_info}}
The {{ site.data.products.cockroachdb-operator }} is in [Preview]({% link {{ page.version.version }}/cockroachdb-feature-availability.md %}).
Expand All @@ -24,6 +24,8 @@ This page describes advanced configurations that override the supported default

The `cockroachdb.crdbCluster.podTemplate` field allows you to override the default pod metadata and specification configured by the {{ site.data.products.cockroachdb-operator }}. The values in this field are merged with the default pod specification, where settings in `podTemplate` override any values in the default.

The `podTemplate` field includes a `containers` field that specifies the container name and image that the template is applied to. By default, this resolves to the `cockroachdb` container name and can be excluded from modifications to the CockroachDB pod template YAML. If needed, you can provide specific images for the `containers` and `initContainers`. For example, the following `podTemplate` configuration specifies a custom init container:

~~~ yaml
cockroachdb:
crdbCluster:
Expand All @@ -35,37 +37,7 @@ cockroachdb:
# initContainers captures the list of init containers for CockroachDB pods.
initContainers:
- name : cockroachdb-init
image: us-docker.pkg.dev/cockroach-cloud-images/data-plane/init-container@sha256:c3e4ba851802a429c7f76c639a64b9152d206cebb31162c1760f05e98f7c4254
# containers captures the list of containers for CockroachDB pods.
containers:
- name: cockroachdb
image: cockroachdb/cockroach:v25.4.0
# imagePullSecrets captures the secrets for fetching images from private registries.
imagePullSecrets: []
~~~

At least one value for `containers` must be specified if any part of `podTemplate` is being modified. For example, the following `podTemplate` configuration overrides pod anti-affinity behavior and specifies a default `cockroachdb/cockroach:v25.4.0` container image:

~~~ yaml
cockroachdb:
crdbCluster:
podTemplate:
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app.kubernetes.io/component
operator: In
values:
- cockroachdb
topologyKey: kubernetes.io/hostname
containers:
- name: cockroachdb
image: cockroachdb/cockroach:v25.4.0
image: us-docker.pkg.dev/cockroach-cloud-images/data-plane/init-container@sha256:example1234567890abcdefghijklmnopqrstuvwxyz
~~~

## Override the default `cockroach start` flags
Expand Down
Loading
Loading