You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: modules/ROOT/pages/index.adoc
+33-33Lines changed: 33 additions & 33 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,32 +1,32 @@
1
1
= {pulsar-operator-full-name}
2
2
:navtitle: About {pulsar-operator-full-name}
3
3
4
-
{pulsar-operator-full-name} simplifies running https://pulsar.apache.org[Apache Pulsar] on Kubernetes by applying the familiar https://kubernetes.io/docs/concepts/extend-kubernetes/operator/[Operator pattern] to Pulsar's components, and horizonally scaling resources up or down based on CPU and memory workloads.
4
+
{pulsar-operator-full-name} simplifies running https://pulsar.apache.org[{pulsar-reg}] on Kubernetes by applying the familiar https://kubernetes.io/docs/concepts/extend-kubernetes/operator/[Operator pattern] to the {pulsar-short}components, and horizontally scaling resources up or down based on CPU and memory workloads.
5
5
6
-
Operating and maintaining Apache Pulsar clusters traditionally involves complex manual configurations, making it challenging for developers and operators to effectively manage the system's lifecycle. However, with the KAAP operator, these complexities are abstracted away, enabling developers to focus on their applications rather than the underlying infrastructure.
6
+
Operating and maintaining {pulsar}clusters traditionally involves complex manual configurations, making it challenging for developers and operators to effectively manage the system's lifecycle. However, with the {pulsar-operator}, these complexities are abstracted away, enabling developers to focus on their applications rather than the underlying infrastructure.
7
7
8
-
Some of the key features and benefits of the KAAP operator include:
8
+
Some of the key features and benefits of the {pulsar-operator} include:
9
9
10
-
- **Easy Deployment**: Deploying an Apache Pulsar cluster on Kubernetes is simplified through declarative configurations and automation provided by the operator.
10
+
- **Easy Deployment**: Deploying an {pulsar} cluster on Kubernetes is simplified through declarative configurations and automation provided by the operator.
11
11
12
-
- **Scalability**: The KAAP operator enables effortless scaling of Pulsar clusters by automatically handling the creation and configuration of new Pulsar brokers and bookies as per defined rules. The broker autoscaling is integrated with the Pulsar broker load balancer to make smart resource management decisions, and bookkeepers are scaled up and down based on storage usage in a safe, controlled manner.
12
+
- **Scalability**: The {pulsar-operator} enables effortless scaling of {pulsar-short} clusters by automatically handling the creation and configuration of new {pulsar-short} brokers and bookies as per defined rules. The broker autoscaling is integrated with the {pulsar-short} broker load balancer to make smart resource management decisions, and bookkeepers are scaled up and down based on storage usage in a safe, controlled manner.
13
13
14
-
- **High Availability**: The operator implements best practices for high availability, ensuring that Pulsar clusters are fault-tolerant and can sustain failures without service disruptions.
14
+
- **High Availability**: The operator implements best practices for high availability, ensuring that {pulsar-short} clusters are fault-tolerant and can sustain failures without service disruptions.
15
15
16
-
- **Lifecycle Management**: The operator takes care of common Pulsar cluster lifecycle tasks, such as cluster creation, upgrade, configuration updates, and graceful shutdowns.
16
+
- **Lifecycle Management**: The operator takes care of common {pulsar-short} cluster lifecycle tasks, such as cluster creation, upgrade, configuration updates, and graceful shutdowns.
17
17
18
-
We also offer the xref:getting-started:stack.adoc[{pulsar-stack}] if you're looking for more Kubernetes-native tooling deployed with your Pulsar cluster. Along with the PulsarCluster CRDs, KAAP stack also includes:
18
+
We also offer the xref:getting-started:stack.adoc[{pulsar-stack}] if you're looking for more Kubernetes-native tooling deployed with your {pulsar-short} cluster. Along with the PulsarCluster CRDs, {pulsar-stack} also includes:
19
19
20
-
* Pulsar Operator
20
+
* {pulsar-operator}
21
21
* Prometheus Stack (Grafana)
22
-
* Pulsar Grafana dashboards
22
+
* {pulsar-short} Grafana dashboards
23
23
* Cert Manager
24
24
* Keycloak
25
25
26
-
Whether you are a developer looking to leverage the power of Apache Pulsar in your Kubernetes environment or an operator seeking to streamline the management of Pulsar clusters, the {pulsar-operator} provides a robust and user-friendly solution.
26
+
Whether you are a developer looking to leverage the power of {pulsar}in your Kubernetes environment or an operator seeking to streamline the management of {pulsar-short} clusters, the {pulsar-operator} provides a robust and user-friendly solution.
27
27
28
28
This guide offers a starting point for {pulsar-operator}.
29
-
We will cover installation and deployment, configuration points, and further options for managing Pulsar components with the {pulsar-operator}.
29
+
We will cover installation and deployment, configuration points, and further options for managing {pulsar-short} components with the {pulsar-operator}.
30
30
31
31
[NOTE]
32
32
====
@@ -36,65 +36,65 @@ If you're upgrading from KAAP v0.1.0 to v0.2.0, you must upgrade the CRDs to v1b
36
36
== Features
37
37
38
38
After a new custom resource type is added to your cluster by installing a CRD, you can create instances of the resource based on its specification.
39
-
The Kubernetes API can be extended to support the new resource type, automating away the tedious aspects of managing a Pulsar cluster.
39
+
The Kubernetes API can be extended to support the new resource type, automating away the tedious aspects of managing a {pulsar-short} cluster.
40
40
41
41
* xref:scaling-components:autoscale-bookies.adoc[Bookkeeper autoscaler] - Automatically scale the number of bookies based on memory usage.
42
42
* xref:scaling-components:autoscale-brokers.adoc[Broker autoscaler] - Automatically scale the number of brokers based on CPU load.
43
43
* xref:resource-sets:index.adoc[Rack-aware bookkeeper placement] - Place bookies in different racks to guarantee high availability.
44
-
* xref:scaling-components:kafka.adoc[Kafka API] - Use the Starlight for Kafka API to bring your Kafka message traffic to Pulsar.
44
+
* xref:scaling-components:kafka.adoc[Kafka API] - Use the Starlight for Kafka API to bring your Kafka message traffic to {pulsar-short}.
45
45
46
-
== How {pulsar-operator} makes Pulsar easier
46
+
== How {pulsar-operator} makes {pulsar-short} easier
47
47
48
48
Operators are a common pattern for packaging, deploying, and managing Kubernetes applications.
49
49
Operators extend Kubernetes functionality to automate common tasks in stateful applications.
50
-
Think of {pulsar-operator} as a manager for the individual components of Pulsar. By implementing the pulsarCluster Custom Resource Definition, the operator knows enough to manage the deployment, configuration, and scaling of Pulsar components with re-usable and automated tasks, such as:
50
+
Think of {pulsar-operator} as a manager for the individual components of {pulsar-short}. By implementing the pulsarCluster Custom Resource Definition, the operator knows enough to manage the deployment, configuration, and scaling of {pulsar-short} components with re-usable and automated tasks, such as:
51
51
52
-
* Deploying a Pulsar cluster
52
+
* Deploying a {pulsar-short} cluster
53
53
* Deploying monitoring and logging components
54
54
* Autoscaling bookies based on memory usage, or brokers based on CPU load
55
55
* Assigning resources to specific availability zones (AZs)
56
56
57
57
{pulsar-operator} is configured, deployed, and packaged with Helm charts and based on the https://quarkiverse.github.io/quarkiverse-docs/quarkus-operator-sdk/dev/index.html[Quarkus Operator SDK].
58
58
59
-
== Pulsar component architecture
59
+
== {pulsar-short} component architecture
60
60
61
-
A typical Pulsar cluster *requires* the following components:
61
+
A typical {pulsar-short} cluster *requires* the following components:
62
62
63
-
* https://pulsar.apache.org/docs/concepts-architecture-overview/#metadata-store[Zookeeper] - This is Pulsar’s meta data store. It stores data about a cluster’s configuration, helps the proxy direct messages to the correct broker, and holds Bookie configurations.
63
+
* https://pulsar.apache.org/docs/concepts-architecture-overview/#metadata-store[Zookeeper] - This is the {pulsar-short} metadata store. It stores data about a cluster's configuration, helps the proxy direct messages to the correct broker, and holds Bookie configurations.
64
64
65
-
* https://pulsar.apache.org/docs/concepts-architecture-overview/#brokers[Broker] - This is Pulsar's message router.
65
+
* https://pulsar.apache.org/docs/concepts-architecture-overview/#brokers[Broker] - This is the {pulsar-short} message router.
66
66
67
-
* https://pulsar.apache.org/docs/concepts-architecture-overview/#apache-bookkeeper[Bookkeeper (bookie)] - This is Pulsar’s data store.
67
+
* https://pulsar.apache.org/docs/concepts-architecture-overview/#apache-bookkeeper[Bookkeeper (bookie)] - This is the {pulsar-short} data store.
68
68
Bookkeeper stores message data in a low-latency, resilient way.
69
69
70
70
In addition to the required components, you might want to include some *optional components*:
71
71
72
-
* https://bookkeeper.apache.org/docs/admin/autorecovery[Bookkeeper AutoRecovery] - This is a Pulsar component that recovers Bookkeeper data in the event of a bookie outage.
73
-
* https://pulsar.apache.org/docs/concepts-architecture-overview/#pulsar-proxy[Pulsar proxy] - The Pulsar proxy is just that - a proxy that runs at the edge of the cluster with public facing endpoints.
74
-
Pulsar proxy also offers special options for cluster extensions, like our [Starlight Suite of APIs].
75
-
* https://pulsar.apache.org/docs/functions-worker-run-separately/[Dedicated functions worker(s)] - You can optionally run dedicated function workers in a Pulsar cluster.
76
-
* xref:luna-streaming:components:admin-console-tutorial.adoc[Pulsar AdminConsole] - This is an optional web-based admin console for managing Pulsar clusters.
77
-
* xref:luna-streaming:components:heartbeat-vm.adoc[Pulsar Heartbeat] - This is an optional component that monitors the health of Pulsar cluster and emits metrics about the cluster that are helpful for observing and debugging issues.
72
+
* https://bookkeeper.apache.org/docs/admin/autorecovery[Bookkeeper AutoRecovery] - This is a {pulsar-short} component that recovers Bookkeeper data in the event of a bookie outage.
73
+
* https://pulsar.apache.org/docs/concepts-architecture-overview/#pulsar-proxy[{pulsar-short} proxy] - The {pulsar-short} proxy is just that - a proxy that runs at the edge of the cluster with public facing endpoints.
74
+
{pulsar-short} proxy also offers special options for cluster extensions, like our [Starlight Suite of APIs].
75
+
* https://pulsar.apache.org/docs/functions-worker-run-separately/[Dedicated functions worker(s)] - You can optionally run dedicated function workers in a {pulsar-short} cluster.
76
+
* xref:luna-streaming:components:admin-console-tutorial.adoc[{pulsar-short} AdminConsole] - This is an optional web-based admin console for managing {pulsar-short} clusters.
77
+
* xref:luna-streaming:components:heartbeat-vm.adoc[{pulsar-short} Heartbeat] - This is an optional component that monitors the health of {pulsar-short} cluster and emits metrics about the cluster that are helpful for observing and debugging issues.
78
78
* Prometheus/Grafana/Alert manager stack - This is the default observability stack for a cluster. The Luna Helm chart includes pre-made dashboards in Grafana and pre-wires all the metrics scraping.
79
79
80
-
To enable these optional components, set them in the Helm chart values. For example, to enable the Pulsar Admin Console in your KAAP stack deployment, add the following configuration to your https://github.com/datastax/kaap/blob/755807d3744735b6f6ae0400365e159e5dcf5dd0/helm/kaap-stack/values.yaml#L25C1-L26C17[values.yaml] file:
80
+
To enable these optional components, set them in the Helm chart values. For example, to enable the {pulsar-short} Admin Console in your {pulsar-stack} deployment, add the following configuration to your https://github.com/datastax/kaap/blob/755807d3744735b6f6ae0400365e159e5dcf5dd0/helm/kaap-stack/values.yaml#L25C1-L26C17[values.yaml] file:
81
81
[source,yaml]
82
82
----
83
83
pulsarAdminConsole:
84
84
enabled: true
85
85
----
86
86
87
-
== How {pulsar-operator} installs Pulsar
87
+
== How {pulsar-operator} installs {pulsar-short}
88
88
89
89
{pulsar-operator} can be installed in two ways.
90
90
91
-
* xref:getting-started:operator.adoc[Pulsar Operator] - Installs just the operator and PulsarCluster CRDs into an existing Pulsar cluster.
91
+
* xref:getting-started:operator.adoc[{pulsar-operator}] - Installs just the operator and PulsarCluster CRDs into an existing {pulsar-short} cluster.
92
92
93
-
* xref:getting-started:stack.adoc[Pulsar Stack] - Installs and deploys the operator, a Pulsar cluster, and a full Prometheus monitoring stack.
93
+
* xref:getting-started:stack.adoc[{pulsar-stack}] - Installs and deploys the operator, a {pulsar-short} cluster, and a full Prometheus monitoring stack.
94
94
95
95
[TIP]
96
96
====
97
-
You can also scan an existing Pulsar cluster and generate an equivalent PulsarCluster CRD. For more, see xref:migration:migrate-cluster.adoc[].
97
+
You can also scan an existing {pulsar-short} cluster and generate an equivalent PulsarCluster CRD. For more, see xref:migration:migrate-cluster.adoc[].
98
98
====
99
99
100
100
To get started, see xref:getting-started:index.adoc[Getting Started].
Copy file name to clipboardExpand all lines: modules/getting-started/pages/index.adoc
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,9 +4,9 @@
4
4
5
5
* xref:getting-started:operator.adoc[{pulsar-operator}] - Installs just the operator pod and PulsarCluster CRDs.
6
6
7
-
* xref:getting-started:stack.adoc[{pulsar-stack}] - Installs and deploys the operator, a Pulsar cluster, and a full Prometheus monitoring stack.
7
+
* xref:getting-started:stack.adoc[{pulsar-stack}] - Installs and deploys the operator, a {pulsar-short} cluster, and a full Prometheus monitoring stack.
8
8
9
9
[TIP]
10
10
====
11
-
If you have an existing Pulsar cluster, you can scan and generate an equivalent PulsarCluster CRD. For more, see xref:migration:migrate-cluster.adoc[].
11
+
If you have an existing {pulsar-short} cluster, you can scan and generate an equivalent PulsarCluster CRD. For more, see xref:migration:migrate-cluster.adoc[].
Copy file name to clipboardExpand all lines: modules/migration/pages/migrate-cluster.adoc
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,15 +1,15 @@
1
-
= Migrate existing cluster to KAAP operator
1
+
= Migrate existing cluster to {pulsar-operator}
2
2
3
-
Migrating an existing Apache Pulsar cluster to one controlled by the {pulsar-operator} is a manual process, but we've included a migration tool to help you along the way.
3
+
Migrating an existing {pulsar-reg} cluster to one controlled by the {pulsar-operator} is a manual process, but we've included a migration tool to help you along the way.
4
4
5
-
The migration tool is a CLI application that connects to an existing Apache Pulsar cluster and generates a valid and equivalent PulsarCluster CRD.
5
+
The migration tool is a CLI application that connects to an existing {pulsar} cluster and generates a valid and equivalent PulsarCluster CRD.
6
6
The migration tool simulates what would happen if the generated PulsarCluster would be submitted, retrieves the Kubernetes resources that would be created, and compares them with the existing cluster's resources, generating a detailed HTML report.
7
7
You can then examine the report and decide if you want to proceed with the cluster migration, or if you need to make some changes first.
8
8
9
9
== Prerequisites
10
10
11
11
* Java 17
12
-
* An existing Apache Pulsar cluster
12
+
* An existing {pulsar} cluster
13
13
* Migration-tool JAR downloaded from the https://github.com/datastax/kaap/releases[latest release].
14
14
15
15
== Scan and generate cluster CRDs
@@ -42,7 +42,7 @@ CURRENT NAME CLUSTER
42
42
+
43
43
Before running the migration tool, {company} recommends switching to K8's current context and ensuring connectivity, such as with `kubectl get pods`.
44
44
45
-
.. The `namespace` is the namespace with the Apache Pulsar resources you wish to scan.
45
+
.. The `namespace` is the namespace with the {pulsar} resources you wish to scan.
46
46
47
47
.. The `clusterName` is the prefix of each pod.
48
48
For example, if the broker pod is `pulsar-prod-cluster-broker-0`, then the `clusterName` is `pulsar-prod-cluster`.
@@ -54,7 +54,7 @@ For example, if the broker pod is `pulsar-prod-cluster-broker-0`, then the `clus
. Find the link to the generated report in the logs, open the generated report in your browser, and then examine the differences between the existing cluster and the KAAP operator.
57
+
. Find the link to the generated report in the logs, open the generated report in your browser, and then examine the differences between the existing cluster and the {pulsar-operator}.
58
58
+
59
59
If everything looks good, proceed to the <<migration-procedure,migration procedure>>.
Copy file name to clipboardExpand all lines: modules/resource-sets/pages/index.adoc
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
= Resource sets
2
2
3
-
The operator allows you to create multiple sets of Pulsar proxies, brokers, and bookies, called resource sets.
3
+
The operator allows you to create multiple sets of {pulsar-short} proxies, brokers, and bookies, called resource sets.
4
4
Each set is a dedicated deployment/statefulset with its own service and configmap.
5
5
When multiple sets are specified, an umbrella service is created as the main entrypoint of the cluster, but otherwise, a dedicated service is created for each set. You can customize the service per set - for example, you might assign different DNS domains for each resource set.
0 commit comments