Installs clickhouse-server and clickhouse-keeper to provide a ClickHouse cluster with replication enabled. Access is via a public-facing NLB. Currently only a single shard is used.
Architecture for a self-hosted clickhouse install supporting replication.
The following can be used to insert some data for testing purposes. Note the use of on cluster <cluster-name> in the database and table creation steps. Clickhouse Cloud abstracts this away from users.
- Create a database
create database db1 on cluster clickhouse_cluster;- Create a table
CREATE TABLE db1.table1 ON CLUSTER clickhouse_cluster
(
`id` UInt64,
`column1` String
)
ENGINE = ReplicatedMergeTree
ORDER BY id- Insert some data
INSERT INTO db1.table1 (id, column1) VALUES (1, 'abc');At this stage the data should be present on all nodes of the cluster given that is it configured as a single shard + n replica cluster.
- Add CI/CD for validating and documenting Terraform
- Add a default user with a password
- Add certificates to the ClickHouse server
- Add support for sharding
| Name | Version |
|---|---|
| terraform | >= 0.13 |
| aws | 5.82.2 |
| random | 3.6.3 |
| tls | ~> 4.0 |
| Name | Version |
|---|---|
| aws | 5.82.2 |
| random | 3.6.3 |
| tls | ~> 4.0 |
| Name | Source | Version |
|---|---|---|
| clickhouse_cluster | terraform-aws-modules/ec2-instance/aws | 5.7.1 |
| clickhouse_keeper | terraform-aws-modules/ec2-instance/aws | 5.7.1 |
| vpc | terraform-aws-modules/vpc/aws | 5.17.0 |
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| admin_user_networks | List of networks allowed to connect as admin user | list(string) |
[ |
no |
| allowed_cidr_blocks | List of CIDR blocks allowed to access the ClickHouse cluster | list(string) |
[ |
no |
| clickhouse_instance_type | The instance type for the ClickHouse servers | string |
"t2.medium" |
no |
| clickhouse_volume_size | The size of the EBS volume for the ClickHouse servers in GB | number |
10 |
no |
| clickhouse_volume_type | The type of EBS volume for the ClickHouse servers | string |
"gp2" |
no |
| cluster_domain | Domain name for the cluster (used for certificates) | string |
"" |
no |
| cluster_name | The name of the ClickHouse cluster | string |
"clickhouse" |
no |
| default_user_networks | List of networks allowed to connect as default user | list(string) |
[ |
no |
| enable_encryption | Enable TLS encryption for all ClickHouse communication | bool |
false |
no |
| enable_nlb | Enable the Network Load Balancer for the ClickHouse cluster | bool |
true |
no |
| enable_nlb_tls | Enable TLS termination at the NLB. Requires either an existing certificate ARN or cluster encryption to be enabled | bool |
false |
no |
| external_ca_cert | PEM-encoded CA certificate for cluster communication. Required if use_external_certs is true | string |
"" |
no |
| external_cert_secret_ids | Map of Secret Manager ARNs containing certificates and private keys for each node. Required if use_external_certs is true | object({ |
null |
no |
| http_port | HTTP default port | number |
8123 |
no |
| https_port | HTTPS default port | number |
8443 |
no |
| interserver_http_port | Inter-server communication port | number |
9009 |
no |
| interserver_https_port | SSL/TLS port for inter-server communications | number |
9010 |
no |
| keeper_instance_type | The instance type for the ClickHouse keepers | string |
"t2.medium" |
no |
| keeper_node_count | The number of ClickHouse keepers to deploy | number |
3 |
no |
| keeper_port | ClickHouse Keeper port | number |
9181 |
no |
| keeper_port_secure | Secure SSL ClickHouse Keeper port | number |
9281 |
no |
| keeper_raft_port | ClickHouse Keeper Raft port | number |
9234 |
no |
| keeper_volume_size | The size of the EBS volume for the ClickHouse keepers in GB | number |
10 |
no |
| keeper_volume_type | The type of EBS volume for the ClickHouse keepers | string |
"gp2" |
no |
| key_name | Name of an AWS key pair to use for SSH access (must exist in the AWS account) | string |
"" |
no |
| nlb_type | Type of NLB to create - internal or external | string |
"internal" |
no |
| prometheus_port | Prometheus metrics port | number |
9363 |
no |
| region | The AWS region to deploy to | string |
n/a | yes |
| retention_period | Log retention period in days | number |
30 |
no |
| shards | List of shards and their configuration. Each shard specifies how many replicas it should have and optionally its weight. | list(object({ |
n/a | yes |
| ssh_access | SSH access configuration. Set enabled=false to disable SSH access, or configure cidr_blocks for access control. | object({ |
{ |
no |
| ssl_cert_days | Validity period for self-signed certificates in days | number |
365 |
no |
| ssl_key_bits | Key size for self-signed certificates | number |
2048 |
no |
| tags | Tags to apply to all resources | map(string) |
{ |
no |
| tcp_port | Native Protocol port for client-server communication | number |
9000 |
no |
| tcp_port_secure | Native protocol SSL/TLS port | number |
9440 |
no |
| tls_certificate_arn | ARN of an existing ACM certificate to use for NLB TLS termination. Required if enable_nlb_tls is true and using an existing certificate | string |
"" |
no |
| use_external_certs | Use externally provided certificates instead of generating them | bool |
false |
no |
| use_generated_cert | Use the cluster's generated CA to create a certificate for the NLB. Requires enable_encryption to be true | bool |
false |
no |
| Name | Description |
|---|---|
| ca_secret_arn | Output the Secret ARN for use in user_data |
| clickhouse_cluster_ips | Output the public and private IP addresses of the ClickHouse cluster |
| clickhouse_credentials_arn | Output the Secret ARN for the clickhouse credentials |
| clickhouse_keeper_ips | Output the public and private IP addresses of the ClickHouse keepers |
| clickhouse_nlb_dns | Output the DNS name of the NLB |