Skip to content

Commit f4a6f4c

Browse files
kate-osbornkate-osborn
authored
HTTP Traffic Splitting (#261)
Support Backend Weight This commit adds support for the BackendRef Weight field which enables traffic splitting use cases. Traffic splitting in the data plane is implemented using the nginx split_clients directive. A split_clients directive is generated for every attached HTTPRoute rule that has multiple backendRefs defined.
1 parent 696a0f0 commit f4a6f4c

40 files changed

+4158
-2151
lines changed

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
NGINX Kubernetes Gateway is an open-source project that provides an implementation of the [Gateway API](https://gateway-api.sigs.k8s.io/) using [NGINX](https://nginx.org/) as the data plane. The goal of this project is to implement the core Gateway APIs -- `Gateway`, `GatewayClass`, `HTTPRoute`, `TCPRoute`, `TLSRoute`, and `UDPRoute` -- to configure an HTTP or TCP/UDP load balancer, reverse-proxy, or API gateway for applications running on Kubernetes. NGINX Kubernetes Gateway is currently under development and supports a subset of the Gateway API.
66

7-
For a list of supported Gateway API resources and features, see the [Gateway API Compatibility](docs/gateway-api-compatibility.md.md) doc.
7+
For a list of supported Gateway API resources and features, see the [Gateway API Compatibility](docs/gateway-api-compatibility.md) doc.
88

99
> Warning: This project is actively in development (beta feature state) and should not be deployed in a production environment.
1010
> All APIs, SDKs, designs, and packages are subject to change.

‎deploy/manifests/nginx-gateway.yaml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,8 @@ spec:
8282
volumes:
8383
- name: nginx-config
8484
emptyDir: { }
85+
- name: var-lib-nginx
86+
emptyDir: { }
8587
- name: njs-modules
8688
configMap:
8789
name: njs-modules
@@ -118,5 +120,7 @@ spec:
118120
volumeMounts:
119121
- name: nginx-config
120122
mountPath: /etc/nginx
123+
- name: var-lib-nginx
124+
mountPath: /var/lib/nginx
121125
- name: njs-modules
122126
mountPath: /usr/lib/nginx/modules/njs

docs/gateway-api-compatibility.md.md renamed to docs/gateway-api-compatibility.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -91,7 +91,7 @@ Fields:
9191
* `type` - supported.
9292
* `requestRedirect` - supported except for the experimental `path` field. If multiple filters with `requestRedirect` are configured, NGINX Kubernetes Gateway will choose the first one and ignore the rest.
9393
* `requestHeaderModifier`, `requestMirror`, `urlRewrite`, `extensionRef` - not supported.
94-
* `backendRefs` - partially supported. Only a single backend ref without support for `weight`. Backend ref `filters` are not supported.
94+
* `backendRefs` - partially supported. Backend ref `filters` are not supported.
9595
* `status`
9696
* `parents`
9797
* `parentRef` - supported.

‎examples/traffic-splitting/README.md‎

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
# Example
2+
3+
In this example we will deploy NGINX Kubernetes Gateway and configure traffic splitting for a simple cafe application.
4+
We will use `HTTPRoute` resources to split traffic between two versions of the application -- `coffee-v1` and `coffee-v2`.
5+
6+
## Running the Example
7+
8+
## 1. Deploy NGINX Kubernetes Gateway
9+
10+
1. Follow the [installation instructions](/docs/installation.md) to deploy NGINX Gateway.
11+
12+
1. Save the public IP address of NGINX Kubernetes Gateway into a shell variable:
13+
14+
```
15+
GW_IP=XXX.YYY.ZZZ.III
16+
```
17+
18+
1. Save the port of NGINX Kubernetes Gateway:
19+
20+
```
21+
GW_PORT=<port number>
22+
```
23+
24+
## 2. Deploy the Coffee Application
25+
26+
1. Create the Cafe Deployments and Services:
27+
28+
```
29+
kubectl apply -f cafe.yaml
30+
```
31+
32+
1. Check that the Pods are running in the `default` namespace:
33+
34+
```
35+
kubectl -n default get pods
36+
NAME READY STATUS RESTARTS AGE
37+
coffee-v1-7c57c576b-rfjsh 1/1 Running 0 21m
38+
coffee-v2-698f66dc46-vcb6r 1/1 Running 0 21m
39+
```
40+
41+
## 3. Configure Routing
42+
43+
1. Create the `Gateway`:
44+
45+
```
46+
kubectl apply -f gateway.yaml
47+
```
48+
49+
1. Create the `HTTPRoute` resources:
50+
51+
```
52+
kubectl apply -f cafe-route.yaml
53+
```
54+
55+
This `HTTPRoute` resource defines a route for the path `/coffee` that sends 80% of the requests to `coffee-v1` and 20% to `coffee-v2`.
56+
In this example, we use 80 and 20; however, the weights are calculated proportionally and do not need to sum to 100.
57+
For example, the weights of 8 and 2, 16 and 4, or 32 and 8 all evaluate to the same relative proportions.
58+
59+
## 4. Test the Application
60+
61+
To access the application, we will use `curl` to send requests to `/coffee`:
62+
63+
```
64+
curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee
65+
```
66+
67+
80% of the responses will come from `coffee-v1`:
68+
69+
```
70+
Server address: 10.12.0.18:80
71+
Server name: coffee-v1-7c57c576b-rfjsh
72+
```
73+
74+
20% of the responses will come from `coffee-v2`:
75+
76+
```
77+
Server address: 10.12.0.19:80
78+
Server name: coffee-v2-698f66dc46-vcb6r
79+
```
80+
81+
### 5. Modify the Traffic Split Configuration
82+
83+
Let's shift more of the traffic to `coffee-v2`. To do this we will update the `HTTPRoute` resource and change the weight
84+
of the `coffee-v2` backend to 80. Backends with equal weights will receive an equal share of traffic.
85+
86+
1. Apply the updated `HTTPRoute` resource:
87+
88+
```
89+
kubectl apply -f cafe-route-equal-weight.yaml
90+
```
91+
92+
2. Test the application again:
93+
94+
```
95+
curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee
96+
```
97+
98+
The responses will now be split evenly between `coffee-v1` and `coffee-v2`.
99+
100+
We can continue modifying the weights of the backends to shift more and more traffic to `coffee-v2`. If there's an issue
101+
with `coffee-v2`, we can quickly shift traffic back to `coffee-v1`.

‎examples/traffic-splitting/cafe-route-equal-weight.yaml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
apiVersion: gateway.networking.k8s.io/v1beta1
2+
kind: HTTPRoute
3+
metadata:
4+
name: cafe-route
5+
spec:
6+
parentRefs:
7+
- name: gateway
8+
sectionName: http
9+
hostnames:
10+
- "cafe.example.com"
11+
rules:
12+
- matches:
13+
- path:
14+
type: PathPrefix
15+
value: /coffee
16+
backendRefs:
17+
- name: coffee-v1
18+
port: 80
19+
weight: 80
20+
- name: coffee-v2
21+
port: 80
22+
weight: 80

‎examples/traffic-splitting/cafe-route.yaml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
apiVersion: gateway.networking.k8s.io/v1beta1
2+
kind: HTTPRoute
3+
metadata:
4+
name: cafe-route
5+
spec:
6+
parentRefs:
7+
- name: gateway
8+
sectionName: http
9+
hostnames:
10+
- "cafe.example.com"
11+
rules:
12+
- matches:
13+
- path:
14+
type: PathPrefix
15+
value: /coffee
16+
backendRefs:
17+
- name: coffee-v1
18+
port: 80
19+
weight: 80
20+
- name: coffee-v2
21+
port: 80
22+
weight: 20

‎examples/traffic-splitting/cafe.yaml‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
name: coffee-v1
5+
spec:
6+
replicas: 1
7+
selector:
8+
matchLabels:
9+
app: coffee-v1
10+
template:
11+
metadata:
12+
labels:
13+
app: coffee-v1
14+
spec:
15+
containers:
16+
- name: coffee-v1
17+
image: nginxdemos/nginx-hello:plain-text
18+
ports:
19+
- containerPort: 8080
20+
---
21+
apiVersion: v1
22+
kind: Service
23+
metadata:
24+
name: coffee-v1
25+
spec:
26+
ports:
27+
- port: 80
28+
targetPort: 8080
29+
protocol: TCP
30+
name: http
31+
selector:
32+
app: coffee-v1
33+
---
34+
apiVersion: apps/v1
35+
kind: Deployment
36+
metadata:
37+
name: coffee-v2
38+
spec:
39+
replicas: 1
40+
selector:
41+
matchLabels:
42+
app: coffee-v2
43+
template:
44+
metadata:
45+
labels:
46+
app: coffee-v2
47+
spec:
48+
containers:
49+
- name: coffee-v2
50+
image: nginxdemos/nginx-hello:plain-text
51+
ports:
52+
- containerPort: 8080
53+
---
54+
apiVersion: v1
55+
kind: Service
56+
metadata:
57+
name: coffee-v2
58+
spec:
59+
ports:
60+
- port: 80
61+
targetPort: 8080
62+
protocol: TCP
63+
name: http
64+
selector:
65+
app: coffee-v2

‎examples/traffic-splitting/gateway.yaml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
apiVersion: gateway.networking.k8s.io/v1beta1
2+
kind: Gateway
3+
metadata:
4+
name: gateway
5+
labels:
6+
domain: k8s-gateway.nginx.org
7+
spec:
8+
gatewayClassName: nginx
9+
listeners:
10+
- name: http
11+
port: 80
12+
protocol: HTTP

‎internal/nginx/config/execute.go‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
package config
2+
3+
import (
4+
"bytes"
5+
"text/template"
6+
)
7+
8+
// executes the template with the given data.
9+
func execute(template *template.Template, data interface{}) []byte {
10+
var buf bytes.Buffer
11+
12+
err := template.Execute(&buf, data)
13+
if err != nil {
14+
panic(err)
15+
}
16+
17+
return buf.Bytes()
18+
}

‎internal/nginx/config/execute_test.go‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
package config
2+
3+
import (
4+
"testing"
5+
6+
"github.com/nginxinc/nginx-kubernetes-gateway/internal/nginx/config/http"
7+
)
8+
9+
func TestExecute(t *testing.T) {
10+
defer func() {
11+
if r := recover(); r != nil {
12+
t.Errorf("execute() panicked with %v", r)
13+
}
14+
}()
15+
16+
bytes := execute(serversTemplate, []http.Server{})
17+
if len(bytes) == 0 {
18+
t.Error("template.execute() did not generate anything")
19+
}
20+
}
21+
22+
func TestExecutePanics(t *testing.T) {
23+
defer func() {
24+
if r := recover(); r == nil {
25+
t.Error("template.execute() did not panic")
26+
}
27+
}()
28+
29+
_ = execute(serversTemplate, "not-correct-data")
30+
}

0 commit comments

Comments
 (0)