8000 Add custom listener controls to VirtualServer (#4271) · nginx/kubernetes-ingress@05ba01b · GitHub
[go: up one dir, main page]
Skip to content

Commit 05ba01b

Browse files
shaun-nxshaun-nx
authored
Add custom listener controls to VirtualServer (#4271)
1 parent 79a3eb9 commit 05ba01b
< 10000 /div>

25 files changed

+3043
-62
lines changed

‎deployments/common/crds/k8s.nginx.org_globalconfigurations.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,5 +45,7 @@ spec:
4545
type: integer
4646
protocol:
4747
type: string
48+
ssl:
49+
type: boolean
4850
served: true
4951
storage: true

‎deployments/common/crds/k8s.nginx.org_virtualservers.yaml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,14 @@ spec:
9797
internalRoute:
9898
description: InternalRoute allows for the configuration of internal routing.
9999
type: boolean
100+
listener:
101+
description: Listener references a custom http and/or https listener defined in GlobalConfiguration.
102+
type: object
103+
properties:
104+
http:
105+
type: string
106+
https:
107+
type: string
100108
policies:
101109
type: array
102110
items:

‎deployments/helm-chart/crds/k8s.nginx.org_globalconfigurations.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,5 +45,7 @@ spec:
4545
type: integer
4646
protocol:
4747
type: string
48+
ssl:
49+
type: boolean
4850
served: true
4951
storage: true

‎deployments/helm-chart/crds/k8s.nginx.org_virtualservers.yaml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,14 @@ spec:
9797
internalRoute:
9898
description: InternalRoute allows for the configuration of internal routing.
9999
type: boolean
100+
listener:
101+
description: Listener references a custom http and/or https listener defined in GlobalConfiguration.
102+
type: object
103+
properties:
104+
http:
105+
type: string
106+
https:
107+
type: string
100108
policies:
101109
type: array
102110
items:

‎examples/custom-resources/custom-listeners/README.md‎

Lines changed: 181 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,181 @@
1+
# Custom HTTP Listeners
2+
3+
In this example, we will configure a VirtualServer resource with custom HTTP listeners.
4+
This will allow HTTP and/or HTTPs based requests to be made on non-default ports.
5+
6+
## Prerequisites
7+
8+
1. Follow the [installation](https://docs.nginx.com/nginx-ingress-controller/installation/installation-with-manifests/)
9+
instructions to deploy the Ingress Controller with custom resources enabled.
10+
2. Ensure the Ingress Controller is configured with the `-global-configuration` argument:
11+
```console
12+
args:
13+
- -global-configuration=$(POD_NAMESPACE)/nginx-configuration
14+
```
15+
16+
3. Save the public IP address of the Ingress Controller into a shell variable:
17+
```console
18+
IC_IP=XXX.YYY.ZZZ.III
19+
```
20+
21+
4. If you have a NodePort or Loadbalancer service deployed, ensure they are updated to include the custom listener ports.
22+
Example YAML for a LoadBalancer:
23+
```yaml
24+
apiVersion: v1
25+
kind: Service
26+
metadata:
27+
name: nginx-ingress
28+
namespace: nginx-ingress
29+
spec:
30+
type: LoadBalancer
31+
ports:
32+
- port: 8083
33+
targetPort: 8083
34+
protocol: TCP
35+
name: http-8083
36+
- port: 8443
37+
targetPort: 8443
38+
protocol: TCP
39+
name: https-8443
40+
selector:
41+
app: nginx-ingress
42+
```
43+
## Step 1 - Deploy the GlobalConfiguration resource
44+
Similar to how listeners are configured in our [basic-tcp-udp](../../examples/custom-resource/basic-tcp-udp) examples,
45+
here we deploy a GlobalConfiguration resource with the listeners we want to use in our VirtualServer.
46+
```yaml
47+
apiVersion: k8s.nginx.org/v1alpha1
48+
kind: GlobalConfiguration
49+
metadata:
50+
name: nginx-configuration
51+
namespace: nginx-ingress
52+
spec:
53+
listeners:
54+
- name: http-8083
55+
port: 8083
56+
protocol: HTTP
57+
- name: https-8443
58+
port: 8443
59+
protocol: HTTP
60+
ssl: true
61+
```
62+
63+
```console
64+
kubectl create -f global-configuration.yaml
65+
```
66+
67+
## Step 2 - Save the custom port numbers
68+
Save the custom HTTP and/or HTTPS ports into a shell variables for later:
69+
70+
```console
71+
IC_HTTP_PORT=8083
72+
IC_HTTPS_PORT=8443
73+
```
74+
75+
## Step 3 - Deploy the Cafe Application
76+
77+
Create the coffee and the tea deployments and services:
78+
79+
```console
80+
kubectl create -f cafe.yaml
81+
```
82+
83+
## Step 4 - Deploy the VirtualServer with custom listeners
84+
The VirtualServer in this example is set to use the listeners defined in the GlobalConfiguration resource
85+
that was deployed in Step 1. Below is the yaml of this example VirtualServer:
86+
87+
```yaml
88+
apiVersion: k8s.nginx.org/v1
89+
kind: VirtualServer
90+
metadata:
91+
name: cafe
92+
spec:
93+
listener:
94+
http: http-8083
95+
https: https-8443
96+
host: cafe.example.com
97+
tls:
98+
secret: cafe-secret
99+
upstreams:
100+
- name: tea
101+
service: tea-svc
102+
port: 80
103+
- name: coffee
104+
service: coffee-svc
105+
port: 80
106+
routes:
107+
- path: /tea
108+
action:
109+
pass: tea
110+
- path: /coffee
111+
action:
112+
pass: coffee
113+
```
114+
115+
1. Create the secret with the TLS certificate and key:
116+
117+
```console
118+
kubectl create -f cafe-secret.yaml
119+
```
120+
121+
2. Create the VirtualServer resource:
122+
123+
```console
124+
kubectl create -f cafe-virtual-server.yaml
125+
```
126+
127+
## Step 5 - Test the Configuration
128+
129+
1. Check that the configuration has been successfully applied by inspecting the events of the VirtualServer:
130+
131+
```console
132+
kubectl describe virtualserver cafe
133+
```
134+
135+
Below you will see the events as well as the new `Listeners` field
136+
```console
137+
. . .
138+
Spec:
139+
Host: cafe.example.com
140+
Listener:
141+
Http: http-8083
142+
Https: https-8443
143+
. . .
144+
Routes:
145+
. . .
146+
Events:
147+
Type Reason Age From Message
148+
---- ------ ---- ---- -------
149+
Normal AddedOrUpdated 7s nginx-ingress-controller Configuration for default/cafe was added or updated
150+
```
151+
152+
2. Since the deployed VirtualServer is using ports `8083` and 8443` in this example. you must explicitly specify these ports
153+
when sending requests to the endpoints of this VirtualServer:
154+
155+
For `/coffee` on `8443`:
156+
157+
```console
158+
curl -k https://cafe.example.com:8443/coffee
159+
```
160+
161+
```text
162+
Server address: 10.32.0.40:8080
163+
Server name: coffee-7dd75bc79b-qmhmv
164+
...
165+
URI: /coffee
166+
...
167+
```
168+
169+
For `/coffee` on `8083`:
170+
171+
```console
172+
curl -k https://cafe.example.com:8083/coffee
173+
```
174+
175+
```text
176+
Server address: 10.32.0.40:8080
177+
Server name: coffee-7dd75bc79b-qmhmv
178+
...
179+
URI: /coffee
180+
...
181+
```

‎examples/custom-resources/custom-listeners/cafe-secret.yaml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
apiVersion: v1
2+
kind: Secret
3+
metadata:
4+
name: cafe-secret
5+
type: kubernetes.io/tls
6+
data:
7+
tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURMakNDQWhZQ0NRREFPRjl0THNhWFdqQU5CZ2txaGtpRzl3MEJBUXNGQURCYU1Rc3dDUVlEVlFRR0V3SlYKVXpFTE1Ba0dBMVVFQ0F3Q1EwRXhJVEFmQmdOVkJBb01HRWx1ZEdWeWJtVjBJRmRwWkdkcGRITWdVSFI1SUV4MApaREViTUJrR0ExVUVBd3dTWTJGbVpTNWxlR0Z0Y0d4bExtTnZiU0FnTUI0WERURTRNRGt4TWpFMk1UVXpOVm9YCkRUSXpNRGt4TVRFMk1UVXpOVm93V0RFTE1Ba0dBMVVFQmhNQ1ZWTXhDekFKQmdOVkJBZ01Ba05CTVNFd0h3WUQKVlFRS0RCaEpiblJsY201bGRDQlhhV1JuYVhSeklGQjBlU0JNZEdReEdUQVhCZ05WQkFNTUVHTmhabVV1WlhoaApiWEJzWlM1amIyMHdnZ0VpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElCRHdBd2dnRUtBb0lCQVFDcDZLbjdzeTgxCnAwanVKL2N5ayt2Q0FtbHNmanRGTTJtdVpOSzBLdGVjcUcyZmpXUWI1NXhRMVlGQTJYT1N3SEFZdlNkd0kyaloKcnVXOHFYWENMMnJiNENaQ0Z4d3BWRUNyY3hkam0zdGVWaVJYVnNZSW1tSkhQUFN5UWdwaW9iczl4N0RsTGM2SQpCQTBaalVPeWwwUHFHOVNKZXhNVjczV0lJYTVyRFZTRjJyNGtTa2JBajREY2o3TFhlRmxWWEgySTVYd1hDcHRDCm42N0pDZzQyZitrOHdnemNSVnA4WFprWldaVmp3cTlSVUtEWG1GQjJZeU4xWEVXZFowZXdSdUtZVUpsc202OTIKc2tPcktRajB2a29QbjQxRUUvK1RhVkVwcUxUUm9VWTNyemc3RGtkemZkQml6Rk8yZHNQTkZ4MkNXMGpYa05MdgpLbzI1Q1pyT2hYQUhBZ01CQUFFd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFLSEZDY3lPalp2b0hzd1VCTWRMClJkSEliMzgzcFdGeW5acS9MdVVvdnNWQTU4QjBDZzdCRWZ5NXZXVlZycTVSSWt2NGxaODFOMjl4MjFkMUpINnIKalNuUXgrRFhDTy9USkVWNWxTQ1VwSUd6RVVZYVVQZ1J5anNNL05VZENKOHVIVmhaSitTNkZBK0NuT0Q5cm4yaQpaQmVQQ0k1ckh3RVh3bm5sOHl3aWozdnZRNXpISXV5QmdsV3IvUXl1aTlmalBwd1dVdlVtNG52NVNNRzl6Q1Y3ClBwdXd2dWF0cWpPMTIwOEJqZkUvY1pISWc4SHc5bXZXOXg5QytJUU1JTURFN2IvZzZPY0s3TEdUTHdsRnh2QTgKN1dqRWVxdW5heUlwaE1oS1JYVmYxTjM0OWVOOThFejM4Zk9USFRQYmRKakZBL1BjQytHeW1lK2lHdDVPUWRGaAp5UkU9Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
8+
tls.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBcWVpcCs3TXZOYWRJN2lmM01wUHJ3Z0pwYkg0N1JUTnBybVRTdENyWG5LaHRuNDFrCkcrZWNVTldCUU5semtzQndHTDBuY0NObzJhN2x2S2wxd2k5cTIrQW1RaGNjS1ZSQXEzTVhZNXQ3WGxZa1YxYkcKQ0pwaVJ6ejBza0lLWXFHN1BjZXc1UzNPaUFRTkdZMURzcGRENmh2VWlYc1RGZTkxaUNHdWF3MVVoZHErSkVwRwp3SStBM0kreTEzaFpWVng5aU9WOEZ3cWJRcCt1eVFvT05uL3BQTUlNM0VWYWZGMlpHVm1WWThLdlVWQ2cxNWhRCmRtTWpkVnhGbldkSHNFYmltRkNaYkp1dmRySkRxeWtJOUw1S0Q1K05SQlAvazJsUkthaTAwYUZHTjY4NE93NUgKYzMzUVlzeFR0bmJEelJjZGdsdEkxNURTN3lxTnVRbWF6b1Z3QndJREFRQUJBb0lCQVFDUFNkU1luUXRTUHlxbApGZlZGcFRPc29PWVJoZjhzSStpYkZ4SU91UmF1V2VoaEp4ZG01Uk9ScEF6bUNMeUw1VmhqdEptZTIyM2dMcncyCk45OUVqVUtiL1ZPbVp1RHNCYzZvQ0Y2UU5SNThkejhjbk9SVGV3Y290c0pSMXBuMWhobG5SNUhxSkpCSmFzazEKWkVuVVFmY1hackw5NGxvOUpIM0UrVXFqbzFGRnM4eHhFOHdvUEJxalpzVjdwUlVaZ0MzTGh4bndMU0V4eUZvNApjeGI5U09HNU9tQUpvelN0Rm9RMkdKT2VzOHJKNXFmZHZ5dGdnOXhiTGFRTC94MGtwUTYyQm9GTUJEZHFPZVBXCktmUDV6WjYvMDcvdnBqNDh5QTFRMzJQem9idWJzQkxkM0tjbjMyamZtMUU3cHJ0V2wrSmVPRmlPem5CUUZKYk4KNHFQVlJ6NWhBb0dCQU50V3l4aE5DU0x1NFArWGdLeWNrbGpKNkY1NjY4Zk5qNUN6Z0ZScUowOXpuMFRsc05ybwpGVExaY3hEcW5SM0hQWU00MkpFUmgySi9xREZaeW5SUW8zY2czb2VpdlVkQlZHWTgrRkkxVzBxZHViL0w5K3l1CmVkT1pUUTVYbUdHcDZyNmpleHltY0ppbS9Pc0IzWm5ZT3BPcmxEN1NQbUJ2ek5MazRNRjZneGJYQW9HQkFNWk8KMHA2SGJCbWNQMHRqRlhmY0tFNzdJbUxtMHNBRzR1SG9VeDBlUGovMnFyblRuT0JCTkU0TXZnRHVUSnp5K2NhVQprOFJxbWRIQ2JIelRlNmZ6WXEvOWl0OHNaNzdLVk4xcWtiSWN1YytSVHhBOW5OaDFUanNSbmU3NFowajFGQ0xrCmhIY3FIMHJpN1BZU0tIVEU4RnZGQ3haWWRidUI4NENtWmlodnhicFJBb0dBSWJqcWFNWVBUWXVrbENkYTVTNzkKWVNGSjFKelplMUtqYS8vdER3MXpGY2dWQ0thMzFqQXdjaXowZi9sU1JxM0hTMUdHR21lemhQVlRpcUxmZVpxYwpSMGlLYmhnYk9jVlZrSkozSzB5QXlLd1BUdW14S0haNnpJbVpTMGMwYW0rUlk5WUdxNVQ3WXJ6cHpjZnZwaU9VCmZmZTNSeUZUN2NmQ21mb09oREN0enVrQ2dZQjMwb0xDMVJMRk9ycW40M3ZDUzUxemM1em9ZNDR1QnpzcHd3WU4KVHd2UC9FeFdNZjNWSnJEakJDSCtULzZzeXNlUGJKRUltbHpNK0l3eXRGcEFOZmlJWEV0LzQ4WGY2ME54OGdXTQp1SHl4Wlp4L05LdER3MFY4dlgxUE9ucTJBNWVpS2ErOGpSQVJZS0pMWU5kZkR1d29seHZHNmJaaGtQaS80RXRUCjNZMThzUUtCZ0h0S2JrKzdsTkpWZXN3WEU1Y1VHNkVEVXNEZS8yVWE3ZlhwN0ZjanFCRW9hcDFMU3crNlRYcDAKWmdybUtFOEFSek00NytFSkhVdmlpcS9udXBFMTVnMGtKVzNzeWhwVTl6WkxPN2x0QjBLSWtPOVpSY21Vam84UQpjcExsSE1BcWJMSjhXWUdKQ2toaVd4eWFsNmhZVHlXWTRjVmtDMHh0VGwvaFVFOUllTktvCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==

‎examples/custom-resources/custom-listeners/cafe-virtual-server.yaml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
apiVersion: k8s.nginx.org/v1
2+
kind: VirtualServer
3+
metadata:
4+
name: cafe
5+
spec:
6+
listener:
7+
http: http-8083
8+
https: https-8443
9+
host: cafe.example.com
10+
tls:
11+
secret: cafe-secret
12+
upstreams:
13+
- name: tea
14+
service: tea-svc
15+
port: 80
16+
- name: c D30D offee
17+
service: coffee-svc
18+
port: 80
19+
routes:
20+
- path: /tea
21+
action:
22+
pass: tea
23+
- path: /coffee
24+
action:
25+
pass: coffee

‎examples/custom-resources/custom-listeners/cafe.yaml‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
name: coffee
5+
spec:
6+
replicas: 2
7+
selector:
8+
matchLabels:
9+
app: coffee
10+
template:
11+
metadata:
12+
labels:
13+
app: coffee
14+
spec:
15+
containers:
16+
- name: coffee
17+
image: nginxdemos/nginx-hello:plain-text
18+
ports:
19+
- containerPort: 8080
20+
---
21+
apiVersion: v1
22+
kind: Service
23+
metadata:
24+
name: coffee-svc
25+
spec:
26+
ports:
27+
- port: 80
28+
targetPort: 8080
29+
protocol: TCP
30+
name: http
31+
selector:
32+
app: coffee
33+
---
34+
apiVersion: apps/v1
35+
kind: Deployment
36+
metadata:
37+
name: tea
38+
spec:
39+
replicas: 1
40+
selector:
41+
matchLabels:
42+
app: tea
43+
template:
44+
metadata:
45+
labels:
46+
app: tea
47+
spec:
48+
containers:
49+
- name: tea
50+
image: nginxdemos/nginx-hello:plain-text
51+
ports:
52+
- containerPort: 8080
53+
---
54+
apiVersion: v1
55+
kind: Service
56+
metadata:
57+
name: tea-svc
58+
spec:
59+
ports:
60+
- port: 80
61+
targetPort: 8080
62+
protocol: TCP
63+
name: http
64+
selector:
65+
app: tea

‎examples/custom-resources/custom-listeners/global-configuration.yaml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
apiVersion: k8s.nginx.org/v1alpha1
2+
kind: GlobalConfiguration
3+
metadata:
4+
name: nginx-configuration
5+
namespace: nginx-ingress
6+
spec:
7+
listeners:
8+
- name: http-8083
9+
port: 8083
10+
protocol: HTTP
11+
- name: https-8443
12+
port: 8443
13+
protocol: HTTP
14+
ssl: true

‎internal/configs/configurator.go‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1241,6 +1241,30 @@ func (cnf *Configurator) UpdateConfig(cfgParams *ConfigParams, resources Extende
12411241
return allWarnings, nil
12421242
}
12431243

1244+
// UpdateVirtualServers updates VirtualServers.
1245+
func (cnf *Configurator) UpdateVirtualServers(updatedVSExes []*VirtualServerEx, deletedKeys []string) []error {
1246+
var errList []error
1247+
for _, vsEx := range updatedVSExes {
1248+
_, err := cnf.addOrUpdateVirtualServer(vsEx)
1249+
if err != nil {
1250+
errList = append(errList, fmt.Errorf("error adding or updating VirtualServer %v/%v: %w", vsEx.VirtualServer.Namespace, vsEx.VirtualServer.Name, err))
1251+
}
1252+
}
1253+
1254+
for _, key := range deletedKeys {
1255+
err := cnf.DeleteVirtualServer(key, true)
1256+
if err != nil {
1257+
errList = append(errList, fmt.Errorf("error when removing VirtualServer %v: %w", key, err))
1258+
}
1259+
}
1260+
1261+
if err := cnf.reload(nginx.ReloadForOtherUpdate); err != nil {
1262+
errList = append(errList, fmt.Errorf("error when updating VirtualServer: %w", err))
1263+
}
1264+
1265+
return errList
1266+
}
1267+
12441268
// UpdateTransportServers updates TransportServers.
12451269
func (cnf *Configurator) UpdateTransportServers(updatedTSExes []*TransportServerEx, deletedKeys []string) []error {
12461270
var errList []error

0 commit comments

Comments
 (0)
0