8000 Add health checks to transport server · nginx/kubernetes-ingress@8e39b6f · GitHub
[go: up one dir, main page]
Skip to content

Commit 8e39b6f

Browse files
LorcanMcVeighLorcanMcVeigh
authored
Add health checks to transport server
1 parent 5811592 commit 8e39b6f

File tree

13 files changed

+489
-9
lines changed

13 files changed

+489
-9
lines changed

‎deployments/common/crds-v1beta1/k8s.nginx.org_transportservers.yaml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,24 @@ spec:
7979
properties:
8080
failTimeout:
8181
type: string
82+
healthCheck:
83+
description: HealthCheck defines the parameters for active Upstream HealthChecks.
84+
type: object
85+
properties:
86+
enable:
87+
type: boolean
88+
fails:
89+
type: integer
90+
intervals:
91+
type: string
92+
jitter:
93+
type: string
94+
passes:
95+
type: integer
96+
port:
97+
type: integer
98+
timeout:
99+
type: string
82100
maxFails:
83101
type: integer
84102
name:

‎deployments/common/crds/k8s.nginx.org_transportservers.yaml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,24 @@ spec:
8080
properties:
8181
failTimeout:
8282
type: string
83+
healthCheck:
84+
description: HealthCheck defines the parameters for active Upstream HealthChecks.
85+
type: object
86+
properties:
87+
enable:
88+
type: boolean
89+
fails:
90+
type: integer
91+
intervals:
92+
type: string
93+
jitter:
94+
type: string
95+
passes:
96+
type: integer
97+
port:
98+
type: integer
99+
timeout:
100+
type: string
83101
maxFails:
84102
type: integer
85103
name:

‎deployments/helm-chart/crds/k8s.nginx.org_transportservers.yaml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,24 @@ spec:
8080
properties:
8181
failTimeout:
8282
type: string
83+
healthCheck:
84+
description: HealthCheck defines the parameters for active Upstream HealthChecks.
85+
type: object
86+
properties:
87+
enable:
88+
type: boolean
89+
fails:
90+
type: integer
91+
intervals:
92+
type: string
93+
jitter:
94+
type: string
95+
passes:
96+
type: integer
97+
port:
98+
type: integer
99+
timeout:
100+
type: string
83101
maxFails:
84102
type: integer
85103
name:

‎docs-web/configuration/transportserver-resource.md‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ This document is the reference documentation for the TransportServer resource. T
1414
- [TransportServer Specification](#transportserver-specification)
1515
- [Listener](#listener)
1616
- [Upstream](#upstream)
17+
- [Upstream.Healthcheck](#upstream-healthcheck)
1718
- [UpstreamParameters](#upstreamparameters)
1819
- [SessionParameters](#sessionparameters)
1920
- [Action](#action)
@@ -189,7 +190,68 @@ failTimeout: 30s
189190
- Sets the `time <https://nginx.org/en/docs/stream/ngx_stream_upstream_module.html#fail_timeout>`_ during which the specified number of unsuccessful attempts to communicate with the server should happen to consider the server unavailable and the period of time the server will be considered unavailable. The default is ``10s``.
190191
- ``string``
191192
- No
193+
* - ``healthCheck``
194+
- The health check configuration for the Upstream. See the `health_check <https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html#health_check>`_ directive. Note: this feature is supported only in NGINX Plus.
195+
- `healthcheck <#upstream-healthcheck>`_
196+
- No
197+
198+
```
199+
200+
### Upstream.Healthcheck
201+
202+
The Healthcheck defines an [active health check](https://nginx.org/en/docs/stream/ngx_stream_upstream_hc_module.html?#health_check). In the example below we enable a health check for an upstream and configure all the available parameters:
203+
204+
```yaml
205+
name: secure-app
206+
service: secure-app
207+
port: 8443
208+
healthCheck:
209+
enable: true
210+
interval: 20s
211+
timeout: 30s
212+
jitter: 3s
213+
fails: 5
214+
passes: 5
215+
port: 8080
216+
217+
```
218+
219+
```eval_rst
220+
.. list-table::
221+
:header-rows: 1
192222
223+
* - Field
224+
- Description
225+
- Type
226+
- Required
227+
* - ``enable``
228+
- Enables a health check for an upstream server. The default is ``false``.
229+
- ``boolean``
230+
- No
231+
* - ``interval``
232+
- The interval between two consecutive health checks. The default is ``5s``.
233+
- ``string``
234+
- No
235+
* - ``timeout``
236+
- This overrides the timeout set by `proxy_timeout <http://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_timeout>`_ which is set in `SessionParameters` for health checks. The default value is ``5s``.
237+
- ``string``
238+
- No
239+
* - ``jitter``
240+
- The time within which each health check will be randomly delayed. By default, there is no delay.
241+
- ``string``
242+
- No
243+
* - ``fails``
244+
- The number of consecutive failed health checks of a particular upstream server after which this server will be considered unhealthy. The default is ``1``.
245+
- ``integer``
246+
- No
247+
* - ``passes``
248+
- The number of consecutive passed health checks of a particular upstream server after which the server will be considered healthy. The default is ``1``.
249+
- ``integer``
250+
- No
251+
* - ``port``
252+
- The port used for health check requests. By default, the port of the upstream is used. Note: in contrast with the port of the upstream, this port is not a service port, but a port of a pod.
253+
- ``integer``
254+
- No
193255
```
194256

195257
### UpstreamParameters

‎internal/configs/transportserver.go‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,8 @@ func generateTransportServerConfig(transportServerEx *TransportServerEx, listene
3434

3535
upstreams := generateStreamUpstreams(transportServerEx, upstreamNamer, isPlus)
3636

37+
healthCheck := generateTransportServerHealthCheck(transportServerEx.TransportServer.Spec.Action.Pass,
38+
transportServerEx.TransportServer.Spec.Upstreams)
3739
var proxyRequests, proxyResponses *int
3840
var connectTimeout, nextUpstreamTimeout string
3941
var nextUpstream bool
@@ -80,6 +82,7 @@ func generateTransportServerConfig(transportServerEx *TransportServerEx, listene
8082
ProxyNextUpstream: nextUpstream,
8183
ProxyNextUpstreamTimeout: generateTime(nextUpstreamTimeout, "0"),
8284
ProxyNextUpstreamTries: nextUpstreamTries,
85+
HealthCheck: healthCheck,
8386
},
8487
Upstreams: upstreams,
8588
}
@@ -115,6 +118,48 @@ func generateStreamUpstreams(transportServerEx *TransportServerEx, upstreamNamer
115118
return upstreams
116119
}
117120

121+
func generateTransportServerHealthCheck(upstreamHealthCheckName string, upstreams []conf_v1alpha1.Upstream) *version2.StreamHealthCheck {
122+
var hc *version2.StreamHealthCheck
123+
for _, u := range upstreams {
124+
if u.Name == upstreamHealthCheckName {
125+
if u.HealthCheck == nil || !u.HealthCheck.Enabled {
126+
return nil
127+
}
128+
hc = generateTransportServerHealthCheckWithDefaults(u)
129+
130+
hc.Enabled = u.HealthCheck.Enabled
131+
hc.Interval = generateTime(u.HealthCheck.Interval)
 B465
132+
hc.Jitter = generateTime(u.HealthCheck.Jitter)
133+
hc.Timeout = generateTime(u.HealthCheck.Timeout)
134+
135+
if u.HealthCheck.Fails > 0 {
136+
hc.Fails = u.HealthCheck.Fails
137+
}
138+
139+
if u.HealthCheck.Passes > 0 {
140+
hc.Passes = u.HealthCheck.Passes
141+
}
142+
143+
if u.HealthCheck.Port > 0 {
144+
hc.Port = u.HealthCheck.Port
145+
}
146+
}
147+
}
148+
return hc
149+
}
150+
151+
func generateTransportServerHealthCheckWithDefaults(up conf_v1alpha1.Upstream) *version2.StreamHealthCheck {
152+
return &version2.StreamHealthCheck{
153+
Enabled: false,
154+
Timeout: "5s",
155+
Jitter: "0",
156+
Port: up.Port,
157+
Interval: "5s",
158+
Passes: 1,
159+
Fails: 1,
160+
}
161+
}
162+
118163
func generateStreamUpstream(upstream *conf_v1alpha1.Upstream, upstreamNamer *upstreamNamer, endpoints []string, isPlus bool) version2.StreamUpstream {
119164
var upsServers []version2.StreamUpstreamServer
120165

‎internal/configs/transportserver_test.go‎

Lines changed: 117 additions & 3 deletions
10815
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ import (
44
"reflect"
55
"testing"
66

7+
"github.com/google/go-cmp/cmp"
78
"github.com/nginxinc/kubernetes-ingress/internal/configs/version2"
89
conf_v1alpha1 "github.com/nginxinc/kubernetes-ingress/pkg/apis/configuration/v1alpha1"
910
meta_v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -134,6 +135,7 @@ func TestGenerateTransportServerConfigForTCP(t *testing.T) {
134135
ProxyNextUpstreamTries: 0,
135136
ProxyNextUpstreamTimeout: "0s",
136137
ProxyTimeout: "50s",
138+
HealthCheck: nil,
137139
},
138140
}
139141

@@ -217,6 +219,7 @@ func TestGenerateTransportServerConfigForTLSPasstrhough(t *testing.T) {
217219
ProxyNextUpstreamTimeout: "0s",
218220
ProxyNextUpstreamTries: 0,
219221
ProxyTimeout: "10m",
222+
HealthCheck: nil,
220223
},
221224
}
222225

@@ -244,9 +247,10 @@ func TestGenerateTransportServerConfigForUDP(t *testing.T) {
244247
},
245248
Upstreams: []conf_v1alpha1.Upstream{
246249
{
247-
Name: "udp-app",
248-
Service: "udp-app-svc",
249-
Port: 5001,
250+
Name: "udp-app",
251+
Service: "udp-app-svc",
252+
Port: 5001,
253+
HealthCheck: &conf_v1alpha1.HealthCheck{},
250254
},
251255
},
252256
UpstreamParameters: &conf_v1alpha1.UpstreamParameters{
@@ -304,6 +308,7 @@ func TestGenerateTransportServerConfigForUDP(t *testing.T) {
304308
ProxyNextUpstreamTimeout: "0s",
305309
ProxyNextUpstreamTries: 0,
306310
ProxyTimeout: "10m",
311+
HealthCheck: nil,
307312
},
308313
}
309314

@@ -345,6 +350,115 @@ func TestGenerateUnixSocket(t *testing.T) {
345350
}
346351
}
347352

353+
func TestGenerateTransportServerHealthChecks(t *testing.T) {
354+
upstreamName := "dns-tcp"
355+
tests := []struct {
356+
upstreams []conf_v1alpha1.Upstream
357+
expected *version2.StreamHealthCheck
358+
msg string
359+
}{
360+
{
361+
upstreams: []conf_v1alpha1.Upstream{
362+
{
363+
Name: "dns-tcp",
364+
HealthCheck: &conf_v1alpha1.HealthCheck{
365+
Enabled: false,
366+
Timeout: "30s",
367+
Jitter: "30s",
368+
Port: 80,
369+
Interval: "20s",
370+
Passes: 4,
371+
Fails: 5,
372+
},
373+
},
374+
},
375+
expected: nil,
376+
msg: "health checks disabled",
377+
},
378+
{
379+
upstreams: []conf_v1alpha1.Upstream{
380+
{
381+
Name: "dns-tcp",
382+
HealthCheck: &conf_v1alpha1.HealthCheck{},
383+
},
384+
},
385+
expected: nil,
386+
msg: "empty health check",
387+
},
388+
{
389+
upstreams: []conf_v1alpha1.Upstream{
390+
{
391+
Name: "dns-tcp",
392+
HealthCheck: &conf_v1alpha1.HealthCheck{
393+
Enabled: true,
394+
Timeout: "40s",
395+
Jitter: "30s",
396+
Port: 88,
397+
Interval: "20s",
398+
Passes: 4,
399+
Fails: 5,
400+
},
401+
},
402+
},
403+
expected: &version2.StreamHealthCheck{
404+
Enabled: true,
405+
Timeout: "40s",
406+
Jitter: "30s",
407+
Port: 88,
408+
Interval: "20s",
409+
Passes: 4,
410+
Fails: 5,
411+
},
412+
msg: "valid health checks",
413+
},
414+
{
415+
upstreams: []conf_v1alpha1.Upstream{
416+
{
417+
Name: "dns-tcp",
418+
HealthCheck: &conf_v1alpha1.HealthCheck{
419+
Enabled: true,
420+
Timeout: "40s",
421+
Jitter: "30s",
422+
Port: 88,
423+
Interval: "20s",
424+
Passes: 4,
425+
Fails: 5,
426+
},
427+
},
428+
{
429+
Name: "dns-tcp-2",
430+
HealthCheck: &conf_v1alpha1.HealthCheck{
431+
Enabled: false,
432+
Timeout: "50s",
433+
Jitter: "60s",
434+
Port: 89,
435+
Interval: "10s",
436+
Passes: 9,
437+
Fails: 7,
438+
},
439+
},
440+
},
441+
expected: &version2.StreamHealthCheck{
442+
Enabled: true,
443+
Timeout: "40s",
444+
Jitter: "30s",
445+
Port: 88,
446+
Interval: "20s",
447+
Passes: 4,
448+
Fails: 5,
449+
},
450+
msg: "valid 2 health checks",
451+
},
452+
}
453+
454+
for _, test := range tests {
455+
result := generateTransportServerHealthCheck(upstreamName, test.upstreams)
456+
if diff := cmp.Diff(test.expected, result); diff != "" {
457+
t.Errorf("generateTransportServerHealthCheck() '%v' mismatch (-want +got):\n%s", test.msg, diff)
458+
}
459+
}
460+
}
461+
348462
func intPointer(value int) *int {
349463
return &value
350464
}

‎internal/configs/version2/nginx-plus.transportserver.tmpl‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,12 @@ server {
3030

3131
proxy_pass {{ $s.ProxyPass }};
3232

33+
{{ if $s.HealthCheck }}
34+
health_check interval={{ $s.HealthCheck.Interval }} port={{ $s.HealthCheck.Port }}
35+
passes={{ $s.HealthCheck.Passes }} jitter={{ $s.HealthCheck.Jitter }} fails={{ $s.HealthCheck.Fails }} {{ if $s.UDP }}udp{{ end }};
36+
health_check_timeout {{ $s.HealthCheck.Timeout }};
37+
{{ end }}
38+
3339
proxy_timeout {{ $s.ProxyTimeout }};
3440
proxy_connect_timeout {{ $s.ProxyConnectTimeout }};
3541

0 commit comments

Comments
 (0)
0