You likely know that Services are how Kubernetes provides load balancing, exposing an application running on a set of Pods as a network service, and providing a single DNS name for that application.
If you have used a non-Kubernetes load balancer, you will also know that while active/active load balancing is probably the most common approach, active/passive load balancing is also a thing in that world. There can be a number of reasons one might want to do active/passive load balancing, for example licensing constraints, application requirements for consistent writes to local file systems, etc.
At first glance, however, it would appear that Kubernetes Services support only active/active approaches. There are ways to do active/passive load-balancing, however, if you are sufficiently motivated — but first you should consider some alternatives:
So, how can we do active/passive load-balancing? One of the easiest and most effective ways relies on 3 key facts:
What does the solution look like?
command: - bash - -c - | while true; do active=$(getActivePod) passive=$(getPassivePod) # If neither pod is active, then arbitrarily set the first # Pod to active (we'll check its status and adjust if # necessary immediately after if there is no active pod use kubectl to patch pod helloworld-0 set label role: active use kubectl to patch pod helloworld-1 set label role: passive active="helloworld-0" passive="helloworld-1" fi # Detect if active pod is unhealthy. This logic will likely # be application-dependent in your implementation. In this # example, only if the response from the HTTP request is 200, # is the pod considered healthy active_http_response=$(use curl to get status of active pod) echo "Active pod HTTP response: $active_http_response" passive_http_response=$(use curl to get status of passive pod) echo "Passive pod HTTP response: $passive_http_response" if [ "$active_http_response" = "200" ]; then echo "Active node passed healthcheck" else if [ "$passive_http_response" = "200" ]; then echo "Enabling $passive" use kubectl to patch $active pod to set label role: passive use kubectl to patch pod $passive to set label role: active else echo "Passive pod failing health check as well... leaving active pod enabled" fi fi # sleep for however long you want to wait between polling # Pod status echo "Sleeping for 60 seconds..." sleep 60 done
A word of caution — the example above is meant to be illustrative of the concept, not to be a production-ready implementation:
Hopefully, however, the provided code illustrates the general concept.
After deploying and waiting for just a moment or two, the helloworld-svc has one endpoint (which happens to correspond to Pod helloworld-statefulset-0):
>kubectl -n local-demo-activepassive4-ns describe svc helloworld-svc Name: helloworld-svc Namespace: local-demo-activepassive4-ns Labels: <none> Annotations: <none> Selector: app.kubernetes.io/instance=main,app.kubernetes.io/name=helloworld-statefulset,role=active Type: ClusterIP IP Family Policy: SingleStack IP Families: IPv4 IP: None IPs: None Port: http 8080/TCP TargetPort: 8080/TCP Endpoints: 10.244.120.66:8080 <-- pod 0’s IP Session Affinity: None Events: <none>
After a while, the loadbalancer pod shows the following log:
helloworld-0 is the active pod helloworld-1 is the passive pod Active pod HTTP response: 500 Passive pod HTTP response: 200 Detected application health failure Enabling helloworld-statefulset-1 pod/helloworld-statefulset-0 patched pod/helloworld-statefulset-1 patched helloworld-1 is the active pod helloworld-0 is the passive pod Active pod HTTP response: 200 Passive pod HTTP response: 200 Active node passed healthcheck ...
and describing the helloworld-svc again shows the following:
>kubectl -n local-demo-activepassive4-ns describe svc helloworld-svc Name: helloworld-svc Namespace: local-demo-activepassive4-ns Labels: <none> Annotations: <none> Selector: app.kubernetes.io/instance=main,app.kubernetes.io/name=helloworld-statefulset,role=active Type: ClusterIP IP Family Policy: SingleStack IP Families: IPv4 IP: None IPs: None Port: http 8080/TCP TargetPort: 8080/TCP Endpoints: 10.244.120.73:8080 <-- Pod 1’s IP Session Affinity: None Events: <none>
As you can see, initially Pod helloworld-statefulset-0 was active. Its health check page at some point returned a 500 to signal that it was no longer healthy (in this example, that is done randomly, but presumably you would replace that health-check with a real implementation of something more applicable to your application). The script toggled the label value on the active pod to the role: passive, thereby removing it from the Service and the passive pod to the role: active, thereby adding it to the Service.
The full source code for the example is available over on GitHub.
Join other developers and claim your FAUN account now!
Architect, Sun Life Financial@psdally
Only registered users can post comments. Please, login or signup.