Ingress: Obviating Platform Load Balancers
https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/
- Load balancers are useful for service discovery
- Kubernetes provides
Service
natively, which does this just as well!- Platform load balancers have an associated cost, per-instance
- Ingress unifies HTTP traffic behind a single Platform Load Balancer
- Deis/Hephy Workflow exposes two services in standard configuration:
-
builder
(:2222 ssh)
-
controller-api-server
(80, 443 http/s)
- (and all of your applications hosted on Workflow too, of course)
-
builder
- Workflow's default configuration includes a
router
component- Router is an optional component that fulfilled the role of Ingress circa
K8S 1.1-1.4
era - Kubernetes provides Ingress natively in
networking.k8s.io/v1beta1
API since1.14
, or theextensions/v1beta1
API since at least1.6
(extensions/v1beta1
is sunsetting next year)
- Router is an optional component that fulfilled the role of Ingress circa
- Thanks to Experimental Native Ingress support,
Deis Workflow is able to removerouter
and use ingress instead!
But:
- In Hephy
v2.21.0
(the latest release), the documented Native Ingress mode with norouter
requirement,still requires cluster to provision one separate Platform Load Balancer for serving Builder traffic. ☹️
Using nginx-ingress
, a configuration is possible that does not provision two separate load balancers
Because of hostNetwork mode, bootstrapped clusters might not even need one LoadBalancer type service!
For bare-bones clusters, this type of configuration is a cost-saving measure. Each node of the cluster serves the role of Load Balancer and ingress serves a Reverse Proxy which routes traffic based on a Host header from the client.
Unfortunately, Kubernetes Ingress spec currently does not provide an L7 TCP configuration mode.
- Good news! nginx-ingress[1] has an annotation which can be used for Exposing TCP and UDP services
apiVersion: v1
kind: ConfigMap
metadata:
name: tcp-services
namespace: ingress-nginx
data:
2222: "deis/deis-builder:2222"
TODO: test this configuration on a live cluster
TODO: document steps to remove the load balancer for deis-builder
TODO: make this a first-class configuration in an upcoming release of Hephy Workflow
[1]: https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/
[2]: https://github.com/kubernetes/ingress-nginx/blob/master/deploy/provider/aws/service-l7.yaml