Deis/Hephy Workflow Pad » Ingress: Obviating Pla...
Export

Ingress: Obviating Platform Load Balancers

https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/

  • Load balancers are useful for service discovery
  • Kubernetes provides Service natively, which does this just as well!
    • Platform load balancers have an associated cost, per-instance
    • Ingress unifies HTTP traffic behind a single Platform Load Balancer
  • Deis/Hephy Workflow exposes two services in standard configuration:
    • builder (:2222 ssh)
    • controller-api-server (80, 443 http/s)
    • (and all of your applications hosted on Workflow too, of course)
  • Workflow's default configuration includes a router component
    • Router is an optional component that fulfilled the role of Ingress circa K8S 1.1-1.4 era
    • Kubernetes provides Ingress natively in networking.k8s.io/v1beta1 API since 1.14, or the extensions/v1beta1 API since at least 1.6 (extensions/v1beta1 is sunsetting next year)
  • Thanks to Experimental Native Ingress support,
    Deis Workflow is able to remove router and use ingress instead!

But:

  • In Hephy v2.21.0 (the latest release), the documented Native Ingress mode with no router requirement,
    still requires cluster to provision one separate Platform Load Balancer for serving Builder traffic. ☹️

Using nginx-ingress, a configuration is possible that does not provision two separate load balancers

Because of hostNetwork mode, bootstrapped clusters might not even need one LoadBalancer type service!

For bare-bones clusters, this type of configuration is a cost-saving measure.  Each node of the cluster serves the role of Load Balancer and ingress serves a Reverse Proxy which routes traffic based on a Host header from the client.

Unfortunately, Kubernetes Ingress spec currently does not provide an L7 TCP configuration mode.

  • Good news! nginx-ingress[1] has an annotation which can be used for Exposing TCP and UDP services
apiVersion: v1
kind: ConfigMap
metadata:
  name: tcp-services
  namespace: ingress-nginx
data:
  2222: "deis/deis-builder:2222"

TODO: test this configuration on a live cluster

TODO: document steps to remove the load balancer for deis-builder

TODO: make this a first-class configuration in an upcoming release of Hephy Workflow

[1]: https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/
[2]: https://github.com/kubernetes/ingress-nginx/blob/master/deploy/provider/aws/service-l7.yaml

Back to top