I’m a huge proponent of Kubernetes, but I’m not afraid to admit it has flaws. It has plenty. Like most people, I tend to use the NGINX Ingress Controller. It is simple and works well. Set it up, add an annotation to your ingress, and you’re golden. If you’re using cert-manager, it also plays nicely.

The challenges arise when you try to use other ingress controllers which don’t conform to the standard kubernetes ingress model. It is understandable that people want to extend beyond that model. The current model (which has been in beta forever) often leads to annotation hell. So other tools have gotten inventive to get around this.

Traefik for instance has a CustomResourceDefinition (CRD) which enables you to create IngressRoutes which as you can see below contains no annotations. As of the current version, they also still support the Kubernetes ingress resource with a traefik annotation.

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
  name: ingressroutebar

    - web
  - match: Host(`bar.com`) && PathPrefix(`/`)
    kind: Rule
    - name: whoami
      port: 80

Istio, which is first a service mesh, takes a different approach. You expose Gateways and setup VirtualServices (kind of like an Apache VirtualHost file). Rather than having separate ingress resources for each service, you define a VirtualService for each gateway.

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
  name: httpbin-gateway
    istio: ingressgateway # use Istio default gateway implementation
  - port:
      number: 80
      name: http
      protocol: HTTP
    - "httpbin.example.com"
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
  name: httpbin
  - "httpbin.example.com"
  - httpbin-gateway
  - match:
    - uri:
        prefix: /status
    - uri:
        prefix: /delay
    - destination:
          number: 8000
        host: httpbin

What is the problem?

It’s really cool that people are building these alternative options for Kubernetes ingress. But part of what makes Kubernetes so great is the standardized way of working. Introducing new ways of working makes it more challenging to build platforms on top of Kubernetes.

For instance, I work with an application which dynamically provisions other services and ingresses on my Kubernetes cluster. I can modify it to have special annotations like telling it to use Nginx or Traefik. But to use an altogether new pattern for setting up Kubernetes ingress such as Istio would require some adding new functionality.

It makes it challenging to build Kubernetes-native software when customers will want to use all ingress options out there. Sure, you can say that you will only support the nginx ingress controller, but then you are limiting the set of viable customers.

Write A Comment