I’m a huge proponent of Kubernetes, but I’m not afraid to admit it has flaws. It has plenty. Like most people, I tend to use the NGINX Ingress Controller. It is simple and works well. Set it up, add an annotation to your ingress, and you’re golden. If you’re using cert-manager, it also plays nicely.
The challenges arise when you try to use other ingress controllers which don’t conform to the standard kubernetes ingress model. It is understandable that people want to extend beyond that model. The current model (which has been in beta forever) often leads to annotation hell. So other tools have gotten inventive to get around this.
Traefik for instance has a CustomResourceDefinition (CRD) which enables you to create IngressRoutes which as you can see below contains no annotations. As of the current version, they also still support the Kubernetes ingress resource with a traefik annotation.
--- apiVersion: traefik.containo.us/v1alpha1 kind: IngressRoute metadata: name: ingressroutebar spec: entryPoints: - web routes: - match: Host(`bar.com`) && PathPrefix(`/`) kind: Rule services: - name: whoami port: 80
Istio, which is first a service mesh, takes a different approach. You expose Gateways and setup VirtualServices (kind of like an Apache VirtualHost file). Rather than having separate ingress resources for each service, you define a VirtualService for each gateway.
--- apiVersion: networking.istio.io/v1alpha3 kind: Gateway metadata: name: httpbin-gateway spec: selector: istio: ingressgateway # use Istio default gateway implementation servers: - port: number: 80 name: http protocol: HTTP hosts: - "httpbin.example.com" --- apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: httpbin spec: hosts: - "httpbin.example.com" gateways: - httpbin-gateway http: - match: - uri: prefix: /status - uri: prefix: /delay route: - destination: port: number: 8000 host: httpbin
What is the problem?
It’s really cool that people are building these alternative options for Kubernetes ingress. But part of what makes Kubernetes so great is the standardized way of working. Introducing new ways of working makes it more challenging to build platforms on top of Kubernetes.
For instance, I work with an application which dynamically provisions other services and ingresses on my Kubernetes cluster. I can modify it to have special annotations like telling it to use Nginx or Traefik. But to use an altogether new pattern for setting up Kubernetes ingress such as Istio would require some adding new functionality.
It makes it challenging to build Kubernetes-native software when customers will want to use all ingress options out there. Sure, you can say that you will only support the nginx ingress controller, but then you are limiting the set of viable customers.