Internal HTTP Application Routing on Azure Kubernetes Service

The HTTP application routing solution configures an Ingress controller in your AKS cluster. It is a quick and dirty way to create an associated Azure DNS zone and automatically assign DNS names for your applications’ endpoints.

Although it should not be used in production environments (because of it lacks the necessary number of replicas and scaling to handle large amounts of trafiic), I find it very usefull for Dev/Test deployments of AKS, when I want to have Ingress capabilities with the least amount of effort.

To enable this addon on your AKS cluster:

az aks enable-addons --resource-group myResourceGroup --name myAKSCluster --addons http_application_routing

Or to enable it at cluster creation:

az aks create --resource-group myResourceGroup --name myAKSCluster --enable-addons http_application_routing

More details on how to use this addon can be found here.

Publishing applications on an internal load balancer

Once you’ve got the http-application-routing addon deployed, you can now publish your Ingress on the AKS external load balancer. But what if you want to publish it on the internal load balancer instead?

In some scenarios this is important to publish the service internally within the VNet and not expose it to the internet. It can be very usefull for development and testing scenarios for applications which are sensitive.

For that you’re going to have to change the way the http-application-routing addon works, and switch it to use the internal rather than the external load balancer.

Steps

Step 1: list the services

kubectl get service --all-namespaces

And find the ingress service, it should be of type LoadBalancer and it should have a publicly accessible External-IP. The goal here is to switch this to a private External-IP provided by the internal load balancer.

step1

Step 2: Export the YAML file for the ingress service

kubectl get service addon-http-application-routing-nginx-ingress --namespace=kube-system -o yaml > addon-ingress.yaml

The output should look something like this:

apiVersion: v1
kind: Service
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"addonmanager.kubernetes.io/mode":"Reconcile","app":"addon-http-application-routing-nginx-ingress","kubernetes.io/cluster-service":"true"},"name":"addon-http-application-routing-nginx-ingress","namespace":"kube-system"},"spec":{"externalTrafficPolicy":"Local","ports":[{"name":"http","port":80,"targetPort":"http"},{"name":"https","port":443,"targetPort":"https"}],"selector":{"app":"addon-http-application-routing-nginx-ingress"},"type":"LoadBalancer"}}
  creationTimestamp: "2020-05-20T18:45:31Z"
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
    app: addon-http-application-routing-nginx-ingress
    kubernetes.io/cluster-service: "true"
  name: addon-http-application-routing-nginx-ingress
  namespace: kube-system
  resourceVersion: "2253"
  selfLink: /api/v1/namespaces/kube-system/services/addon-http-application-routing-nginx-ingress
  uid: e2d1c65d-09c5-48f4-bf78-c8cfc57c4c7a
spec:
  clusterIP: 10.0.60.114
  externalTrafficPolicy: Local
  healthCheckNodePort: 31162
  ports:
  - name: http
    nodePort: 32258
    port: 80
    protocol: TCP
    targetPort: http
  - name: https
    nodePort: 31091
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    app: addon-http-application-routing-nginx-ingress
  sessionAffinity: None
  type: LoadBalancer
status:
  loadBalancer:
    ingress:
    - ip: 102.133.139.219

We want to modify this YAML file to use the internal load balancer, and then re-apply it.

The modifications are simple:

We first add the annotation service.beta.kubernetes.io/azure-load-balancer-internal: "true" to the annotations.
From the metadata node, delete the creationTimestamp, resourceVersion, selfLink, and uid fields
Delete the status node and all its children

The output should be something like this:

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/azure-load-balancer-internal: "true"
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"addonmanager.kubernetes.io/mode":"Reconcile","app":"addon-http-application-routing-nginx-ingress","kubernetes.io/cluster-service":"true"},"name":"addon-http-application-routing-nginx-ingress","namespace":"kube-system"},"spec":{"externalTrafficPolicy":"Local","ports":[{"name":"http","port":80,"targetPort":"http"},{"name":"https","port":443,"targetPort":"https"}],"selector":{"app":"addon-http-application-routing-nginx-ingress"},"type":"LoadBalancer"}}
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
    app: addon-http-application-routing-nginx-ingress
    kubernetes.io/cluster-service: "true"
  name: addon-http-application-routing-nginx-ingress
  namespace: kube-system
spec:
  clusterIP: 10.0.60.114
  externalTrafficPolicy: Local
  healthCheckNodePort: 31162
  ports:
  - name: http
    nodePort: 32258
    port: 80
    protocol: TCP
    targetPort: http
  - name: https
    nodePort: 31091
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    app: addon-http-application-routing-nginx-ingress
  sessionAffinity: None
  type: LoadBalancer

Step 3: Apply the modified service YAML

When you run the following command you should get the desired result:

kubectl apply -f addon-ingress.yaml

step3

Note that if you disable then enable the addon, you’ll have to do this all over again.

I hope this helps.

zoomspeaks.tech