Kubernetes Nginx Ingress Controller Logs

Posted by

This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Community Hub.

By default, the Kubernetes Nginx Ingress Controller writes logs and traces to stdout / stderr.


These logs and traces are stored in the ContainerLog table. You can navigate to an autogenerated query from inside the Container Insights blade from the Azure Portal.



During an engagement, we faced an issue with viewing these logs and traces. We could view the logs in the “Live Logs” tab inside the Portal, but none of these got sent to Log Analytics workspace and we were unable to retain and run Kusto queries against that data.


As it turned out, after a troubleshooting session with a colleague, the reason the Nginx Ingress Controller logs and traces were not sent in Log Analytics workspace was the fact that the controller had been deployed to the kube-system namespace, which happens to be excluded by default from the container log collection, based on the below note.


To solve this, the recommended approach, based also on the example in the official Microsoft Learn documentation, is to redeploy the Ingress Controller to a different namespace.

Taking it to the next level, we wanted to do this with zero downtime. For that, a secondary Ingress Controller needs to be created. Then, the ingress routes need to be duplicated to use the secondary ingress controller. Finally, you should do a swap in the Firewall Appliance to redirect traffic to the new Ingress IP address.

Below you can find the detailed steps to deploy and configure a secondary ingress controller:

  • Create an additional ingress controller on a different / appropriate namespace, providing a custom ingressClass (e.g., “nginx-v2”), given the default “nginx” ingressClass will already be in use by the existing ingress controller. The helm install command is simplified for readability, you will probably have additional parameters as per your script, but the highlighted ones are required:

helm install nginx-ingress ingress-nginx/ingress-nginx \

    --namespace ingress-nginx-v2 \

    --create-namespace \

    --set controller.ingressClass=nginx-v2 \

    --set controller.ingressClassResource.name=nginx-v2 \

    --set controller.ingressClassResource.controllerValue="k8s.io/nginx-v2"

  • At this point, there should be two ingress controllers on different namespaces and two different ingressClass definitions:

kubectl get svc --selector=app.kubernetes.io/name=ingress-nginx -A


NAMESPACE          NAME                                               TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)                      AGE

ingress-nginx-v2   nginx-ingress-ingress-nginx-controller             LoadBalancer   80:32746/TCP,443:31787/TCP   20m

ingress-nginx-v2   nginx-ingress-ingress-nginx-controller-admission   ClusterIP   <none>           443/TCP                      20m

ingress-nginx      ingress-nginx-controller                           LoadBalancer   80:32040/TCP,443:30183/TCP   15d

ingress-nginx      ingress-nginx-controller-admission                 ClusterIP    <none>           443/TCP                      15d



kubectl get ingressclass



nginx      k8s.io/ingress-nginx   <none>       15d

nginx-v2   k8s.io/nginx-v2        <none>       23m

  • Duplicate the existing ingress routes, as they will be referencing the original ingress controller:

k get ingress -A



contoso     contoso-ingress   nginx   *   80      15d

  • This means you need to create an exact copy of all ingress routes, changing the “ingressClassName” to the newly created ingressClass:

cat << EOF | kubectl apply -f -

apiVersion: networking.k8s.io/v1

kind: Ingress


  name: contoso-ingress-v2

  namespace: contoso


    nginx.ingress.kubernetes.io/ssl-redirect: "false"

    nginx.ingress.kubernetes.io/use-regex: "true"

    nginx.ingress.kubernetes.io/rewrite-target: /$2


  ingressClassName: nginx-v2


  - http:


      - path: /contoso(/|$)(.*)

        pathType: Prefix



            name: contoso-webapp


              number: 80



ingress.networking.k8s.io/contoso-ingress-v2 created



kubectl get ingress -A


NAMESPACE   NAME                 CLASS      HOSTS   ADDRESS          PORTS   AGE

contoso     contoso-ingress      nginx      *          80      15d

contoso     contoso-ingress-v2   nginx-v2   *          80      85s

At this point, you can access the applications using both ingress controllers, and you should be able to redirect all traffic to the new ingress after performing the necessary validations.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.