Karpenter: Run your Workloads upto 80% Off using Spot with AKS

Posted on by — No Comments ↓

This post has been republished via RSS; it originally appeared at: Microsoft Tech Community - Latest Blogs - .

Introduction

At last year's KubeCon North America, Microsoft announced the adoption of Karpenter in Azure Kubernetes Service (AKS) as an alternative to the Cluster Autoscaler (CA), referred to as Node Autoprovisioning (NAP). While Cluster Autoscaler has been the default node scaler in AKS/Kubernetes, there have been significant challenges that led to the adoption of Karpenter. This post delves into these challenges and explores how Karpenter addresses them.


Challenges with Cluster Autoscaler

Here is Node Autosclaing flow chart for Cluster-Autoscaler

OsamaSheikh_0-1716376503500.png

 

  • Limited to VMSS Groups: Cluster Autoscaler can only operate with Virtual Machine Scale Sets (VMSS) in AKS. Each VMSS consists of a specific group of VM instances with a specific VM SKU, hardware, and CPU:Memory ratio (e.g., Standard D4sv5 with 4 CPUs and 16 GB RAM).

    2. Node Latency: CA triggers the node pool API, which calls the VMSS instance API. This scaling process has latency, taking over a minute for a node to be ready in AKS.

    3. Node Pool Constraints: When deploying new pods, if the existing node capacity is exhausted, CA attempts to spin up a new node of the same VMSS SKU type. If that instance is unavailable, pods remain in a pending state.

    4. Scalability Limitations: CA can only scale up based on specific node pool SKU VMSS availability. It cannot leverage the capacity of other VM SKUs even if they have available resources.

Introducing Karpenter (Node Autoprovisioning)

Karpenter is an efficient node autoscaler for Kubernetes clusters, designed to optimize performance and cost. It can scale up and down worker nodes faster than Cluster Autoscaler and can launch appropriate individual nodes without creating traditional node groups in AKS.

OsamaSheikh_1-1716376503497.png

 

  • Key Features of Karpenter:

    • Efficiency: Faster scaling of Kubernetes nodes.

    • Flexibility: Launches nodes without needing VMSS.

    • Cost Optimization: Reduces overall costs and helps with patching of node images and Kubernetes versions.

    • Nodepool YAML based config which defined what types of nodes it can provision

Handling Disruptions

Disruption Controller responsible for Terminating/Replacing nodes in kubernetes cluster.
its uses one of 3 automated methods to finalise which nodes to handle via Disruption controller

  • Expiration: Karpenter will mark nodes as expired and disrupt them after they have lived a set number of seconds. this parameters act as TTL for k8s nodes

spec:
  disruption:
    consolidationPolicy: WhenUnderutilized
    expireAfter: 300s

  • Consolidateafter: it used to configure disrupiton interval,amount of time it should wait before considering disruption cycle again

  • Consolidation: It operates to actively reduce cluster cost by analyzing nodes
    Consolidation policy has two modes
    a)When Empty: Karpenter will only disrupt nodes with no workloads pods
    b)Whenunderutilized: It will attempt to reduce/replace nodes when underutilised

apiVersion: karpenter.sh/v1beta1
kind: NodePool
metadata:
  name: ondemand
spec:
  disruption:
    consolidationPolicy: WhenEmpty
    consolidateAfter: 60s

 

Enable NAP(Karpenter) on AKS

There are few pre requisites to enable NAP on AKS

  • Install Az CLI with preview extension of version greater than 0.5.17

  • Regsiter NAP provides called "NodeAutoProvisioningPreview"

  • AKS with network configuration as Cilium + Overlay

Enable NAP on existing AKS cluster
Make sure existing AKS cluster has 'Azure' network plugin with Cilium as Network Policy. Key thing in this command is feature flag '--node-provisioning-mode Auto', which set NAP as default Node Autoscaler

az aks update --name aksclustername --resource-group rgname --node-provisioning-mode Auto

Deploy NAP with new AKS cluster

az aks create --name aksclustername--resource-group rgname--node-provisioning-mode Auto --network-plugin azure --network-plugin-mode overlay --network-dataplane cilium

Verify Karpenter Enablement:

kubectl api-resources | grep -e aksnodeclasses -e nodeclaims -e nodepools

aksnodeclasses                     aksnc,aksncs                        karpenter.azure.com/v1alpha2           false        AKSNodeClass
nodeclaims                                                             karpenter.sh/v1beta1                   false        NodeClaim
nodepools                                                              karpenter.sh/v1beta1                   false        NodePool

Disabling Cluster-Autoscaler

To switch from Cluster-Autoscaler to Karpenter, disable Cluster-Autoscaler on your AKS cluster:

az aks update --name aksclustername --resource-group aksrg --disable-cluster-autoscaler

Deploying a Sample Application

To see Node-Autoprovisioning in action, deploy a sample application:

osama [ ~ ]$ kubectl get nodes
NAME                                STATUS   ROLES   AGE     VERSION
aks-default-h2jxh                   Ready    agent   35m     v1.27.9
aks-nodepool1-41633911-vmss000000   Ready    agent   3d19h   v1.27.9

Scale replicas of Vote Application to trigger scale out events

osama [ ~ ]$ kubectl scale deployment azure-vote-front --replicas=12 -n karpenter-demo-ns
^[[Adeployment.apps/azure-vote-front scaled
osama [ ~ ]$ kubectl scale deployment azure-vote-back --replicas=12 -n karpenter-demo-ns
deployment.apps/azure-vote-back scaled

Verify auto scaling of nodes by reading via karpenter using below kubectl cmd

kubectl get events -A --field-selector source=karpenter --sort-by='.lastTimestamp' -n 10
NAMESPACE           LAST SEEN   TYPE     REASON                  OBJECT                                  MESSAGE
default             50m         Normal   Unconsolidatable        nodeclaim/default-95f54                 SpotToSpotConsolidation is disabled, can't replace a spot node with a spot node
default             50m         Normal   Unconsolidatable        node/aks-default-95f54                  SpotToSpotConsolidation is disabled, can't replace a spot node with a spot node
default             38m         Normal   DisruptionBlocked       nodepool/default                        No allowed disruptions due to blocking budget
default             5m33s       Normal   Unconsolidatable        nodeclaim/default-h2jxh                 Can't remove without creating 2 candidates
default             5m33s       Normal   Unconsolidatable        node/aks-default-h2jxh                  Can't remove without creating 2 candidates
default             2m12s       Normal   DisruptionBlocked       nodepool/system-surge                   No allowed disruptions due to blocking budget
karpenter-demo-ns   63s         Normal   Nominated               pod/azure-vote-front-6855444955-bnq7p   Pod should schedule on: nodeclaim/default-mrh7w
karpenter-demo-ns   63s         Normal   Nominated               pod/azure-vote-front-6855444955-gbwk6   Pod should schedule on: nodeclaim/default-mrh7w
karpenter-demo-ns   63s         Normal   Nominated               pod/azure-vote-front-6855444955-l2bgj   Pod should schedule on: nodeclaim/default-mrh7w
karpenter-demo-ns   63s         Normal   Nominated               pod/azure-vote-front-6855444955-nvc56   Pod should schedule on: nodeclaim/default-mrh7w
karpenter-demo-ns   63s         Normal   Nominated               pod/azure-vote-front-6855444955-22glj   Pod should schedule on: nodeclaim/default-mrh7w
karpenter-demo-ns   63s         Normal   Nominated               pod/azure-vote-front-6855444955-sxdl6   Pod should schedule on: nodeclaim/default-mrh7w
karpenter-demo-ns   63s         Normal   Nominated               pod/azure-vote-front-6855444955-t69w4   Pod should schedule on: nodeclaim/default-mrh7w

Customise Karpenter Config

Karpenter leverage new resource type in kubernetes Kind i.e. Nodepools

  • Customise Nodepools: Specific specific VM series or VM family or even Specific CPU or Memory ratio.

  • Select node based on features sets like GPU enable or Network Acceleration

  • Defined Archiecture of CPU type either ARM or AMD based on capablity of specfic workload

  • Architect your nodes for resiliency by configure zone topology

  • Limit numbers of CPU & Memory could be utilised from nodes on nodelevel

Here is default Nodepool Yaml for karpenter(NAP), Which has confiuration on Node SKU types and Capacity, Also limit on nodes CPU:Memory along with Weight incase of Multiple nodepools

apiVersion: karpenter.sh/v1beta1
kind: NodePool
metadata:
  name: default
spec:
  disruption:
    consolidationPolicy: WhenUnderutilized
    expireAfter: 10s
  template:
    spec:
      nodeClassRef:
        name: default

      # Requirements that constrain the parameters of provisioned nodes.
      # These requirements are combined with pod.spec.affinity.nodeAffinity rules.
      # Operators { In, NotIn, Exists, DoesNotExist, Gt, and Lt } are supported.
      # https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#operators
      requirements:
      - key: kubernetes.io/arch
        operator: In
        values:
        - amd64
      - key: kubernetes.io/os
        operator: In
        values:
        - linux
      - key: karpenter.sh/capacity-type
        operator: In
        values:
        - ondemand
      - key: karpenter.azure.com/sku-family
        operator: In
        values:
        - E
        - D
      - key: karpenter.azure.com/sku-name
        operator: In
        values:
        - Standard_E2s_v5
        - Standard_D4s_v3
limits:
  cpu: "1000"
  memory: 1000Gi
weight: 100

 

Using Spot Node with Karpenter

  • Add toleration in Sample AKS-Vote application i.e. "karpenter.sh/disruption:NoSchedule" which comes as default in spot node when provision with AKS Cluster

  • Please refer my github repo for Application yaml and sample nodepool config

    spec:
      nodeSelector:
        "kubernetes.io/os": linux
      tolerations:
      - key: "kubernetes.azure.com/scalesetpriority"
        operator: "Equal"
        value: "spot"
        effect: "NoSchedule"
      containers:
      - name: azure-vote-front
        image: mcr.microsoft.com/azuredocs/azure-vote-front:v1

  • Scale down your application replicas to allow Karpenter to evict existing on-demand nodes and replace them with Spot nodes:

    osama [ ~/karpenter ]$ kubectl get nodes
NAME                                STATUS   ROLES   AGE     VERSION
aks-nodepool1-41633911-vmss000000   Ready    agent   3d21h   v1.27.9
aks-nodepool1-41633911-vmss00000b   Ready    agent   24m     v1.27.9

osama [ ~/karpenter ]$ kubectl get pods -n karpenter-demo-ns -o wide
No resources found in karpenter-demo-ns namespace.

osama [ ~/karpenter ]$ kubectl scale deployment azure-vote-back --replicas=10 -n karpenter-demo-ns
deployment.apps/azure-vote-back scaled
osama [ ~/karpenter ]$ kubectl scale deployment azure-vote-front --replicas=10 -n karpenter-demo-ns
deployment.apps/azure-vote-front scaled
osama [ ~/karpenter ]$

  • Deploy and scale vote application replicas so that karpenter spins up spot nodes based on nodepool configuration and schedule pods after toleration validation on spot

  • Karpenter spins up new spot nodes and Nominate that node for sceduling sample vote-app

    
osama [ ~/karpenter ]$ kubectl get events -A --field-selector source=karpenter --sort-by='.lastTimestamp'
NAMESPACE           LAST SEEN   TYPE      REASON                       OBJECT                                    MESSAGE
karpenter-demo-ns   104s        Normal    Nominated                    pod/azure-vote-back-687ddb67bd-pz8sp      Pod should schedule on: nodeclaim/default-52gbg
karpenter-demo-ns   104s        Normal    Nominated                    pod/azure-vote-back-687ddb67bd-ckdcq      Pod should schedule on: nodeclaim/default-52gbg
karpenter-demo-ns   104s        Normal    Nominated                    pod/azure-vote-back-687ddb67bd-v9nqj      Pod should schedule on: nodeclaim/default-52gbg
karpenter-demo-ns   104s        Normal    Nominated                    pod/azure-vote-back-687ddb67bd-vswvs      Pod should schedule on: nodeclaim/default-52gbg
karpenter-demo-ns   104s        Normal    Nominated                    pod/azure-vote-back-687ddb67bd-lnxmp      Pod should schedule on: nodeclaim/default-52gbg
karpenter-demo-ns   104s        Normal    Nominated                    pod/azure-vote-back-687ddb67bd-jc2jz      Pod should schedule on: nodeclaim/default-52gbg
karpenter-demo-ns   104s        Normal    Nominated                    pod/azure-vote-back-687ddb67bd-hwnbh      Pod should schedule on: nodeclaim/default-52gbg
karpenter-demo-ns   104s        Normal    Nominated                    pod/azure-vote-back-687ddb67bd-r7msb      Pod should schedule on: nodeclaim/default-52gbg
karpenter-demo-ns   104s        Normal    Nominated                    pod/azure-vote-back-687ddb67bd-96lm9      Pod should schedule on: nodeclaim/default-52gbg
karpenter-demo-ns   104s        Normal    Nominated                    pod/azure-vote-back-687ddb67bd-5qcvk      Pod should schedule on: nodeclaim/default-52gbg
default             1s          Normal    DisruptionLaunching          nodeclaim/default-bkz6c                   Launching NodeClaim: Expiration/Replace
default             1s          Normal    DisruptionWaitingReadiness   nodeclaim/default-bkz6c                   Waiting on readiness to continue disruption
default             1s          Normal    DisruptionBlocked            nodepool/system-surge                     No allowed disruptions due to blocking budget
default             1s          Normal    DisruptionWaitingReadiness   nodeclaim/default-5vp7x                   Waiting on readiness to continue disruption
default             1s          Normal    DisruptionLaunching          nodeclaim/default-5vp7x                   Launching NodeClaim: Expiration/Replace

    Configuring Multiple NodePools

  • To configure separate NodePools for Spot and On-Demand capacity:

    Spot nodes configure with E series VM "Standard E2s_v5" and OnDemand with D series VM as "Standard_D4s_v5"

  • In multi-nodepool scenario each nodepool needs to be configured with 'Weight' attribute, nodepool with highest weight would be priotized over another, here we have Spot node with weight:100 and ondemand with weight:60

osama [ ~ ]$ kubectl get nodepool default -o yaml
apiVersion: karpenter.sh/v1beta1
kind: NodePool
metadata:
  name: default
spec:
  disruption:
    budgets:
    - nodes: 100%
    consolidationPolicy: WhenUnderutilized
    expireAfter: 720h
  template:
    spec:
      nodeClassRef:
        name: default
      requirements:
      - key: kubernetes.io/arch
        operator: In
        values:
        - amd64
      - key: kubernetes.io/os
        operator: In
        values:
        - linux
      - key: karpenter.sh/capacity-type
        operator: In
        values:
        - spot
      - key: karpenter.azure.com/sku-family
        operator: In
        values:
        - B
      - key: karpenter.azure.com/sku-name
        operator: In
        values:
        - Standard_B2s_v2
  weight: 100

  • If we do not specify an explicit SKU name, Karpenter will consider the entire VM series.

  • To validate that the sample VoteApp is running on Spot nodes, use the following commands:

  • The output should indicate that the nodes are of capacity type "spot":

    osama [ ~ ]$ kubectl get pods -n karpenter-demo-ns -o wide
NAME                                READY   STATUS    RESTARTS   AGE   IP             NODE                NOMINATED NODE   READINESS GATES
azure-vote-back-687ddb67bd-w7ghm    1/1     Running   0          63m   10.244.3.11    aks-default-5cr5f   <none>           <none>
azure-vote-front-6855444955-64558   1/1     Running   0          63m   10.244.3.168   aks-default-5cr5f   <none>           <none>
osama [ ~ ]$ kubectl describe node aks-default-5cr5f | grep karpenter.sh
                    karpenter.sh/capacity-type=spot
                    karpenter.sh/initialized=true
                    karpenter.sh/nodepool=default
                    karpenter.sh/registered=true
                    karpenter.sh/nodepool-hash: 12393960163388511505
                    karpenter.sh/nodepool-hash-version: v2


    Simulating Spot Node Eviction

    To test the spot eviction scenario, simulate a spot eviction using the Azure CLI:

    osama [ ~ ]$ az vm simulate-eviction --resource-group MC_aks-lab_aks-karpenter_eastus --name aks-default-5cr5f
osama [ ~ ]$ date
Tue May 21 06:20:02 PM IST 2024

  • Monitor the availability of your VoteApp using a simple curl command:

    while true; do echo "$(date) $(curl -s -v -o /dev/null -w 'HTTP %{http_code}\n' http://voteapp.com 2>&1 | grep 'HTTP')"; sleep 2; done

  • After running the spot simulation, the existing node will be marked for termination, and a new Spot node will be created to schedule the VoteApp pods. Within less than a minute, the VoteApp should start responding with HTTP 200 status codes.

  •     root@MININT-8C81HDE:/home/osamaex while true; do echo "$(date) $(curl -s -v -o /dev/null -w 'HTTP %{http_code}\n' http://voteapp.com 2>&1 | grep 'HTTP')"; sleep 2; done
    Tue May 21 18:20:04 IST 2024 > GET / HTTP/1.1
    < HTTP/1.1 200 OK
    HTTP 200
    Tue May 21 18:20:07 IST 2024 > GET / HTTP/1.1
    < HTTP/1.1 200 OK
    HTTP 200
    Tue May 21 18:20:09 IST 2024 > GET / HTTP/1.1
    < HTTP/1.1 200 OK
    HTTP 200
    Tue May 21 18:20:12 IST 2024 HTTP 000  $Failure-Alert
    Tue May 21 18:21:14 IST 2024 > GET / HTTP/1.1
    < HTTP/1.1 200 OK                      $Successful-Response
    HTTP 200
    Tue May 21 18:22:58 IST 2024 > GET / HTTP/1.1
    < HTTP/1.1 200 OK
    HTTP 200

  • Check the events logged by Karpenter:

  •     kuctl get events -A --field-selector source=karpenter --sort-by='.lastTimestamp'

  • Results of events logged by karpenter to replace spot noded with ondemand

  •     osama [ ~ ]$ 
    NAMESPACE           LAST SEEN   TYPE      REASON           OBJECT                                  MESSAGE
    default             23s         Warning   FailedDraining   node/aks-default-5cr5f                  Failed to drain node, 10 pods are waiting to be evicted
    karpenter-demo-ns   22s         Normal    Evicted          pod/azure-vote-back-687ddb67bd-w7ghm    Evicted pod
    karpenter-demo-ns   22s         Normal    Evicted          pod/azure-vote-front-6855444955-64558   Evicted pod
    karpenter-demo-ns   21s         Normal    Nominated        pod/azure-vote-back-687ddb67bd-tb2pv    Pod should schedule on: nodeclaim/default-6zkkl
    karpenter-demo-ns   21s         Normal    Nominated        pod/azure-vote-front-6855444955-7wzss   Pod should schedule on: nodeclaim/default-6zkkl

  • Verify that the pods are running on the new Spot node:

    kubectl get pods -n karpenter-demo-ns -o wide
NAME                                READY   STATUS    RESTARTS   AGE   IP             NODE                NOMINATED NODE   READINESS GATES
azure-vote-back-687ddb67bd-tb2pv    1/1     Running   0          18m   10.244.2.103   aks-default-6zkkl   <none>           <none>
azure-vote-front-6855444955-7wzss   1/1     Running   0          18m   10.244.2.47    aks-default-6zkkl   <none>           <none>

Save Cost by utilizing Reserved Instance VM's

  • NodePool configuration allows you to specify different VM series along with multiple VM SKUs. Create a separate NodePool with the highest weight value and specify all Reserved Instance VM SKU families or explicit SKU names using the karpenter.azure.com/sku-name or karpenter.azure.com/sku-familyparameter.

       spec:
      nodeClassRef:
        name: default
      requirements:
      - key: kubernetes.io/arch
        operator: In
        values:
        - amd64
      - key: kubernetes.io/os
        operator: In
        values:
        - linux
      - key: karpenter.sh/capacity-type
        operator: In
        values:
        - on-demand
      - key: karpenter.azure.com/sku-family
        operator: In
        values:
        - D
      - key: karpenter.azure.com/sku-name
        operator: In
        values:
        - [Standard_D2s_v3, Standard_D4s_v3, Standard_D8s_v3, Standard_D16s_v3, Standard_D32s_v3, Standard_D64s_v3, Standard_D96s_v3]
  weight: 90

Conclusion

The adoption of Karpenter in AKS signifies a major advancement in node scaling efficiency, flexibility, and cost optimization. By addressing the limitations of the Cluster Autoscaler and introducing dynamic, rapid provisioning of nodes, Karpenter provides a robust solution for managing Kubernetes clusters. Its flexibility in handling different VM types, faster scaling capabilities, and cost optimization make it a valuable addition to Kubernetes cluster management. By leveraging Karpenter, organizations can achieve more responsive and cost-effective Kubernetes deployments.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

This site uses Akismet to reduce spam. Learn how your comment data is processed.