Tip: running a cheap AKS lab

This post has been republished via RSS; it originally appeared at: Azure Developer Community Blog articles.

I am currently running three AKS labs in one of my subscriptions and I constantly try to minimize costs so I'm quite regularly spinning a "cluster" with only one worker node (Standard DS2 V2) which I stop whenever I don't use it. Lately, when working with KEDA, I enabled the Virtual Nodes (see below) when creating the cluster:

which makes use of Azure Container Instances (ACI) behind the scenes. Then I realized that, I could leverage Virtual Nodes more often in order to reduce costs. Therefore, I thought of spinning up the cluster with the smallest possible Virtual Machine as the main worker node.

While virtual machines represent a fixed cost, the pricing model of ACIs is based on actual consumption as they are only charged per second of execution which may be just enough for test scenarios/demos/lab-kind of activities. By default, if you do not specify any resource request in your container, AKS will provision ACIs with 1 CPU and 1.5 GB of memory. When specifying resources, for instance like this:

apiVersion: apps/v1 kind: Deployment metadata: labels: app.kubernetes.io/name: serverless app.kubernetes.io/part-of: serverless app.kubernetes.io/version: v9 name: serverless namespace: default spec: replicas: 1 selector: matchLabels: app: serverless template: metadata: labels: app: serverless spec: containers: - name: serverless image: akspremiumregistry.azurecr.io/serverlesscontainer:1 resources: requests: memory: "4000Mi" cpu: "4" imagePullSecrets: - name: regsecret tolerations: - key: virtual-kubelet.io/provider operator: Equal value: azure effect: NoSchedule

, AKS applies the requested resources to the ACI config as illustrated here:

Benefits of such a setup

Cost-friendly, providing you do not let ACIs hanging around
Allows for resource-greedy scenarios

Limitations of such a setup

Although interesting from a financial perspective, it comes with a few limitations:

Suitable for lab-only activities
Suitable for short-running containers (again, perfect in a demo scenario, etc.). When used like this, you might end up with only a few euros/dollars to pay at the end of the month instead of much more for a full blown worker node.
Container startup is a little delayed because of the ACI provisioning.
By default, only 100 concurrent ACIs are allowed per subscription, meaning that you can't have more than 100 concurrent pods. This can be changed by reaching out to support.
Make sure to clean your stuff after use, else the ACIs will keep running forever which then will become very costly. The easiest way is to run a kubectl delete namespace .... but the bare minimum is to run a kubectl scale deploy --all --replicas=0 so as to let AKS destroy the corresponding ACIs.

To wrap it up, to get a cheap AKS lab, just proceed the following steps:

Create the cluster & enable Virtual Nodes
Choose the smallest possible VM as your main worker node

Leave a Reply Cancel reply