Site icon TheWindowsUpdate.com

Use Azure tags in Azure Kubernetes Service (AKS)

This post has been republished via RSS; it originally appeared at: Microsoft Tech Community - Latest Blogs - .

As documented in Use Azure tags in Azure Kubernetes Service (AKS), you can use Azure tags on an AKS cluster to associate its related resources to a given workload or tenant. For some resources, such as a managed data disk created via a persistent volume claim or an Azure Public IP created by a public Kubernetes service, you can also use Kubernetes manifests to set Azure tags. Azure tags are a helpful mechanism to track resource usage and charge back their costs to separate tenants or business units within an organization. This article explains how to set Azure tags for AKS clusters and related resources.

 

Azure Tags

Tags are metadata elements that you apply to your Azure resources. They're key-value pairs that help you identify resources based on settings relevant to your organization. If you want to track the deployment environment for your resources, add a key named Environment. To identify the resources deployed to production, give them a value of Production. Fully formed, the key-value pair becomes Environment = Production. You can apply tags to your Azure resources, resource groups, and subscriptions. For more information on Azure Tags, see Use tags to organize your Azure resources and management hierarchy.

You can use Azure Policy to create tagging rules and conventions for Azure resources, including AKS clusters or its resources. By creating a policy, you avoid the scenario of resources being deployed to your subscription that don't have the expected tags for your organization. Instead of manually applying tags or searching for resources that aren't compliant, you create a policy that automatically applies the needed tags during deployment. Tags can also now be applied to existing resources with the new Modify effect and a remediation task. For more information, see Assign policy definitions for tag compliance.

 

How tags work in AKS

Before diving into how using Azure tags in AKS, let's see what happens when you set and update Azure tags with AKS clusters and their related resources:

Prerequisites

Limitations

 

Add tags to the cluster

When you create or update an AKS cluster with the --tags parameter, the following are assigned the Azure tags that you've specified:

NOTE: Azure Private DNS only supports 15 tags. tag resources.

To create a cluster and assign Azure tags, you can run az aks create with the --tags parameter, as shown in the following command. Running the command creates a myAKSCluster in the myResourceGroup with the tags dept=IT and costcenter=9999.

 

az aks create \
    --resource-group myResourceGroup \
    --name myAKSCluster \
    --tags dept=IT costcenter=9999 \
    --generate-ssh-keys

 

NOTE: To set tags on the initial node pool, the node resource group, the virtual machine scale set, and each virtual machine scale set instance that's associated with the initial node pool, also set the --nodepool-tags parameter.

az aks create \
    --resource-group myResourceGroup \
    --name myAKSCluster \
    --tags dept=IT costcenter=9999 \
    --nodepool-tags dept=IT costcenter=9999 \
    --generate-ssh-keys

IMPORTANT: If you're using existing resources when you're creating a new cluster, such as an IP address or route table, az aks create overwrites the set of tags. If you delete that cluster later, any tags set by the cluster will be removed.

You can use the az aks show command to verify that the tags have been properly applied to the cluster and related resources. The cluster tags for myAKSCluster are shown in the following example:

 

 

$ az aks show -g myResourceGroup -n myAKSCluster --query '[tags]'
{
  "clusterTags": {
    "costcenter": "9999",
    "dept": "IT"
  }
}

 

To update the tags on an existing cluster, you can run the az aks update command with the --tags parameter. Running the command updates the myAKSCluster with the tags team=alpha and costcenter=1234.

 

az aks update \
    --resource-group myResourceGroup \
    --name myAKSCluster \
    --tags team=alpha costcenter=1234

 

You can use the az aks show command to verify that the tags have been applied to the cluster. For example:

 

$ az aks show -g myResourceGroup -n myAKSCluster --query '[tags]'
{
  "clusterTags": {
    "costcenter": "1234",
    "team": "alpha"
  }
}

 

IMPORTANT: Setting tags on a cluster by using az aks update overwrites the set of tags. For example, if your cluster has the tags dept=IT and costcenter=9999 and you use az aks update with the tags team=alpha and costcenter=1234, the new list of tags would be team=alpha and costcenter=1234.

In the following example, the businessUnit=FastTrack for Azure and supportedBy=Paolo Salvatori tags of the AKS cluster were created via Azure Policy, while the IaC=Bicep and workload=tags tags were created at creation time via Bicep.

 

Adding tags to node pools

You can apply one or more Azure tags to a new or existing node pool in your AKS cluster. Tags applied to a node pool are applied to each node within the node pool and are persisted through upgrades. Tags are also applied to new nodes that are added to a node pool during scale-out operations. Adding a tag can help with policy tracking or cost estimation tasks.

When you create or update a node pool with the --tags parameter, the tags that you specify are assigned to the following resources:

 

To create a node pool with an Azure tag, run az aks nodepool add with the --tags parameter. Running the following command creates a contosoNodePool node pool with the tags tenant=contoso and costcenter=5555 in the myAKSCluster.

 

az aks nodepool add \
    --resource-group myResourceGroup \
    --cluster-name myAKSCluster \
    --name contosoNodePool \
    --node-count 1 \
    --tags tenant=contoso costcenter=5555 \
    --no-wait

 

You can use the az aks show command to verify that the tags have been applied to the contosoNodePool node pool.

 

$ az aks show -g myResourceGroup -n myAKSCluster --query 'agentPoolProfiles[].{nodepoolName:name,tags:tags}'
[
  {
    "nodepoolName": "nodepool1",
    "tags": null
  },
  {
    "nodepoolName": "contosoNodePool",
    "tags": {
      "tenant": "contoso",
      "costcenter": "5555"
    }
  }
]

 

To update a node pool with an Azure tag, you can run the az aks nodepool update command with the --tags parameter. Running the following command updates the contosoNodePool node pool with the tags appversion=1.0.0 and costcenter=4444 in the myAKSCluster, which already has the tags tenant=contoso and costcenter=5555.

 

az aks nodepool update \
    --resource-group myResourceGroup \
    --cluster-name myAKSCluster \
    --name contosoNodePool \
    --tags appversion=1.0.0 costcenter=4444 \
    --no-wait

 

IMPORTANT: Setting tags on a node pool by using the az aks nodepool update command overwrites the set of tags. For example, if your node pool has the tags tenant=contoso and costcenter=5555, and you use the az aks nodepool update command with the tags appversion=1.0.0 and costcenter=4444, the new list of tags would be appversion=1.0.0 and costcenter=4444.

You can use the az aks show command to verify that the tags have been appropriately updated on the node pool.

 

$ az aks show -g myResourceGroup -n myAKSCluster --query 'agentPoolProfiles[].{nodepoolName:name,tags:tags}'
[
  {
    "nodepoolName": "nodepool1",
    "tags": null
  },
  {
    "nodepoolName": "contosoNodePool",
    "tags": {
      "appversion": "1.0.0",
      "costcenter": "4444"
    }
  }
]

 

In the following example, the osDiskType=ephemeral and osType=Windows2022 tags of the node pool VMSS were created using the az aks nodepool create command with the --tags parameter at provisioning time.

 

Add tags by using Kubernetes manifests

You can apply Azure tags to Azure public IPs, managed disks, and Azure files by using a Kubernetes YAML manifest.

Azure Public IPs

The service.beta.kubernetes.io/azure-pip-tags annotation can be to determines what Azure tags should be applied to the Azure public IP used by a Kubernetes service. The supported format is a=b,c=d,.... After updated, the old user-assigned tags would not be replaced by the new ones.

For example, the following YAML file creates a Kubernetes deployment that uses NGINX container image, and exposes it via a public LoadBalancer Kubernetes service. The service definition uses the costcenter=1234,tenant=Contoso annotation to set Azure tags on the Azure public IP resource created by the cloud controller manager.

 

---
apiVersion: apps/v1 
kind: Deployment
metadata:
  name: nginx
spec:
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: nginx
  replicas: 3 
  template: 
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
        resources:
          requests:
            memory: "128Mi"
            cpu: "250m"
          limits:
            memory: "256Mi"
            cpu: "500m"
        livenessProbe:
          httpGet:
            path: /
            port: 80
          failureThreshold: 1
          initialDelaySeconds: 1
          periodSeconds: 30
          timeoutSeconds: 5
        readinessProbe:
          httpGet:
            path: /
            port: 80
          failureThreshold: 1
          initialDelaySeconds: 1
          periodSeconds: 30
          timeoutSeconds: 5
        startupProbe:
          httpGet:
            path: /
            port: 80
          failureThreshold: 1
          initialDelaySeconds: 1
          periodSeconds: 30
          timeoutSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    app: nginx
  annotations:
    service.beta.kubernetes.io/azure-pip-tags: costcenter=1234
spec:
  externalTrafficPolicy: Local
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

 

In the following example, the costcenter=1234 tag of the Azure Public IP in the node resource group of the AKS cluster was created by the service.beta.kubernetes.io/azure-pip-tag annotation in the YAML manifest used to deploy the Kubernetes service.

 

 

IMPORTANT: Setting tags on files, disks, and public IPs by using Kubernetes updates the set of tags. For example, if your disk has the tags dept=IT and costcenter=5555, and you use Kubernetes to set the tags team=beta and costcenter=3333, the new list of tags would be dept=IT, team=beta, and costcenter=3333. Any updates that you make to tags through Kubernetes will retain the value that's set through Kubernetes. For example, if your disk has tags dept=IT and costcenter=5555 set by Kubernetes, and you use the portal to set the tags team=beta and costcenter=3333, the new list of tags would be dept=IT, team=beta, and costcenter=5555. If you then remove the disk through Kubernetes, the disk would have the tag team=beta.

Azure Disks and Files

You can apply Azure tags to Azure managed disks and Azure files by using a Kubernetes YAML manifest created via a persistent volume claim PVC. Today you cannot specify metadata and, in particular, tags in a PVC that are passed down to a CSI driver. Today, the only way to tag managed resources dynamically created by a PVC on a cloud platform is using the tags parameter of the Azure Disks CSI driver:

 

 

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: contoso-managed-csi-premium
provisioner: disk.csi.azure.com
parameters:
  skuname: Premium_LRS
  tags: costcenter=1234,tenant=Contoso
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true

 

 

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: contoso-managed-csi-premium
spec:
  accessModes:
  - ReadWriteOnce
  storageClassName: contoso-managed-csi-premium
  resources:
    requests:
      storage: 50Gi

 

In a multi-tenant environment, this approach can lead to two kinds of problems:

You can use the following workaround to avoid the proliferation of storage classes:

 

In the following example, the costcenter=1234 and tenant=Contoso tags of the Azure managed disk in the node resource group of the AKS cluster were created by the tags parameter in the YAML manifest used to create the contoso-managed-csi-premium storage class used by the persistent volume claim (PVC).

 

Exit mobile version