This post has been republished via RSS; it originally appeared at: Microsoft Tech Community - Latest Blogs - .
Using Azure Kubernetes Service on Azure Stack HCI to deploy your workload can provide significant infrastructural advantages, such as flexible scaling, management of distributed components, and control over different versions of your application. However, as infrastructure and applications continue to rise with business need, increased control will bring about increased complexity. By enabling GitOps using Flux v2 in your AksHci cluster, you can manage these complexities by declaring the desired state of your Kubernetes clusters in files in Git repositories.
For those who are new to the concept, GitOps is an applications development and operations methodology. It is a strategy that enables you to declaratively describe your entire system, have the desired system state versioned in Git, allow approved changes to be automatically applied to the system, and presents a way of reconciling to ensure correctness. With GitOps, you can use your Git repository as the source of truth for cluster configuration and application deployment.
Typical Git repositories may contain the following files to include:
- Yaml formatted manifests that describe Kubernetes resources such as Namespaces, Secrets, Deployments and others.
- Helm charts for deploying applications.
- Kustomize files to describe environment specific changes. Note: Using Kustomize for helm to apply taints, coloration, and other application configuration should work fine, however, using Kustomize to configure the kubernetes cluster has limited support and some of the changes might be lost during update. E.g., Crictl config
Azure provides configuration management capability using GitOps in Azure Kubernetes Service (AKS) managed clusters and Azure Arc-enabled Kubernetes connected clusters as a Microsoft.KubernetesConfiguration/extensions/microsoft.flux cluster extension resource. You can easily enable and use GitOps on AksHci clusters.
As at the time of writing this documentation, GitOps using Flux v2 has been announced to be generally available for cloud and hybrid environments Announcing General Availability for GitOps with Flux v2 in Azure - Microsoft Tech Community. The process to implement this is as follows:
Supported Regions: These include East US, Southeast Asia, West Europe and Australia east.
Implementation steps using Azure CLI
1. Ensure you have an existing AksHci cluster or create one if you don’t. You can also create one through the steps in Create AksHci cluster. You can check for the status of your cluster by running Get-AksHciCluster.
PS C:\Windows\system32> Get-AksHciCluster
Status : {ProvisioningState, Details}
ProvisioningState : Deployed
KubernetesVersion : v1.22.6
PackageVersion : v1.22.6-kvapkg.2
NodePools : linuxpool
WindowsNodeCount : 0
LinuxNodeCount : 3
ControlPlaneNodeCount : 3
ControlPlaneVmSize : Standard_A2_v2
AutoScalerEnabled : False
AutoScalerProfile :
LoadBalancer : {VMSize, Count, Sku}
Name : mycluster1
2. Ensure that your cluster is Arc connected by running Enable-AksHciArcConnection -name mycluster1, or follow steps to connect an existing Kubernetes cluster to Azure Arc Quickstart: Connect an existing Kubernetes cluster to Azure Arc - Azure Arc | Microsoft Docs.
3. Ensure you have Azure CLI downloaded and check version to be certain you have Azure CLI version 2.15 or later. Run az version
PS C:\Windows\system32> az version
{
"azure-cli": "2.36.0",
"azure-cli-core": "2.36.0",
"azure-cli-telemetry": "1.0.6",
"extensions": {
"k8s-configuration": "1.5.1",
"k8s-extension": "1.2.0"
}
4. Login to azure to register your subscription by running - az login.
PS C:\Windows\system32> az login
A web browser has been opened at https://login.microsoftonline.com/organizations/oauth2/v2.0/authorize. Please continue the login in the web browser. If no web browser is available or if the web browser fails to open, use device code flow with `az login --use-device-code`.
[
{
"cloudName": "AzureCloud",
"homeTenantId": "ff547155-eb0a-4b14-afe1-129411efeb4c",
"id": "5807bea4-cd17-4b81-b4d8-d4e94caab82d",
"isDefault": true,
"managedByTenants": [],
"name": "Visual Studio Enterprise Subscription",
"state": "Enabled",
"tenantId": "ff547155-eb0a-4b14-afe1-129411efeb4c",
"user": {
"name": "ekelearthur@outlook.com",
"type": "user"
}
}
]
5. Check to see that you are registered with the Azure Service providers. You can run az provider show --n Microsoft.Kubernetes -o table to see registration status, or run the following commands to register
- az provider register --namespace Microsoft.Kubernetes.
- az provider register --namespace Microsoft.ContainerService
- az provider register --namespace Microsoft.KubernetesConfiguration
PS C:\Windows\system32> az provider show -n Microsoft.Kubernetes -o table
>> az provider show -n Microsoft.KubernetesConfiguration -o table
Namespace RegistrationPolicy RegistrationState
-------------------- -------------------- -------------------
Microsoft.Kubernetes RegistrationRequired Registered
Namespace RegistrationPolicy RegistrationState
--------------------------------- -------------------- -------------------
Microsoft.KubernetesConfiguration RegistrationRequired Registered
6. You can see the list of Az CLI extensions installed and version with the command. Run az extension list -o table
PS C:\Windows\system32> az extension list -o table
Experimental ExtensionType Name Path Preview Version
-------------- --------------- ----------------- ----------------------------------------------------- --------- ---------
False whl k8s-configuration C:\Users\ekele\.azure\cliextensions\k8s-configuration False 1.5.1
False whl k8s-extension C:\Users\ekele\.azure\cliextensions\k8s-extension False 1.2.0
7. You can now apply the Flux configuration to use GitOps in your Arc-enabled Kubernetes cluster through the k8s-configuration Azure CLI extension or Azure Portal. The sample repo used in this article is https://github/Azure/gitops-flux2-kustomize-helm-mt. If the Microsoft.Flux extension is not already installed in the cluster, it will be installed.
- Run az k8s-configuration flux create -g cluster-config -c <clustername> -n <namespace> -t <managedClusters or connectedClusters> --scope cluster –u https://github/Azure/gitops-flux2-kustomize-helm-mt --branch main –kustomization --kustomization name=infra path=./infrastructure prune=true --kustomization name=apps path=./apps/staging prune=true dependsOn=["infra"]
'Microsoft.Flux' extension not found on the cluster, installing it now. This may take a few minutes...
'Microsoft.Flux' extension was successfully installed on the cluster
Creating the flux configuration 'cluster-config' in the cluster. This may take a few minutes...
…
},
"name": "cluster-config",
"namespace": "cluster-config",
"provisioningState": "Succeeded",
"repositoryPublicKey": "",
"resourceGroup": "<>",
"scope": "cluster",
"sourceKind": "GitRepository",
"sourceSyncedCommitId": "main/cc2e6a384fb...",
"sourceUpdatedAt": "2022-08-23T20:29:15+00:00",
"statusUpdatedAt": "2022-08-23T20:29:21.599000+00:00",
"statuses": [
{
"appliedBy": null,
"complianceState": "Compliant",
"helmReleaseProperties": null,
"kind": "GitRepository",
"name": "cluster-config",
"namespace": "cluster-config",
"statusConditions": [
{
"lastTransitionTime": "2022-08-23T20:29:15+00:00",
"message": "stored artifact for revision 'main/cc2e6a384fb...'",
"reason": "Succeeded",
"status": "True",
"type": "Ready"
},
…
The following namespaces were created:
- flux-system: Holds the Flux extension controllers.
- cluster-config: Holds the Flux configuration objects.
- nginx, podinfo, redis: Namespaces for workloads described in manifests in the Git repository.
PS C:\Windows\system32> kubectl get namespaces
NAME STATUS AGE
azure-arc Active 9h
cluster-config Active 86m
default Active 17h
flux-system Active 88m
kube-node-lease Active 17h
kube-public Active 17h
kube-system Active 17h
nginx Active 86m
podinfo Active 83m
redis Active 86m
The Git repository tracks all changes made to files in your project and configuration information usually stored in a .git folder.
The Git repository contains the following top directories:
- apps dir contains Helm releases with a custom configuration per cluster
- infrastructure dir contains common infra tools such as NGINX ingress controller and Helm repository definitions
- clusters dir contains the Flux configuration per cluster
The apps configuration is structured into:
- apps/base/ dir contains namespaces and Helm release definitions
- In apps/base/podinfo/ dir we have a HelmRelease with common values for both clusters
- apps/production/ dir contains the production Helm release values
- In apps/production/ dir we have a Kustomize patch with the production specific values
- apps/staging/ dir contains the staging values
- In apps/staging/ dir we have a Kustomize patch with the staging specific values
In infrastructure/sources/ dir we have the Helm repositories definitions
The clusters dir contains the Flux configuration.
- In clusters/staging/ dir the Kustomization definitions are made
The flux-system namespace contains flux extension objects such as Azure Flux controllers (fluxconfig-agent, fluxconfig-controller), and OSS Flux controllers (source-controller, kustomize-controller, helm-controller, notification-controller). The Flux agent and controller pods are in a running state. Run kubectl get pods -n flux-system
PS C:\Windows\system32> kubectl get pods -n flux-system
NAME READY STATUS RESTARTS AGE
fluxconfig-agent-5d98b7c568-d9xsz 2/2 Running 0 92m
fluxconfig-controller-cd6444dd9-k8p84 2/2 Running 0 92m
helm-controller-75988fc95c-8q7rs 1/1 Running 0 92m
kustomize-controller-5d7d58477f-wng26 1/1 Running 0 92m
notification-controller-88879667f-9st8k 1/1 Running 0 92m
source-controller-7896dfc4b7-4c9nw 1/1 Running 0 92m
The cluster-config namespace has the Flux configuration objects. Run kubectl get crds
PS C:\Windows\system32> kubectl get crds
NAME CREATED AT
alerts.notification.toolkit.fluxcd.io 2022-08-23T20:27:12Z
arccertificates.clusterconfig.azure.com 2022-08-23T12:24:20Z
azureclusteridentityrequests.clusterconfig.azure.com 2022-08-23T12:24:20Z
azureextensionidentities.clusterconfig.azure.com 2022-08-23T12:24:20Z
bgpconfigurations.crd.projectcalico.org 2022-08-23T04:29:59Z
bgppeers.crd.projectcalico.org 2022-08-23T04:29:59Z
blockaffinities.crd.projectcalico.org 2022-08-23T04:29:59Z
buckets.source.toolkit.fluxcd.io 2022-08-23T20:27:12Z
clusterinformations.crd.projectcalico.org 2022-08-23T04:29:59Z
connectedclusters.arc.azure.com 2022-08-23T12:24:20Z
csinodeinfos.csi.storage.k8s.io 2022-08-23T04:29:58Z
customlocationsettings.clusterconfig.azure.com 2022-08-23T12:24:20Z
extensionconfigs.clusterconfig.azure.com 2022-08-23T12:24:20Z
felixconfigurations.crd.projectcalico.org 2022-08-23T04:29:59Z
fluxconfigs.clusterconfig.azure.com 2022-08-23T20:27:12Z
gitconfigs.clusterconfig.azure.com 2022-08-23T12:24:20Z
gitrepositories.source.toolkit.fluxcd.io 2022-08-23T20:27:12Z
globalnetworkpolicies.crd.projectcalico.org 2022-08-23T04:29:59Z
globalnetworksets.crd.projectcalico.org 2022-08-23T04:29:59Z
helmcharts.source.toolkit.fluxcd.io 2022-08-23T20:27:12Z
helmreleases.helm.toolkit.fluxcd.io 2022-08-23T20:27:13Z
helmrepositories.source.toolkit.fluxcd.io 2022-08-23T20:27:12Z
hostendpoints.crd.projectcalico.org 2022-08-23T04:29:59Z
imagepolicies.image.toolkit.fluxcd.io 2022-08-23T20:27:12Z
imagerepositories.image.toolkit.fluxcd.io 2022-08-23T20:27:12Z
imageupdateautomations.image.toolkit.fluxcd.io 2022-08-23T20:27:12Z
ipamblocks.crd.projectcalico.org 2022-08-23T04:30:00Z
ipamconfigs.crd.projectcalico.org 2022-08-23T04:30:00Z
ipamhandles.crd.projectcalico.org 2022-08-23T04:30:00Z
ippools.crd.projectcalico.org 2022-08-23T04:30:00Z
kubecontrollersconfigurations.crd.projectcalico.org 2022-08-23T04:30:00Z
kustomizations.kustomize.toolkit.fluxcd.io 2022-08-23T20:27:12Z
networkpolicies.crd.projectcalico.org 2022-08-23T04:30:00Z
networksets.crd.projectcalico.org 2022-08-23T04:30:00Z
providers.notification.toolkit.fluxcd.io 2022-08-23T20:27:12Z
receivers.notification.toolkit.fluxcd.io 2022-08-23T20:27:12Z
renewrevocations.certificates.microsoft.com 2022-08-23T04:30:02Z
PS C:\Windows\system32> kubectl get fluxconfigs -A
NAMESPACE NAME SCOPE URL PROVISION AGE
cluster-config cluster-config cluster https://github.com/Azure/gitops-flux2-kustomize-helm-mt Succeeded 97m
PS C:\Windows\system32> kubectl get gitrepositories -A
NAMESPACE NAME URL AGE READY STATUS
cluster-config cluster-config https://github.com/Azure/gitops-flux2-kustomize-helm-mt 98m True stored artifact for revision 'main/cc2e6a384fb...'
PS C:\Windows\system32> kubectl get helmreleases -A
NAMESPACE NAME AGE READY STATUS
cluster-config nginx 99m True Release reconciliation succeeded
cluster-config podinfo 96m True Release reconciliation succeeded
cluster-config redis 99m True Release reconciliation succeeded
PS C:\Windows\system32> kubectl get kustomizations -A
NAMESPACE NAME AGE READY STATUS
cluster-config cluster-config-apps 100m True Applied revision: main/cc2e6a384fbfa...
cluster-config cluster-config-infra 100m True Applied revision: main/cc2e6a384fbfa...
Workloads would be deployed from manifests in the Git repository
PS C:\Windows\system32> kubectl get deploy -n nginx
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-ingress-controller 1/1 1 1 100m
nginx-ingress-controller-default-backend 1/1 1 1 100m
PS C:\Windows\system32> kubectl get deploy -n podinfo
NAME READY UP-TO-DATE AVAILABLE AGE
podinfo 1/1 1 1 98m
PS C:\Windows\system32> kubectl get all -n redis
NAME READY STATUS RESTARTS AGE
pod/redis-master-0 1/1 Running 0 101m
pod/redis-replicas-0 1/1 Running 0 101m
pod/redis-replicas-1 1/1 Running 0 100m
pod/redis-replicas-2 1/1 Running 0 99m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/redis-headless ClusterIP None <none> 6379/TCP 101m
service/redis-master ClusterIP 10.111.146.68 <none> 6379/TCP 101m
service/redis-replicas ClusterIP 10.98.161.85 <none> 6379/TCP 101m
NAME READY AGE
statefulset.apps/redis-master 1/1 101m
statefulset.apps/redis-replicas 3/3 101m
Supporting documents: For more information on deploying configurations on your AksHci cluster using GitOps with Flux v2, see Tutorial: Use GitOps with Flux v2 in Azure Arc-enabled Kubernetes or Azure Kubernetes Service (AKS) clusters - Azure Arc | Microsoft Docs