This post has been republished via RSS; it originally appeared at: Microsoft Tech Community - Latest Blogs - .
CycleCloud provides the easiest way to provision High Performance Computing (HPC) clusters based on the most widely used schedulers on the market such as Slurm, OpenPBS or IBM Spectrum LSF in Azure. In general, an instance of CycleCloud is deployed as a virtual machine from Azure Marketplace and after that, to complete the installation, it is necessary to follow a web-based wizard of three simple steps.
In most cases, manual deployment is enough as it is a one-time process. However, in some cases we may want to automate the installation without having to use the graphical interface. Especially in Infrastructure as Code (IaC) scenarios to deploy multiple or ephemeral environments.
This article is based on the Ansible configuration scripts used by Azure HPC On-Demand Platform and allows you to deploy your CycleCloud server automatically. In the first part of the article the necessary configuration is explained step by step, after that, a minimum implementation in Bicep ready to be deployed is provided in the second part.
Data required for CycleCloud initial configuration
To have a functional installation of CycleCloud, it is necessary to provide the following data:
- Username and password of the initial user account that will be assigned the server administrator role.
- Public SSH key associated with that user that will be used to configure remote access to cluster nodes.
- Information of the Azure subscription that will be used to deploy the resources associated with the clusters.
Set up of administrator’s account
CycleCloud allows you to modify server configuration dynamically through JSON files. These files, when placed in the /config/data folder within the CycleCloud installation directory, are automatically imported and the changes applied. If the configuration has been successfully imported, you will see that the name of the file will be modified with the extension .json.imported. This feature allows you to provide the data needed to automate the installation.
To configure the administrator account, a new file called account_data.json should be created in that directory. The name is indifferent so it is possible to choose any other name. In a CycleCloud standard installation, the full path for the folder is /opt/cycle_server/config/data. File content will be as follows:
[
{
"AdType": "Application.Setting",
"Name": "cycleserver.installation.initial_user",
"Value": <YourUserName>
},
{
"AdType": "AuthenticatedUser",
"Name": <YourUserName>
"RawPassword": <YourUserPassword>,
"Superuser": true
},
{
"AdType": "Credential",
"CredentialType": "PublicKey",
"Name": "<YouUserName>/public"
"PublicKey": <YourPublicKeyInformation>,
},
{
"AdType": "Application.Setting",
"Name": "cycleserver.installation.complete",
"Value": true
}
]
The first three properties define CycleCloud administrator username, its password and the associated public access key. The last property instructs CycleCloud not to display the initial configuration screen because the configuration is already done.
As soon as the file is saved, CycleCloud will process it and you should check that its extension is modified to .imported indicating that the changes have been applied.
(Optional) Set up Active Directory integration
CycleCloud supports Active Directory as an external identity provider through a LDAP connection. It is also possible to automate this step creating a file configure_ad.json in the same /config/data folder with the following properties:
[
{
"AdType": "Application.Authenticator",
"DefaultDomain": <Domain name>,
"Disabled": False,
"Label": "Active Directory",
"Method": "active_directory",
"Name": "active_directory",
"Order": 100,
"URL": <The ldap url to connect to (i.e. ldap(s)://dc)>
},
{
"AdType": "Application.Setting",
"Name": "authorization.check_datastore_permissions",
"Value": True
}
]
You need to replace the DefaultDomain property with the name of your Active Directory domain, and the URL with the LDAP or LDAPS endpoint of your domain controller.
Set up your Azure subscription information
Next step is to provide the Azure subscription information where CycleCloud would deploy your clusters. We will create a JSON file that we will call azure_data.json. This time, however, we will not do it in that directory but in our $HOME folder. File content will be as follows. Properties are self-explanatory:
{
"Environment": "public",
"AzureRMUseManagedIdentity": true,
"AzureResourceGroup": <ResourceGroupWhereForCycleCloudResources>,
"AzureRMApplicationId": " ",
"AzureRMApplicationSecret": " ",
"AzureRMSubscriptionId": <AzureSubscriptionId>,
"AzureRMTenantId": <AzureTenantId>,
"DefaultAccount": true,
"Location": <DefaultLocation>,
"Name": "azure",
"Provider": "azure",
"ProviderId": "fd6abe95-c55e-44c8-9085-68002a27c1bb",
"RMStorageAccount": <ExistinStorageAccountName>
"RMStorageContainer": <CycleCloudLockerContainerName>
}
It is important to mention that CycleCloud supports two of Azure’s authentication methods:
- Using a Service Principal (Application) and its password (Secret)
- Using an Azure-managed identity.
The second option is recommended since all identity and secrets lifecycle management is automatically managed by Azure. In this example we will use this option. If for some reason you need to use a Service Principal, you need to use AzureRMApplicationId and AzureRMApplicationSecret properties, and modify the AzureRMUseManagedIdentity property to false
In either option remember to assign RBAC permissions within our subscription to be able to deploy later our resources. In this example, we'll use the Contributor role at the subscription level. However, referring to CycleCloud documentation, it is possible also to create a custom role with a more restricted set of permissions.
After ensuring that the permissions are correctly configured, execute the following command:
/usr/local/bin/cyclecloud account create -f $HOME/azure_data.json
You can verify that everything is working by trying to access CycleCloud we portal with the username and password configured in the previous step. After that, check that inside the Settings (1) > Subscriptions option (2), the Azure subscription has been correctly configured (3) and CycleCloud is retrieving the required information from Azure’s APIs (4).
Using Bicep for automating the whole configuration process
Previous explanation has helped to understand which the required steps are to automate the installation without using CycleCloud’s user interface. However, we have manually performed each of the steps.
As a reminder, those are:
- Creating the virtual machine from the Marketplace with the CycleCloud base image.
- Configuring the virtual machine to make use of managed identities.
- Assigning RBAC permissions to the managed identity so that it has access to the Azure subscription.
- Generating the configuration file of the CycleCloud administrator user in the /data/config folder.
- Importing Azure settings with CycleCloud account
The first three steps can be automated with a Bicep template. For the last two, we can write our own scripts, or, as we mentioned at the beginning, take advantage of the existing work made available by az-hop.
We want to avoid reinventing the wheel so we will use their configure.py file that implements this process and it incorporates extra validations against possible failures. As an extra point, it also initializes the CycleCloud CLI in case that we want to use it.
Script can’t be used directly in our case without a minimal change. It is necessary to modify line 230 replacing:
"--url=https://localhost/cyclecloud"
by
"--url=https://localhost/"
CycleCloud’s installation using the Azure Marketplace image is done at the root level of the web server instead of a subfolder.
If you want to try it, code it is available at jangelfdez/cyclecloud-bicep. Deploy it into your subscription executing the following command replacing the values of these parameters:
- location: Azure region where the deployment will take place.
- resourceGroupname: Resource group where the CycleCloud virtual machine will be created.
- vnetName and subnetName: Data from the network where the virtual machine will be deployed.
- vnetResourceGroupName: If your virtual network is in a different resource group, you need to indicate this with this parameter. If it is in the same, you can omit it.
- storageAccountName: Name of the storage account that CycleCloud will use as a locker.
- tenantId: identifier of your tenant *.onmicrosoft.com.
- adminUsername, adminPassword, publicKey: The access data of the CycleCloud administrator user that will match those of the VM in this case.
az deployment sub create --location <location> --template-file .\main.bicep --parameters resourceGroupName=<rgName> vnetName=<vnetName> subnetName=<subnetName> vnetResourceGroupName=<netRgName> storageAccountName=<saname> tenantId=<tenantId> adminUsername=<username> publicKey='<publicKey>'
If you want to understand what is happening under the hoods, the structure of the Bicep template is as follows:
The main.bicep file orchestrates the deployment because we need to deploy resources both at the subscription level and the resource group level. In the first step, CycleCloud’s virtual machine is deployed along with its disks, network card, and public IP. Once finished, the Contributor role is assigned at the subscription level for the managed identity associated with the VM. Finally, deployment finishes executing the CustomScript to initialize CycleCloud.
If you only need to assign resources at the resource group level, it would be possible to simplify the deployment in a single file with all the resources. We have selected the other option because it is a little bit more complex to implement and it covers the common scenario based on our experience.
That’s all folks
If you've made it this far, congratulations, you already know the basics of how to automate the deployment of CycleCloud and how you can integrate it into your own scripts.
#AzureHPCAI