This post has been republished via RSS; it originally appeared at: Azure Developer Community Blog articles.
SharePoint Online (SPO) is Microsoft’s enterprise-class document management service for Microsoft365. The SharePoint Online (SPO) team at Microsoft has created a solution, built on Azure DevTest Labs (DTL), to make SPO testing environments readily available to SharePoint engineers, leveraging the strengths of DevTest Labs and Azure. This is the second in a series of blog entries enabling anyone to build, configure and run a similar system using Azure and DTL. For part 1, see How the SharePoint Online Team Leverages DevTest Labs.
In this blog post, two use cases of the solution’s expected VM lifecycle are described. A simple architecture is proposed that involves hooking up an Azure Function App to DevTest Lab Events by way of Azure Event Grid to implement the two use cases. There is a brief walk-through of the implementation intended to be used by other Azure customers to build similar solutions. If you are building an event-driven DTL application, consider using Azure Functions and Event Grid as the SPO team did. This article is of interest to you!
Scenario
As introduced in Part 1 of this blog series, environments in Azure DevTest Labs are a natural fit for SPO integration testing. SPO engineers can claim test environments that are miniaturized data centers, with all components shrunk down and interacting on a single Azure VM inside DTL. Pools of these VMs are made available for SPO engineers so that test environments are readily available at any time. When finished, they unclaim them.
There are two features that the SPO team has built on top of existing DevTest Labs Claim/Unclaim operations:
Feature 1 – Tag a Claimed VM
By default, DTL does not allow an easy way for the admin to know the ownership of claimed VMs. A common way to do this is to add an “Owner” tag to the VM using Azure Tags. Tags are flexible and allow lab admins to easily organize Azure resources and make them easy to query, for example, to list VMs assigned to a given user or set of users. In this solution, when a user claims a VM, an event is fired, and the handler attaches an “Owner” tag, with associated user name as data, to the VM. The tag shows up in the VM’s properties in the Azure Portal.
Feature 2 – Delete an Unclaimed VM
In SPO’s use case, VMs are not re-usable. Once a VM is claimed by a lab user, the state on the VM may change in such a way as to make the VM unsuitable for future users’ needs. For example, users can change the state of one of the SharePoint web applications, or introduce state into SharePoint by simply adding a document to a document library as part of a test. (For a better understanding of SharePoint’s document libraries, try: What is a document library?) The engineer may also update the code running on the VM, which can change behavior in a way undesirable for other engineers.
In general, once a VM is assigned to a lab user, its validity as a clean test environment can no longer be trusted. With this use case in mind, VMs in the SPO solution are single-use. Once unclaimed, they are deleted.
Note: Replenishment of the pool of available VMs is described in a future blog entry in this series, entitled “Maintaining a Pool of Claimable VMs”.
Both of these features can be added to DevTest Labs using Azure Event Grid, with an Azure Function listening and handling events. The solution is simple and elegant; please read on to see how this is configured.
Event Handling with Azure Functions
Azure Functions are a flexible, serverless computing platform in Azure. Functions can be configured to trigger when an Azure event fires, and DevTest Labs fires “Claim” and “Unclaim” events when VMs are claimed or unclaimed, respectively. When configured properly through Event Grid, events are dispatched to an Azure Function subscriber.
For simplicity and brevity, the solution described herein implements a single Azure function to handle both Claim and Unclaim events. The function is written in PowerShell and implements the tagging and delete business logic for these events. It is deployed in a Function App, triggered when events fire, in accordance with this table:
|
DevTestLab Event |
Event Name |
Action |
|
Claim |
microsoft.devtestlab/labs/virtualmachines/claim/action |
Add an "Owner" tag to the VM |
|
Unclaim |
microsoft.devtestlab/labs/virtualmachines/unclaim/action |
Delete the VM |
Note: You can find a complete list of DevTestLab events here: DevTest Lab Events. There are several useful ones that, when paired with the technique described in this post, can be used to solve other event-driven scenarios using the template given here.
Solution Architecture
This solution builds on the ideas presented in the article Use Azure Functions to extend DevTest Labs. A key difference is how the events are triggered: In the article, the Functions are triggered via an HTTP call. In this example, Claim/Unclaim events are generated from the DevTest Lab and routed via Event Grid to the function app. We will also be building on the coding concepts in Azure Event Grid trigger for Azure Functions.
Walkthrough
The following solution assumes a pre-created Azure DevTest Lab, such as one created using the steps in Create a lab in Azure DevTest Labs. In this example, the Lab has been freshly deployed with name “PetesDTL” and resource group “PetesDTL_rg” with no VMs yet created.
Step 1: Create a Function App
Use the “Create a Resource” UI and find “Function App” in your Azure subscription, and hit “Create”. Select the resource group for your DevTest Lab. This is optional but is a convenience for grouping your app with the other resources for your Lab. In this example, the Name of the app is "PetesFnApp" and is in the “PetesDTL_rg” resource group with a Runtime stack that is PowerShell Core. PowerShell Core is a flexible, operating system independent control language that is easy to get to work with Azure.
Leaving the majority of the settings with their defaults, hit “Review + create”, then “Create” to deploy the function app after the verification step.
Step 2: Create the FnDTLClaimUnclaim Function
Navigate to the newly-deployed Function App, hit “Functions” and then “+ Add” to add a new function to the App. Leave the “Development environment” to be “Develop in portal” and select “Azure Event Grid trigger” for the template. In this example, the new function is named “FnDTLClaimUnclaim”. The name indicates that this Powershell Core function is dual-purpose and will handle both the “Claim” and “Unclaim” events from the DevTest Lab. Click “Add”.
Step 3: Add logic to the function
Navigate to the newly-created function, and hit “Code + Test”. There is some sample code in the edit window, which you can replace with the following code:
Click “Save” to save the function.
The code snippet contains an ApplyOwnerTag function and a main body. The ApplyOwnerTag function gets the user from variety of sources, preferring first the identity from the passed-in claims, then looking at the owner or created-by user from the DevTestLabs VM. Once it has a valid user name, it adds the “Owner” tag to the VM.
The main body of the Azure Function first determines which event is being handled. On a Claim event, the ApplyOwnerTag function is called to add the “Owner” tag to the given VM. On Unclaim, the VM is removed. This logic implements the behavior desired for the SharePoint Online application for these two events.
Step 4: Assign an Identity and Role for the Azure Function
The Azure function has a call to Connect-AzAccount, which requires that the function app uses a System-assigned identity. (User-assigned identities can also be configured, however for simplicity this example uses a System-assigned identity.) Further, the identity in question needs access to resources in the subscription in order to add tags and to delete VMs. To configure this, click on the Function App, then Identity. Change the Status to “On”.
Next, Click on “Azure role assignments” to configure role-based access control for this Function App. In this example, Contributor role is assigned for the entire Azure subscription. While it’s possible to configure more fine-grained access to the Identity, this example in order to keeps things simple.
The resultant role assignment should look like this:
Step 5: Create an Event Subscription for the Function App
At this point we have logic ready to handle the DevTestLabs events, but the Function App is not configured to subscribe to those events. Azure Event Grid is infrastructure for mapping Azure events to logic and can be used to create rich applications. To get deeper into Azure Event Grid, please check out Azure Event Grid Overview.
Navigate to the subscription, then click Events. Under “Get Started”, click on “Azure Function”:
Enter “DTLEventSub” as the name. Use “ClaimUnclaimVM” for the System Topic Name. For Endpoint Type select “Azure Function” with Endpoint “FnDTLClaimUnclaim”.
Hit Create. This will create the ClaimUnclaimVM Topic and the Event Subscription.
Step 6: Test the Claim functionality
Navigate to the DevTest Lab in your subscription, “PetesDTL” in this example, and click “+ Add” to create a new VM. The type of VM you choose is not relevant for this example – you can select the operating system and resources that are appropriate for your application. In this example the VM is named “petesvm001”.
In “Advanced Settings”, select “Yes” for “Make this machine claimable” under “Claim options”, then in “Basic Settings” click “Create”. The reason is that we expect to Claim the VM through the UI to trigger the tagging behavior. Later, when we Unclaim the VM, we will expect to see the VM being deleted.
After the VM is created (this may take several minutes, perhaps tens of minutes, to complete) it should show in the “Claimable virtual machines” section of the DevTest Labs UI:
Note that this VM does not have an “Owner” tag yet, since it has not yet been claimed:
Now, click the ellipsis (…) on the VM in the Claimable virtual machines, and select “Claim”. This operation will also take several tens of minutes before the VM is successfully claimed and the Azure event has made its way through the Event Grid infrastructure and called our Azure Function. You can monitor when your Function has been called by selecting the Function in the Function App UI and seeing Total and Successful Execution Count in the Overview section:
You can also see what the Azure Function logs in near real-time by using the “Code + Test” UI and opening the Logs popup at the bottom:
After several minutes, the Function is called, and the ApplyOwnerTag PowerShell function adds the current user’s name in the Owner tag, as expected:
Now navigate to “My virtual machines” in the DevTest Labs UI, click the ellipsis (…) and select “Unclaim”:
Once again, it will take several minutes for the event to make its way through Azure to call FnDTLClaimUnclaim, but when it does the Remove-AzResource call will ensure the VM gets deleted:
This completes the full Claim/Unclaim cycle that a SharePoint Online user would experience and demonstrates the two features built on top of DevTest Labs.
Some Notes on Performance
The Azure Function App used in this example uses the PowerShell Core runtime. PowerShell was chosen for its strength as a simple control language for Azure, and its amenability to concise code samples. However, there is overhead to the boot time and resource consumption for PowerShell-based Function Apps over, say, C#-based Apps that should not be overlooked for performance-sensitive applications.
Eventing in Azure has its own set of performance characteristics, and you will notice with these samples delays of minutes and sometimes up to tens of minutes for events to fire and be handled by the Function App. This can be improved somewhat by upgrading from a Consumption Plan to an Dedicated App Service Plan, but one should bear in mind that there is always inherent and unavoidable latency due to the Azure’s eventing model, and this should be accounted for in the design of the Azure application.
What's Next?
The next blog post will describe how the SharePoint team built an Azure VPN that complements the DevTest Lab, securing the connection between lab users and the VMs they connect to. If you are interested in securing your environment, this is not one you will want to miss!
If you run into any problems with the content, or have any questions, please feel free to drop a comment below this post and we will respond. Your feedback is always welcome.
- Pete Harwood, Principal Engineering Manager, OneDrive and SharePoint Engineering Fundamentals at Microsoft