Simplify Windows Server management in Azure with Automanage & Hotpatch

QUICK LINKS: 

00:00 — Introduction 

01:34 — Automanage capabilities 

03:26 — Behind Machine Configuration setting 

05:00 — Steps to get new or existing servers to Automanaged state 

06:22 — Correct a non-conformant state 

08:17 — Hotpatch capabilities 

11:56 — Wrap up

Link References: 

More about running Windows Server on Azure at https://aka.ms/wsonazure 

Unfamiliar with Microsoft Mechanics? 

As Microsoft’s official video series for IT, you can watch and share valuable content and demos of current and upcoming tech from the people who build it at Microsoft. 

• Subscribe to our YouTube: https://www.youtube.com/c/MicrosoftMechanicsSeries 
• Talk with other IT Pros, join us on the Microsoft Tech Community: https://techcommunity.microsoft.com/t5/microsoft-mechanics-blog/bg-p/MicrosoftMechanicsBlog
• Watch or listen from anywhere, subscribe to our podcast: https://microsoftmechanics.libsyn.com/podcast

Keep getting this insider knowledge, join us on social: 

• Follow us on Twitter: https://twitter.com/MSFTMechanics
• Share knowledge on LinkedIn: https://www.linkedin.com/company/microsoft-mechanics/
• Enjoy us on Instagram: https://www.instagram.com/msftmechanics/
• Loosen up with us on TikTok: https://www.tiktok.com/@msftmechanics 

Video Transcript:

- Coming up, we'll take an inside look at updates for managing your Windows Servers from Automanage as a unified solution to automate operations, apply consistent best practices and enforce policies on your servers that you manage via Azure and Azure Arc, to the new Hotpatch capabilities for Windows Server Azure Edition VMs to speed up deployment and updates to minimize reboots, so you can apply security updates faster. And joining me once again is Dean Wells, Lead Engineer on the Azure control plane, welcome back.

- Hey Jeremy, happy to be here.

- Thanks so much for joining us today. We both have a lot of experience with systems management operations, policy controls, and patch management, especially when it comes to Windows. So given the massive set of controls and tools that we have available today, what are we really solving for with Automanage and Hotpatch?

- Yeah, there certainly is a lot you can do with all the management tools, scripting options, the bolt-on services, but when you put everything together, it's just not the easiest thing to wrap your head around. So if you're experienced with server management, you might want to save the time and effort to apply the controls you need to make sure your servers don't drift from their desired state in the first place. Automanage in Azure assembles all of these options as a set of best practices that you can apply as policy to every server you provision, things like security baselines, monitoring, backups, anti-malware, connection to the Windows Admin Center to name just a few. Then with Hotpatch, we're delivering one of the top requests from server admins to install monthly cumulative updates and minimize the reboots required, so it's not always a monthly occurrence, and that way you can apply required security updates faster.

- And these really are both things that our server admins have been asking for for a while, and I'd love to see what's possible with Automanage and Hotpatch, so can you show us?

- Absolutely, I'll start with Automanage. So you can get to it in the Azure Portal by searching for Automanage, not surprisingly. From here, you'll see the machine best practices and controls to manage configuration profiles, and we highlight Automanage for Windows Server here, and that's the focus for our conversation today, but to be clear, this also works with Linux Servers. Config Profiles define what services and settings get applied. I'll start by showing what's in the config profiles view. You'll see that there are two default configuration profiles, one using Azure best practices for Dev/Test machines, and another for your production machines, and I have six others I've created below. I'm going to create a new profile in this case to highlight a few of the options, and you'll see what gets configured here goes beyond your standard GPO or basic management settings. After the standard fields for name, subscription, resource group, and region, you'll find all of the services that you can choose to keep enabled or disable the ones you might not need. First, you'll see something important to just about any stateful workload, backup. This is where you can set the frequency, timing, time zone even, restore behavior, and the retention range for your backup. Next, you'll find all the configuration options for Microsoft antimalware. Here you can set up exclusions for files and locations, file extensions, and specific process names, then the protection and scanning behavior, along with scan frequency. Then by enabling insights monitoring, you can see the health of your server through Azure Monitor. This also uses the Log Analytics Workspace service in the background. And for all of the machine policies, this next option for machine configuration will apply as baseline audit settings. So this is using the policies in machine configuration in the backend, which consists of hundreds of settings that you'll likely be familiar with if you've ever used Group Policy.

- So how can you see then what's behind the machine configuration settings?

- Yeah, you can look at everything within the Guest Assignments page in Azure. I'll take a look at the server here and go into the Machine Configuration blade. This row shows the Azure Windows baseline selected, and you can see the status is non-compliant. When I click into it, by default, it's filtered to show just non-compliance settings, so I'll change the filter to show compliance settings as well. And there are almost 300 settings, so I won't go through all of them, but as you can look down this list, you'll get an idea of just how granular these ones are. That said, we're still not through our list of the services you typically want to attach to your managed servers, so I'll head back over to my Automanage Configuration Profile view. Now you'll see the rest of the options, like Update Management, which uses an automation account and Log Analytics infrastructure to apply the updates, change tracking and inventory also, so you'll always know what's running in your service. Then Microsoft Defender for Cloud, which goes above and beyond mere anti-malware. It continually assesses your infrastructure security posture, provides additional security recommendations, and helps defend workloads in real-time. Finally, you'll see Azure Automation, Log Analytics, and Boot Diagnostics. And one of my favorite new capabilities here with Windows Admin Center, which you can use directly to interact with your Windows Servers right from the Azure Portal. And of course, you have the option to configure everything yourself manually, but we also provide the default best practices for Dev/Test and Production that you can use. The primary difference between the two is that with Dev/Test, we don't buy default, enable Backup or Insights Monitoring, like we would with Production machines.

- Okay, so now that you've described what's behind Automanage, what do you need to do then to get new or existing servers into the Automanaged state?

- That is super simple, in fact, that was one of our goals. As long as your machines are in Azure or connected via Azure Arc enabled server, you can onboard them individually or in bulk. I'll show you the process from the Azure Portal. So here we are in Automanage machines. You can see which machines you currently have enabled along with the configuration profile for each and their current status. I'll hit Enable on Existing Machine, and this is where you can choose the configuration profile you want to use from before. I can also view the Azure Best Practices here for production or use this dropdown to look at the Dev/Test equivalent. I'll close this and stick with the default one using best practices for production. Next, in the machine view, if I have a lot of machines, I can filter to the machines I want by name. In my case though, I'll pick these three on the first page, review and create, then confirm, and that's it.

- So, could you automate also the enablement of Automanage if you're also using things like Azure Policy or maybe easy to deploy new resources with Resource Manager templates?

- Absolutely, of course, yeah. There are policy definitions available for the best practice and custom configurations as well. To find them, you just need to filter the category to Automanage, and you'll see two different options for standard Automanage onboarding or custom onboarding. For ARM templates, it's literally just a few lines to add to your existing VM templates. And if you use either of these approaches, your new VMs will be enabled for Automanage right as they're provisioned.

- Okay, so with your existing and new machines now enabled for Automanage, how would this help then, for example, if a server was put out of a conformant state, maybe it's non-conformant?

- That's the best part. A lot of times people might have the best intentions, for example, as they run internal services or developing apps and testing solutions, where they might want to disable certain protections, and that can sometimes leave these machines as a blind spot for management, which is obviously never a good thing. So you'll see that right now everything is good, and we're in a conformant state, so here I'll pretend I'm an internal dev with permissions to change the VM configuration. From this blade, I'm going to uninstall a few extensions. I'll start by removing ConfigurationforWindows. That manages the things we saw earlier. Then I don't think I need this Dependency Agent, so I'll uninstall that too. Now, like my favorite book's name, I'll just hit Refresh and just like that, two of the five extensions are gone. So now the machine is missing a few important extensions to maintain configuration and monitoring tools, like Azure Monitor and App Insights. Now, while that's not a good thing at the moment, the good news is that Automanage will also see that configuration drift and force the machine back into conformance. Let's take a look at our server again, and immediately we see its conformant. I'll dig into the Status Report, and in the raw JSON logs, you'll see the status is actually ConformantCorrected. We pride ourselves in our simplistic status naming. The term, as it implies, means the server was once non-conformant, then later corrected. Now I'll search the log for our Dependency agent, and you'll see it also has a ConformantCorrected status, as does the Windows configuration extension. Now I'll close the Status Report and head back to the Extensions and Applications blade, and you'll see that all five extensions are running again, so our server is healthy and back in a conformance state.

- And it all happened automatically. And because services like Defender for Cloud and also log analytics were also enabled, everyone then on your team has what they need to monitor and manage this VM. Why don't we switch gears though to servicing and keeping those servers up to date? That's always been a challenge for IT.

- Absolutely, yeah, and as we all know, this is something that's a necessity to ensure that you have the latest security and quality fixes, but it also means that the servers running critical workloads will typically incur some downtime to apply these cumulative updates and then reboot, and that's where the new Hotpatch capabilities for Windows Server Azure Edition VMs come in. This works for servers running Server Core and will come soon to the full desktop experience. Whereas before, you'd have to reboot every month as you apply cumulative updates, with Hotpatch, reboot events are reduced dramatically. Part of the magic is that it can patch in-memory code for running processes without requiring a shutdown. This way, processes can continue to run during a cumulative update event, which means your servers get the security updates you need without the downtime that would normally result with a cumulative update.

- And that's something a lot of us have been waiting for, so can you show us how this might work in practice?

- Indeed, I can. I'll show you with an example, using a server with a process running, where we want to avoid downtime and even any packet loss. So here in Azure, I have a Windows Server 2022 Data Center: Azure Edition VM, running with SQL Server installed, and this is a Windows 11 jumpbox VM with SQL Server Management Studio and Wireshark installed. Now, here's the cumulative Hotpatch update I want to install, it's KB5019080. You'll see it doesn't require a reboot. Remember, my server is running Server Core, so I've got a command prompt open, and I'll run a Get-Hotfix command to see the updates already applied, and you'll see that that particular KB is not installed. Now I just need to start a process that before would've needed to be interrupted by a reboot. So from my jumpbox VM, I'll head into SQL Server Management Studio and start a load test in SQL running on my Windows server, and I'll also start SQL Server Profiler to monitor events. Now I don't want to lose any packets either in terms of networking activity, so I'll run Wireshark to monitor for packet loss. Now with our monitoring tools running, let's apply the Hotpatch. And to do that, I'm back in the updates blade of my Windows server VM. I'll hit One-Time Update on top, select my virtual machine, and hit Next. There's our update, and again, you'll see reboot status is set to NeverReboots. In the next screen, I have an option here to override the reboot behavior in case I do want to reboot, but that's no fun, so let's keep it as NeverReboots and leave the other defaults in place. From here I can kick off the in install.

- And, unlike other cumulative updates, this process is going to take a few minutes to run, so why don't we speed that up a bit?

- Indeed, yeah, nobody wants to watch paint dry. But if I head over into the Windows Admin Center to look at my server's activity, you'll see that it's very busy running our SQL process and installing the update. And in our jumpbox, everything is healthy. And you'll see in Wireshark here for networking, events are running in our SQL Server Profile and in the SQL Server Management Studio console, our query is still executing. Back in our VM blade in Azure, the update should almost be complete. There, it succeeded. And let's look at our applied updates in our command prompt, and you'll see there is our KB5019080 as the second one in the list. And our query is still executing. Events never stopped firing in the profiler, and if I open Wireshark one more time, jump into Statistics and then Capture File Properties, here we can see that we dropped zero packets. So the server stayed up the entire time, processes kept running, we didn't lose any packets, and our cumulative update Hotpatch with the security fixes we needed has been applied, zero downtime. And when you add this to what we've shown with Automanage, Azure has the most powerful set of services to simplify management for your Windows Servers, and you have all of the control you need.

- And it's really great to see all of these capabilities for managing Windows VMs on Azure. And like you mentioned, Automanage also works for your Linux VMs as well. So for anyone who's watching looking to get started, what do you recommend?

- I only have one suggestion really, start enabling Automanage on your Azure VMs. It's really easy to start with a few and just expand your coverage over time and even integrate it as part of your automated provisioning processes. And for Hotpatch, start creating your Windows server VMs using the Azure Edition images, and this even works as you build custom images yourselves. To find out more about running Windows Server on Azure, check out aka.ms/wsonazure.

- Thanks so much for joining us today, Dean, and of course keep checking back to Microsoft Mechanics for all the latest updates. Subscribe to our channel if you haven't already, and as always, thank you for watching.