This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Community Hub.

Howdy Readers! Welcome to my post!

 

I get lot of questions around how to secure Monitor services and why Azure monitor needs to be secured. So, I decided to write this blog which explains about securing monitoring services and why we care about Azure Monitor Private Link Scope (AMPLS).

 

Let’s talk about Azure Monitor first, it is one of the most critical and must-have services for monitoring data for your cloud and on-premises environments. You can learn more about Azure Monitor. Microsoft includes a few different services into Azure monitor like Log Analytics, App Insights, VM Insights, Container Insights, Network Insights etc. Azure Monitor collects two main types of details from Azure Services - 1) Metrics, and 2) Logs.  Once it’s being collected, this data can be used to trigger alerts, diagnose issues, analyze trends, etc.

 

The following diagram shows a high-level architecture view of Azure Monitor.

 

Having said that we need to understand why Azure Monitor needs to be secured. Why is it so important? Azure monitor will have many critical information types including usernames, IP addresses, server names, etc. You may all agree on why this data needs security. Also, Microsoft have their own measures to secure the ingested log data to be secured (link)

 

There are a few options to secure this service -

• Network Isolation
• Access Controls (IAM)

In this blog, I will be talking about network isolation only. I will cover Azure Monitor Access controls in my other blogs. You can refer to this article - https://learn.microsoft.com/en-us/azure/azure-monitor/roles-permissions-security

 

Azure Monitor Network Isolation

Network Isolation can be configured with the help of private endpoints. But the interconnection with the Azure monitor and private endpoint is slightly different from how we normally work with private endpoints. For that, we use an intermediate resource called Azure private link.

With Azure Private Link, we can securely connect Azure resources to a virtual network by using private endpoints. Azure Monitor is a group of different interconnected services (as mentioned before) that work as one to monitor the workloads. Azure Monitor Private Link connects a private endpoint to a set of Azure Monitor resources, outlining the boundaries of the monitoring network. This is known as an Azure Monitor Private Link Scope or AMPLS.

 

 

What is Private Link again?

Let’s recap: a Private Link is a unidirectional connection between two resources in different VNETs. Unidirectional means that there is a “client side” (called Private Endpoint) and a “server side” (called Private Link Service). However, for Microsoft-managed services, the “server side” is not visible by the user, and you only see the Private Endpoint in your Virtual Network as a representation of the service (Azure SQL in this example) with a private IP address in your VNet range:

Private Link to an Azure SQL service architecture

 

You might be wondering what is that green pipe between the VM and Azure SQL? the private endpoint (and its associated NIC) is a “representation” of the service, but the actual traffic flows straight from the Virtual Machine to the destination service encapsulated in IPv6.

 

So, what is an Azure Monitor Private Link Scope (AMPLS)?

Certain services (today Azure Monitor and Arc) do not support the architecture above. For example, in Azure Monitor you cannot just create a Private Endpoint directly connected with a Log Analytics Workspace. Instead, you use something in between: the Private Link Scope: This Azure Monitor Private Link Scope will be associated on one side to the Private Endpoint, and on the other to certain Log Analytics and Application Insight workspaces. From the perspective of the Virtual Machines in the VNETs, nothing changes: they talk to the private IP address of the private endpoint, and traffic is magically sent using private IP addresses to the destination Log Analytics workspace.

Azure Monitor Private Link Scope architecture

 

It’s not like other Azure PaaS services i.e. one endpoint per resource. AMPLS has at least ten Azure endpoints involved, and some of them are global. When configuring private endpoint target resource, instead of using existing application insights or log analytics workspace, AMPLS must be created as the target resource. Furthermore, DNS configuration can be challenging, as there is an issue called DNS override. You can avoid DNS overrides by using a single AMPLS - https://learn.microsoft.com/en-us/azure/azure-monitor/logs/private-link-design#guiding-principle-avoid-dns-overrides-by-using-a-single-ampls

Personally, I prefer the below hub and spokes architecture. Azure Monitor Private Link Scope (AMPLS) is connected to the hub network and the traffic from the spoke network will also traverse through the hub network. If your organizations use a single global or regional DNS to have a single Private Link for managing traffic of all Azure Monitor resources, across all global, or regional networks via the hub network. You might prefer to create separate private links for your spoke virtual networks, for example, to allow each virtual network to access a limited set of monitoring resources. In such cases, you can create a dedicated private endpoint and AMPLS for each virtual network. You must also verify they don't share the same DNS zones to avoid DNS overrides.

In addition, log services like log analytics workspace accepts inbound traffic from the cloud resources but also from on-prem services. When you enable private links, all the resources within the VNET or networks peered with the particular network can send the diagnostics logs to the monitoring services.

 

Hub & spoke networks architecture

 

AMPLS Limits

Azure monitor component must be connected to AMPLS to access via private link, but one AMPLS can only connect to maximum 50 resources. Each VNet can only connect to one AMPLS. Please refer this doc - https://docs.microsoft.com/en-us/azure/azure-monitor/logs/private-link-security#consider-limits

As per the information provided in this article, when you have hundreds or thousands of log analytics workspaces or application insights, you will need multiple Azure monitor private endpoints and multiple private link scopes. That means following global FQDNs, each one will need to be resolved to a different private IP, depending on the specific VNet.

• api.monitor.azure.com
• global.in.ai.monitor.azure.com
• profiler.monitor.azure.com
• live.monitor.azure.com
• snapshot.monitor.azure.com
• eastus2–0.in.ai.monitor.azure.com

Why we care about AMPLS?

By using Azure Monitor Private Link Scope, you can:

1. Connect privately to Azure Monitor without opening any public network access.
2. Ensure your monitoring data is only accessed through authorized private networks.
3. Prevent data exfiltration from your private networks by defining specific Azure Monitor resources that connect through your private endpoint.
4. Keep all traffic inside the Microsoft Azure backbone network.

This provides an additional layer of security to your monitoring data and helps prevent unauthorized access.

 

Tips: when configuring AMPLS

There are a few things to keep in mind when configuring Azure monitor private link scopes:

• Private links rely on private DNS zones.
  • Make sure you connect the Private link to the correct private DNS zone.
  • Make sure you have the correct connection from the DNS zone to the required virtual network.
  • The DNS configuration needed for Azure Monitor can be found here.

 

• After adding the appropriate rules to my DNS forwarding ruleset, I can move on the local DNS server and configure the forwarder to send the queries for the domains needed by the agent to the private resolver.

 

• Microsoft does have separate documentation for the supported network scenarios - https://learn.microsoft.com/en-us/azure/azure-monitor/logs/private-link-design. Please make sure you understand the supported scenarios otherwise logs from the resources will not reach the insights or workspaces.
• There can be some cases you still need log ingestion to be possible via public networks for this also you do have a couple of options available.

• When you disable public access for data ingestion, no machine can send data to this component except those that are configured to send traffic through AMPLS. But in my opinion, if you enable these options, it defeats the purpose of having a private link scope and making the log vaults private. So, try your best not to turn on these features.

Feature Assessment

I created a lab environment in order to better understand product feature based on this article - https://docs.microsoft.com/en-us/azure/azure-monitor/logs/private-link-security.

Test environment:

• Connect activity log to a log analytics workspace for injection test.
• Teams’ SSO bot web app is linked to an Application Insights.
• To save cost, I provisioned Windows server 2019 with DS1_v2 VM SKU.

 

Application Insights and Log analytics workspace are converted to private link.

In this test, assume application insights and log analytics workspace are converted to private endpoint at same time, network isolation is configured to deny public injection and public query. Follow this article- https://docs.microsoft.com/en-us/azure/azure-monitor/logs/private-link-security#example-connection

From a public network device (e.g. home PC), I lost the access to both the application insights and log analytics workspace, but from the Azure VM running from the VNET, all works.

 

 

Private link for log analytics only and allow public access for application insights.

I experimented the Azure monitor private link for log analytics only. When you configure DNS records for Azure monitor private endpoint, I omitted *.monitor.azure.com, and only configured log analytics workspace FQDN.

 

Now, for application insights, allow public network access in Virtual networks access configuration.

 

For Log analytics workspace, deny public network access in Virtual networks access configuration.

 

From my private network, I’m not able to query log analytics any more from Azure portal. Even though application insight has workspace specific FQDN, it still relies on the global Azure monitor endpoint.

 

Is log analytics workspace injection still working via private link? Now, let’s allow query from public network, so we can validate if recent log is captured.

 

Now from my Azure VM, I’m able to query the log from workspace, and it shows most recent log entries.

Allow public network access to application insight works fine.

 

Test concludes –

As MS document says, once Azure monitor private link is turned on, it’s all or nothing for application insight, since both query and ingestion endpoints are global. Please refer to this article - https://docs.microsoft.com/en-us/azure/azure-monitor/logs/private-link-security#azure-monitor-private-link-applies-to-all-azure-monitor-resources

Now, can we turn on Azure monitor private link for just log analytics and leave application insight for future phases? When log analytics workspace specific FQDN points to private IP, and you leave *.monitor.azure.com FQDN unchanged as public, Azure portal log analytics query stops working, since query depends on Azure monitor global FQDN. For e.g., api.loganalytics.io (Part of CNAME chain api.monitor.azure.com). Log analytics workspace query works, when you allow public network access for queries in log analytics workspace network isolation blade.

 

Summary

As I explained above, the configuration of Azure Monitor Private Link Scope (AMPLS) is a must-have in your environments. It provides more protection to the log and metric data by ensuring that monitoring data is only accessed through authorized private networks, preventing data exfiltration, and keeping all traffic inside the Microsoft Azure backbone network.