This post has been republished via RSS; it originally appeared at: Core Infrastructure and Security Blog articles.
Azure Monitor Logs, also known as Log Analytics, is a fundamental tool for monitoring and reporting on your Azure, multi-cloud, and hybrid resources. It supports such a vast array of Microsoft cloud services that it has become one of the most used Azure services for all sorts of customers. Consequently, Azure Monitor Logs has also become an important cost driver for many Azure customers and being able to allocate or split those costs across the right cost centers in the organization is a pressing need most customers have. This is especially relevant for customers who centralize as much as possible their Log Analytics workspaces, following Microsoft’s recommended best practices. The question this article answers is: how can we sort out which logs belong to which cost center in a simple manner? My colleague @Bruno Gabrielli recently described the logic behind cost allocation by subscription, resource group or resource. What I am going to describe next is how to do it based on the resource tags.
Important: this article describes how to allocate Analytics ingestion costs only. Other cost factors such as Basic logs ingestion, log retention beyond 31 days or Basic Logs queries are not included but can follow a strategy like the one described here. For more details about Azure Monitor Logs pricing, visit the pricing page.
The keys to allocate Log Analytics ingestion costs
Most of the billable Azure Monitor logs come with a _ResourceId column identifying the Azure or Azure Arc resource who was responsible for emitting the logs. For example, a Virtual Machine running the Azure Monitor agent which is collecting Syslog events will send its logs to the Syslog table in a specific Log Analytics workspace and all its logs will identify the Virtual Machine through the _ResourceId column. This is also true for other Azure platform services using Diagnostic Settings to export their logs to Log Analytics. For example, Key Vault audit logs or Azure Firewall network event logs also come with a resource identifier.
When speaking of logs cost allocation, a typical and most of the time acceptable reasoning should be to allocate the logs costs to the resources emitting those logs. If you agree with me, then you just need to ensure all your Azure and Azure Arc resources are tagged according to your cost allocation needs, which is one of the fundamental practices of cost management in Azure. With all this set up, we “just” have to correlate Azure Monitor logs with resource tags. The next section describes an Azure Monitor Workbook that does this plumbing.
Log Analytics Ingestion Usage by Tag Workbook
The Log Analytics Ingestion Usage by Tag Workbook puts into practice the rationale I described above, by scanning all the Log Analytics workspace logs and correlating them with Azure tags through the respective Azure resource Id. Needless to say, the larger the Azure environment and the larger the logs time range, the longer it takes to generate the results. Therefore, as a rule of thumb, start small and progressively increase the scope of the report.
First thing, you must adjust the Ingestion Price parameter according to your Azure price sheet and the Azure region of the Log Analytics workspaces in scope. Next, select the desired tag for cost allocation by setting the Tag Name parameter. Once you finish setting the parameters, the Workbook will scan the selected workspaces and present a couple of tables like the ones below. The Tag Value column presents the value of the desired tag associated to the logs-emitting resource.
The left table shows ingestion costs for all identifiable resources, i.e., resources with a non-empty _ResourceId column. In case the Tag Value comes empty, this means that the respective resource does not have the desired tag associated.
The right table shows ingestion costs for non-identifiable resources, i.e., where the logs did not contain the _ResourceId column. In this case, we do our best, splitting instead the costs by the Log Analytics workspace where the logs were ingested, and classifying them with the Tag Value associated to each workspace.
The last step is to export the results to Excel, by clicking on the export button, and do the aggregations you need to allocate your Log Analytics costs according to the selected tag. Please remember the costs may not represent exactly what you see in your Azure bill. Use this data only to determine the approximate percentage of Azure Monitor Logs costs to allocate to each cost center.
Installing the Workbook
To install this Workbook in your environment, open the Monitor blade in the Azure Portal and select Workbooks. Then click on the Empty workbook button.
Finally, click on the Workbook code editor button, replace the JSON code by the one available here, click on “Apply”, then “Done Editing”, and that is it. If you want to reuse it often, do not forget to save the Workbook (you need at least Workbook Contributor permission somewhere in Azure to do it).
Happy cost allocation!