Using NXLog to enhance Azure Sentinel’s ingestion capabilities

<Output AzureHTTP>
Module om_http
URL https://%WORKSPACE%.%SUBDOMAIN%/%RESOURCE%?%APIVER%
ContentType application/json
HTTPSAllowUntrusted TRUE
HTTPSCAFile %INSTALLDIR%\cert\ca-certificates.crt
<Exec>
create_stat(‘ec’, ‘COUNT’);
create_stat(‘bc’, ‘COUNT’);
create_var(‘batch’);
create_var(‘nextbatch’);
add_stat(‘ec’,1);

#—BEGIN— the enrichment of this event with any new fields:
# The following can be used for debugging batch mode if needed:
# $BatchNumber = get_stat(‘bc’);
# $EventNumber = get_stat(‘ec’);
# to_json();
#—END— the enrichment of this event

if (size(get_var(‘batch’)) + size($raw_event) + 3) > %SIZELIMIT%
# Flush this batch of events
{
set_var(‘nextbatch’, $raw_event);
$raw_event = ‘[‘ + get_var(‘batch’) + ‘]’;
add_stat(‘bc’,1);
set_var(‘batch’,get_var(‘nextbatch’));
$Workspace = “%WORKSPACE%”;
$SharedKey = “%SHAREDKEY%”;
$ContentLength = string(size($raw_event));
$dts = strftime(now(),’YYYY-MM-DDThh:mm:ssUTC’);
$dts_no_tz = replace($dts,’Z’,”);
$parsedate_utc_false = parsedate($dts_no_tz,FALSE);
$x_ms_date = strftime($parsedate_utc_false, ‘%a, %d %b %Y %T GMT’);
plxm->call(“genauth”);
add_http_header(‘Authorization’,$authorization);
add_http_header(‘Log-Type’,$SourceModuleName);
add_http_header(‘x-ms-date’,$x_ms_date);
}
else
{
$delimiter = get_stat(‘ec’) == 1 ? ” : “,\n”;
set_var(‘batch’, get_var(‘batch’) + $delimiter + $raw_event);
drop();
}
</Exec>
</Output>

The values for the three HTTP headers Authorization, Log-Type, and x-ms-date are set using the add_http_header procedure as shown above on lines 41-43. Log-Type is dynamically set to $SourceModuleName, the name of the input instance we chose at the beginning. Since all REST API events are categorized by Azure Monitor as Custom Logs, Azure appends _CL to the value of Log-Type in order to prevent naming conflicts with other Azure tables thus the name we originally chose, DNS_Logs, appears in Azure Sentinel as DNS_Logs_CL.

By leveraging $SourceModuleName for defining Log-Type, we have created a completely generic output instance that can be used with any other log sources.

Configuration checklist
To prepare for testing, let’s run through the steps needed to ensure success:

1. Use the output instance in this example nxlog.conf configuration in your current C:\Program Files\nxlog\conf\nxlog.conf NXLog configuration file.


2. Ensure that you have changed the values of WORKSPACE and SHAREDKEY to match those of your Log Analytics workspace.


3. Download the sentinelauth.pl Perl script. Copy it to the location defined by the PerlCode directive in the xm_perl instance (plxm, lines 1-4 above) and rename it to sentinelauth.pl.


4. Read about the Windows requirements for Perl in the Perl (xm_perl) in the NXLog Reference Manual.


5. Once the Perl requirements for Windows have been met, restart the nxlog service via Windows Services.

To test DNS Server logging of audit events, we added an A record for R04LRC13.example.com and reloaded the example.com zone. This logs an event with EventID 515 (Record Create) and another one with EventID 561 (Zone Reload).

Now it’s time to log into the Azure Log Analytics workspace that was defined in the DNS_Logs output instance and open Logs. After expanding Custom Logs the DNS_Logs_CL table should be visible. With a simple query, the newly ingested events are visible.

Expanding the first event’s details shows the complete set of fields and their values:

Temporary output instance

For testing purposes, you may want to add a temporary output instance for validating the integrity of your configuration. This lets you compare the events and their fields with what Azure Sentinel is ingesting. As you can see here, by adding a new output instance named TempFile as an additional destination to the route, this allows you to view the events in JSON format that will be stored in the file defined by the File directive.

nxlog.conf (Section: file output instance and modified route)