Unit Testing of Analytic Rules for Microsoft XDR

Microsoft’s SIEM solution for Azure cloud environments is Sentinel. Sentinel allows
for “analytics rules” to be created, which are used to search for patterns and
indicators in the SIEM data. Creating these analytics rules can either be done in the
Sentinel GUI which can be tedious, or via synchronization with a Git repository.
When using the latter option, the analytics rules come in the form of JSON
formatted ARM templates. These templates define the analytics rule as code, just like
any other deployment as code in the Azure cloud. Unfortunately, this comes with the
downside that a deployment is not tested in any way before it is deployed.
This thesis presents the implementation of an automated Azure DevOps pipeline to
validate Sentinel analytics rules to some extent before deployment, mitigating the risk
of errors of deployments greatly and introducing continuous testing into the process.
The solution also abstracts the complexity of the ARM template structure, which can
be hard to read, by accepting YAML rules as input which mainly focus on the most
crucial content of the rule itself, without having to care about metadata.

To achieve this, the following tools have been integrated:

• Maester as an automated testing framework.
• SentinelARConverter which converts YAML rules into ARM templates.
• ARM Template Test Toolkit to test for syntactical and to some extent semantic correctness of the finalized ARM template.
• KQL Analyzer to test the syntactical correctness of the KQL query provided with the rule (Kusto Query Language)

Keywords: Microsoft Sentinel, Azure DevOps, Automated Testing, ARM
Templates, KQL Validation, Maester Framework