Building a Production-Ready DMARC and TLS-RPT Analytics Platform on Azure

Email authentication posture is relatively easy to assess. Operationalizing it is harder. A point-in-time review can identify an incomplete DMARC policy, questionable DNS configuration, unauthorized sending infrastructure, or other exposure, but the finding itself does not remediate the condition or prove that it improves over time.

I designed and engineered a reusable Azure-based DMARC and TLS-RPT analytics platform to close that gap. The system securely ingests reports from external providers, preserves original evidence, resolves heterogeneous packaging and compression formats, normalizes DMARC and SMTP TLS reporting data, exposes an analytical model through serverless SQL, and presents operational telemetry through Azure Managed Grafana.

The larger objective was productization. Rather than solving the problem once for one environment, I designed the platform as a reusable Gold Plate: a secure, production-ready capability that can be deployed repeatedly with standardized architecture, security controls, application configuration, observability, dashboards, and validation while keeping client-specific configuration isolated.

The Problem Was Not Parsing DMARC XML

Parsing an XML document is a small implementation task. The systems problem is receiving untrusted reports from multiple external organizations and turning them into reliable, actionable, client-specific telemetry without creating an operational burden.

External Reporting Sources
        |
        v
Authenticated Ingestion
        |
        v
Preserve Original Evidence
        |
        v
Resolve MIME / ZIP / GZip / Format Variations
        |
        v
Normalize DMARC + TLS-RPT
        |
        v
Analytics-Ready Storage
        |
        v
Serverless SQL
        |
        v
Operational + Business Telemetry
        |
        v
Measure Improvement Over Time

That changed the engineering problem from "read a report" to designing a system with explicit contracts for identity, transport, serialization, binary content, asynchronous processing, evidence retention, normalization, analytics, authorization, network access, deployment, and operations.

Requirements and Architectural Drivers

• Security: authenticate ingestion, minimize secrets, use managed identity where practical, apply least privilege, and isolate client deployments.
• Reliability: preserve original reports before transformation, make rejected data observable, and maintain correlation identifiers.
• Interoperability: tolerate legitimate differences in provider packaging.
• Operability: expose actionable telemetry rather than raw reports.
• Business visibility: provide evidence that controls work and posture improves.
• Repeatability: automate resources, SQL, permissions, dashboards, networking, and application configuration.
• Cost efficiency: favor consumption/serverless components for a naturally bursty workload.

Reference Architecture

                    External Reporting Organizations
                    DMARC Aggregate / SMTP TLS Reports
                                  |
                                  v
                         Ingestion Mailbox
                                  |
                                  v
                           Power Automate
                      Classification / Transport
                                  |
                         Entra-authenticated
                                  |
                                  v
                         Azure Functions
                  +---------------+---------------+
                  |                               |
                  v                               v
             Report Ingest                 Normalize Report
                  |                               |
                  v                               v
             Raw Evidence                  Normalized Data
                  |                               |
                  +----------- ADLS Gen2 ---------+
                                                  |
                                                  v
                                       Synapse Serverless SQL
                                                  |
                                          Managed Identity
                                                  |
                                                  v
                                       Azure Managed Grafana
                                                  |
                                  +---------------+---------------+
                                  |                               |
                                  v                               v
                         Engineering / TechOps           Business Stakeholders
                         Failure Investigation           Posture / Improvement

Each boundary has a specific responsibility. Power Automate handles mailbox-oriented workflow and classification. The ingestion Function exposes a small authenticated API boundary. ADLS Gen2 provides durable raw and normalized storage. A separate normalization stage handles variable report packaging. Synapse Serverless provides an analytical SQL surface without dedicated database compute, and Managed Grafana provides operational visualization without maintaining another server platform.

Architecture Decisions and Tradeoffs

• Preserve raw reports: supports troubleshooting, reprocessing, parser evolution, and auditability.
• Separate ingestion from normalization: keeps the external API small and allows asynchronous processing.
• ADLS Gen2: creates a durable boundary between evidence and analytics.
• Synapse Serverless: supplies SQL analytics without continuously provisioned compute.
• Azure Managed Grafana: provides operational dashboards without another server estate.
• Managed identity: reduces stored credentials and credential lifecycle burden.
• Deterministic Grafana egress: supports controlled Synapse firewall allowlisting.
• Infrastructure plus application bootstrap: makes deployment reproducible beyond resource creation.

Designing a Deterministic Ingestion Contract

One subtle integration boundary was attachment transport between Power Automate and the ingestion Function. Connectors can expose attachments differently depending on packaging. Binary reports may appear Base64 encoded while MIME-wrapped content can surface as textual MIME containing quotation marks, boundaries, headers, and nested content.

Directly interpolating arbitrary attachment content into JSON created an unsafe contract. I standardized the boundary so report content crosses the HTTP interface as Base64 and the ingestion Function reconstructs the original bytes.

Mail Attachment
      |
      v
Power Automate
      |
      | Standardized Base64 transport
      v
Authenticated HTTP Contract
      |
      v
ReportIngest
      |
      v
Original Artifact Bytes

The broader lesson is that encoding, serialization, escaping, content type, and downstream expectations are part of API design, not incidental implementation details.

Defensive and Recursive Normalization

Real-world reporting invalidated the assumption that a filename extension reliably identifies the underlying report. A mail platform can present an attachment as a .p7m object even when the content is a textual signed MIME container containing a report, signature artifacts, and another compressed payload.

Instead of creating a sender-specific exception, I designed normalization around container semantics: identify the wrapper, extract supported report artifacts, recursively resolve additional compression or MIME layers, and only then invoke the DMARC or TLS-RPT parser.

MIME
  +-- Signature artifacts
  +-- ZIP
       +-- XML -> DMARC parser

MIME
  +-- GZip
       +-- JSON -> TLS-RPT parser

Raw, Normalized, and Rejected Data

raw/         Original report exactly as received
normalized/  Stable analytics-oriented representation
rejected/    Processing metadata for failed normalization

This separation prevents parser evolution from destroying evidence. Retained raw artifacts can be reprocessed when normalization improves, while rejected reports remain observable instead of silently disappearing.

Security Architecture

Power Automate -- Entra authenticated API --> Azure Function
                                                |
                                         Managed Identity
                                                v
                                             ADLS Gen2

Azure Managed Grafana -- Managed Identity --> Synapse Serverless --> ADLS Gen2

The ingestion API uses Microsoft Entra application authentication. Azure workloads use managed identities where supported. SQL access is granted to the Grafana identity through a dedicated database principal and constrained permissions. Secrets that cannot be eliminated belong in appropriate secret storage rather than source code or reusable configuration.

Network access is a separate control. Managed Grafana uses deterministic outbound addresses so Synapse firewall access can be restricted to the workspace's known egress set.

Analytics Must Preserve Meaning

A dashboard can be technically correct and still communicate the wrong thing. I treated the SQL model and Grafana panels as architecture because aggregation semantics affect operational decisions.

For DMARC, pass/fail interpretation must respect aligned SPF and DKIM. For TLS-RPT, policy-level session totals cannot be attributed in full to every MX host listed in a policy without creating misleading counts. The analytical layer therefore has to answer not only "does this query run?" but "what exactly does this number represent?"

• DMARC message volume and authentication pass rates.
• Source IPs associated with authentication results.
• Authentication failures over time.
• TLS successful and failed session counts.
• TLS reporting organizations and policy visibility.
• Receiving MX hosts associated with actual TLS failures.
• Failure reasons suitable for engineering investigation.

Different Stakeholders Need Different Evidence

Engineering and TechOps need to know which sources fail, whether SPF or DKIM alignment is involved, which receiving MX hosts are implicated, and whether a new failure pattern requires investigation.

Account-management and business stakeholders need evidence that posture is improving, unauthorized or misconfigured senders are being eliminated, secure transport is succeeding, and remediation has produced measurable progress from the initial assessment.

Initial Assessment
      |
      v
Remediation / Gold Plate Deployment
      |
      v
Operational Telemetry
      |
      +-- Engineering Action
      +-- Stakeholder Evidence
      |
      v
Measured Improvement

Productizing the Architecture

Once the first implementation was functional, I changed the question from "How do I deploy this?" to "How should an organization repeatedly deliver and support this capability?"

Prerequisite Validation
        |
        v
Azure / Identity Validation
        |
        v
Entra Application Configuration
        |
        v
Infrastructure as Code
        |
        v
Managed Identities + RBAC
        |
        v
Function Publication
        |
        v
Synapse / SQL Bootstrap
        |
        v
Grafana Data Source + Dashboard
        |
        v
Network Allowlisting
        |
        v
Deployment Validation
        |
        v
Power Platform Deployment Artifact

Standardization vs. Client Configuration

• Standardized: architecture, security patterns, storage organization, Functions, normalized schemas, analytical views, dashboard structure, deployment workflow, naming conventions, validation, and operational patterns.
• Parameterized: client identifier, environment, domain, mailbox, Azure subscription/resource group, region, Power Platform environment, and deployment-specific values.
• Generated: resource names, endpoints, managed identities, deployment outputs, and Azure-assigned network information.
• Secret: credentials or tokens only where workload identity cannot eliminate the requirement.

The Gold Plate Delivery Model

DMARC Analytics is one implementation of a broader engineering model I use for repeatable client solutions. A Gold Plate combines architecture, deployment automation, security controls, operational telemetry, documentation, and validation into a repeatable capability rather than a collection of scripts.

ASSESS
  Establish current posture
        |
        v
IDENTIFY
  Quantify risk, gap, or opportunity
        |
        v
DEPLOY
  Implement standardized remediation / controls
        |
        v
OBSERVE
  Collect operational telemetry
        |
        v
ACT
  Drive engineering response
        |
        v
MEASURE
  Demonstrate improvement

A pre-sales or client assessment can evaluate areas such as DMARC posture, DNS posture, or external exposure. Where it identifies a material gap, an appropriate Gold Plate can provide remediation, monitoring, automation, reporting, and ongoing controls. The reusable element is not a mandatory technology stack; it is the engineering discipline around assessment, security, deployment, observability, ownership, and measurable outcomes.

Failure Modes Became Architecture Inputs

• Attachment representation: MIME content demonstrated why arbitrary dynamic content should not be interpolated directly into JSON contracts.
• Provider packaging: nested and signed containers reinforced content-semantic normalization.
• Cloud identity: Entra and SQL authorization introduced propagation and deployment-order requirements.
• Network egress: Managed Grafana can expose multiple deterministic outbound addresses, so firewall automation must reconcile the full published set rather than one observed IP.
• Regional constraints: cloud service availability and provisioning behavior must be treated as deployment inputs.

The reusable lesson is to turn production observations into generalized controls. Fixing one IP, sender, or attachment solves an incident; improving deployment or normalization for the entire class of condition improves the product.

Known Boundaries and Current Tradeoffs

• Some Power Platform connection authorization remains an interactive identity boundary.
• Legacy non-solution Power Automate packaging is less mature than full solution-based ALM.
• Serverless SQL and Grafana time-filter behavior requires validation before all dashboard time semantics can be treated as authoritative.
• Public Synapse connectivity with deterministic allowlisting is pragmatic for the current design; private networking presents a different security/complexity tradeoff.
• Provider diversity means parser regression testing should expand as new report samples are encountered.

Production readiness does not require pretending every future capability exists. It requires knowing which risks are controlled, which tradeoffs are accepted, and what conditions should trigger the next architectural change.

How I Would Evolve the Platform

• Migrate Power Automate to Power Platform Solution-based ALM with connection references and environment variables.
• Build a broader provider regression corpus and automated normalization tests.
• Expand deployment verification across network, identity, SQL, Functions, storage, and dashboards.
• Add explicit historical baselines and before/after posture measurements.
• Version Gold Plate deployments and define upgrade paths for existing environments.
• Strengthen alerting for rejected reports, ingestion failures, and material authentication changes.
• Allow assessment output to seed deployment configuration where secure and appropriate.

What This Project Demonstrates

The most important result is not an Azure Function or Grafana dashboard. It is the transformation of a security observation into an operational capability.

"We need better DMARC visibility."
               |
               v
       Security Assessment
               |
               v
      Architecture + Controls
               |
               v
       Secure Data Ingestion
               |
               v
    Resilient Normalization
               |
               v
       Analytical Model
               |
               v
    Operational Telemetry
               |
               v
     Automated Deployment
               |
               v
     Repeatable Gold Plate
               |
               v
     Measurable Improvement

The implementation required software engineering, but the larger work was architectural: decomposing requirements, defining system boundaries, selecting managed services, designing identity and network controls, defining the data lifecycle, establishing deployment boundaries, separating standardized from configurable elements, and connecting telemetry to operational and business outcomes.

Lessons Learned

• Point-in-time assessment is more valuable when connected to an operational remediation path.
• Integration boundaries are architecture. Encoding, serialization, authentication, and contracts deserve deliberate design.
• Preserving raw evidence creates options. Reprocessing and parser improvement remain possible.
• Provider diversity should be modeled, not patched around.
• A metric is only useful when its semantics are correct.
• Managed identity reduces credential ownership but does not replace authorization design.
• Deployment automation must configure the application, not merely create resources.
• Production failures should improve the reusable architecture.
• Different stakeholders need different views of the same evidence.
• Repeatability is an architectural property. A solution becomes a product when another environment can receive the same capability without reconstructing tribal knowledge.

Final Thoughts

What is the current security posture?
        |
        v
What material gap exists?
        |
        v
What control or remediation should be deployed?
        |
        v
How do we know it is operating?
        |
        v
What action follows from the telemetry?
        |
        v
Can we demonstrate improvement?
        |
        v
Can we deploy and operate the same capability again?

Those are not simply DMARC questions. They are questions about security architecture, platform engineering, operational ownership, and productized cloud delivery.

The technology supports that lifecycle. The larger engineering challenge is designing a system that can move from assessment to remediation, from remediation to evidence, and from one successful implementation to a repeatable capability.

Tags: Cloud Architecture, Security Architecture, Azure, DMARC, TLS-RPT, Email Security, Azure Functions, ADLS Gen2, Synapse Serverless, Azure Managed Grafana, Power Automate, Bicep, Managed Identity, Platform Engineering, Infrastructure as Code, Observability

Published: August 30, 2026
Author: Robert H. Osborne