Microsoft AZ-104: Azure Monitor Data Collection Rules
Azure Monitor Data Collection Rules define how supported telemetry is collected, transformed, and sent to destinations. Microsoft describes the DCR model as an ETL-like data-collection process that standardizes configuration across Azure Monitor scenarios. Instead of embedding collection details separately in every agent or resource, a DCR can define data sources, data streams, transformations, and destinations, while Data Collection Rule Associations connect resources to the appropriate rule.
DCRs are now foundational to many Azure Monitor Agent scenarios, custom log ingestion, transformations, and other modern collection paths. Data Collection Endpoints are related but separate resources: a DCE provides endpoints for configuration access and/or ingestion in scenarios that require it. Microsoft explicitly notes that a DCE is not required for every DCR scenario, so architects should add one only when the collection design needs it.
DCR design belongs inside Azure Architecture in Practice.
Think of a DCR as a data pipeline contract
A DCR specifies what telemetry enters, which stream it belongs to, how it can be transformed, and where it is sent.
Azure Monitor design becomes more manageable when collection logic is centralized and versioned instead of hidden across individual virtual machines.
The rule should be owned like other production configuration because a change can affect data volume, cost, alerting, and incident evidence.
Separate data sources from destinations
Data sources define what the monitored resource or ingestion client supplies.
Destinations can include supported Azure Monitor stores such as Log Analytics or Azure Monitor workspaces depending on scenario.
Data flows map streams to destinations and optional transformations, which lets one collection rule express a clear route from source to target.
Use DCR associations deliberately
A Data Collection Rule Association links a monitored resource to a DCR for supported agent-driven collection.
This allows teams to reuse one rule across resources that need the same collection behavior.
Associations should follow workload, environment, sensitivity, and ownership boundaries instead of assigning one giant enterprise rule to every resource automatically.
Use transformations to control telemetry
DCR transformations can use KQL-style transformation logic for supported streams before data lands in the destination.
This can filter records, select fields, normalize values, or reshape data to reduce noise and cost.
Transformations should be tested carefully because dropping or renaming data at ingestion can remove evidence needed later for incident response or troubleshooting.
Use a DCE only when the scenario needs it
A Data Collection Endpoint provides regional endpoints for configuration access and data ingestion.
Some private-link, custom ingestion, or network-isolation designs need a DCE, while other DCR scenarios can use platform endpoints without one.
Microsoft’s current guidance explicitly says DCEs are not required universally, so avoid creating them by habit.
Design for regionality
DCRs, monitored resources, destination workspaces, and DCE components have regional relationships.
Configuration endpoints can need to align with monitored-resource regions, while ingestion endpoints align with destination region requirements.
Failure-domain design should include monitoring itself because telemetry collection can be affected by regional or network outages.
Control cost at collection time
Collecting every event at maximum verbosity can create significant ingestion and retention cost.
Use DCR scope and transformations to collect signals that support operations, security, audit, or performance.
Do not filter solely for cost: preserve evidence required for high-value investigations and regulatory obligations.
Version and test rule changes
A DCR change can alter several thousand resources if the rule is shared broadly.
Use infrastructure-as-code, peer review, test resources, and staged association changes where the blast radius is large.
Monitor ingestion volume and schema after rollout so a transformation error does not silently create a telemetry gap.
Keep collection architecture understandable
For AZ-104, the durable mental model is source → DCR → optional transformation → destination, with DCRA connecting resources and DCE supplying endpoints only where required.
This model helps teams distinguish agent configuration, ingestion routing, endpoint networking, and storage—four layers that are often confused when telemetry stops arriving.
DCR naming and ownership should reflect the workload or control objective. Names such as prod-windows-security or aks-platform-logs are easier to review than generic rules called dcr01. Tags and deployment scope can make responsibility visible when many teams share one Azure tenant.
Migration from legacy agent or collection methods should be staged. Compare event coverage, data schema, alert dependencies, dashboards, and retention before turning off the previous path. A monitoring migration is successful only when operational consumers confirm they still receive the signals they need.
Private Link designs should test DNS and endpoint reachability for DCEs and workspaces. A correct DCR cannot compensate for a network path that blocks the agent from retrieving configuration or sending data.
Monitoring the monitoring pipeline is essential: alert on sudden ingestion drops, association failures, agent health, and unexpected volume spikes so telemetry problems are detected before a real application incident depends on missing logs.
DCR design should begin with a telemetry contract. List the events, counters, logs, or custom records the workload needs, who consumes them, the destination table, retention purpose, and expected volume. This prevents teams from enabling broad default collection and then trying to reduce cost after large amounts of low-value data have already become dependencies.
Azure Monitor Agent configuration is one of the major DCR scenarios. A resource can have associations that determine which rules supply its agent collection behavior. When several teams share a VM or scale set, review overlapping DCRs so the same event or counter is not collected multiple times accidentally.
Data streams should remain understandable. Different sources can produce streams destined for standard or custom tables, and transformations operate within those supported stream paths. Keep naming and documentation clear enough that an operator can trace one record from source through transformation into the final table.
Transformations can support normalization across diverse sources. For example, a custom application can send a richer raw payload while the DCR keeps only required fields and maps names into a stable schema. This can simplify downstream queries and reduce ingestion, but it also makes the DCR a schema boundary that should be versioned and tested.
DCE network design becomes important in private or controlled environments. Configuration access and ingestion can depend on different endpoint components and regional placement. Validate DNS resolution, private endpoints, firewall policy, and agent connectivity from the monitored subnet rather than assuming resource creation means the collection path works.
Cross-region architectures should account for where monitoring data is stored and where agents obtain configuration. Monitoring should remain available enough to diagnose a regional incident. In some designs, a single destination workspace can become a dependency whose outage removes visibility across several workloads.
Access control should separate DCR administration from log-reading where possible. Changing a collection rule can remove security telemetry or increase cost across many resources. Use RBAC, deployment pipelines, and change review appropriate to the blast radius of shared rules.
Data minimization also has privacy value. Do not collect secret-bearing command lines, sensitive payloads, or user data simply because the source exposes it. Transform or exclude fields where they do not serve an operational, security, or compliance purpose.
Migration should include query and alert compatibility. If a DCR transformation changes table or field names, dashboards and Sentinel/Monitor alerts may stop working even while data ingestion remains healthy. Run side-by-side collection or compatibility tests for high-value monitoring paths.
The production goal is a small understandable set of rules aligned with workloads and controls. DCRs should make telemetry collection more consistent and scalable—not become another layer of configuration that nobody can trace when an alert loses data.
DCRs should be organized around stable control objectives rather than individual server names. A Windows security rule, Linux performance rule, or application-specific custom-log rule can be reused and associated with the appropriate resource groups or machines. This reduces configuration drift and makes changes easier to review.
When several DCRs apply to one resource, teams should understand whether streams overlap. Duplicate collection can increase cost and create confusing repeated records. Inventory associations before adding another rule to a shared VM fleet.
Custom log ingestion should validate schema at the boundary. If an application changes field types or names, transformations can fail or data can land incorrectly. Treat the custom table schema and DCR transformation as a versioned contract between the application and monitoring platform.
Finally, keep DCR deployment and rollback automated. Monitoring configuration should move through the same engineering discipline as application infrastructure because a bad collection rule can remove the evidence operators need precisely when a production release fails.
Change review should include downstream consumers such as alerts, dashboards, Sentinel analytics, workbooks, and retention policy. A collection rule that filters one field can break several operational products even though ingestion remains technically healthy.
Use deployment rings for high-impact DCR changes. Associate the candidate rule with a small set of representative resources, compare volume and schema, then expand. Monitoring pipelines deserve canary rollout because their failures can be silent until an incident occurs.
For critical workloads, document the telemetry minimum required to operate safely. This baseline protects against well-intended cost optimization that removes authentication logs, dependency failures, or diagnostic fields needed during response.