Microsoft DP-600: KQL Databases in Fabric
KQL databases are the core query and storage unit inside Microsoft Fabric Eventhouse for high-volume, time-oriented, and event-driven data. An Eventhouse can contain multiple KQL databases that share capacity and management, while each database contains tables, functions, materialized views, policies, and related real-time assets that can be queried with Kusto Query Language.
Current Fabric Real-Time Intelligence guidance places KQL databases alongside eventstreams, Eventhouse, Real-Time Dashboards, Activator, and other streaming capabilities. The design is useful when data arrives continuously and teams need fast exploration, filtering, aggregation, correlation, and operational analysis without waiting for a batch pipeline.
KQL database design belongs inside Microsoft Data Platform Engineering.
Use KQL databases for event-shaped data
Logs, telemetry, clickstreams, IoT events, application events, and operational records often arrive with timestamps and append-heavy patterns.
Real-Time Intelligence changes the engineering contract by making ingestion delay and query responsiveness part of the product expectation.
KQL is optimized for exploring and aggregating this kind of data at scale.
Use Eventhouse as the operating container
An Eventhouse can manage multiple KQL databases together and provides unified monitoring and resource management.
Use separate databases when data ownership, retention, access, schema, or workload isolation justify a boundary.
Do not split every table into its own database; the database should reflect a meaningful operational and governance scope.
Design tables around query patterns
Good KQL table design starts with event grain, timestamp, high-value dimensions, and fields users filter or aggregate frequently.
KQL for SQL users is easier to adopt when table names and columns express familiar business meaning instead of raw ingestion payloads.
Nested dynamic fields are useful, but important dimensions can be easier to query when promoted into clear columns.
Use materialized views selectively
Materialized views can precompute aggregations or transformed projections that are queried repeatedly.
They trade additional maintenance work for faster repeated queries.
Use them for stable high-value patterns, not as a substitute for understanding whether a raw query can be improved first.
Set retention according to operational value
KQL databases support data-retention policies, and retention should reflect how long detailed event data remains useful.
Long retention can increase cost and governance burden, while very short retention can remove evidence needed for investigation or trend analysis.
Use Lakehouse or other storage when long-term historical retention has a different cost and query profile.
Ingest through the right path
Eventstreams, connectors, APIs, pipelines, and custom producers can feed KQL databases depending on the source and timing requirements.
Fabric eventstreams are useful when the team needs routing and transformation before the data lands in Eventhouse.
Batch ingestion can remain appropriate for sources that do not require continuous delivery.
Use OneLake integration where it fits
KQL databases can work with OneLake features, including supported shortcuts and visibility into data.
This helps connect real-time analysis with the broader Fabric data estate without forcing every consumer into one storage engine.
OneLake shortcuts should still preserve source ownership and permission boundaries when event data is reused elsewhere.
Monitor ingestion and query health
Operators should track ingestion rate, failures, query latency, retention, hot data, resource use, and downstream consumers.
Streaming telemetry is most useful when teams can distinguish source outage, ingestion lag, schema errors, and query pressure.
A real-time platform can be unavailable in practice even when the database itself reports as healthy if the data is arriving late.
Design for operational questions
KQL databases should answer the questions the business needs in minutes or seconds: what changed, where is the anomaly, which customer or device is affected, and what pattern preceded the event?
For engineers working toward DP-700, the durable pattern is to use KQL databases for time-oriented event data, model tables for the questions operators ask, set retention deliberately, integrate with eventstreams and OneLake where useful, and keep ingestion and query health observable.
As the Eventhouse estate grows, database boundaries, ownership, retention, and query conventions should be standardized enough that real-time data remains reusable rather than becoming another collection of isolated telemetry silos.
Eventhouse capacity design should reflect ingestion and query concurrency. A database that receives constant telemetry while analysts run heavy aggregations can create different pressure from a mostly read-only event store. Monitor ingest and query workloads together before deciding whether several databases should share one Eventhouse.
Retention policy can be tiered conceptually even when the data lives across different Fabric stores. Keep high-value recent event data in KQL for fast investigation, while older history can be retained in Lakehouse or another economical store if the organization still needs long-term analysis. The right split depends on response-time requirements and regulatory retention.
Update policies and materialized views can move repeated transformation closer to ingestion. This can simplify consumer queries, but it also turns the database into an active processing layer. Document those transformations and their owners so analysts know whether a field came from the producer or from a database-side policy.
KQL functions can standardize recurring logic such as entity normalization, severity mapping, or reusable filters. Keep business-critical functions versioned and documented; otherwise important meaning becomes hidden inside ad hoc query text that only one analyst understands.
Access control should be designed per database and workload. Security teams, operations teams, and business analysts may need different permissions even when they consume the same event source. Avoid granting broad write access merely because KQL makes exploration easy.
Real-time dashboards should be considered consumers of a data product, not the data model itself. A good KQL schema can support dashboards, alerting, notebooks, APIs, and investigations simultaneously. If table design exists only to satisfy one visual, reuse becomes difficult.
Operational runbooks should cover source interruption, ingestion failure, schema mismatch, retention changes, and query saturation. Real-time systems create value only if teams can distinguish “no event happened” from “the pipeline stopped receiving events.”
For high-volume environments, establish naming and folder conventions for tables, functions, materialized views, and querysets. KQL’s flexibility can produce a large estate quickly, so consistency reduces analyst confusion and helps automation discover the correct objects.
The mature KQL database is therefore a governed event data product: clear grain, known producer, intentional retention, reusable query logic, measurable ingestion health, controlled access, and a place in the wider OneLake and Real-Time Intelligence architecture.
Query conventions help large teams share KQL. Reusable functions, naming standards, saved querysets, and comments can turn one analyst’s investigation into a supportable operational capability rather than a one-off query copied through chat.
Hot and cold analytical needs can also be separated. Keep recent high-value telemetry optimized for fast KQL exploration, while older data can move to lower-cost storage when the business mostly uses it for occasional historical analysis.
Materialized views and update policies should be monitored after schema changes. A source table can continue ingesting while a dependent transformation begins failing, leaving users with incomplete derived data even though the database itself appears healthy.
For incident-response scenarios, preserve timestamps and identifiers precisely enough to correlate KQL data with Azure Monitor, application logs, and other systems. Real-time data creates the most value when it can be joined into a trustworthy timeline.
Data ingestion should preserve event time separately from ingestion time where late-arriving events matter. Operational analysis often needs to know when the business event occurred and when the platform received it. Mixing those concepts can distort windows and incident timelines.
Schema changes should be tested against dashboards, materialized views, functions, and downstream consumers. Streaming producers can evolve independently, so compatibility discipline is as important for KQL databases as it is for message brokers.
Use query limits and optimization where shared interactive workloads can affect others. One expensive exploratory query should not make operational dashboards unusable during an incident.
KQL databases can also support SQL analytical endpoints and notebooks in supported scenarios, which helps teams bring event data into broader analytical workflows without moving every dataset first.
For large estates, review whether database boundaries still match ownership and retention. Consolidation can improve reuse, while separation can improve isolation. The right answer depends on actual workload behavior and governance, not one universal database-per-team rule.
For high-value operational queries, keep a small validated query library so responders can start from trusted patterns during incidents instead of assembling complex KQL under pressure.
Keep those trusted query patterns versioned and owned.
Review query and retention standards regularly.
Keep ingestion, query, and retention operations observable.
Keep the incident query library tested against real schemas so responders do not discover syntax, permissions, or renamed-table problems during a live outage.
Keep schema ownership explicit and reviewed.
Review retention again as event volume and investigation needs grow.