Microsoft DP-600: Vector Search in Azure SQL

Azure SQL Database now supports native vector data, vector functions, and vector indexing for AI application patterns. As of the current 2026 platform state, vector indexes and VECTOR_SEARCH are generally available in Azure SQL Database and SQL database in Microsoft Fabric. The latest vector-index implementation uses DiskANN and supports full DML, iterative filtering, optimizer-driven plan choice, and approximate nearest-neighbor search with the newer SELECT TOP (N) WITH APPROXIMATE syntax.

This matters because teams can keep embeddings beside relational business data and use one engine for semantic similarity plus transactional filters, joins, permissions, and current state. Vector search becomes a database capability rather than a separate service requirement for every RAG or semantic application.

Vector search therefore sits naturally inside Microsoft Data Platform Engineering.

Store embeddings in the vector type

Azure SQL provides a native fixed-dimension vector data type that stores vector values efficiently while exposing familiar JSON-style representation to clients.

Database embeddings should remain tied to source text, model version, dimensions, and business identifiers so they can be rebuilt and audited.

Do not mix incompatible embedding models in one search space.

Use exact distance for small or diagnostic workloads

VECTOR_DISTANCE can calculate distance directly and is useful for exact search, benchmarks, and smaller datasets.

Exact search provides a useful quality baseline because it examines the candidate set without approximation.

Retrieval evaluation should compare important approximate-search changes against a stable known-relevance baseline.

Use DiskANN indexes for scale

Azure SQL vector indexes use DiskANN for approximate nearest-neighbor search.

The index is designed to balance memory, storage, CPU, and query performance over larger vector sets.

Vector design still needs representative benchmarking because index speed is valuable only when relevant records remain near the top.

Use the current approximate-search syntax

The latest vector-index generation uses SELECT TOP (N) WITH APPROXIMATE together with VECTOR_SEARCH.

The older TOP_N parameter is maintained for backward compatibility with earlier vector-index versions and is deprecated for new implementations.

Version-aware SQL matters because copy-pasting an older example can produce errors against a current index.

Combine vector search with relational filters

Modern Azure SQL vector search can apply predicates during approximate search, which makes hybrid business queries more practical.

Hybrid retrieval should use ordinary columns for tenant, status, date, category, or permission-related constraints and vector similarity for semantic ranking.

Hard business rules belong in structured predicates, not inside the embedding space.

Keep write-heavy workloads in scope

The latest vector indexes support INSERT, UPDATE, DELETE, and MERGE while maintaining the index.

This improves fit for applications where source data changes continuously rather than being loaded once as a static knowledge base.

Measure index maintenance cost on production-like write patterns before assuming a design optimized for read-heavy RAG will fit an operational workload unchanged.

Use the database for current business state

A semantic result can be joined immediately to current inventory, entitlement, customer, or transaction data stored in Azure SQL.

AI database design is strongest when semantic search extends the system of record instead of creating a disconnected copy that becomes stale.

Re-read authoritative state before high-impact actions even if the vector match came from the same row earlier in the conversation.

Plan index migration

Microsoft documents migration from earlier vector-index versions to the latest DiskANN format.

Keep index version visible, test the current syntax, and compare retrieval quality before removing the previous path.

Parallel rebuild and controlled cutover are safer than changing a production search surface without a benchmark.

Keep vector search as one query capability

Azure SQL remains a relational database first. Use vector search where semantic similarity improves the application, and keep ordinary indexes, constraints, Query Store, access controls, and transaction design intact.

For Azure and Fabric teams, the durable pattern is to store vectors with relational context, index them for scale, combine approximate search with business predicates, benchmark quality, and manage vector lifecycle like any other derived production data.

The attraction of Azure SQL vector search is not that every application should become a RAG system. It is that applications that already depend on relational truth can add semantic access without abandoning the database controls they already trust.

Application teams should decide when exact and approximate search are appropriate. Exact distance is useful for validation and small datasets, while DiskANN approximate search is designed for scalable low-latency retrieval. Keeping an exact benchmark gives engineers a way to quantify recall loss when index or filter settings change.

Vector indexes now support full DML in the current Azure SQL implementation, but write-heavy applications should still measure index-maintenance overhead. Frequent updates to embeddings can change transaction cost and storage behavior, especially when the same table also serves latency-sensitive relational transactions.

Iterative filtering is important because enterprise vector search often includes predicates. A nearest-neighbor query may need to return only records for one tenant, active product set, or permitted classification. Applying filters during search can improve the usefulness of approximate results compared with retrieving a broad set and discarding ineligible rows afterward.

The optimizer can choose between DiskANN and exact search according to the query and index state. Teams should use Query Store and execution evidence rather than assuming the approximate index is always selected or always faster. Query behavior can vary with data size, predicate selectivity, and workload.

Vector score should not be treated as confidence. Distance indicates similarity under one embedding and metric, not correctness, authority, or business eligibility. The application still needs source metadata, structured filters, and possibly reranking before the result becomes model context.

Index rebuilds should be part of migration planning when embedding dimensions or models change. A parallel table, column, or index can let the team compare a new representation without corrupting the current search path. Cut over only after quality and latency both meet the target.

Security remains relational. Use users, roles, row-level patterns, application APIs, and stored procedures to enforce access; do not rely on the vector index to hide restricted records. Semantic similarity is a ranking feature, not an authorization model.

Operational telemetry should include query latency, approximate-search usage, index size, DML performance, failed embedding generation, and stale-vector backlog. A vector feature can appear healthy to users until the derived data silently stops refreshing.

Azure SQL vector search is most compelling when the application already depends on transactional business state. In that situation, semantic retrieval can join directly to current truth, use ordinary predicates, and remain inside familiar security and operational tooling instead of creating another datastore that must be synchronized.

Embedding generation can be performed outside the database or through supported SQL AI integration patterns. Whichever approach is used, keep model credentials and network access out of ad hoc user queries and put generation behind a governed service or database workflow.

Filtering strategy should be benchmarked. Highly selective predicates can change whether exact or approximate search is optimal, and the optimizer may choose differently as data grows. Query Store and execution plans remain valuable for vector workloads.

Applications should preserve source provenance in results. Return the business key, source text or summary, update timestamp, and distance alongside the match so downstream systems can explain what was retrieved and verify whether the record is current.

As vector search becomes generally available in Azure SQL, avoid treating it as a reason to collapse every search workload into one database. Use it where relational and semantic access belong together; use specialized search platforms when full-text ranking, document enrichment, or other search capabilities dominate the requirement.

Vector-index build and maintenance should be scheduled with database workload in mind. Large index creation can consume resources, and production teams should understand how build parallelism, table size, and concurrent transactions affect the maintenance window.

Use traditional indexes alongside vector indexes when relational filters are selective. Azure SQL’s optimizer can combine structured access paths and vector search, so good relational indexing remains valuable for hybrid queries.

Embedding refresh should avoid partial-model mixtures. If the model version changes, track which rows have been re-embedded and keep the application from comparing incompatible vectors during a long migration.

Application-level reranking can still add value after SQL vector retrieval. Business authority, recency, user preferences, or model-based ranking can refine the candidate set, provided the extra latency is justified.

Finally, vector search should have a clear success metric such as recall at k, answer groundedness, case-match usefulness, or recommendation quality. Database feature adoption is worthwhile when semantic retrieval improves the product, not simply because the engine supports a new index type.

Keep retrieval evidence versioned.