Microsoft AI-103: Vector Search Design in Azure
Vector search can retrieve documents that describe a concept without repeating the user’s exact words, but a successful similarity search does not guarantee relevant, current or authorized evidence. The design depends on chunk boundaries, embedding models, vector field configuration, filters and the way candidates reach a reranker or language model. Azure AI Search provides indexing and query capabilities, yet engineering choices determine whether they behave well as a corpus changes. A good vector-search architecture starts with labeled user questions and a defined access boundary, then treats index design, query strategy, evaluation and migrations as one system. Similarity is useful evidence discovery, not factual authority.
On this page
- Define retrieval tasks and unacceptable mistakes
- Choose an embedding space deliberately
- Design fields, algorithms and filters for real queries
- Handle document structure before embedding
- Use hybrid and semantic ranking when justified
- Enforce document rights before generation
- Plan migrations and operations from the start
Define retrieval tasks and unacceptable mistakes
Collect questions users actually ask and identify the passages that answer them. Distinguish exact-identifier needs, broad conceptual searches, multi-part procedures and queries requiring the most recent document version. A support agent that retrieves the wrong firmware revision can give dangerous instructions even when the passage is semantically close. Establish task-level metrics for finding correct authorized evidence and define how the application should behave when no relevant source exists. A trustworthy Microsoft AI-103 vector-search design tests chunking, filters, embeddings and index updates together to avoid retrieving stale or unauthorized evidence.
Include negative cases in the benchmark. Search should not invent a match merely to fill a results page, and an assistant should not treat a weak vector neighbor as proof. Record which documents each test user is permitted to see. Authorization failures can masquerade as high retrieval recall in unrestricted test environments, so permission scope belongs in the evaluation corpus from the beginning.
Choose an embedding space deliberately
An embedding model maps text or other supported data into numerical vectors. The index and query vectors must be generated in a compatible embedding space with the expected dimension and normalization behavior. Changing the model or dimension can require a new vector field or index and reembedding of the corpus. Do not assume a newer or larger embedding model automatically improves business retrieval; compare candidates on the same labeled queries and document set.
Track language coverage, unusual identifiers, cost and latency in the decision. Some technical requests depend on literal strings such as error codes; semantic similarity alone may overlook those differences. Hybrid lexical retrieval can complement vectors. Version embedding models in the ingestion pipeline and query service together so the application does not compare new query vectors with old document representations by accident.
Design fields, algorithms and filters for real queries
An Azure AI Search vector index requires key and vector fields plus a configured vector search profile. Keep human-readable content and source metadata where permitted so results can be interpreted and cited. Vector fields have different supported operations from ordinary filterable or sortable text fields; plan separate fields for tenant, document owner, effective date, product version and source link. An index that stores only vectors cannot support meaningful text citations without another trusted source mapping.
A product-support index may need to answer ‘How do I fix intermittent scanner disconnects?’ and also exact queries for a serial code such as SC-0418. Pure vector similarity can broaden recall for the first wording while underweighting the identifier in the second. Preserve the human-readable text, product family, revision and authorization scope alongside the embedding field, and decide when lexical search or hybrid fusion should participate. The embedding model determines the required vector dimensions; changing it is a schema and migration decision, not simply swapping a deployment name. When source parsing splits an error code away from the troubleshooting steps, neither a larger vector index nor a different similarity metric automatically repairs the lost relationship.
Test queries across current and retired manuals, authorized and unauthorized users, exact identifiers and paraphrases. Record which source passage should be available for each case before tuning approximate nearest-neighbor parameters or semantic ranking. If a technician’s question retrieves a retired bulletin with a higher score, consider source-version filters and freshness logic before treating the issue as insufficient vector capacity. Inspect deletion behavior as well: a document removed from the source repository must no longer appear through its child vectors, caches or related indexes. The correct design is one that returns current permitted evidence predictably as the corpus evolves, not an index optimized for one attractive demonstration query.
Choose supported algorithms and compression according to corpus size, update rate, quality and service constraints. Approximate nearest-neighbor approaches trade speed and memory against exhaustive recall in ways that depend on configuration. Test candidate settings with a known query set, including rare but important questions. A configuration that saves storage but consistently misses a critical policy exception is not a good production optimization.
Handle document structure before embedding
A large document can contain several topics and tables whose meaning depends on headings. Use document chunking methods that preserve logical boundaries and relevant context, then attach parent IDs and source locations. Tiny chunks may match a phrase without the condition that qualifies it; enormous chunks may blur multiple topics in one vector. Compare segmentation strategies on real question-to-evidence labels and report both recall and quality of the evidence passed to generation.
For scanned or layout-heavy documents, extract text and structure first. Keep table headers, units and page references where necessary. An embedding cannot reconstruct missing spatial relationships that were lost during parsing. Preserve source revisions and ingestion lineage so a reviewer can determine whether a wrong answer came from OCR, segmentation, ranking or model synthesis. Data preparation is part of retrieval quality, not an invisible precursor.
Use hybrid and semantic ranking when justified
Lexical search can find exact names and identifiers, while vectors broaden recall for paraphrases and related concepts. Azure AI Search can combine results and use Reciprocal Rank Fusion before optional semantic reranking of text candidates. These stages have different score meanings, so do not apply one universal threshold across cosine similarity, fusion and semantic scores. Evaluate candidate counts, text recall size, reranking configuration and latency separately.
Avoid adding complexity without a demonstrated retrieval gain. On a narrow corpus of exact codes, simple keyword search may outperform a vector-heavy design. On long explanatory documents, hybrid ranking may retrieve more useful passages. The correct setup follows measured question patterns, current source governance and service limits. A higher average similarity score is not itself evidence that users receive better answers.
Enforce document rights before generation
Tenant filters and document permissions must apply to every search branch that can contribute evidence. Review specialized filter overrides, query-time vectorization paths and caching for potential bypasses. A model prompt cannot undo a leak after restricted text has entered the inference context. Use trusted authorization attributes from the source repository and test revocations, group changes and cross-tenant queries. A search result should be both relevant and permitted.
Suppose two divisions share an Azure AI Search index of operating manuals, but only one may read the emergency maintenance appendix. A query about a common device might return both manuals by semantic similarity unless an application-enforced access filter is derived from the authenticated caller. Test the same query as a user from each division, plus a newly revoked user and a service identity with no document access. Each response should include only citations to documents available under that identity. Do not trust a language-model instruction to ‘ignore confidential sources’ after prohibited search results have already reached its context. The filter belongs in the retrieval authorization path.
When a document is reclassified or deleted, track its parent ID and all indexed child chunks. A deletion that removes only the visible parent record can leave vectors or cached answer snippets retrievable. Include those asynchronous cleanup windows in the operational design, and treat results during the transition as a security issue rather than a quality metric. If several organizations share one index, compare the isolation and audit benefits against the overhead of separate indexes before finalizing the architecture. An evaluation score that ignores these permission tests is incomplete even when ranking quality looks excellent.
When source documents are withdrawn, reconcile child chunks and embeddings, invalidate caches as required and confirm that no alternate index still serves the old content. Access metadata must survive indexing and migration rather than be inferred from document titles. Security and lifecycle correctness deserve the same release attention as vector quality because a semantically excellent result can still be unauthorized or obsolete.
Plan migrations and operations from the start
A model upgrade, index schema change or chunking revision is a data migration. Build a versioned index or compatible field configuration, reprocess authoritative documents, synchronize updates and deletions, and compare the candidate with the accepted baseline. Preserve source IDs and citation behavior through migration. Shadow or limited traffic tests should verify query quality, permissions, latency, costs and rollback readiness before switching production routing.
Monitor index health, source freshness, query errors, filter behavior, candidate relevance and the quality of final grounded answers. A result set can be technically successful while missing the necessary passage. Vector search earns a place in Azure AI applications when its measurable retrieval benefits are sustained through permissions, source changes and deployment evolution—not merely because an index returns plausible neighbors.