Practice Exams:

Microsoft AI-103: Choosing Embeddings for Azure Search

AI & Machine Learning

Embedding models are often compared by vector dimension or public benchmark score, but those figures do not tell an enterprise search team whether its users will find the right material. A legal assistant must distinguish similar clauses, a support system must recognize technical shorthand, and a multilingual knowledge base must handle queries that use different vocabulary from its documents. Embedding selection affects all of those outcomes and changes how an Azure AI Search index is built and maintained. The best design begins with representative retrieval questions, compatible index settings, and an explicit migration plan. Model size is one input, not the decision itself.

On this page
  1. Define the semantic behavior you actually need
  2. Compare models on the same documents and queries
  3. Treat vector dimension as a schema decision
  4. Understand indexing and query trade-offs
  5. Use hybrid search for exact and conceptual needs
  6. Plan embedding migrations rather than replacing live fields
  7. Monitor drift in source content and questions
  8. Make the decision reproducible

Define the semantic behavior you actually need

Start by collecting question-and-document pairs from real tasks. For support content, include error codes, abbreviations, product versions, and paraphrased symptoms. For policies, include related clauses that differ by one condition. For multilingual search, include queries and sources in the supported languages and dialects. Label passages that truly answer each question; a semantically similar document is not necessarily the correct one. This benchmark lets engineers compare embeddings against observable retrieval quality rather than choosing a vector model because its marketing description seems broad. A Microsoft AI-103 embedding decision should be tested on the questions and documents the application actually needs to retrieve.

Also document the role of embeddings in the full search pipeline. They may support candidate retrieval, clustering, deduplication, or recommendation, and those tasks have different success criteria. A model suitable for broad thematic similarity may be weak at distinguishing nearly identical product identifiers. If exact terms are important, plan lexical retrieval alongside vectors rather than expecting embeddings to preserve every character-level relationship. The task contract should state which failures are unacceptable and how they will be detected.

Compare models on the same documents and queries

Evaluate candidate embedding models using identical source documents, chunk boundaries, query sets, and ranking settings where feasible. Keep a held-out test set so tuning one model does not accidentally reward memorized queries. Measure recall at useful candidate counts, ranking quality, multilingual behavior, and failure cases. Inspect false positives: a retrieved passage might mention the same broad technology while being from the wrong product version. Such errors can appear harmless in a similarity score but mislead a downstream RAG answer.

Do not trust one aggregate number to represent every business area. Report by document type, language, query length, and presence of exact identifiers. A model may improve broad natural-language search while hurting code-like technical strings. Use qualitative case review to understand these differences and decide whether hybrid retrieval can compensate. If a candidate requires heavier processing or more storage, include that operational cost in the comparison. Retrieval quality is useful only when it can be maintained at the required scale.

Treat vector dimension as a schema decision

The embedding output must match the Azure AI Search vector field‘s expected dimensions and related index configuration. Changing an embedding model can require rebuilding or creating new vector fields because existing vectors cannot simply be relabeled as if they were generated by the new model. Record the model name, version, dimension, normalization assumptions, and any configurable dimension setting in the ingestion pipeline’s release manifest. Generate document and query vectors using compatible settings; mixing generations can make distance comparisons meaningless.

Larger vectors increase storage and can affect indexing or query resource consumption, but higher dimension does not guarantee better answers. A smaller embedding may perform better on a narrow domain or reduce operational cost sufficiently to permit more useful candidate retrieval. Test dimensionality choices empirically. If an index has several vector fields for different modalities or models, define which field each query targets and why. Avoid sending a query embedding to a field whose vectors represent another model space.

Understand indexing and query trade-offs

Approximate nearest-neighbor search trades some exhaustiveness for speed and scale. Azure AI Search supports approaches such as HNSW and exhaustive search, subject to index configuration and current service capabilities. Use exhaustive comparisons on a manageable benchmark to estimate what approximate retrieval misses. Tune candidate counts and search parameters against observed recall and latency. A configuration that is perfect for a small development index may change as the production collection grows or document distributions shift.

Filters interact with vector retrieval. If an application restricts results to a particular tenant, product family, date range, or permission set, test retrieval under those filters, not only against the whole corpus. An unauthorized document should never become model context merely because its vector is close to the query. Security and business filters can reduce the available candidate pool; this may require adjusting query behavior or index design. Track filtered recall separately so engineers do not misdiagnose an authorization boundary as a weak embedding model.

Use hybrid search for exact and conceptual needs

Vector similarity is useful for paraphrases and related concepts, but business queries often include exact identifiers that lexical search handles well. Azure AI Search hybrid queries combine full-text and vector candidates, using Reciprocal Rank Fusion for supported retrieval configurations. Semantic ranking may then reorder candidates based on text fields. Embeddings should therefore be chosen and tuned within the entire pipeline. Evaluate whether a model improves candidate recall before concluding that the final answer improves, and keep search ranking and generation behavior separable in traces.

A practical diagnostic case is a customer asking about a specific error code with a descriptive symptom. Keyword search may find the code directly; embeddings may find a resolution guide that describes the symptom without the exact code. Hybrid retrieval can give the generator both pieces of evidence. If the answer still cites an outdated document, investigate source freshness and effective-date filtering. A larger embedding vector will not solve stale content. Retrieval quality emerges from model choice, metadata, ranking, and source management together.

Plan embedding migrations rather than replacing live fields

Changing embeddings across a large index is a data migration. Decide whether to create a new index, a new vector field, or another versioned configuration supported by the application. Recompute document vectors from authoritative source text, not from previously embedded vectors. Keep source IDs and versions stable so results can be compared between old and new retrieval stacks. Synchronize updates and deletions during the migration window. A production switch should not silently combine new query vectors with old document vectors.

Use a fixed regression corpus and limited live or shadow traffic to compare the two index versions. Include permissions and tenant filters in both paths. Record cost, indexing duration, storage growth, query latency, and retrieval gains. Keep the previous index available for rollback until the new configuration has passed acceptance criteria and caught up with source updates. Avoid assuming an alias or route switch alone is reversible if the upstream application schema or permissions were changed incompatibly.

Monitor drift in source content and questions

Even a well-selected embedding model can become less useful as documents and user queries evolve. New product terminology, policy changes, or an expanded language audience may change the distribution of relevant material. Monitor retrieval success and complaint patterns by topic, not just global query rate. Use fresh labeled questions from production incidents to extend the evaluation set. Reevaluate the current embedding choice when the corpus changes materially, but avoid unnecessary migrations triggered solely by the appearance of a newer model.

Watch source ingestion freshness and authorization metadata alongside ranking outcomes. A failure to retrieve new documentation could be an indexing pipeline issue rather than embedding degradation. An increase in long, multi-part questions may demand a different query decomposition strategy. Trace the full path from user question to candidate passages and final answer so a team can diagnose the right subsystem. Model replacement should be one possible remedy among several, chosen only when evidence points to the embedding space itself.

Make the decision reproducible

A useful embedding evaluation report records the source corpus version, question dataset, candidate models, vector settings, search configuration, permission filters, and relevant metrics. Include representative errors and intentionally rejected models. Explain how dimension and cost influenced the outcome without pretending that either determines answer quality alone. Future engineers should be able to reproduce the same test and understand why the model was selected under the then-current constraints.

For Azure AI applications, embeddings are a foundational retrieval component but not a substitute for good document design and governed search. Choose with the questions users actually ask, the evidence they are entitled to see, and the scale the system must support. Then version the choice and reevaluate it when real conditions change. That approach produces a reliable search architecture rather than an index built around an impressive but poorly tested vector benchmark.

Continue learning

Related guides

Azure AI Search for RAGDesign Azure AI Search indexes, retrieval filters, and grounding paths.Vector, Hybrid, and Semantic Search for Better GroundingChoose keyword, vector, hybrid, and reranking strategies for RAG.Blue-Green Releases for AI EndpointsCompare model versions safely before switching all production traffic.Choosing Azure AI Deployment ModelsCompare Azure AI deployment choices against scale, isolation, and cost.

Related Posts

• The Ultimate Overview of Microsoft AI-900: What You Should Know

• How to Use Microsoft Copilot: Your Guide to Boosting Productivity with AI

• Microsoft AI-300: Reproducibility Is the First Test of Production ML

• Microsoft AI-300: From Experiment Tracking to an Auditable ML Lifecycle

• Microsoft AI-300: How to Roll Back a Model Without Rolling Back the App

• Microsoft AI-103: Safe Tool-Using Agents

• Microsoft AI-103: Hybrid Search and Reranking

• Microsoft AB-900: The Microsoft 365 Admin Role Now Includes AI

• Microsoft AI-901: Computer Vision, Language, and Generative AI

• Microsoft AI-103: Identity Boundaries for Foundry Agents