Agent tracing answers the production question that eventually matters most: why did the agent do that? A final response is not enough when the system retrieves documents, calls models, uses tools, writes memory, or coordinates other agents. Operators need to see the sequence of spans that produced the outcome, how long each step took, which tool arguments were sent, and where an error or latency spike entered the flow. Microsoft Foundry tracing uses OpenTelemetry conventions and stores trace data in connected Azure Monitor Application Insights. For agents hosted in Foundry,…

Tool calling turns an Azure AI agent from a conversational model into a system that can retrieve live information, call APIs, run business operations, or interact with external services. Microsoft Foundry supports custom function tools, OpenAPI tools, MCP servers, built-in tools, and reusable toolboxes. The architecture question is not how many tools an agent can reach; it is how to make each tool understandable, permissioned, testable, and safe to invoke. Current Foundry function calling follows a clear loop: define a function schema, let the model request a call, execute the…

Threat modeling an Azure AI application starts by accepting that the attack surface is broader than the model endpoint. User prompts, retrieved documents, memory, tools, agent identities, model outputs, logs, and external services all create trust boundaries. A system can have a secure model deployment and still be vulnerable because untrusted content becomes an instruction, an agent holds excessive permissions, or a tool executes model-generated arguments without validation. Microsoft’s current security guidance for agents recommends mapping data flows, trust boundaries, and control points before implementation. Its Catalog of AI Attack…

Prompt testing should move beyond a few playground examples as soon as a prompt affects production behavior. A prompt can change output quality, refusal behavior, tool selection, latency, token cost, and the way retrieved evidence is interpreted. Structured testing gives the team a repeatable way to decide whether a change is an improvement rather than relying on the most recent manual examples. Microsoft Foundry supports model and agent evaluation using existing datasets, synthetic data, traces, and—in preview for some scenarios—full conversation simulation. Foundry hosted-agent guidance also separates unit tests, local…

Synthetic data is useful when a team needs broader test coverage than it can curate manually, especially before production traffic exists. Microsoft Foundry can generate synthetic evaluation queries, send them to a model or agent, score the responses, and save the generated queries as a reusable dataset. The current SDK workflow for this capability is preview, which matters when deciding whether it belongs in a production quality gate. Synthetic data is not a replacement for real data. It is a coverage tool. The generator can repeat its own assumptions, miss…

Serverless is useful for AI when the application needs event-driven execution, burst handling, lightweight APIs, scheduled jobs, or durable orchestration without managing a permanently running server. Azure Functions provides serverless compute for bounded handlers, while Durable Functions and the broader Durable Task stack add state persistence, retries, coordination, and long-running workflows. It is important to distinguish serverless compute from serverless model APIs. Microsoft Foundry model deployments can be consumed through managed APIs, while Azure Functions is the application compute that handles events or exposes business endpoints. One does not replace…

Securing an Azure AI endpoint requires several controls working together: authentication, authorization, network exposure, rate limits, request validation, monitoring, and deployment governance. An endpoint is not secure merely because it uses HTTPS or sits behind a private network. The caller still needs a trustworthy identity and only the permissions required for the operation. Azure OpenAI and Microsoft Foundry support Microsoft Entra authentication, and current Microsoft guidance includes keyless patterns using Azure Identity libraries and managed identities. Private endpoints can remove public network exposure, while RBAC controls who can invoke or…

Secrets management for AI applications starts with a simple priority: do not create a secret when identity-based authentication can solve the same problem. Managed identities and Microsoft Entra ID can remove API keys and client secrets from many Azure-to-Azure connections. When a workload still needs a secret—for a third-party API, legacy service, or connection that cannot use Entra ID—Azure Key Vault provides controlled storage, access, and rotation. Microsoft Foundry also supports Key Vault-backed connections for scenarios that require stored connection secrets. Current documentation notes important operational limits around bring-your-own Key…

A responsible AI review should happen while architecture can still change, not after a system is already politically or operationally difficult to stop. Microsoft frames responsible AI around six principles: fairness, reliability and safety, privacy and security, inclusiveness, transparency, and accountability. A useful review turns those principles into concrete questions about the application’s users, data, actions, failure modes, and controls. Current Microsoft guidance for agents treats responsible AI as a release gate scaled to risk. Foundry also provides risk and safety evaluations for categories such as hateful and unfair content,…

Reranking improves RAG when the retriever finds the right documents but does not place the best evidence high enough in the result set. Azure AI Search semantic ranker is an L2 reranking stage: it takes an initial candidate set produced by text or hybrid retrieval and applies deeper language understanding to move the most semantically relevant results toward the top. That distinction matters. Reranking cannot recover a document that the first-stage search never returned. It is strongest when recall is already reasonable and the problem is ordering. This makes reranking…

Reproducibility means a team can explain and recreate how a machine learning artifact was produced. Azure Machine Learning pipelines help by breaking a workflow into versioned components with explicit inputs and outputs, while environments, data assets, registries, and job metadata preserve the dependencies around those components. Reproducibility is not identical output from every stochastic training run; it is the ability to reconstruct the conditions and evidence behind the result. Current Azure Machine Learning guidance uses SDK and CLI v2 as the active path for pipeline development. Components are self-contained steps…

REST is the lowest common denominator for Azure AI integration. It is useful when a language SDK does not expose a new capability yet, when a platform team needs one consistent integration surface across languages, or when an application already uses OpenAI-compatible HTTP tooling. The cost of that flexibility is that the application owns more details: authentication, headers, retries, streaming, versioning, pagination, and error interpretation. Microsoft Foundry exposes project-scoped REST endpoints for agents and an OpenAI-compatible Responses API. The project endpoint adds Foundry-specific platform capabilities, while Azure OpenAI endpoints remain…

Python is often the shortest path from an Azure AI prototype to maintainable application code, but SDK choice and client structure matter once the project grows. Microsoft Foundry exposes project-scoped APIs, model inference, agents, evaluations, and platform tools through several Python surfaces. The most useful pattern is to choose one abstraction deliberately instead of mixing portal-generated snippets, low-level HTTP calls, and several SDK generations in the same application. Current Microsoft documentation identifies azure-ai-projects as the Python package for Foundry project operations and supports OpenAI-compatible access through the Foundry project endpoint….

RAG systems can be attacked through the data they retrieve. An attacker does not always need to break the model or compromise the application directly; they may only need to place malicious, misleading, stale, or adversarial content into a source that the retriever trusts. If that content ranks highly, the model can treat it as grounding evidence or even as an instruction. This threat includes classic data poisoning, indirect prompt injection, manipulated documents, untrusted external sources, and compromised ingestion pipelines. Azure AI Content Safety Prompt Shields can detect document attacks,…

Prompt versioning is necessary because prompt text is production behavior. A change to system instructions, examples, tool guidance, or retrieval framing can alter quality, safety, latency, cost, and tool use without any application-code change. Teams that edit prompts directly in a portal without preserving history lose the ability to explain why behavior changed or to restore the last known-good configuration. Current Microsoft Foundry provides immutable versions for prompt agents: after a saved version is created, later edits are saved as a new version. Foundry’s code-first path also allows agent definitions…