Practice Exams:

Microsoft AI-103: Durable AI Jobs with Queues

AI & Machine Learning

A document-processing assistant may take seconds to start and minutes to finish, especially when it extracts data, retrieves references, calls a model, and waits for a reviewer. Keeping the original HTTP request open during that chain creates fragile timeouts and encourages duplicate work when clients retry. A durable queue can separate accepting the job from performing it, but only if the workflow records ownership, retries safely, and gives users a reliable status. Azure Service Bus and Durable Functions provide useful building blocks for such systems. The difficult design question is not how to enqueue an AI task; it is how to preserve correctness through failures and replays.

On this page
  1. Define the job contract before choosing a queue
  2. Choose between a message, an event, and an orchestration
  3. Make delivery at least once safe
  4. Separate transient from permanent failures
  5. Control concurrency and expensive model work
  6. Keep human approvals durable and time-bound
  7. Observe the job from acceptance to final outcome

Define the job contract before choosing a queue

A useful job begins with a stable identifier, caller or tenant identity, authorized purpose, input references, expected output shape, priority, and a realistic completion window. The API should validate scope and credentials before accepting work. Store only the minimum data needed in the message; large files and sensitive documents can remain in protected storage, referenced through short-lived or service-authorized identifiers. Do not place permanent secrets in a queue payload, where they can be copied into telemetry or dead-letter stores later. Durable queues in Microsoft AI-103 do not solve reliability by themselves; retries, idempotency and failed-job handling determine whether work survives interruptions.

Return a durable job ID and a truthful accepted or queued status. Clients can poll, subscribe to notifications, or query an application status endpoint, but those views must reflect committed storage state. If an enqueue operation might have succeeded despite a network timeout, the caller needs an idempotency key and a way to discover the previously accepted job. Generating a new job each retry is a direct path to duplicate charges and possibly duplicate external actions.

Choose between a message, an event, and an orchestration

A queue represents work that one consumer or consumer group should process. An event typically announces that something happened, potentially to many independent listeners. An orchestrator coordinates a known sequence of steps with durable state and retry semantics. These patterns can coexist. An uploaded contract event can trigger creation of one extraction job; a durable workflow can then perform classification, selected document analysis, policy checks, and a human review checkpoint before publishing structured output.

Do not create a complex orchestrator when a single short task with an idempotent output suffices. Equally, avoid passing dozens of ad hoc messages between components when the product requires clear step ordering and auditable compensation. Choose the simplest arrangement that records the business process and its failure behavior. A queued AI task might be resumable without repeating the model inference if its intermediate result is checkpointed under the same input and model version.

Make delivery at least once safe

Most practical queue consumers must tolerate redelivery. A worker can complete a database write but lose the acknowledgment, crash while finalizing a message, or exceed its lock period. Azure Service Bus peek-lock allows a worker to receive and explicitly settle a message; redelivery occurs when the lock is lost or the operation is not completed. Design the downstream operation to be idempotent using a business transaction ID, stable output key, and a record of completed stages.

Suppose an accounts-payable application accepts a batch of invoices and queues document extraction. A submission produces a durable job ID, an upload checksum and the identity of the requesting organization. The worker may reserve a message, call an extraction service and then lose network connectivity before acknowledging completion. The queue can deliver that message again; therefore the downstream write should check whether that invoice checksum and job version already produced an accepted result. Use a state transition such as accepted → processing → validated → completed and make the final result refer to its original input. Counting attempts is useful for diagnosis, but the business outcome should be counted once, not every time a worker resumes.

A malformed invoice and a temporarily throttled extraction endpoint need different recovery paths. The temporary failure can receive bounded retries with backoff and a maximum age, while a permanently invalid file should move to a reviewable failure state with an explanation and source link. Keep poison or dead-letter messages available to authorized operators, but redact customer documents and access tokens from diagnostic messages. A human reviewer should be able to resume a corrected job without creating a second invoice payment. A queue provides resilience only when application state, external side effects and operator remediation all agree on what completion means.

Duplicate detection on sends can reduce repeated enqueue operations within a configured window for supported Service Bus tiers, but it is not a substitute for idempotent receivers. A charge, ticket creation, or policy update must check the business state before repeating. If an inference result is generated twice, identify which version becomes authoritative; if a tool action is repeated twice, the effect may be irreversible. Test both send-acknowledgment loss and worker-crash scenarios rather than assuming a successful queue SDK call establishes exactly-once business behavior.

Separate transient from permanent failures

Temporary model throttling, rate limits, and service outages can justify retries with exponential backoff and jitter. Malformed input, unauthorized access, a missing required document, or an unsupported business action usually requires failure reporting or a human decision instead. Retrying every exception indefinitely wastes tokens and hides defective data. Cap attempts by step and cost budget, then move a repeatedly failing message to a monitored dead-letter workflow with a documented operator response.

A dead-letter queue is not a disposal bin. Store enough sanitized context to explain why the job failed, who owns triage, how corrections are authorized, and whether replay is safe. For some workflows, a human might fix a source document and resubmit a new version; for others, the original result must remain immutable for audit. Expose meaningful status to the user without revealing implementation stack traces or sensitive data embedded in tool requests.

Control concurrency and expensive model work

Queue depth absorbs demand bursts, but unlimited consumers can overwhelm model quotas, retrieval services, and downstream databases. Set concurrency based on measured end-to-end throughput and the permitted budget for retries. A model call may have low average latency yet high tail latency under long prompts; a worker may also fan out to multiple tool calls. Use separate controls for job arrivals, active inference calls, and side-effecting operations so scaling one stage does not silently exceed another service’s limits.

Priority and tenant fairness matter. A large batch from one team should not starve urgent tasks from another if the product promises independent service levels. Separate queues or partitioning strategies may be useful, but introduce monitoring and operational cost. Use backpressure to avoid accepting work that cannot be completed within the advertised window, and inform the caller when the queue is full. A durable queue protects task state, not the application’s promise to process unbounded demand.

Keep human approvals durable and time-bound

Many AI tasks have a step that cannot proceed safely on model judgment alone. The system may need a reviewer to approve an invoice field, confirm a deployment change, or authorize an email to a customer. Persist the pending decision with its context, approver identity, expiry time, and permitted options. An approval arriving after the request has been canceled or superseded should not resurrect obsolete work. Timeouts and escalations are workflow states, not reasons to call the model repeatedly hoping it will authorize itself.

Version the review artifact as carefully as model results. When an input changes while waiting for a human, invalidate or explicitly revisit earlier approval instead of applying it to new content. Where a workflow crosses transactional boundaries, define compensation for partially completed side effects and document which operations cannot be undone. A durable orchestrator can retain progress, but the business system still needs a policy for approval conflicts and manual recovery.

Observe the job from acceptance to final outcome

Correlate the incoming request, queued message, orchestration instance, model invocation, search queries, tool calls, reviewer decisions, and result publication under stable trace identifiers. Record timing for each stage and the number of attempts, not only the overall duration. Useful metrics include accepted versus completed jobs, age of the oldest queued job, failure categories, dead-letter backlog, cost per correct completion, and cases that needed human correction.

A job may be marked technically completed while delivering a wrong extraction or missing source citation. Put domain-level validation before final publication and retain evidence for the user or operator. Reconciliation jobs should detect stuck or orphaned states and decide whether safe replay is possible. Durable AI workflows succeed when the system can truthfully explain where work is, what went wrong, and why a user will not receive duplicate or unauthorized outcomes after a failure.

Continue learning

Related guides

Rate Limits, Cost, and Scaling Azure AI ApplicationsPlan Azure AI retries, rate limits, cost budgets, and scaling.Building Multi-Agent Workflows on AzureCoordinate agent handoffs, shared state, permissions, and failure recovery.Capacity Planning for Azure AIEstimate throughput, throttling, concurrency, and provisioned capacity needs.Event-Driven AI Workflows on AzureTrigger AI processing from events while controlling retries and ordering.

Related Posts

• Mastering the Azure AI Engineer (AI-102) Exam

• Mastering the Basics: Your Guide to Microsoft Azure AI

• Microsoft AI-300: How to Roll Back a Model Without Rolling Back the App

• Microsoft AI-300: Machine Learning CI/CD Needs More Than a Build Pipeline

• Microsoft AI-103: Managed Identities for AI Apps

• Microsoft AI-901: Computer Vision, Language, and Generative AI

• Microsoft AI-900: Skills That Still Matter in the Current AI Path

• Microsoft AI-103: Identity Boundaries for Foundry Agents

• Microsoft AI-103: Choosing Safety Controls for Azure AI

• Microsoft AI-103: Controlling the Cost of Azure AI Apps