1 Overview: From Working Demo to Production Citizen

Tutorials 11 and 12 left you with something valuable and fragile: a console app that calls Azure OpenAI, uses tools, and handles a 429 — with the client built by hand, the configuration read inline, errors logged with Console.WriteLine, and everything welded together in Program.cs. That shape is perfect for learning and unacceptable for a team codebase. An AI call is just another remote dependency, and enterprises already know how to integrate remote dependencies well.

This tutorial applies that knowledge to AI: a service layer that hides the SDK behind an interface your domain understands; dependency injection to construct and share the client correctly; the options pattern and layered configuration providers for secure configuration management; Key Vault and Managed Identity for securing API keys and secrets; structured logging so every AI call is traceable and measurable; and resilience — timeouts, retries with exponential backoff, and the circuit breaker — so a slow or rate-limited AI service degrades your app gracefully instead of hanging it.

2 Learning Objectives

• Design a service-layer abstraction (IChatService) that isolates AI SDK types behind domain-shaped methods and makes them mockable in tests.
• Register AI services with dependency injection using correct service lifetimes — a singleton client — and constructor injection throughout.
• Bind AI settings with the options pattern, layer configuration providers correctly, and validate configuration at startup.
• Secure API keys with user secrets in development and Key Vault plus Managed Identity in production, with secret rotation in mind.
• Emit structured logging for every AI call — correlation ids, latency, token usage — without leaking prompts or secrets into logs.
• Add resilience with timeouts, retries with exponential backoff for transient faults, and a circuit breaker for sustained failure.

3 Prerequisites

• Tutorials 11–12: calling chat completions from C#, the orchestration loop for tools, and basic retry-on-429 handling.
• Working familiarity with ASP.NET Core or the .NET Generic Host: Program.cs, builder.Services, appsettings.json.
• C# interfaces and constructor injection as language features (this tutorial supplies the architectural discipline around them).
• Your Azure OpenAI endpoint and deployment from earlier tutorials, with credentials currently in user secrets or environment variables.

4 Key Concepts: The Six Disciplines

Each subtopic of this tutorial answers one production question. Who constructs the AI client, and how many exist? — dependency injection with a singleton lifetime. What does the rest of the codebase call? — a service layer interface, not the SDK. Where do settings live? — layered configuration providers bound through the options pattern. Where does the API key live? — ideally nowhere (Managed Identity); otherwise Key Vault with secret rotation. What happened at runtime? — structured logging with correlation ids. What when the service is slow or failing? — timeouts, retries, and a circuit breaker.

The six reinforce each other: DI is what lets the resilience policy wrap the service layer; the options pattern is what lets Key Vault swap in for user secrets without code changes; structured logging is what proves the retries and the circuit breaker are doing their jobs. You will build them as one coherent composition in the step-by-step section.

5 Deep Dive 1: The Service-Layer Abstraction

The first refactoring is architectural: no controller, page, or domain class should ever see a ChatClient, a ChatMessage, or a BinaryData of tool arguments. Instead the application defines a service layer — an interface owned by your code, speaking your domain: Task<OrderAnswer> AskAboutOrderAsync(string question, CancellationToken ct). One implementation class translates between that domain language and the SDK: building messages, running the orchestration loop from tutorial 12, and mapping results back to plain domain types.

The payoffs compound. Testability: consumers depend on the interface, so unit tests substitute a mock and run instantly, offline, at zero token cost — without it, every test of an AI feature is a slow, flaky, billable network call. Swappability: when the SDK's types shift between versions (they have), or a second model provider appears, or tutorial 17 replaces the hand-rolled loop with Semantic Kernel, the change is confined to one class. Reviewability: prompt text, tool definitions, and parameters live in one place a reviewer can audit, not scattered across features. Policy: temperature, max tokens, and the system message become decisions the service makes consistently, not copy-pasted defaults.

6 Deep Dive 2: Dependency Injection for AI Services

Dependency injection inverts construction: classes declare what they need in their constructors, and the container builds the object graph at startup. For AI services the pattern resolves three practical questions. Construction: the AzureOpenAIClient needs an endpoint and a credential — that wiring happens once, in the composition root, not wherever someone needs a completion. Sharing: the client is thread-safe and manages HTTP connections, so its service lifetime must be singleton; per-request construction wastes sockets and handshakes and can exhaust the connection pool under load. Substitution: because consumers receive IChatService through their constructors, tests and future implementations slot in without touching consumers.

// Program.cs — the only place that knows how the AI stack is assembled.
var builder = Host.CreateApplicationBuilder(args);

// 1. Options: bind and validate the AzureOpenAI section (deep-dive 3).
builder.Services.AddOptions<AzureOpenAIOptions>()
    .Bind(builder.Configuration.GetSection(AzureOpenAIOptions.SectionName))
    .ValidateDataAnnotations()
    .ValidateOnStart();   // fail at startup, not at first user request

// 2. The SDK client: one singleton for the whole application.
builder.Services.AddSingleton(sp =>
{
    var opts = sp.GetRequiredService<IOptions<AzureOpenAIOptions>>().Value;
    return new AzureOpenAIClient(new Uri(opts.Endpoint),
        new AzureKeyCredential(opts.ApiKey));   // or DefaultAzureCredential — see deep-dive 3
});

// 3. The service layer: consumers depend on the interface only.
builder.Services.AddSingleton<IChatService, ChatService>();

var app = builder.Build();

Lifetime reasoning for the rest of the stack: the service-layer class is stateless (all per-call state lives in locals), so it can be a singleton too; anything holding per-request state — a conversation history builder, say — belongs scoped in a web app. The rule that catches most mistakes: a singleton may not depend on a scoped service. Keep the AI stack stateless-singleton and inject per-call context (user id, cancellation token) as method parameters instead.

7 Deep Dive 3: Secure Configuration and Secrets

Secure configuration management has two halves: how settings reach the app, and where secrets live. The first half is the host's configuration provider chain: appsettings.json (safe defaults — endpoint, deployment name, temperature), appsettings.{Environment}.json, user secrets in development, environment variables, and finally a Key Vault provider in production. Later providers override earlier ones, so the same key — AzureOpenAI:ApiKey — can come from user secrets on a laptop and from Key Vault in production with zero code differences. The options pattern then binds the merged result to a typed, validated class, so a missing endpoint stops the app at startup with a clear message instead of surfacing as a 401 at 2 a.m.

The second half is minimizing what counts as a secret at all. The strongest move is Managed Identity: the app authenticates to Azure OpenAI (and to Key Vault itself) as an Azure identity via DefaultAzureCredential, a role assignment grants access, and no API key exists anywhere in the system. Where a key is unavoidable — third-party services, local tooling — Key Vault gives it an access policy, an audit log, and automated secret rotation, so exposure has a short shelf life. The hierarchy to remember: no secret beats a vaulted secret beats an environment secret beats anything in a file — and a secret in source control is an incident, not a configuration choice.

8 Deep Dive 4: Resilience and Structured Logging

AI calls are the slowest, most rate-limited remote calls most apps make, which makes two runtime disciplines mandatory. Resilience first. A timeout converts a hung call into a prompt, handleable failure — without one, a stalled request pins a thread and a user for minutes; generation is slow, so budget generously (say 30–60 seconds) but finitely, and flow a CancellationToken through every async signature so cancellation actually reaches the SDK. Retries handle transient faults — 429 and 5xx — with exponential backoff and jitter, capped at a few attempts, exactly as built in tutorial 11. The circuit breaker completes the set: when failures are sustained rather than momentary, it opens after a threshold and fails fast for a cooldown, sparing the struggling deployment a retry storm and sparing your users the full timeout wait; after the cooldown it lets a trial request probe recovery.

Structured logging is how you see any of this working. The principle: log message templates with named properties — logger.LogInformation("AI call {Operation} finished in {ElapsedMs} ms using {TotalTokens} tokens", ...) — never interpolated strings. The logging system stores Operation, ElapsedMs, and TotalTokens as fields, so questions like 'p95 latency of order questions this week' or 'token spend by feature' become queries, not archaeology. Stamp every entry with a correlation id (ASP.NET Core's request id, or an Activity/trace id) so one user request's completion calls, tool executions, retries, and breaker events line up as one story. And log about prompts — lengths, token counts, deployment, finish reason — not the prompt text itself, which is user data; if you must capture bodies for debugging, gate it behind a flag and redact.

9 Ecosystem and Tools

Everything in this table is standard .NET infrastructure — none of it was invented for AI. Polly deserves a special note: this course taught the retry loop by hand so you understand it, but production code should generally express timeout, retry, and circuit breaker as a composed Polly pipeline registered in DI, giving you tested policies, jitter strategies, and telemetry hooks for free.

10 Use Cases

• Team onboarding: a new developer adds an AI feature by injecting IChatService — without reading SDK docs, holding a key, or knowing which deployment is live.
• Provider migration: switching a workload to a different model deployment (or a second provider for comparison) by changing one implementation class and zero consumers.
• Cost governance: structured logs of token usage per operation feed a dashboard that catches a prompt change doubling spend — the day it ships, not on the invoice.
• Incident response: a correlation id from one user complaint pulls up the exact request's retries, breaker state, latency, and finish reason across services.
• Security review: the audit is short because the answer is boring — no key in code, user secrets in dev, Key Vault plus Managed Identity in production, rotation automated.
• Load spikes: a marketing campaign triples traffic; backoff absorbs the 429s, the breaker sheds load when a deployment saturates, and the app degrades to a friendly fallback instead of a hung UI.
• Test suites: hundreds of unit tests run against a mock of the service layer in seconds — zero tokens billed, zero network flakiness in CI.

11 Code Examples

These examples form one coherent stack: an options class, a service-layer interface with its SDK-facing implementation, and the logging plus resilience wrapper. Deep-dive 2 already showed the Program.cs registration that wires them together.

using System.ComponentModel.DataAnnotations;

public sealed class AzureOpenAIOptions
{
    public const string SectionName = "AzureOpenAI";

    [Required, Url]
    public string Endpoint { get; set; } = string.Empty;

    [Required]
    public string ChatDeployment { get; set; } = string.Empty;

    // Empty when using Managed Identity — see the factory in Program.cs.
    public string ApiKey { get; set; } = string.Empty;

    [Range(0.0, 2.0)]
    public float Temperature { get; set; } = 0.2f;

    [Range(1, 4000)]
    public int MaxOutputTokens { get; set; } = 400;

    [Range(1, 10)]
    public int MaxRetryAttempts { get; set; } = 4;

    [Range(1, 300)]
    public int TimeoutSeconds { get; set; } = 45;
}

// The abstraction consumers depend on: no SDK types anywhere in sight.
public interface IChatService
{
    Task<OrderAnswer> AskAboutOrderAsync(string question, CancellationToken ct = default);
}

public sealed record OrderAnswer(string Text, bool Truncated);

// The one class allowed to know the SDK exists.
public sealed class ChatService : IChatService
{
    private readonly ChatClient _chat;
    private readonly AzureOpenAIOptions _options;
    private readonly ILogger<ChatService> _logger;

    public ChatService(AzureOpenAIClient client,
                       IOptions<AzureOpenAIOptions> options,
                       ILogger<ChatService> logger)
    {
        _options = options.Value;
        _chat = client.GetChatClient(_options.ChatDeployment);
        _logger = logger;
    }

    public async Task<OrderAnswer> AskAboutOrderAsync(string question, CancellationToken ct = default)
    {
        var messages = new ChatMessage[]
        {
            new SystemChatMessage("You are an order-support assistant. " +
                                  "Use tools for order data; never invent details."),
            new UserChatMessage(question)
        };
        var chatOptions = new ChatCompletionOptions
        {
            Temperature = _options.Temperature,
            MaxOutputTokenCount = _options.MaxOutputTokens
        };

        ChatCompletion completion =
            await CompleteResilientlyAsync(messages, chatOptions, ct);   // Example 3

        return new OrderAnswer(
            Text: completion.Content[0].Text,
            Truncated: completion.FinishReason == ChatFinishReason.Length);
    }
}

private async Task<ChatCompletion> CompleteResilientlyAsync(
    ChatMessage[] messages, ChatCompletionOptions chatOptions, CancellationToken ct)
{
    var jitter = Random.Shared;
    for (int attempt = 1; ; attempt++)
    {
        // Timeout: this attempt gets a bounded window, linked to the caller's token.
        using var timeoutCts = CancellationTokenSource.CreateLinkedTokenSource(ct);
        timeoutCts.CancelAfter(TimeSpan.FromSeconds(_options.TimeoutSeconds));

        var watch = System.Diagnostics.Stopwatch.StartNew();
        try
        {
            ChatCompletion completion = await _chat.CompleteChatAsync(
                messages, chatOptions, timeoutCts.Token);

            // Structured logging: named properties, queryable fields — no prompt text.
            _logger.LogInformation(
                "AI call {Operation} ok in {ElapsedMs} ms: {PromptTokens}+{CompletionTokens} tokens, finish {FinishReason}, attempt {Attempt}",
                nameof(AskAboutOrderAsync), watch.ElapsedMilliseconds,
                completion.Usage.InputTokenCount, completion.Usage.OutputTokenCount,
                completion.FinishReason, attempt);
            return completion;
        }
        catch (ClientResultException ex)
            when ((ex.Status == 429 || ex.Status >= 500) && attempt < _options.MaxRetryAttempts)
        {
            // Transient fault: exponential backoff with jitter, and a Warning you can query.
            int delayMs = (int)(Math.Pow(2, attempt - 1) * 1000) + jitter.Next(0, 250);
            _logger.LogWarning(
                "AI call {Operation} transient {Status} on attempt {Attempt}; retrying in {DelayMs} ms",
                nameof(AskAboutOrderAsync), ex.Status, attempt, delayMs);
            await Task.Delay(delayMs, ct);
        }
        catch (OperationCanceledException) when (timeoutCts.IsCancellationRequested && !ct.IsCancellationRequested)
        {
            _logger.LogWarning("AI call {Operation} timed out after {TimeoutSeconds} s on attempt {Attempt}",
                nameof(AskAboutOrderAsync), _options.TimeoutSeconds, attempt);
            if (attempt >= _options.MaxRetryAttempts) throw;
        }
    }
}

12 Step by Step: Refactoring the Order Assistant to Enterprise Shape

This walkthrough takes tutorial 12's single-file order assistant and rebuilds it with every discipline from this tutorial. The observable behavior does not change — that is the point of a refactoring — but every seam a production team needs appears.

1. Create the skeleton: 'dotnet new console -n OrderAssistant', then add Microsoft.Extensions.Hosting, Azure.AI.OpenAI, and Azure.Identity. The Generic Host brings configuration, DI, and logging in one builder.
2. Add AzureOpenAIOptions (Example 1). Put Endpoint, ChatDeployment, Temperature, and MaxOutputTokens in appsettings.json under an 'AzureOpenAI' section — and confirm the section contains no ApiKey line.
3. Move the key: 'dotnet user-secrets init', then 'dotnet user-secrets set AzureOpenAI:ApiKey <your-key>'. Delete any environment-variable reading code left from tutorial 11.
4. Register the stack in Program.cs exactly as in deep-dive 2: AddOptions with ValidateDataAnnotations and ValidateOnStart, the singleton AzureOpenAIClient factory, and AddSingleton<IChatService, ChatService>.
5. Create the service layer (Example 2): IChatService plus OrderAnswer in their own files; move the system message, the tool definitions, and the orchestration loop from Program.cs into ChatService. Program.cs shrinks to host setup plus a read-ask-print loop that only sees IChatService.
6. Wrap the SDK call with Example 3's resilience and logging: timeout from options, exponential backoff on 429/5xx, structured Warning per retry, structured Information per success with token counts.
7. Run 'dotnet run' and ask about ORD-1042. Confirm the answer still works — and that the console now shows a structured log line with latency, token counts, and finish reason.
8. Prove fail-fast: temporarily rename Endpoint in appsettings.json and run again. The host must refuse to start with a validation error naming the missing field — that is ValidateOnStart earning its keep.
9. Prove testability: add an xUnit test project referencing the app, write a FakeChatService implementing IChatService that returns a canned OrderAnswer, and unit-test the console loop's truncation warning logic with zero network calls.
10. Sketch production: add the Key Vault configuration provider behind an if (builder.Environment.IsProduction()) guard, using DefaultAzureCredential — the options class and every consumer stay untouched, which is the layering payoff.
11. Commit, then diff against tutorial 12's version: same feature, but now the SDK lives in one file, the key lives in no file, and every AI call leaves a queryable trace.

13 Limitations and Caveats

• SDK naming caveat: examples target Azure.AI.OpenAI 2.x (AzureOpenAIClient, ChatClient, CompleteChatAsync with a CancellationToken, ChatCompletion.Usage.InputTokenCount/OutputTokenCount, ClientResultException.Status). Names shift between versions — if a member does not compile, check the package's current samples; the pattern stands even where a property name moved.
• The hand-rolled resilience in Example 3 omits a real circuit breaker; breakers need shared state across calls and are genuinely easier to get right with Polly than by hand. Treat Example 3 as pedagogy, Polly as production.
• Timeouts cancel your wait, not the server's work — a cancelled generation may still bill tokens for what was produced before cancellation reached it.
• Retries interact with cost and idempotency: a retried completion is a second billable call, and a retried tool-using conversation may re-execute tools — another reason tool implementations must be idempotent (tutorial 12).
• ValidateOnStart catches missing configuration, not wrong configuration — a valid-looking endpoint for the wrong resource still fails at first call. A startup health probe that makes one cheap real call closes that gap.
• Key Vault adds a startup dependency and latency, and cached secrets mean rotation is not instantaneous — understand your provider's reload behavior before rotating under load.
• Structured logging of token counts is not a billing system; reconcile against the portal's usage data. And logging prompt bodies, even truncated, can put user data in logs — default to metadata only.
• DI lifetimes shown here assume the stateless-singleton service design from deep-dive 2; if you add per-conversation state to ChatService, its lifetime (and thread safety) must be redesigned, not just re-registered.

14 Best Practices

• One interface, one implementation file, zero SDK types outside it — enforce with code review and, at scale, an architecture test that fails on forbidden references.
• Register the AI client once as a singleton in the composition root; inject everything, construct nothing inline.
• Bind all AI settings through the options pattern with DataAnnotations and ValidateOnStart; treat a magic string deployment name in code as a bug.
• Follow the secrets hierarchy: Managed Identity if possible, Key Vault if not, user secrets in development — and never a secret in appsettings.json or source.
• Flow CancellationToken through every async AI signature, and give every call a finite timeout.
• Retry only transient faults, with exponential backoff, jitter, and a bounded attempt count; add a circuit breaker (via Polly) for sustained failure.
• Log structured events for every call — operation, latency, token counts, finish reason, attempt — with a correlation id, and never log prompt text or keys.
• Unit-test consumers against a mock of the service layer; keep one small integration test that exercises the real deployment nightly, not per-commit.

20 Summary

• Treat the model as a remote dependency and every standard .NET discipline applies: the fastest route to production-ready AI is patterns your team already trusts.
• A service layer (IChatService) hides all SDK types behind domain-shaped methods — one file to change for SDK upgrades, one interface to mock for free, fast, offline tests.
• Dependency injection assembles the stack in the composition root: validated options, a singleton AzureOpenAIClient, and stateless singleton services with per-call context as parameters.
• Secure configuration layers providers (appsettings → user secrets → environment variables → Key Vault) under the options pattern with ValidateOnStart; secrets follow the hierarchy Managed Identity > Key Vault > user secrets > never source control.
• Structured logging with named properties and a correlation id turns every AI call into queryable fields — latency, token spend, retries, finish reasons — which is observability and cost governance in one habit.
• Resilience composes three policies for three failure shapes: timeouts for hangs, backoff retries for transient faults, a circuit breaker for sustained outage — expressed in production as a Polly pipeline, with an unbroken CancellationToken chain making all of it real.

The order assistant does exactly what it did yesterday — but today the SDK lives in one file, the key lives in no file, a missing setting stops startup with a field name, a hung call fails in forty-five seconds instead of never, and every request leaves a trace you can query. None of it required anything AI-specific; all of it is what makes the AI parts safe to build on. That foundation is precisely what the next tutorial assumes as it moves the assistant into a real ASP.NET Core web application.

21 Next Steps

Next tutorial: AI in ASP.NET Core Web Apps (ai-in-aspnet-web-apps). The enterprise-shaped stack you built slots straight into a web application — and gains new dimensions there: scoped lifetimes and per-user context, request cancellation wired to real browsers, streaming responses to the UI, and the middleware pipeline as a home for cross-cutting AI concerns.

• Practice: replace Example 3's hand-rolled loop with a Polly resilience pipeline (timeout + retry + circuit breaker) registered in DI, and verify the structured log events still tell the story.
• Practice: break your configuration on purpose five ways (missing endpoint, malformed URL, out-of-range temperature, absent key, wrong deployment name) and confirm which are caught at startup versus first call — then add the startup health probe that closes the gap.
• Practice: write the FakeChatService test from step 9 of the walkthrough, then add a second fake that always returns Truncated = true and assert your UI's warning path.
• Practice: switch the client factory to DefaultAzureCredential and run keyless locally via your development tool sign-in — then delete the ApiKey from user secrets entirely.
• Read: the official documentation for 'Options pattern in ASP.NET Core', 'Safe storage of app secrets in development', 'Azure Key Vault configuration provider', 'Logging in .NET', and the Polly project's resilience strategy docs.