1 Overview: From a Working System to a Running Service

Every tutorial to this point has built something that works — an agent, a RAG pipeline, a multi-agent system with governance and evaluation. Working is necessary but not sufficient: a GenAI application also has to be deployed, monitored, and operated as a real service that real users depend on, that changes safely over time, and that someone can debug at 2 a.m. when something goes wrong. This tutorial covers that operational layer: deployment architecture for GenAI apps, cloud deployment options, monitoring and observability, enabling logging and telemetry specifically for AI calls, and the rollout and configuration concerns that separate a demo from production.

Much of this is standard .NET operations — deployment, monitoring, configuration are not new problems GenAI invented. What's new is what to monitor and configure specifically for AI: token usage and cost, model deployment health, retrieval quality, and the rollout risk of changing a prompt or swapping a model version, none of which a traditional web app's operations playbook covers by default. This tutorial extends the operational practices you already know with the AI-specific additions that make the difference.

2 Learning Objectives

• Design deployment architecture for a GenAI app spanning web tier, model access, and background/agent work.
• Choose an appropriate cloud deployment target for a .NET GenAI application and understand its operational tradeoffs.
• Establish monitoring and observability covering both ordinary application health and AI-specific signals.
• Enable logging and telemetry for AI calls specifically — tokens, latency, finish reasons, retrieval quality — building on tutorial 13.
• Apply rollout and configuration practices (canary releases, feature flags, secrets management) suited to changes that affect model behavior.

3 Prerequisites

• Tutorial 13: structured logging, resilience, and dependency injection for AI services — this tutorial's monitoring section extends that foundation.
• Tutorial 14: the ASP.NET Core web layer an AI feature typically deploys as part of.
• Basic familiarity with cloud deployment concepts (containers, managed platforms) and CI/CD pipelines is helpful but not required.
• Tutorial 22's evaluation/golden-dataset practice, referenced here for validating a rollout's safety.

4 Key Concepts: Operating an AI Application Like Any Production Service

A GenAI application in production has all the operational needs of any web service — deployment architecture, a hosting environment, monitoring, logging, and a safe way to roll out changes — plus AI-specific additions layered on top of each. The five subtopics of this tutorial map directly onto standard operational concerns, each with what changes when AI is involved.

None of this replaces ordinary .NET deployment practice — it extends it. A GenAI app still needs a deployment architecture, a cloud target, dashboards, logs, and a rollout strategy exactly as any production service does; this tutorial's job is showing what to add so those practices actually cover the AI-specific ways this kind of application can fail or regress.

5 Deep Dive 1: Deployment Architecture for GenAI Apps

A typical GenAI application's deployment architecture has several distinct pieces beyond a plain web app: the web tier (tutorial 14's ASP.NET Core app serving REST endpoints and streaming responses), model access (an Azure OpenAI or Azure AI Foundry connection, ideally via Managed Identity, tutorial 13/16), background/agent work (tutorial 22's long-running or multi-agent workflows, which may need their own hosting separate from the request-serving web tier), and any MCP servers (tutorial 21) the app depends on, which are themselves independently deployed services with their own lifecycle.

The key architectural decision is what deploys together versus separately. A simple app might deploy web tier and lightweight background work as one unit. A more complex system separates them: the web tier scales for request volume, background/agent workers scale for job volume, and each MCP server scales independently based on its own consumers — mirroring tutorial 22's lesson that scaling one component doesn't automatically help another. Documenting this architecture (even a simple diagram) is worth doing before choosing a cloud deployment target, since the target needs to accommodate every piece, not just the web tier.

6 Deep Dive 2: Cloud Deployment

For a .NET GenAI application, two common Azure targets cover most cases. Azure App Service hosts the web tier directly — .NET-native, straightforward deployment from CI/CD, built-in scaling and slots for rollout (deep-dive 5) — a strong default for a web app with no unusual infrastructure needs. Azure Container Apps runs containerized applications with built-in scaling (including scale-to-zero) and is well suited when the app is packaged as a container, needs to run background/agent workloads alongside the web tier, or needs more control over the runtime environment than App Service's model provides.

Whichever target is chosen, the app connects to model access (Azure OpenAI or a Foundry project) the same way regardless of host — via Managed Identity where possible, following tutorial 13's secrets discipline, so the hosting choice doesn't change how credentials are managed. Environment separation matters here too: development, staging, and production should be distinct environments, ideally with their own model deployments or at least clearly separated quota, so a staging load test or a broken prompt in staging cannot exhaust production's token budget or leak staging data into a production index.

7 Deep Dive 3: Monitoring, Observability, Logging, and Telemetry

Observability is the ability to understand a running system's internal state from its outputs — logs, metrics, and traces — and enabling logging and telemetry is how you produce those outputs in the first place. For a GenAI app, this means everything a normal .NET app needs (request rates, error rates, CPU/memory, response times) plus AI-specific telemetry: token usage per request (the direct driver of cost), model call latency separate from total request latency, finish reasons (catching truncated or filtered responses), and for RAG systems, retrieval quality signals like which passages were retrieved and their relevance scores.

// Building on tutorial 13's structured logging: the same fields also feed
// Application Insights custom metrics for dashboards and alerts.
using var activity = _activitySource.StartActivity("AI.ChatCompletion");
activity?.SetTag("ai.deployment", deploymentName);

var watch = Stopwatch.StartNew();
ChatCompletion completion = await chat.CompleteChatAsync(messages, options, ct);
watch.Stop();

_telemetryClient.GetMetric("AI.TokensUsed").TrackValue(
    completion.Usage.InputTokenCount + completion.Usage.OutputTokenCount);
_telemetryClient.GetMetric("AI.LatencyMs").TrackValue(watch.ElapsedMilliseconds);

_logger.LogInformation(
    "AI call {Operation} finished in {ElapsedMs} ms using {TotalTokens} tokens, finish {FinishReason}",
    "ChatCompletion", watch.ElapsedMilliseconds,
    completion.Usage.InputTokenCount + completion.Usage.OutputTokenCount, completion.FinishReason);

8 Deep Dive 4: Rollout and Configuration Concerns

A GenAI application has more kinds of 'changes' than ordinary code deployments: a code change, a prompt or instruction change, a model version upgrade, and a configuration change (temperature, max tokens, which tools are enabled) can each independently alter behavior — and only the first is caught by traditional code review and unit tests. Rollout practices need to account for all four. Canary release routes a small percentage of traffic to a new version (code or prompt) first, catching regressions before they reach everyone. Blue-green deployment keeps two full environments and switches traffic only once the new one is verified — well suited when a full behavioral comparison before cutover matters more than gradual exposure.

// A feature flag lets a prompt or model change roll out without a new deployment,
// and lets it be instantly reverted if evaluation (tutorial 22) flags a regression.
bool useNewPrompt = await featureFlags.IsEnabledAsync("NewGroundingPrompt", userId);

string systemPrompt = useNewPrompt
    ? PromptLibrary.GroundingPromptV2
    : PromptLibrary.GroundingPromptV1;

var messages = new ChatMessage[]
{
    new SystemChatMessage(systemPrompt),
    new UserChatMessage(userQuestion)
};

Before widening a canary or flipping a feature flag, run the change against tutorial 22's golden dataset and compare evaluation metrics between old and new — a prompt change that looks fine on a handful of manual tries can still regress accuracy on cases the golden dataset covers but a developer didn't happen to try. If regression appears, a rollback should be as fast as flipping the feature flag or reverting the canary weight, not a full redeploy. Configuration itself needs its own discipline: secrets management (Managed Identity, Key Vault, never in source, per tutorial 13) prevents credential leaks, and guarding against configuration drift — an environment's actual settings silently diverging from what's documented — is what infrastructure as code (deep-dive 2) is for.

9 Ecosystem and Tools

This is standard .NET cloud operations tooling. The GenAI-specific value-add is what you point it at: custom metrics for tokens and cost, evaluation as a pipeline gate, and feature flags scoped to prompt/model changes as much as to code features.

10 Use Cases

• A RAG-based support assistant deployed to Azure App Service, with Application Insights dashboards tracking token spend and retrieval relevance alongside ordinary request metrics.
• A multi-agent document-processing system (tutorial 22) deployed to Azure Container Apps, with the web tier and background workers scaled independently.
• A canary rollout of a new grounding prompt: 5% of traffic for a day, evaluated against the golden dataset, widened only after metrics hold steady.
• An incident where token spend spikes unexpectedly: an alert fires, the on-call engineer checks the dashboard, traces the spike to a specific endpoint via distributed tracing, and finds a recent prompt change resent too much conversation history.
• A staging environment with its own Azure OpenAI deployment and quota, so a load test never risks exhausting production's token budget.
• A feature flag instantly reverting a newly-deployed model version after evaluation detects a groundedness regression, without waiting for a new deployment.
• Infrastructure as code defining the whole deployment architecture (web tier, background workers, Key Vault, Application Insights) so a new environment can be stood up identically to production.

11 Code Examples

These examples show custom telemetry for AI-specific dashboards, a health check covering model connectivity, and a rollout guard tied to evaluation results.

// Program.cs
builder.Services.AddApplicationInsightsTelemetry();
builder.Services.AddSingleton<TelemetryClient>();

// A small wrapper service records AI-specific metrics on every call,
// reusing the resilient ChatService from tutorial 13.
builder.Services.AddSingleton<IChatService>(sp =>
    new TelemetryEnrichedChatService(
        inner: sp.GetRequiredService<ChatService>(),
        telemetry: sp.GetRequiredService<TelemetryClient>()));

// A cheap real call proves the deployment is reachable and authorized --
// closing the gap ValidateOnStart (tutorial 13) leaves for connectivity issues.
builder.Services.AddHealthChecks()
    .AddCheck<AzureOpenAIHealthCheck>("azure-openai", tags: new[] { "ready" });

public sealed class AzureOpenAIHealthCheck : IHealthCheck
{
    private readonly ChatClient _chat;
    public AzureOpenAIHealthCheck(ChatClient chat) => _chat = chat;

    public async Task<HealthCheckResult> CheckHealthAsync(
        HealthCheckContext context, CancellationToken ct = default)
    {
        try
        {
            await _chat.CompleteChatAsync(
                new UserChatMessage("ping"),
                new ChatCompletionOptions { MaxOutputTokenCount = 5 }, ct);
            return HealthCheckResult.Healthy();
        }
        catch (Exception ex)
        {
            return HealthCheckResult.Unhealthy("Model deployment unreachable", ex);
        }
    }
}

// A CI/CD pipeline step (conceptual): widen the canary only if evaluation passes.
EvaluationReport report = await EvaluateAsync(goldenDataset, candidateService: newVersion);

if (report.Accuracy < baselineAccuracy - 0.02)   // more than a 2-point regression
{
    logger.LogWarning("Evaluation regression detected: {Accuracy} vs baseline {Baseline}",
        report.Accuracy, baselineAccuracy);
    await featureFlags.SetAsync("NewGroundingPrompt", enabled: false);   // auto-rollback
    throw new InvalidOperationException("Rollout blocked: evaluation regression.");
}

await canaryController.WidenAsync("NewGroundingPrompt", toPercent: 25);

12 Step by Step: Deploying and Monitoring a GenAI Web App

This walkthrough deploys the tutorial-14 streaming assistant to Azure, wires up monitoring, and performs a governed rollout of a prompt change.

1. Sketch the deployment architecture: web tier (the ASP.NET Core app), model access (Azure OpenAI via Managed Identity), and any background work — decide what deploys together and what deploys separately.
2. Provision the target with infrastructure as code: an Azure App Service (or Container Apps) instance, an Application Insights resource, and a Key Vault, all defined in Bicep or Terraform rather than manual steps.
3. Deploy the app via CI/CD, confirming Managed Identity grants it access to the Azure OpenAI deployment with no key in configuration.
4. Add the health check from Example 2 and confirm the platform's readiness probe reflects actual model connectivity, not just process liveness.
5. Add the telemetry wrapper from Example 1, recording tokens and latency as custom metrics alongside tutorial 13's structured logs.
6. Build a dashboard in Application Insights showing request rate, error rate, p95 latency, and daily token spend; set an alert on a token-spend threshold.
7. Prepare a prompt change (e.g. a revised grounding instruction) behind a feature flag, and run it against tutorial 22's golden dataset before enabling it for any real traffic.
8. Enable the flag for a small percentage of traffic (a canary), monitor the dashboard and evaluation metrics for a defined period, and only then widen it — following Example 3's gating logic.
9. Simulate a regression: deliberately degrade the new prompt, confirm the evaluation gate catches it, and confirm the feature flag rollback actually reverts behavior for all traffic within seconds.
10. Document the whole architecture, environments, and rollout process — the artifact a new team member or an on-call engineer would need to operate this system without you.

13 Limitations and Caveats

• This tutorial's guidance is Azure-centric and .NET-centric by design (matching the course); other clouds and stacks have direct equivalents (containers, APM services, secret stores) but different specific names and setup steps.
• Custom telemetry and health-check code shown are illustrative; exact Application Insights/telemetry SDK APIs evolve, and production code should be verified against current package documentation.
• Canary releases and feature flags add operational complexity (traffic splitting, flag management) that a very low-traffic or internal-only app may not need — match rollout rigor to actual risk and blast radius.
• Evaluation-gated rollout (Example 3) assumes a maintained, representative golden dataset (tutorial 22); a stale or narrow dataset gives false confidence that a regression was caught when it wasn't.
• Health checks that make a real model call (Example 2) consume tokens and count against rate limits on every check — tune check frequency accordingly rather than checking on every request.
• Separating web tier and background/agent work into different deployment units adds coordination overhead (versioning, communication) that a very simple app may not need yet.
• Monitoring dashboards and alerts require someone to actually watch and respond to them; tooling alone does not provide operational readiness without a team practice around it.
• Cost monitoring via token telemetry approximates spend; always reconcile against the cloud provider's actual billed usage, since telemetry counts and billing can diverge in edge cases.

14 Best Practices

• Document the deployment architecture explicitly — what deploys together, what deploys separately, and why — before choosing a cloud target.
• Deploy long-running background/agent work separately from the request-serving web tier once it's more than trivially short.
• Provision infrastructure as code, not manual portal steps, to prevent configuration drift across environments.
• Monitor AI-specific signals (tokens, model latency, finish reasons, retrieval quality) alongside ordinary application metrics, not as an afterthought.
• Never log prompt or response bodies by default; capture metadata for dashboards and alerts instead.
• Treat prompt, instruction, and model-version changes with the same rollout caution as code changes — canary or feature-flag them, and gate on evaluation results.
• Keep secrets in Managed Identity or Key Vault, never in source or plain configuration, across every environment.
• Test the rollback path deliberately, before an incident forces you to discover whether it actually works.

20 Summary

• Deployment architecture for a GenAI app includes the web tier, model access, background/agent work, MCP servers, and data stores — deployed together or separately based on differing scaling and lifecycle needs.
• Cloud deployment options like Azure App Service and Azure Container Apps both reach model access the same way (Managed Identity); the choice follows from the architecture, not the reverse.
• Monitoring and observability, plus enabling logging and telemetry, must cover AI-specific signals — tokens, model latency, finish reasons, retrieval quality — alongside standard application metrics, without ever logging prompt/response bodies by default.
• Rollout and configuration concerns treat prompt, instruction, and model-version changes with the same rollout caution as code — canary releases and feature flags gated on evaluation results (tutorial 22), with a deliberately tested rollback path.
• Environment separation, infrastructure as code, and secrets management (Managed Identity, Key Vault) round out the operational discipline a GenAI app needs to run safely over time.
• None of this is new mechanism: it's standard .NET deployment and operations practice extended with the specific additions AI workloads require.

You now have the operational layer that turns a capable AI system into a production one: an architecture that accommodates AI's extra moving parts, a cloud deployment suited to it, monitoring that watches what's specifically worth watching in an AI system, and a rollout discipline that treats behavior changes — prompts and models, not just code — with real rigor. With the system deployed, monitored, and safely changeable, the course turns next to a concern that applies regardless of how well any of this is done: security and responsible use, because an operable system that isn't secure or responsibly used isn't actually safe to run.

21 Next Steps

Next tutorial: AI Security and Responsible Usage (ai-security-responsible-usage). With deployment, monitoring, and rollout established, the next tutorial addresses the security threats specific to AI applications (like prompt injection, hinted at in earlier tutorials) and the responsible-AI practices that must accompany a system now capable of running in production at scale.

• Practice: sketch the deployment architecture for a system you've built in this course, identifying which pieces would deploy together versus separately and why.
• Practice: add the health check and telemetry examples from this tutorial to an existing project, and confirm a dashboard reflects token usage and latency accurately.
• Practice: implement a feature flag around a prompt change, run it against a golden dataset, and practice the full canary-then-widen-then-rollback cycle deliberately.
• Practice: write an infrastructure-as-code definition (Bicep or Terraform) for a simple GenAI deployment architecture, even if you don't apply it to a real subscription.
• Read: the official documentation for 'Azure App Service', 'Azure Container Apps', 'Application Insights for .NET', and any current guidance on deployment slots and canary releases in Azure.