1 Overview: From Portal to Program

In the previous tutorial you provisioned an Azure OpenAI resource and created a model deployment. This tutorial makes it real: you will call that deployment from C# code. We cover the two operations you will use most — the chat completion API for generating text and the embeddings API for turning text into a vector — and the practical engineering around them: tuning temperature and max tokens, handling errors and retries safely, and keeping your endpoint and API key out of source code.

By the end you will have built a complete .NET console app that reads a prompt from the user, sends it to Azure OpenAI, and displays the response. Everything here is foundation for the rest of the course: function calling, RAG, and Semantic Kernel all sit on top of the same client, the same parameters, and the same error-handling discipline you learn now.

2 Learning Objectives

• Call the chat completion API from C# with system and user messages and read the generated reply.
• Call the embeddings API and understand what the returned vector represents and what it is used for.
• Explain what temperature and max tokens do, and choose sensible values for different kinds of tasks.
• Recognize transient faults such as HTTP 429 rate limit errors and implement retries with exponential backoff.
• Build and run a .NET console app that sends a prompt and displays the response end to end.
• Manage configuration and API keys safely using environment variables, user secrets, and Managed Identity — never hardcoded strings.

3 Prerequisites

• C# fundamentals: classes, async/await, exception handling, and using the .NET CLI (dotnet new, dotnet run).
• .NET 8 SDK installed on your machine.
• An Azure OpenAI resource with a chat model deployment (note its deployment name) and the resource's endpoint URL and API key from the Azure portal.
• Concepts from earlier tutorials: what a token is, what a prompt is, and what an Azure OpenAI deployment is.

4 Key Concepts: The Client, the Deployment, and Two APIs

All access goes through a client object from the official SDK, installed from NuGet as the Azure.AI.OpenAI package. You construct the client once with two pieces of information: the endpoint of your resource and a credential (an API key or an Azure identity). From that client you ask for an operation-specific sub-client bound to a deployment name — a chat client for text generation, an embedding client for vectors.

The distinction between model and deployment matters in code: you never pass 'gpt-4o' to the client. You pass the name you chose when you created the deployment (which might also be 'gpt-4o', but is your label, not the model id). This indirection is what lets operations teams swap model versions without changing application code.

Both APIs are metered in tokens and both are governed by the rate limit assigned to your deployment. That is why parameters like max tokens and habits like exponential backoff retries are not optional extras — they are how you keep a real application within quota, on budget, and resilient.

5 Deep Dive 1: The Chat Completion API

A chat completion request is a conversation snapshot: an ordered list of messages, each tagged with a role. The system message sets the assistant's behavior and constraints ('You are a concise assistant for .NET developers. Answer in C#.'). User messages carry what the human said. Assistant messages carry what the model said earlier — you include them when you want the model to see conversation history, because the API itself is stateless: it remembers nothing between calls.

The response contains more than text. The finish reason tells you why generation stopped: 'stop' means the model completed naturally, 'length' means it was truncated by the max tokens limit, and a content-filter reason means Azure's Responsible AI filters intervened. Usage counts report prompt tokens and completion tokens — log them, because they are exactly what you are billed for and what your rate limit consumes.

6 Deep Dive 2: The Embeddings API

The embeddings API does not generate text at all. You send it a string and it returns an embedding — a vector of floating-point numbers (1,536 dimensions for text-embedding-3-small) that encodes the meaning of the text. Two texts that mean similar things produce vectors that point in similar directions, which you can measure with cosine similarity. That single property powers semantic search: instead of matching keywords, you match meaning.

In C#, the flow is symmetrical with chat: get an embedding client bound to your embedding deployment, call it with text, and read back the vector as a ReadOnlyMemory of float. Embeddings are cheap and fast compared with chat completions, but they still consume tokens and rate limit, and one rule is absolute: vectors are only comparable when produced by the same embedding model. If you switch models, re-embed everything.

7 Deep Dive 3: Key Parameters — Temperature and Max Tokens

Temperature controls how the model picks each next token. At low temperature (near 0) it almost always picks the most probable token, giving focused, consistent, near-deterministic output. At higher temperature (0.8 and up) it samples more freely, giving varied and creative output — and a higher chance of drifting off course. Temperature does not make the model smarter or dumber; it changes how adventurous its choices are.

Max tokens caps the length of the generated response — it limits output tokens only, not your prompt. It is a hard stop: when the cap is reached the reply is truncated mid-thought and the finish reason comes back as 'length'. Set it deliberately: high enough for a complete answer, low enough to bound cost and latency. A one-line classifier might need 20 tokens; a code explanation might need 800. Remember that in .NET SDK code the property is named MaxOutputTokenCount, but the concept everyone says out loud is max tokens.

These two parameters interact with cost directly: temperature costs nothing extra, but sloppy max tokens settings do. A safe pattern is to set temperature per task type in configuration, set max tokens to about double your typical expected answer length, and log actual usage so you can tighten both with data.

8 Deep Dive 4: Errors, Retries, and Protecting Your API Key

Calls to Azure OpenAI fail in two fundamentally different ways, and your code must treat them differently. A transient fault — HTTP 429 when you exceed your rate limit, or an occasional 5xx service hiccup — will likely succeed if retried after a short wait. A permanent error — 401 (bad API key), 404 (wrong endpoint or deployment name), 400 (malformed request or prompt blocked by content filtering) — will fail identically every time, and retrying it is pure waste. Retry the first kind; fix the second.

The second discipline of this section is configuration. Your endpoint and API key must never appear in source code or appsettings.json committed to a repository. In local development, use an environment variable or the .NET user secrets store (dotnet user-secrets), both of which live outside the project folder. In production, prefer Managed Identity: your app authenticates as an Azure identity with a role assignment on the Azure OpenAI resource, and there is simply no key to leak, rotate, or steal. When a key is unavoidable, keep it in Azure Key Vault and rotate it on a schedule.

9 Ecosystem and Tools

The Azure.AI.OpenAI package builds on the general OpenAI .NET library, so chat message types like SystemChatMessage and UserChatMessage come from the OpenAI.Chat namespace while the Azure-specific entry point, AzureOpenAIClient, handles the endpoint and credential. Install both Azure.AI.OpenAI and Azure.Identity via NuGet and you have everything the code in this tutorial needs.

10 Use Cases

• Internal help desk assistant: a console or web tool where staff ask policy questions and a chat completion call answers from a grounded prompt.
• Document summarizer: a batch job that feeds long reports through the chat completion API with low temperature and a generous max tokens budget.
• Semantic FAQ search: embed every FAQ entry once, embed each incoming user question, and use cosine similarity to surface the right answer without any keyword matching.
• Support ticket router: an embedding of each new ticket compared against category examples to auto-assign teams — cheaper and faster than asking a chat model to classify.
• Code review helper: a CLI tool that sends a diff to your deployment and prints suggested improvements, with retries so CI does not fail on a momentary rate limit.
• Duplicate detection: embeddings of bug reports compared pairwise to flag likely duplicates before a human triages them.

Notice the pattern across all of these: chat completions where text must be produced, embeddings where texts must be compared, tuned temperature and max tokens per task, and retry logic anywhere the call sits in an automated pipeline.

11 Code Examples

Three complete examples follow. They assume the environment variables AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_KEY are set, and a chat deployment named gpt-4o-mini plus an embedding deployment named text-embedding-3-small exist on your resource. Install packages first: dotnet add package Azure.AI.OpenAI and dotnet add package Azure.Identity.

using Azure;
using Azure.AI.OpenAI;
using OpenAI.Chat;

// Read the endpoint and API key from environment variables — never hardcode them.
string endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT")
    ?? throw new InvalidOperationException("Set AZURE_OPENAI_ENDPOINT first.");
string apiKey = Environment.GetEnvironmentVariable("AZURE_OPENAI_KEY")
    ?? throw new InvalidOperationException("Set AZURE_OPENAI_KEY first.");

// One client for the resource; a chat sub-client bound to a deployment name.
AzureOpenAIClient azureClient = new(new Uri(endpoint), new AzureKeyCredential(apiKey));
ChatClient chat = azureClient.GetChatClient("gpt-4o-mini");

ChatCompletion completion = await chat.CompleteChatAsync(
    new SystemChatMessage("You are a concise assistant for .NET developers."),
    new UserChatMessage("Explain async/await in two sentences."));

Console.WriteLine(completion.Content[0].Text);
Console.WriteLine($"Finish reason: {completion.FinishReason}");

using OpenAI.Chat;

// Low temperature for a factual task; a tight output cap to bound cost.
ChatCompletionOptions options = new()
{
    Temperature = 0.2f,
    MaxOutputTokenCount = 300   // the SDK name for the max tokens limit
};

ChatCompletion completion = await chat.CompleteChatAsync(
    new ChatMessage[]
    {
        new SystemChatMessage("Answer with a short, accurate definition."),
        new UserChatMessage("What is dependency injection?")
    },
    options);

if (completion.FinishReason == ChatFinishReason.Length)
{
    Console.WriteLine("[Warning] Reply was truncated by the max tokens limit.");
}
Console.WriteLine(completion.Content[0].Text);

using OpenAI.Embeddings;

EmbeddingClient embedder = azureClient.GetEmbeddingClient("text-embedding-3-small");

OpenAIEmbedding a = await embedder.GenerateEmbeddingAsync("How do I read a file in C#?");
OpenAIEmbedding b = await embedder.GenerateEmbeddingAsync("Reading file contents with .NET");

float[] va = a.ToFloats().ToArray();
float[] vb = b.ToFloats().ToArray();

// Cosine similarity: dot product over the product of magnitudes.
double dot = 0, magA = 0, magB = 0;
for (int i = 0; i < va.Length; i++)
{
    dot += va[i] * vb[i];
    magA += va[i] * va[i];
    magB += vb[i] * vb[i];
}
double similarity = dot / (Math.Sqrt(magA) * Math.Sqrt(magB));

Console.WriteLine($"Vector dimensions: {va.Length}");
Console.WriteLine($"Similarity: {similarity:F4}");  // similar meanings score close to 1

using System.ClientModel;
using OpenAI.Chat;

async Task<ChatCompletion> CompleteWithRetryAsync(
    ChatClient chat, ChatMessage[] messages, ChatCompletionOptions options,
    int maxAttempts = 4)
{
    var random = new Random();
    for (int attempt = 1; ; attempt++)
    {
        try
        {
            return await chat.CompleteChatAsync(messages, options);
        }
        catch (ClientResultException ex)
            when ((ex.Status == 429 || ex.Status >= 500) && attempt < maxAttempts)
        {
            // Transient fault: wait 1s, 2s, 4s... plus jitter, then retry.
            int delayMs = (int)(Math.Pow(2, attempt - 1) * 1000) + random.Next(0, 250);
            Console.WriteLine($"Attempt {attempt} got {ex.Status}; retrying in {delayMs} ms...");
            await Task.Delay(delayMs);
        }
        // 401/404/400 fall through and throw immediately — retrying cannot fix them.
    }
}

12 Step by Step: A Console App That Sends a Prompt and Displays the Response

This walkthrough builds the tutorial's deliverable from an empty folder: a console app that asks the user for a prompt, calls your chat deployment, and prints the reply — with safe configuration and a retry loop.

1. Create the project: run 'dotnet new console -n AskAzureOpenAI' and 'cd AskAzureOpenAI'.
2. Add packages from NuGet: 'dotnet add package Azure.AI.OpenAI' and 'dotnet add package Azure.Identity'.
3. Store your secrets outside the code. Enable the store with 'dotnet user-secrets init', then run 'dotnet user-secrets set AzureOpenAI:Endpoint https://<your-resource>.openai.azure.com' and 'dotnet user-secrets set AzureOpenAI:Key <your-api-key>'. (Alternatively, set the two environment variables from Example 1.)
4. Load configuration in Program.cs using Microsoft.Extensions.Configuration with AddUserSecrets and AddEnvironmentVariables so either source works.
5. Create the client: new AzureOpenAIClient(new Uri(endpoint), new AzureKeyCredential(key)), then GetChatClient with your deployment name.
6. Read the user's prompt with Console.ReadLine(), build a SystemChatMessage plus a UserChatMessage, and set ChatCompletionOptions with Temperature = 0.3f and MaxOutputTokenCount = 400.
7. Send the request through the CompleteWithRetryAsync helper from Example 4 so a 429 rate limit becomes a short wait instead of a crash.
8. Display the response text, and if the finish reason is Length, warn that the answer hit the max tokens cap.
9. Run it with 'dotnet run', type a question such as 'What is a C# record type?', and confirm the model's answer prints.
10. Verify safety: check that 'git status' shows no file containing your key, and that deleting the environment variables plus user secrets makes the app fail fast with a clear error message.

using Azure;
using Azure.AI.OpenAI;
using Microsoft.Extensions.Configuration;
using OpenAI.Chat;

var config = new ConfigurationBuilder()
    .AddUserSecrets<Program>()          // dev-time secrets, outside the repo
    .AddEnvironmentVariables()          // or environment variables in CI / prod
    .Build();

string endpoint = config["AzureOpenAI:Endpoint"] ?? config["AZURE_OPENAI_ENDPOINT"]
    ?? throw new InvalidOperationException("No endpoint configured.");
string key = config["AzureOpenAI:Key"] ?? config["AZURE_OPENAI_KEY"]
    ?? throw new InvalidOperationException("No API key configured.");

var azureClient = new AzureOpenAIClient(new Uri(endpoint), new AzureKeyCredential(key));
ChatClient chat = azureClient.GetChatClient("gpt-4o-mini");

Console.Write("Ask me anything: ");
string question = Console.ReadLine() ?? string.Empty;

var options = new ChatCompletionOptions { Temperature = 0.3f, MaxOutputTokenCount = 400 };
var messages = new ChatMessage[]
{
    new SystemChatMessage("You are a helpful assistant for .NET developers."),
    new UserChatMessage(question)
};

ChatCompletion reply = await CompleteWithRetryAsync(chat, messages, options);

Console.WriteLine();
Console.WriteLine(reply.Content[0].Text);
if (reply.FinishReason == ChatFinishReason.Length)
    Console.WriteLine("[Note] Answer truncated — raise MaxOutputTokenCount for longer replies.");

13 Limitations and Caveats

• SDK surface changes between versions. The examples target the Azure.AI.OpenAI 2.x line (AzureOpenAIClient, GetChatClient, CompleteChatAsync, MaxOutputTokenCount, ClientResultException). Exact type, method, and property names differ in the older 1.x SDK and may evolve — treat the examples as correct in spirit and check the package's current documentation if a name does not compile.
• The chat completion API is stateless: no conversation memory exists unless you resend history, and resent history is billed as prompt tokens on every call.
• Max tokens limits output only; it does not protect you from oversized prompts. The model's total context window bounds prompt plus response together.
• Embedding vectors from different models (or different versions of a model) are not comparable — switching embedding deployments means re-embedding your whole corpus.
• Retries multiply cost and latency: a request that consumed tokens before failing may still have counted against your quota. Cap attempts (3–5) and never retry 400/401/404.
• Content filtering can block a prompt or a completion; your code must handle a filtered result as a normal outcome, not an exception path only.
• Environment variables and user secrets are development conveniences, not production security. Production apps should use Managed Identity or Key Vault.
• Deployment names in this tutorial (gpt-4o-mini, text-embedding-3-small) are examples — yours are whatever you named them in the portal.

14 Best Practices

• Create one client per application (it is thread-safe) rather than one per request — register it as a singleton in dependency injection.
• Always send a system message; an unanchored model drifts. Keep it short, specific, and version it with your code.
• Set temperature and max tokens explicitly for every call path — defaults tuned for chat demos are rarely right for production tasks.
• Check the finish reason on every completion and handle 'length' and content-filter outcomes deliberately.
• Log token usage (prompt and completion counts) from day one; it is your cost dashboard and your early warning for rate limit pressure.
• Retry only transient faults (429, 5xx) with exponential backoff plus jitter and a bounded attempt count; fail fast on everything else.
• Keep credentials in the environment, user secrets, Key Vault, or best of all eliminate them with Managed Identity — and rotate any long-lived API key on a schedule.
• Validate configuration at startup and fail with a clear message, rather than surfacing a confusing 401 deep inside a user request.

20 Summary

• One client (AzureOpenAIClient from the Azure.AI.OpenAI NuGet package), built from an endpoint plus a credential, hands out chat and embedding sub-clients bound to deployment names.
• The chat completion API generates text from role-tagged messages and is stateless — your app owns conversation memory and pays for resent history in prompt tokens.
• The embeddings API turns text into a vector; cosine similarity between vectors powers semantic search, and vectors are only comparable within one embedding model.
• Temperature tunes randomness (low for code and facts, high for creativity); max tokens caps output length and truncates with finish reason 'length' when hit.
• Retry only transient faults — 429 rate limit and 5xx — using exponential backoff with jitter and a bounded attempt count; fail fast on 400/401/404.
• Keep the endpoint and API key in an environment variable or user secrets during development, and prefer Managed Identity in production so no key exists at all.

You now have the complete request path from C# code to a model deployment and back, including the two parameters you will tune most and the failure modes you will meet first. The console app you built — configuration, client, prompt, retry, display — is the skeleton every later project in this course grows from. Keep the discipline you practiced here: explicit parameters, checked finish reasons, logged token usage, and secrets that never touch the repository.

21 Next Steps

Next tutorial: Function Calling and Tools (function-calling-and-tools). So far the model can only answer with text; function calling lets it request that your C# code run a method — check an order, query a database, call an API — and then use the result in its reply. That turns the model from a text generator into the reasoning core of an application, and it is the doorway to agents later in the course.

• Practice: extend the console app into a multi-turn chat loop that resends history, then add a '/reset' command that clears it — watch prompt token usage grow and shrink in your logs.
• Practice: embed ten of your own FAQ entries, then embed three test questions and rank the FAQs by cosine similarity — verify the best match is the right one.
• Practice: temporarily set MaxOutputTokenCount to 30, observe the 'length' finish reason, and make the app print a clear truncation warning.
• Practice: switch the console app from AzureKeyCredential to DefaultAzureCredential and authenticate with your developer identity instead of a key.
• Read: the official 'Azure OpenAI Service documentation' quickstarts for C#, the 'Azure SDK for .NET' guidance on retries, and the 'Microsoft identity platform' pages on Managed Identity.