1 Overview: Teaching the Model What It Doesn't Know

A model knows only what it saw in training. It has never read your product manuals, your internal policies, last quarter's report, or the ticket a customer opened yesterday — and when asked about them it will often produce a confident hallucination rather than admit ignorance. RAG, Retrieval-Augmented Generation, closes that gap without retraining anything. The idea is simple and powerful: before the model answers, retrieve the most relevant passages from your own data and put them into the prompt, then instruct the model to answer from that supplied context and cite it. The model's fluency meets your organization's facts.

This tutorial builds the whole pipeline in .NET. You will see why enterprises reach for RAG over fine-tuning; how a vector database and embeddings make retrieval by meaning possible; how Azure AI Search provides that retrieval as a managed service; how to create an index and define its fields and schema; the difference between keyword and semantic search and why hybrid usually wins; how to integrate search results into a RAG pipeline behind the same IChatService from earlier tutorials; how to enhance prompts with retrieved context; and how to enforce grounding and return citations so answers are trustworthy and verifiable.

2 Learning Objectives

• Explain why enterprises need RAG and when it beats fine-tuning or a bigger context window.
• Describe how a vector database, embeddings, and vector search retrieve passages by meaning.
• Use Azure AI Search as a retrieval backend, creating an index and defining its fields and schema.
• Contrast keyword search, semantic search, and hybrid search, and choose the right mode.
• Integrate search results into a RAG pipeline in .NET, retrieving context before calling the model.
• Enhance prompts with retrieved context and a grounding system prompt so the model answers only from sources.
• Return grounding and citations so users can trace and verify every claim in an answer.

3 Prerequisites

• Tutorial 11: the embeddings API — turning text into vectors and comparing them with cosine similarity.
• Tutorials 13–14: an IChatService behind DI with resilience and logging, exposed over a streaming web endpoint.
• C# comfort with async/await, records, and calling an Azure SDK client (you will add the Azure AI Search client).
• An Azure OpenAI resource with a chat deployment and an embedding deployment (text-embedding-3-small), plus an Azure AI Search service to create an index in.

4 Key Concepts: Retrieve, Augment, Generate

RAG is three steps named in its expansion. Retrieve: given the user's question, find the most relevant passages from your indexed data. Augment: insert those passages into the prompt as context, alongside a system prompt telling the model to answer only from them. Generate: the model produces an answer grounded in the supplied passages, with citations back to the sources. Everything in this tutorial is an elaboration of those three steps and the machinery that makes retrieval fast and accurate.

Two ingredients from earlier tutorials power retrieval. Embeddings turn both your documents and each question into vectors, so similarity of meaning becomes distance in vector space; a vector database indexes those vectors and answers nearest-neighbor queries quickly. But retrieval by meaning alone misses exact terms — a product code, an error number — which is where keyword search and the combination called hybrid search come in. Azure AI Search bundles all of it, so most of your work is preparing data well (chunking, schema) and constructing the augmented prompt carefully.

5 Deep Dive 1: Why Enterprises Need RAG

Enterprises need answers grounded in private, current, verifiable knowledge, and a raw model provides none of those. Its training data is frozen at a cutoff, so it cannot know this morning's policy change. It never saw your internal documents at all. And it cannot show its work — an ungrounded answer is a claim with no source, unusable where being right and being checkable both matter. RAG addresses all three: it injects current, private passages at question time, and it makes every answer traceable through citations.

The natural question is why not fine-tune the model on company data instead. Fine-tuning changes a model's style and behavior, but it is a poor and expensive way to inject facts: it must be redone whenever the data changes, it blurs rather than stores exact information (so it still hallucinates specifics and cannot cite), and it risks memorizing sensitive data into weights. RAG keeps knowledge in an index you can update in seconds, add and remove documents from, apply access control to, and cite precisely. As a rule: fine-tune to change how the model behaves; use RAG to change what it knows.

6 Deep Dive 2: Vector Databases and Retrieval by Meaning

Keyword systems match words; RAG needs to match meaning, because a user asking 'how do I get my money back?' should find a passage titled 'Refund Policy' that shares not one keyword with the question. Embeddings make this possible: the embedding API maps text to a vector such that similar meanings produce nearby vectors. Embed the refund policy and embed the question, and their vectors sit close together even with no shared words. Closeness is measured by cosine similarity — the cosine of the angle between vectors, near 1 for very similar meaning.

A vector database is the store built to exploit this. It indexes embedding vectors and answers the query 'give me the k nearest vectors to this one' efficiently — the operation called vector search — even across millions of vectors, using approximate nearest-neighbor algorithms so it stays fast at scale. Conceptually you could compute cosine similarity against every stored vector yourself (as in tutorial 11), but that is linear and collapses at scale; a vector database is what makes retrieval by meaning practical in production.

7 Deep Dive 3: Azure AI Search — Index, Fields, and Schema

You could run a vector database yourself, but Azure AI Search provides retrieval as a managed service that does keyword, vector, and hybrid search plus a semantic ranker in one place — which is why it is a common RAG backend. Its central object is the index: a searchable collection of documents. Before you add anything you define the index's schema — the set of fields, each with a name, a type, and attributes controlling how it behaves. Getting the schema right is most of the setup work.

A field's attributes decide its role. Searchable makes a text field full-text searchable (keyword search). A field typed as a collection of singles with a vector-search configuration holds an embedding for vector search. Filterable lets you restrict results (by category, or by an allowed-users field for access control). Retrievable means the value is returned with results — your content and any source id for citations must be retrievable. Key marks the unique document id. A typical RAG index carries at least: an id (key), the chunk content (searchable, retrievable), a content vector (vector-searchable), a title or source path (retrievable, for citations), and often filter fields.

using Azure.Search.Documents.Indexes;
using Azure.Search.Documents.Indexes.Models;

// One searchable document = one chunk of a source file.
var index = new SearchIndex("knowledge-index")
{
    Fields =
    {
        new SimpleField("id", SearchFieldDataType.String) { IsKey = true },
        new SearchableField("content") { IsFilterable = false },   // full-text keyword search
        new SearchableField("title") { IsFilterable = true },       // shown in citations, filterable
        new SimpleField("sourcePath", SearchFieldDataType.String) { IsFilterable = true }, // citation link
        new VectorSearchField("contentVector", 1536, "vector-profile")  // embedding for vector search
    },
    // Vector search + semantic ranking configuration attach to the index.
    VectorSearch = new() { /* HNSW algorithm + profile named 'vector-profile' */ },
    SemanticSearch = new() { /* semantic configuration naming title + content */ }
};

var indexClient = new SearchIndexClient(endpoint, credential);
await indexClient.CreateOrUpdateIndexAsync(index);

Before indexing you must chunk your documents — split them into passage-sized pieces (a few hundred tokens, often with slight overlap) — and embed each chunk. Chunking matters because retrieval returns whole documents' worth of noise if chunks are too big, and loses context if too small; passage-sized chunks give focused, citable context that fits the prompt budget. Each chunk becomes one index document: its text in content, its embedding in contentVector, and its origin in title/sourcePath so the answer can cite it.

8 Deep Dive 4: Keyword vs Semantic Search — and Hybrid

Retrieval has two complementary strategies, and understanding when each wins is central to good RAG. Keyword search (lexical) matches the actual terms of the query against the indexed text, ranked by algorithms such as BM25. It is unbeatable for exact strings — a product code 'RX-4400', an error like 'ORA-00942', a person's name, an acronym — because those must match literally, and it needs no embedding. Its weakness is vocabulary mismatch: 'get my money back' and 'refund' share no words, so keyword search misses the connection.

Semantic search retrieves by meaning. In Azure AI Search this means vector search over embeddings, optionally sharpened by the semantic ranker, which re-scores the top candidates with a language model and can return concise captions. Semantic search handles paraphrase and synonym effortlessly — the refund case just works — but can underperform on exact tokens, sometimes returning a passage that is topically close yet misses the specific code the user typed. The strengths are mirror images, which points straight at the answer: use both.

Hybrid search — issuing keyword and vector search together and fusing the results, then optionally applying the semantic ranker — is the recommended default for RAG because real user questions mix exact terms and natural language. Azure AI Search performs the fusion for you; your job is to supply both the query text and the query embedding and enable the semantic configuration. This is the retrieval quality that sets your RAG system's ceiling, so it is worth tuning.

9 Ecosystem and Tools

You can build the pipeline at two levels. Hand-rolled, you chunk and embed documents in your own code, upload them to the index, and at query time embed the question, run a hybrid query, and construct the prompt — full control, and the version this tutorial teaches so the mechanics are visible. Higher up, Azure AI Search's integrated vectorization and indexers can chunk and embed for you during ingestion, and orchestration frameworks wrap retrieval as reusable 'memory'. Understand the hand-rolled pipeline first; the conveniences then read as shortcuts over parts you already know.

10 Use Cases

• Internal knowledge assistant: employees ask natural-language questions and get answers grounded in policies, wikis, and runbooks, each with a citation to the source page.
• Customer support deflection: a help widget answers from the product documentation and knowledge base, citing articles so users can read more.
• Contract and document Q&A: upload a set of contracts and ask 'what is the termination notice period?', getting the exact clause with a citation.
• Technical support with exact codes: hybrid search shines when a user pastes an error code and describes the symptom in words — keyword catches the code, vector catches the description.
• Onboarding helper: new staff query scattered documentation through one assistant instead of hunting across systems.
• Research assistant over a report library: ask across many documents and get a synthesized, cited answer rather than a list of links.
• Compliance-aware search: retrieval filtered by the user's permissions so answers only ever draw on documents they are allowed to see.

The common thread: a body of private or specialized text, natural-language questions over it, and a hard requirement that answers be correct and verifiable. Wherever those three meet, RAG is the fitting pattern — and the citation requirement is usually what rules out an ungrounded model.

11 Code Examples

These examples build the query-time RAG pipeline in .NET: embed the question, run a hybrid search against Azure AI Search, construct a grounded prompt, and generate a cited answer — all inside the IChatService from earlier tutorials. Indexing (schema in deep dive 3, plus chunk-embed-upload) is assumed done.

using Azure.Search.Documents;
using Azure.Search.Documents.Models;

public sealed record Passage(string Content, string Title, string SourcePath);

// Embed the question, then run keyword + vector search in one hybrid query.
async Task<IReadOnlyList<Passage>> RetrieveAsync(string question, int k, CancellationToken ct)
{
    // Same embedding model used when indexing (deep dive 2's rule).
    OpenAIEmbedding qEmbedding =
        await _embedder.GenerateEmbeddingAsync(question, cancellationToken: ct);

    var options = new SearchOptions
    {
        Size = k,
        VectorSearch = new()
        {
            Queries = { new VectorizedQuery(qEmbedding.ToFloats())
                        { KNearestNeighborsCount = k, Fields = { "contentVector" } } }
        },
        // Enable semantic ranking over the fused hybrid results.
        QueryType = SearchQueryType.Semantic,
        SemanticSearch = new() { SemanticConfigurationName = "default-semantic" }
    };
    options.Select.Add("content");
    options.Select.Add("title");
    options.Select.Add("sourcePath");

    // Passing both the text (question) and the vector query makes it hybrid.
    SearchResults<SearchDocument> results =
        await _search.SearchAsync<SearchDocument>(question, options, ct);

    var passages = new List<Passage>();
    await foreach (SearchResult<SearchDocument> r in results.GetResultsAsync())
    {
        passages.Add(new Passage(
            (string)r.Document["content"],
            (string)r.Document["title"],
            (string)r.Document["sourcePath"]));
    }
    return passages;
}

// Number each passage so the model can cite it as [1], [2], ...
static (string context, IReadOnlyList<Passage> sources) BuildContext(IReadOnlyList<Passage> passages)
{
    var sb = new System.Text.StringBuilder();
    for (int i = 0; i < passages.Count; i++)
    {
        sb.AppendLine($"[{i + 1}] {passages[i].Title}");
        sb.AppendLine(passages[i].Content);
        sb.AppendLine();
    }
    return (sb.ToString(), passages);
}

const string GroundingSystemPrompt =
    "You are a knowledge assistant. Answer ONLY using the numbered sources below. " +
    "Cite the sources you use with their bracket numbers, e.g. [1]. " +
    "If the sources do not contain the answer, say you don't know — do not use outside knowledge.";

public async Task<GroundedAnswer> AskGroundedAsync(string question, CancellationToken ct = default)
{
    // 1. Retrieve
    IReadOnlyList<Passage> passages = await RetrieveAsync(question, k: 5, ct);
    if (passages.Count == 0)
        return new GroundedAnswer("I couldn't find anything relevant to answer that.", Array.Empty<Passage>());

    // 2. Augment
    (string context, var sources) = BuildContext(passages);
    var messages = new ChatMessage[]
    {
        new SystemChatMessage(GroundingSystemPrompt),
        new UserChatMessage($"Sources:\n{context}\nQuestion: {question}")
    };

    // 3. Generate (reusing tutorial 13's resilient call; low temperature for grounding)
    var options = new ChatCompletionOptions { Temperature = 0.1f, MaxOutputTokenCount = 500 };
    ChatCompletion completion = await CompleteResilientlyAsync(messages, options, ct);

    // Return the answer AND the sources so the UI can render citations.
    return new GroundedAnswer(completion.Content[0].Text, sources);
}

public sealed record GroundedAnswer(string Text, IReadOnlyList<Passage> Sources);

12 Step by Step: A Grounded, Cited Assistant

This walkthrough builds an end-to-end RAG assistant: index a small document set, then answer questions from it with citations, streamed into the tutorial-14 page.

1. Provision: an Azure AI Search service plus your existing Azure OpenAI chat and embedding deployments. Add the Azure.Search.Documents package to the project.
2. Create the index: define the schema from deep dive 3 (id key, searchable content, retrievable title and sourcePath, a 1536-dim contentVector, plus vector and semantic configurations) and call CreateOrUpdateIndexAsync.
3. Prepare data: take a handful of documents (policies, FAQs, a manual) and chunk them into passage-sized pieces of a few hundred tokens with slight overlap.
4. Embed and upload: for each chunk, call the embedding deployment, then upload a document per chunk (id, content, title, sourcePath, contentVector) with the index's upload/merge API.
5. Implement retrieval (Example 1): embed the incoming question with the same model and run a hybrid query — text plus vector — with semantic ranking, selecting content, title, and sourcePath.
6. Build the grounded prompt (Example 2): number the retrieved passages, and write a system prompt that forbids outside knowledge and requires bracket citations.
7. Generate (Example 3): call the chat deployment at low temperature through tutorial 13's resilient wrapper, returning both the answer text and the source list as a GroundedAnswer.
8. Expose it: add AskGroundedAsync to IChatService and a web endpoint that returns the answer plus a citations array; render citations under the answer in the tutorial-14 page, linking each [n] to its sourcePath.
9. Test grounding: ask a question whose answer is in your documents and confirm the answer cites the right source; then ask something absent from the documents and confirm the assistant says it doesn't know rather than inventing an answer.
10. Test hybrid retrieval: ask once using an exact code or name from a document and once by paraphrase, and confirm both retrieve the correct passage — keyword catching the code, vector catching the paraphrase.
11. Stream it: switch generation to the streaming path from tutorial 14 so the grounded answer types into the page, then append the citations when the stream completes.

13 Limitations and Caveats

• SDK naming caveat: the Azure.Search.Documents SDK's field and query construction (SearchableField, VectorSearchField, VectorizedQuery, SemanticSearch/VectorSearch configuration objects) varies across versions — the examples are correct in shape and intent, but verify exact type and property names against the current package before relying on them.
• Retrieval sets the ceiling: if the right passage is not retrieved, the answer cannot be correct no matter the model or prompt. Most RAG quality work is retrieval and chunking work, not prompt tweaking.
• Chunking is a real design decision: chunks too large dilute relevance and waste the context window; too small lose the surrounding meaning. Passage-sized with slight overlap is a starting point, not a universal answer.
• Grounding is instructed, not guaranteed: a model told to answer only from context will usually comply, but can still leak training knowledge or miscite. Low temperature, a firm system prompt, and evaluation reduce but do not eliminate this.
• Citations can be wrong: the model may attribute a claim to the wrong bracket number. Treat citations as strong hints to verify, and consider post-checking that cited passages actually support the claim.
• Context window limits how many passages you can include; more retrieval is not always better, and stuffing the prompt raises cost and can bury the relevant passage. Tune k.
• Access control lives in retrieval: you must filter the index by the user's permissions, because the model will quote any context it is given regardless of who is asking.
• Freshness depends on your indexing pipeline: RAG is only as current as the last time you re-indexed changed documents — an ingestion schedule or change feed is part of the system, not an afterthought.
• Embedding-model consistency is mandatory: index and query must share the same embedding model, so changing it forces a full re-embed and re-index.

14 Best Practices

• Default to hybrid search with the semantic ranker; reach for pure keyword or pure vector only when you have measured a reason to.
• Invest in chunking and schema before prompt tuning — retrieval quality dominates RAG outcomes.
• Always ground explicitly: a system prompt that says 'answer only from the sources, cite them, and say you don't know otherwise' — and keep temperature low.
• Return sources with every answer and render citations in the UI; verifiability is usually the whole point of choosing RAG.
• Filter retrieval by the user's permissions so answers never draw on documents the user cannot access.
• Keep index and query on the same embedding model, and version your index so a model change is a deliberate re-index, not a silent regression.
• Log retrieved passages and scores per query; it is the fastest way to tell a retrieval failure from a generation failure.
• Tune k (number of passages) against your context window and cost — enough to cover the answer, few enough to keep the relevant passage prominent.

20 Summary

• RAG retrieves relevant passages from your data, augments the prompt with them, and generates a grounded, cited answer — supplying private, current, verifiable knowledge without retraining. Fine-tune for behavior; use RAG for knowledge.
• Embeddings map text to vectors so meaning becomes distance; a vector database indexes those vectors and returns nearest neighbors fast, enabling retrieval by meaning at scale — with index and query bound to the same embedding model.
• Azure AI Search provides retrieval as a managed service; its index is defined by a field schema (key, searchable content, a vector field, retrievable citation fields, filterable access-control fields), and documents are passage-sized chunks.
• Keyword search nails exact terms, semantic search nails meaning, and hybrid search fuses both with a semantic ranker — the recommended default because real questions mix codes and natural language.
• The .NET pipeline lives behind IChatService: embed the question, run hybrid search, build a grounded prompt with numbered sources, and generate at low temperature — returning the answer plus its sources.
• Grounding (answer only from sources) is enforced by a firm system prompt and low temperature but not guaranteed; citations (references back to passages) make answers verifiable, and retrieval quality — not the model — sets the system's ceiling.

RAG is the workhorse pattern of enterprise GenAI, and you have now built it end to end: retrieval by meaning over an Azure AI Search index, hybrid search for the best of keyword and semantic, and a grounded, cited answer streamed into the web app from earlier tutorials — all behind the same clean interface. The disciplines that make it production-grade are the retrieval ones: chunk well, search hybrid, ground firmly, cite honestly, filter by permission, and keep the index fresh. With grounded answers in hand, the course now turns to the platform that unifies these building blocks, as the next tutorial introduces Azure AI Foundry.

21 Next Steps

Next tutorial: Azure AI Foundry (azure-ai-foundry). You have assembled AI capabilities piece by piece — deployments, embeddings, function calling, and now retrieval. Azure AI Foundry is the unified platform for building, evaluating, and managing these AI applications: model catalog and deployments, playgrounds, evaluation, and the tooling that ties the building blocks of this course into a managed workflow.

• Practice: build the full pipeline — create the index, chunk and embed a small document set, and answer questions with citations, confirming an out-of-scope question yields an honest 'I don't know'.
• Practice: run the same question as an exact code and as a paraphrase, and log the retrieved passages to see keyword and vector each earn their keep in hybrid search.
• Practice: deliberately vary chunk size (very large vs very small) and observe the effect on retrieval relevance and answer quality.
• Practice: add a permission filter to retrieval and verify that a restricted document never appears in answers for an unauthorized user.
• Read: the official documentation for 'Azure AI Search', 'Vector search in Azure AI Search', 'Semantic ranking', 'Retrieval-Augmented Generation (RAG) in Azure AI Search', and the Azure.Search.Documents .NET client library.