Prompting for Testing and Documentation
Prompting for Testing and Documentation
1 Overview
Tests and documentation are the two artifacts developers most consistently under-produce — not because they are hard, but because they are repetitive, and repetition is exactly what language models excel at. A model will happily write the fifteenth boundary test or the twentieth parameter description with the same care as the first. Used well, prompting turns 'we should really have tests for this' into an afternoon's reviewed reality.
Used carelessly, it produces something worse than nothing: test suites that pass while verifying nothing, and documentation that states plausible falsehoods with authority. The difference — as everywhere in this course — is the prompt's context and your review discipline.
This tutorial covers the four skills: generating NUnit unit tests that actually test behavior, systematically eliciting test cases and edge scenarios, generating API documentation from code, and — the load-bearing skill — reviewing generated tests for coverage and correctness so you can trust green.
2 Learning Objectives
After completing this tutorial, you will be able to:
- Write test-generation prompts that produce NUnit suites grounded in the behavior spec, not just the implementation.
- Elicit test cases and edge scenarios systematically — boundaries, empties, duplicates, and failure paths — before any test code exists.
- Use parameterized tests to turn boundary tables into compact [TestCase] rows.
- Generate XML documentation comments, endpoint docs, and READMEs from code, and verify their claims.
- Review generated tests for the failure modes that matter: assertion-free tests, tautological tests, and bug-enshrining characterization.
- Apply the break-the-code check (manual mutation testing) to prove a test can actually fail.
- Read code coverage numbers for what they are — execution evidence, not verification evidence.
- Keep documentation from drifting by regenerating and reviewing it alongside code changes.
3 Prerequisites
- Tutorial 4 (Prompt Engineering for Coding Tasks) — context packs, exemplars, standards profiles, and the review bar.
- Basic familiarity with a .NET test framework (NUnit examples are used; xUnit/MSTest knowledge transfers directly).
- Comfortable C#: attributes, async, exceptions; ability to run `dotnet test`.
- Optional but ideal: a real method of your own that lacks tests — the walkthrough lands best on real code.
4 Why Tests and Docs Are Ideal — and Dangerous — GenAI Tasks
Test and documentation generation share three properties that make them model-friendly: the output is highly structured (a unit test has a fixed shape; a parameter description has a fixed shape), the input is complete (the code is right there in the prompt), and the volume is high (many cases, many members — exactly where human patience fails first). This is why test generation is consistently among the highest-satisfaction uses of AI coding tools.
The danger is equally structural. A test is only as good as its test oracle — the source of truth it checks against. If your prompt offers only the implementation, the model derives expectations from the implementation, and the resulting tests verify that the code does what the code does: green today, green after you break it the same way, tautological forever. The fix is supplying the behavior specification — what the method should do, in rules and examples — so expectations derive from intent, not from code.
Documentation inherits the same split. Docs generated purely from code describe mechanism ('sets the flag to true') and can only restate what the code says; docs generated from code plus intent describe purpose ('marks the invoice ready for the nightly export'). And both artifacts share one lifecycle risk — the code changes and they silently don't — which is why generation is never once-and-done but part of the change workflow.
5 Prompts That Generate NUnit Unit Tests
A production-grade test-generation prompt packs four things: the code under test (the method plus the types it touches), the behavior spec (rules and expected outcomes in plain language — the oracle), an exemplar test (one existing test class, so naming and style match your suite), and the framework constraints (NUnit, one test fixture per class under test, naming convention like MethodName_Scenario_ExpectedResult, arrange-act-assert layout, one behavior per test).
Structure the ask in two turns for anything non-trivial: first 'list the test cases you would write — name, scenario, expected outcome — as a table; no code yet', then, after you've added the cases it missed and cut the redundant ones, 'now implement exactly this list as NUnit tests'. The case list is reviewable in a minute and turns test generation into spec review — the highest-value minute in this whole workflow.
[TestFixture]
public class DiscountCalculatorTests
{
[Test]
public void Calculate_OrderBelowThreshold_ReturnsZeroDiscount()
{
// Arrange
var calculator = new DiscountCalculator(threshold: 100m, rate: 0.1m);
// Act
decimal discount = calculator.Calculate(orderTotal: 99.99m);
// Assert
Assert.That(discount, Is.EqualTo(0m));
}
[TestCase(-1)]
[TestCase(-100.50)]
public void Calculate_NegativeTotal_ThrowsArgumentOutOfRange(decimal total)
{
var calculator = new DiscountCalculator(threshold: 100m, rate: 0.1m);
Assert.Throws<ArgumentOutOfRangeException>(() => calculator.Calculate(total));
}
}
6 Generating Test Cases and Edge Scenarios
Models are superb edge-case brainstormers — they have seen every category of input that has ever broken code. The technique is to ask for scenarios before tests, systematically: 'For this method, list test cases in these categories: happy paths, boundary values for every numeric/size rule, empty/null/whitespace inputs, duplicates and ordering, culture and encoding, concurrency if applicable, and failure paths for each dependency. Table format: case name, input, expected outcome.'
Boundary value analysis deserves explicit demand. For every rule with a limit — 'age must be 18 to 65' — the cases are the values on each side of each edge: 17, 18, 65, 66 (plus the extremes: 0, negative, int.MaxValue). Prompted for a boundary table, the model produces it mechanically and completely; unprompted, it samples a few boundaries and moves on. Boundary tables then map one-to-one onto NUnit parameterized tests — one test method, one [TestCase] row per boundary — keeping the suite compact.
| Category | Prompt cue | Example cases for ParseQuantity(string s) → int (1–999) |
|---|---|---|
| Happy path | typical valid inputs | "1" → 1, "500" → 500, "999" → 999 |
| Boundaries | each side of every limit | "0" → throws, "1" → 1, "999" → 999, "1000" → throws |
| Empty/null | the absent inputs | null → throws ArgumentNullException, "" → throws, " " → throws |
| Format | almost-valid shapes | "12.5" → throws, "+12" → ?, " 12 " → ? (spec decides!) |
| Culture/encoding | locale-sensitive parses | "١٢" (Arabic digits) → spec decides, "1٢" mixed → throws |
| Hostile | adversarial inputs | very long string, "999999999999999999999" → throws, not overflows |
Notice the two 'spec decides!' cells — this is the second gift of case generation. When the model lists a case and you realize you don't know the expected outcome, you have found a hole in the specification, before the code ships. Feed the answer back into both the spec and the tests. Edge-scenario prompting is requirements discovery wearing a testing costume.
7 Generating API Documentation from Code
Documentation generation runs on the same grounding rule as everything else: code alone yields mechanical descriptions; code plus intent yields useful ones. The workable prompt shape: paste the type or endpoint, state the audience ('consumers of our NuGet package', 'frontend developers calling this API'), supply the intent context the code can't carry (units, invariants, why parameters exist), and fix the format (XML documentation comments, an OpenAPI-style endpoint description, or a README section).
- XML documentation comments: 'Write /// docs for every public member: summary (purpose, not mechanism), param (meaning, units, valid range), returns, exception (each condition), one example for non-obvious members.' These feed IntelliSense and doc generators, so they compound.
- Endpoint documentation: paste the controller action plus DTOs; ask for description, parameter table, status codes with conditions, and a realistic request/response example pair. Feed the result into your OpenAPI annotations so rendered docs and code share one source.
- README sections: give the model the public surface plus two working usage snippets and ask for a quick-start — models write excellent 'first five minutes' docs when given real, runnable examples to build around.
- Explanatory docs from tests: pasting the test suite alongside the code is a power move — well-named tests are executable examples of intended behavior, and the docs inherit their precision.
Verification for docs means checking claims, not prose quality: every stated default, unit, range, exception condition, and status code must be confirmed against the code or a run. Models fluently document what code plausibly does, including behaviors it doesn't have. And schedule for doc drift: regenerate member docs in the same pull request that changes the member — a diff-sized doc update reviews in seconds, while 'update all the docs someday' never happens.
8 Reviewing Generated Tests for Coverage and Correctness
Generated tests fail in ways human tests rarely do, because the model optimizes for looking like a test suite. The review targets three specific pathologies. Assertion-free tests: code runs, nothing meaningful is asserted (or only 'is not null') — coverage rises, verification doesn't. Tautological tests: expectations derived from the implementation, so the test proves the code equals itself — including its bugs. Bug-enshrining characterization: when generating tests for existing code, current wrong behavior gets pinned as 'expected'; that's a feature for refactoring safety but a trap when you believe the suite validates correctness.
The coverage review is a different axis: not 'are these tests honest' but 'what behavior has no test at all'. Read the test names against your case list from section 6 — every category (boundaries, empties, failure paths) should be represented. A code coverage tool then finds unexecuted branches, but read its number correctly: coverage proves execution, not verification. An assertion-free suite can hit 95% coverage while catching zero bugs. Coverage tells you where tests are missing; only the honesty check tells you whether the existing ones matter.
9 The .NET Testing and Docs Toolchain Around Your Prompts
- Test frameworks: NUnit (used here — [TestFixture], [Test], [TestCase], Assert.That), xUnit, MSTest. Every prompt shape in this tutorial transfers; only attributes and assertion syntax change — say which framework in every prompt.
- Runner: `dotnet test` executes the generated suite; it is the first honesty gate (does it compile, does it pass) and the sabotage-check runner.
- Coverage: Coverlet + report generators produce the line/branch numbers — the where-are-tests-missing map, never the quality score.
- GitHub Copilot: generates tests inline from open code (tutorials 8–9); the review discipline here applies unchanged to its output.
- XML docs pipeline: /// comments → compiler XML file → IntelliSense, DocFX sites, and NuGet package docs — one generated artifact, three surfaces.
- OpenAPI pipeline: ASP.NET Core annotations + Swashbuckle render endpoint docs; generated endpoint descriptions belong in the annotations so docs live with code.
- CI: run tests and (optionally) coverage thresholds on every PR — generated tests enter the same gate as human ones.
The toolchain's role mirrors tutorial 4's: prompts produce candidates; deterministic tools judge them. `dotnet test` proves the suite runs; your sabotage check proves it verifies; coverage maps the gaps; CI keeps all three honest over time.
10 The Testing & Docs Task Catalog
| Task | Prompt shape | Non-negotiable ingredient |
|---|---|---|
| New method needs tests | Two-turn: case list first, then NUnit implementation of the approved list | Behavior spec as the oracle |
| Legacy class, no tests | Characterization tests: 'pin current behavior, flag anything that looks like a bug rather than intent' | Knowing these prove stability, not correctness |
| Bug just fixed | Regression test: failing input + fixed expectation, named after the bug | Test fails on the pre-fix code |
| Boundary-heavy validation | Boundary table → [TestCase] parameterized test | Every limit's both sides |
| Public API surface | XML documentation comments per member, purpose-not-mechanism | Claims verified against code |
| HTTP endpoints | Description + parameter table + status codes + example pair → OpenAPI annotations | Realistic examples that actually run |
| Package/README | Quick-start from public surface + two working snippets | Snippets compile and run |
| PR with code changes | Regenerate docs for changed members in the same PR | Diff-sized doc review |
One composite workflow ties the catalog together: when fixing a bug, generate the regression test first from the bug report (it must fail), fix the code (test goes green), then regenerate the affected member's docs — three generated artifacts, each verified by the one before it.
11 Code Examples in C#
The test-generation prompt template, packing oracle, style, and framework constraints:
public static class TestPrompts
{
// Turn 1: cases only — reviewed before any code exists.
public static string BuildCaseListPrompt(string codeUnderTest, string behaviorSpec) => $"""
List the test cases you would write for the method below.
Categories: happy paths, boundary values (both sides of every limit),
empty/null/whitespace, format edge cases, failure paths per dependency.
Output a table: CaseName | Input | ExpectedOutcome. NO test code yet.
Behavior specification (source of truth for expectations):
{behaviorSpec}
Code under test:
```csharp
{codeUnderTest}
```
""";
// Turn 2: implement the human-approved list.
public static string BuildTestImplPrompt(string approvedCases, string exemplarTest) => $"""
Implement EXACTLY these approved test cases as NUnit tests:
{approvedCases}
Rules: [TestFixture] class; MethodName_Scenario_ExpectedResult names;
arrange-act-assert with comments; one behavior per [Test];
use [TestCase] rows for the boundary table; Assert.That syntax.
Match the style of this existing test class:
```csharp
{exemplarTest}
```
""";
}
The documentation prompt for a public member, with intent the code cannot carry:
public static string BuildXmlDocPrompt(string publicMembers, string intentNotes) => $"""
Write XML documentation comments (///) for each public member below.
- <summary>: the PURPOSE (why callers use it), not the mechanism.
- <param>: meaning, units, valid range.
- <returns> and <exception> for every documented throw condition.
- <example> for any member whose use is not obvious.
Document ONLY behavior visible in the code or stated in the notes —
do not invent defaults, limits, or thread-safety claims.
Intent notes (context the code cannot express):
{intentNotes}
Members:
```csharp
{publicMembers}
```
""";
12 Step-by-Step: Test-Covering a Real Method
This walkthrough covers an untested `ParseQuantity(string input) → int` (valid range 1–999) with a trustworthy NUnit suite. Replay it on any method you own.
- Write the behavior spec in five lines: valid range 1–999; leading/trailing whitespace tolerated; null/empty/non-numeric throws ArgumentException; out-of-range throws ArgumentOutOfRangeException; no culture-specific digits accepted.
- Turn 1 — request the case list (template from section 11): the model returns ~18 cases across the categories. Review it: it missed the mixed-digit case ('1٢') and duplicated two happy paths. Add one, cut one.
- Notice the spec holes the list exposed: what about '+12'? The spec didn't say. Decide (reject it), update the spec, add the case — requirements discovery, on schedule.
- Turn 2 — request implementation of the approved list with your exemplar test class pasted; receive the [TestFixture] with parameterized boundary rows.
- Run `dotnet test`: 17 pass, 1 fails — the whitespace case. Investigate: the test is right, the CODE never trimmed input. You just found a real bug with a generated test; fix the code, not the test.
- Honesty check: sabotage the range check (999 → 998) — boundary tests go red ✓; remove the null guard — null test goes red ✓. Restore. The suite has earned its green.
- Coverage pass: Coverlet shows the culture-digit branch unexecuted — the model implemented the case but a copy-paste error asserted on the wrong input. One targeted Improve turn fixes it.
- Finish the artifact set: generate XML documentation comments for ParseQuantity from the now-verified spec — the docs and tests now tell the same story.
13 Limitations and Caveats
- Implementation-derived expectations are the root failure: given only code, the model writes tests proving code == code. Always supply a behavior spec; when generating from legacy code deliberately (characterization), label the suite as stability protection, not correctness validation.
- Assertion quality varies: watch for 'Is.Not.Null' where a value check belongs, and for asserting on the mock instead of the outcome. The sabotage check catches both.
- Coverage numbers flatter generated suites: high execution with weak assertions is the signature pathology. Coverage locates missing tests; it cannot score existing ones.
- Async, time, and randomness need explicit prompting: demand deterministic tests — injected clocks, seeded randomness, awaited assertions — or you'll inherit flaky tests that erode suite trust.
- Mocking depth is a design signal: if generated tests need five mocks per test, the model is faithfully reflecting your coupling — fix the design, don't blame the tests.
- Docs hallucinate specifics: invented defaults, ranges, and thread-safety claims read exactly like real ones. Verify every stated fact; keep the 'do not invent' clause in doc prompts.
- Doc drift resumes the moment you stop: regenerate member docs in the same PR that changes the member, or accept that your docs describe a previous version.
14 Best Practices and Common Mistakes
Practices that make generated tests and docs trustworthy:
- Spec before suite: the behavior specification is the test oracle — write it first, even five lines of it.
- Two turns, always: case list → human review → implementation. The list review is requirements discovery.
- Demand the boundary table for every limit, and implement it as [TestCase] rows.
- Sabotage twice before trusting: a test earns green the day you watch it fail.
- Name tests MethodName_Scenario_ExpectedResult so the name list doubles as the coverage review.
- Docs state purpose, not mechanism; every claimed default/range/exception is verified; 'do not invent' stays in the prompt.
- Regenerate docs and add regression tests in the same PR as the change they concern.
Mistakes that produce theatrical green:
- Generating tests from implementation only — the suite becomes a mirror, bugs included.
- Accepting a 40-test suite wholesale because it's long — volume is the model's cheapest trick.
- Reading 95% coverage as 95% verified — coverage measures execution.
- Keeping a test that never goes red under sabotage 'because it documents intent' — it documents nothing.
- Letting characterization tests of legacy bugs masquerade as correctness validation.
- Publishing generated docs without fact-checking the specifics.
- Treating tests and docs as generation targets once, instead of regeneration targets on every change.
20 Summary & Key Takeaways
- Tests and docs are the model-friendliest artifacts in development — structured, grounded, voluminous — and the easiest to generate convincingly wrong.
- The oracle decides everything: supply a behavior spec, or the model derives expectations from the implementation and certifies its bugs.
- Two turns always: case list → human review (which doubles as requirements discovery) → implementation of the approved list.
- Demand boundary tables for every limit and implement them as [TestCase] rows; ask for edge scenarios by category, not by vibe.
- Green is earned, not observed: read assertions against the spec and sabotage the code until the right tests go red.
- Coverage maps missing tests; it cannot score existing ones — an assertion-free suite posts 95% while verifying nothing.
- Docs need intent the code can't carry, a 'do not invent' clause, fact-checking of every specific, and regeneration bound to the changing PR.
- Regression tests must fail before the fix; characterization suites must be labeled for what they are; flakiness constraints go in the prompt.
You can now direct a model to produce the two artifacts teams chronically lack — and, more importantly, you know exactly how to make their green trustworthy. Next, the course turns to reliability and safety in prompting itself: defending against injection, managing failure modes, and building prompting practices that hold up in production.
21 Next Steps
Continue with the next tutorial in the path: Reliability, Safety & Best Practices in Prompting — prompt injection and defenses, failure-mode handling, output validation at scale, and the operational discipline that makes prompting production-grade.
- Practice: pick one untested method you own and run the full section-12 walkthrough — spec, case list, implementation, sabotage. Count what the workflow catches.
- Practice: take an existing generated (or inherited) test suite and run the 15-minute review — names, assertions, sabotage, coverage. Report to yourself how much of its green was earned.
- Practice: generate XML documentation comments for one public class using the intent-notes template, then fact-check every specific claim it made.
- Practice: on your next bug fix, generate the regression test first and verify it fails before fixing.
- Reading: the NUnit documentation ([TestCase], Assert.That), Coverlet for coverage, and Stryker.NET if you want automated mutation testing.
15 Quiz
Pick an answer for each question, then press Check answer. (Notes are disabled in this tab.)
1. Why are testing and documentation especially good fits for model generation?
2. What is a test oracle, and why does it decide the value of generated tests?
3. Which NUnit attribute marks a class containing tests?
4. What is the natural NUnit tool for implementing a boundary-value table?
5. Why request a test-case list BEFORE any test code?
6. For the rule 'quantity must be 1–999', which is the correct minimal boundary set?
7. What is an assertion-free test?
8. What is a tautological test?
9. What is the 'break the code' honesty check?
10. A generated suite reaches 95% code coverage. What do you now know?
11. When generating tests for legacy code with no spec, what are you actually creating?
12. What's the key grounding difference between mediocre and useful generated docs?
13. The riskiest defect in generated API documentation is:
14. What is the best defense against doc drift?
15. You fixed a bug. What's the correct generated-artifact sequence?
16 Exam Questions
Try answering each question yourself before expanding the model answer.
1. Explain why test generation is simultaneously one of the best and one of the riskiest uses of code-generation models.
2. Define 'test oracle' and trace how oracle choice determines whether a generated suite catches or reproduces bugs, with an example.
3. Specify the complete context pack for an NUnit test-generation prompt and justify each element.
4. Describe the two-turn (case list → implementation) protocol and argue why the intermediate review is 'the highest-value minute in the workflow'.
5. Apply boundary value analysis to 'password length must be 8–64 characters' — derive the full case set and show how it becomes NUnit code.
6. Contrast the three generated-test pathologies — assertion-free, tautological, and bug-enshrining characterization — including detection for each.
7. Explain manual mutation testing (the sabotage check): protocol, what each outcome means, and its relation to tooling.
8. Argue precisely what code coverage does and does not tell you about a generated suite, and derive the correct workflow use of it.
9. Design the documentation-generation approach for a public NuGet library: inputs, prompt constraints, verification, and pipeline integration.
10. Why is 'docs from code + tests' stronger than 'docs from code alone'? Explain the mechanism.
11. A generated suite of 40 tests all pass on first run. Give a complete 15-minute review plan with the rationale for each step.
12. How should async code, time, and randomness be handled in generated tests? What must the prompt demand?
13. Explain the regression-test workflow for a bug fix and why the pre-fix failure is non-negotiable.
14. Your team adopted AI test generation; coverage rose to 90% but production bugs didn't fall. Diagnose systematically and prescribe.
15. Scenario: an untested OrderTotalCalculator (money, discounts, tax rounding) must be test-covered and documented this sprint using AI assistance. Prescribe the complete workflow.
17 Flashcards
Click a card to reveal the back.
Why tests/docs suit model generation
Test oracle
Test-gen context pack
Two-turn protocol
Boundary value analysis
'Spec decides!' moments
Assertion-free test
Tautological test
Sabotage (honesty) check
Code coverage — honest reading
Characterization tests
NUnit essentials
Docs grounding rule
Doc drift defense
Regression test rule
15-minute suite review
18 Interview Questions & Answers
1. How do you use AI for unit testing without ending up with a suite that verifies nothing?
2. Walk me through your actual workflow for getting tests generated for a method.
3. What's the most common mistake teams make with AI test generation?
4. How do you review a generated test suite efficiently?
5. Explain boundary value analysis and how you'd get a model to do it properly.
6. What is mutation testing and do you actually use it?
7. How much do you trust code coverage as a metric?
8. You're asked to generate tests for a legacy class with no documentation. What do you produce and how do you label it?
9. How do you keep generated tests from being flaky?
10. What makes AI-generated documentation good instead of mechanical?
11. How do you stop documentation going stale?
12. A bug was just fixed. What's your artifact protocol?
13. Tests from code, or docs from code — which benefits more from including the other artifact?
14. Your team's coverage went from 40% to 90% with AI generation, but escaped bugs didn't drop. What happened?
15. Where do generated tests and docs fit in your definition of done?
19 Glossary
- Unit test
- An automated test verifying one unit of behavior in isolation; the primary target of test-generation prompts.
- NUnit
- A mainstream .NET test framework: [TestFixture] classes, [Test] methods, [TestCase] parameterized rows, Assert.That assertions.
- Test fixture
- The class grouping related tests and shared setup; NUnit's [TestFixture].
- Arrange-Act-Assert
- The canonical test structure — setup, invocation, verification — demanded explicitly in generation prompts.
- Assertion
- The verifying line of a test; assertion quality, invisible to coverage, is what separates verification from decoration.
- Test case
- One input/expected-outcome pair; listed and human-reviewed as a table before implementation in the two-turn protocol.
- Parameterized test
- One test body run over many [TestCase] rows — the natural implementation of a boundary table.
- Edge case
- Boundary-of-validity input (empty, null, zero, max, duplicates, unicode) where bugs concentrate; elicited systematically by category.
- Boundary value analysis
- Deriving cases from both sides of every limit (17/18, 65/66 for 18–65) plus structural extremes.
- Test oracle
- The truth source expectations derive from; spec-derived oracles catch bugs, implementation-derived oracles certify them.
- Behavior specification
- Plain-language rules of what code should do — the prompt ingredient that turns generated tests from mirror into judge.
- Assertion-free test
- A generated-test pathology: executes code, verifies nothing; inflates coverage; exposed by the sabotage check.
- Tautological test
- Expectations restate the implementation, proving code equals itself — bugs included; caused by implementation-only prompting.
- Characterization test
- Pins current behavior of legacy code as expected — refactoring protection that must never be mislabeled as correctness proof.
- Mutation testing
- Verifying tests by breaking code and demanding red — manually (the sabotage check) or via tools that score mutant kill rates.
- Code coverage
- Executed-line/branch percentage; the map of where tests are missing, structurally blind to whether executed tests verify anything.
- Regression test
- A test locking in a bug fix; must demonstrably fail on pre-fix code to prove its detection power.
- XML documentation comments
- C# /// comments feeding IntelliSense, DocFX, and package docs; generated per-member with purpose-not-mechanism summaries.
- API documentation
- Consumer-facing reference for public surface; generated from code plus intent notes, with every specific claim fact-checked.
- OpenAPI
- The standard machine-readable HTTP API description; generated endpoint docs belong in its annotations so docs live with code.
- Doc drift
- Documentation rotting as code evolves; prevented by regenerating changed members' docs in the same PR.
- Flaky test
- A test that passes or fails nondeterministically (time, randomness, async); forbidden via injected clocks, seeds, and awaited assertions.