Your Swagger Doc Is More Than Documentation — It's a Test Suite Waiting to Happen

AI Summary10 min read

TL;DR

Swagger documentation can be leveraged as a source of truth for automated contract testing, ensuring API providers and consumers adhere to defined schemas. This approach validates request and response formats, catching breaking changes early and reducing manual test maintenance.

Key Takeaways

  • Contract testing verifies that API providers and consumers conform to shared contracts, preventing silent breaks when APIs evolve.
  • Using Swagger as a foundation automates validation of request and response schemas, reducing manual assertion writing and improving test efficiency.
  • The implementation includes loading and caching Swagger documents, extracting schemas for requests and responses, and validating JSON payloads with descriptive error handling.
  • This method integrates easily into existing test projects, supports both request and response validation, and automatically updates tests when the Swagger spec changes.
  • Future enhancements could include support for path parameters, schema versioning, CI/CD integration, and auto-generating test cases from Swagger.

Tags

apiautomationtestingcsharp

In a previous article, I showed how we can use Swagger and AI to generate test cases automatically. This time, we're going to explore another powerful use for Swagger: using it as the source of truth for contract tests.


Understanding Contract Testing

Before jumping into code, let's understand what contract testing actually means in the context of APIs.

According to the ISTQB (International Software Testing Qualifications Board) syllabus:

Contract testing is a type of interface testing that verifies that a service provider and a service consumer conform to a shared contract specifying request and response formats, data types, status codes, and interaction rules.

In plain terms, contract testing ensures that:

  • The API provider delivers responses in the format it promised
  • The consumer sends requests in the expected format
  • Both sides can evolve independently without breaking each other

Think of it like a formal agreement between two people. The contract testing job is simply to check: "Is everyone still honoring the deal?"

Why does this matter?

Imagine your team changes the response of an endpoint — maybe they rename a field, change a data type, or remove a property. Without contract tests, that kind of change can silently break consumers who depend on that contract. Contract testing catches this early, before it reaches production.


Why Use Swagger for This?

There are several tools for contract testing, such as:

  • Pact – Consumer-driven contract testing
  • Spring Cloud Contract – JVM-based microservices
  • Swagger / OpenAPI – API schema validation
  • Postman – Can validate schema contracts

Some teams adopt an API First approach, where API specifications are defined using OpenAPI before the API is even built. This prevents teams from being blocked during development and improves communication between frontend, backend, and QA.

That's precisely why using Swagger as our source of truth makes a lot of sense — if the contract is already documented, why not automate the validation against it?


What We're Building

Here's a quick overview of what we'll build:

  1. Swagger loader that fetches and caches the OpenAPI document
  2. schema extractor for responses
  3. schema extractor for requests (request body validation)
  4. validator that checks JSON payloads against those schemas
  5. NUnit tests that tie everything together

We'll be using the public FakeRESTApi as our API under test, and the NSwag package to parse the Swagger document.


Step 1 — Loading the Swagger Document

The first thing we need is a way to fetch and cache the OpenAPI document. We don't want to download it on every test run — that would be slow and fragile.

public static async Task<OpenApiDocument> LoadDocument(string url)
{
    _cachedDocument = await OpenApiDocument.FromUrlAsync(url);
    return _cachedDocument;
}
Enter fullscreen mode Exit fullscreen mode

Why cache it? The Swagger document usually doesn't change during a test run. Fetching it once and reusing it keeps your tests fast and avoids unnecessary network calls.


Step 2 — Extracting the Response Schema

With the document loaded, we need to navigate through it and find the specific schema for a given path, HTTP method, and status code.

private static JsonSchema GetSchemaResponse(
    OpenApiDocument doc,
    string path,
    string method,
    int statusCode,
    string contentType = "application/json")
{
    if (!doc.Paths.TryGetValue(path, out var pathItem))
        throw new Exception($"Path '{path}' not found in the swagger provided");

    if (!pathItem.TryGetValue(method, out var op))
        throw new Exception($"The method {method} isn't available for the path {path}");

    var statusKey = statusCode.ToString();

    if (!op.ActualResponses.TryGetValue(statusKey, out var response))
        throw new Exception($"Response {statusCode} isn't available for this path {path}");

    if (response.Content == null || !response.Content.Any())
        throw new Exception($"Response {statusCode} for {path} has no content.");

    var mediaType = response.Content
        .FirstOrDefault(c =>
            c.Key.StartsWith(contentType, StringComparison.OrdinalIgnoreCase));

    if (mediaType.Value == null)
        throw new Exception(
            $"The content type {contentType} isn't available for this path {path}. " +
            $"Available: {string.Join(", ", response.Content.Keys)}");

    return mediaType.Value.Schema;
}
Enter fullscreen mode Exit fullscreen mode

Notice that we're throwing descriptive exceptions at every possible failure point. This is intentional — when a test fails, you want to know exactly why without digging through stack traces.


Step 3 — Extracting the Request Body Schema

Contract testing isn't only about what the API returns — it's also about what it receives. Let's add a method to extract the schema for request bodies:

private static JsonSchema GetSchemaRequest(
    OpenApiDocument doc,
    string path,
    string method,
    string contentType = "application/json")
{
    if (!doc.Paths.TryGetValue(path, out var pathItem))
        throw new Exception($"Path '{path}' not found.");

    if (!pathItem.TryGetValue(method, out var operation))
        throw new Exception($"Method '{method}' not found for '{path}'.");

    var requestBody = operation.RequestBody;

    if (requestBody == null)
        throw new Exception($"No request body defined for {method} {path}");

    var mediaType = requestBody.Content
        .FirstOrDefault(c =>
            c.Key.StartsWith(contentType, StringComparison.OrdinalIgnoreCase));

    if (mediaType.Value == null)
        throw new Exception($"Content type {contentType} not found for request.");

    return mediaType.Value.Schema;
}
Enter fullscreen mode Exit fullscreen mode

Step 4 — Validating the JSON Payload

Now that we have the schema, we need to validate a real JSON payload against it. This is where the actual contract check happens:

private static void ValidateSchema(string bodyResponse, JsonSchema schema)
{
    var json = JToken.Parse(bodyResponse);
    var validator = new JsonSchemaValidator();

    var errors = validator.Validate(json, schema);
    if (errors.Count == 0) return;

    var msg = string.Join("", errors.Select(e => $"{e.Path} -> {e.Kind}"));
    throw new Exception($"The response isn't valid against the schema: {msg}");
}
Enter fullscreen mode Exit fullscreen mode

If there are any violations — wrong types, missing required fields, unexpected values — this will surface all of them in a single readable message.


Step 5 — The Main Validation Entry Point

Let's wrap everything into one clean public method that the tests will call:

public async Task ContractValidation(
    string swaggerUrl,
    string path,
    string method,
    int expectedStatusCode,
    string bodyResponse,
    object? requestBody = null)
{
    // Load (or use cached) Swagger document
    var doc = await LoadDocument(swaggerUrl);

    // Validate request body if provided
    if (requestBody != null)
    {
        var requestSchema = GetSchemaRequest(doc, path, method.ToLower());
        var requestJson = JsonConvert.SerializeObject(requestBody);
        ValidateSchema(requestJson, requestSchema);
    }

    // Validate the API response
    var responseSchema = GetSchemaResponse(doc, path, method.ToLower(), expectedStatusCode);
    ValidateSchema(bodyResponse, responseSchema);
}
Enter fullscreen mode Exit fullscreen mode

The requestBody parameter is optional — if you're testing a GET endpoint, you simply don't pass it, and only the response will be validated.


Step 6 — Writing the Tests

Now let's put it all together in NUnit tests:

using NUnit.Framework;
using SwaggerIntegration.Common;
using Assert = NUnit.Framework.Assert;

namespace SwaggerIntegration.Tests;

[TestFixture]
public class CheckSwaggerIntegration
{
    private SwaggerContractValidator _validator;
    private Requests _requests;

    private readonly string _baseUrl = "https://fakerestapi.azurewebsites.net/";
    private string _swaggerUrl;

    [SetUp]
    public void Setup()
    {
        _requests = new Requests(_baseUrl);
        _validator = new SwaggerContractValidator();
        _swaggerUrl = $"{_baseUrl}swagger/v1/swagger.json";
    }

    // Validates that the GET response matches the Swagger schema
    [Test]
    public async Task CheckSwaggerGetMethod()
    {
        var response = await _requests.GetAsync<string>("api/v1/Activities");
        Assert.That(response.StatusCode, Is.EqualTo(System.Net.HttpStatusCode.OK));

        await _validator.ContractValidation(
            _swaggerUrl,
            "/api/v1/Activities",
            "get",
            200,
            response.Content);
    }

    // Validates both the request body and the response against the Swagger schema
    [Test]
    public async Task CheckSwaggerPostMethod()
    {
        var bodyRequest = new
        {
            id = 0,
            title = "Test Activity",
            dueDate = DateTime.UtcNow,
            completed = true
        };

        var response = await _requests.PostAsync<string>("api/v1/Activities", bodyRequest);
        Assert.That(response.StatusCode, Is.EqualTo(System.Net.HttpStatusCode.OK));

        await _validator.ContractValidation(
            _swaggerUrl,
            "/api/v1/Activities",
            "post",
            200,
            response.Content,
            bodyRequest);
    }

    // This test intentionally sends an invalid body (id as string instead of int)
    // It should fail the contract validation, proving our validator works
    [Test]
    public async Task CheckSwaggerPostMethodInvalidBody()
    {
        var bodyRequest = new
        {
            id = "Sending as string", // Wrong type — should be int
            title = "Test Activity",
            dueDate = DateTime.UtcNow
        };

        var response = await _requests.PostAsync<string>("api/v1/Activities", bodyRequest);

        await _validator.ContractValidation(
            _swaggerUrl,
            "/api/v1/Activities",
            "post",
            200,
            response.Content,
            bodyRequest);
    }
}
Enter fullscreen mode Exit fullscreen mode

Note on the invalid body test: This test is expected to fail — that's the point. It demonstrates that our validator is actually catching type mismatches. In a real project, you might want to assert the exception explicitly using Assert.ThrowsAsync to make the intent clear and keep the test green in CI.


Conclusion

Using Swagger as the foundation for contract testing is a smart move for teams that already follow an API First approach — or want to start. Instead of writing assertions manually for every field in every response, you let the OpenAPI spec do the heavy lifting.

What we built here is lightweight, reusable, and easy to plug into any existing test project:

  • One method handles both request and response validation
  • Descriptive errors tell you exactly what broke and where
  • Schema caching keeps things fast
  • Works for any endpoint defined in your Swagger doc

The best part? When the API contract changes, you update the Swagger spec — and your tests automatically reflect the new expectations. No need to hunt down and update dozens of individual assertions.

Future Improvements

There's a lot of room to evolve this approach. Here are some ideas worth exploring:

1. Support for path parameters and query strings Currently the approach focuses on request bodies and responses. A natural next step is validating path parameters (e.g., /api/v1/Activities/{id}) and query string types against the Swagger spec.

2. Schema versioning and drift detection Automatically compare two versions of a Swagger document (e.g., current vs. production) and generate a diff report. This could be used as a pre-merge gate to detect breaking changes before they reach consumers.

3. Integration with CI/CD pipelines These tests are already a great fit for pipelines, but you could take it further by generating a contract validation report as an artifact — showing which endpoints were validated, which passed, and which failed.

4. Support for multiple content types The current implementation defaults to application/json. Extending it to handle multipart/form-dataapplication/xml, and others would make it applicable to a wider range of APIs.

6. Auto-generating test cases from Swagger Going full circle with the first article in this series — the Swagger document could be used to automatically generate contract test skeletons for every endpoint and status code defined in the spec, reducing the manual effort to near zero.

You can check this full project here.

Visit Website