Testing Principles

We deliberately test at the edges of the system this gives us:

Freedom to refactor
Confidence to optimise
Stable, long-lived tests
Reduced coupling to implementation details

We write high-level, black-box integration tests that focus only on observable outcomes (behaviour not implementation details).

If a refactor breaks a test without changing observable behaviour, the test was too coupled.

Test Projects

We maintain two test suites:

./Tests        → CLI tests
./Tests.E2e    → Blazor Web UI tests (Playwright)

All tests must pass before a PR can be merged into main.

If something is worth testing manually, it’s worth automating.

When adding features:

Add tests with them
If tests are missing, add them
If behaviour changes, update tests intentionally

When fixing bugs:

Reproduce manually
Reproduce repeatably with a failing test
Make the test pass

Web UI (E2E) Tests

We use Playwright to test the Blazor Server app.

These tests are:

Slow(ish)
Primarily happy-path + critical flows

Avoid:

Testing styling details
Over-asserting UI minutiae

Running E2E Tests

You must build the Docker image before running:

cd RackPeek
docker build -t rackpeek:ci -f RackPeek.Web/Dockerfile .

cd Tests.E2e
dotnet tool install --global Microsoft.Playwright.CLI
dotnet build

playwright install
dotnet test

Rebuild the image whenever the Web project changes.

Page Object Model (POM)

Each page/component has a POM (Page Object Model) abstraction.

The POM:

Encapsulates selectors
Encapsulates browser interactions
Exposes intent-level methods (AddDesktopAsync, GotoHardwareAsync)
Shields tests from UI churn

Tests should read like workflows — not like browser scripts.

Example:

[Fact]
public async Task User_Can_Add_And_Delete_Desktop()
{
    var (context, page) = await CreatePageAsync();
    var resourceName = $"e2e-ap-{Guid.NewGuid():N}"[..16];

    try
    {
        await page.GotoAsync(fixture.BaseUrl);

        var layout = new MainLayoutPom(page);
        await layout.AssertLoadedAsync();
        await layout.GotoHardwareAsync();

        var hardwarePage = new HardwareTreePom(page);
        await hardwarePage.AssertLoadedAsync();
        await hardwarePage.GotoDesktopsListAsync();

        var listPage = new DesktopsListPom(page);
        await listPage.AssertLoadedAsync();
        await listPage.AddDesktopAsync(resourceName);
        await listPage.AssertDesktopExists(resourceName);
        await listPage.DeleteDesktopAsync(resourceName);
        await listPage.AssertDesktopDoesNotExist(resourceName);
    }
    catch (Exception)
    {
        _output.WriteLine("TEST FAILED — Capturing diagnostics");
        _output.WriteLine($"Current URL: {page.Url}");

        var html = await page.ContentAsync();
        _output.WriteLine("==== DOM SNAPSHOT START ====");
        _output.WriteLine(html);
        _output.WriteLine("==== DOM SNAPSHOT END ====");

        throw;
    }
    finally
    {
        await context.CloseAsync();
    }
}

Good E2E Test Traits

Single responsibility
Independent (no ordering dependencies)
Idempotent
Generates unique test data
Cleans up after itself
Fails with useful diagnostics

Debugging E2E Tests

You may temporarily modify:

Tests.E2e/infra/PlaywrightFixture.cs

To debug visually:

Browser = await _playwright.Chromium.LaunchAsync(new BrowserTypeLaunchOptions
{
    Headless = false,
    SlowMo = 500,
    Args = new[]
    {
        "--disable-dev-shm-usage",
        "--no-sandbox"
    }
});

⚠ Before committing, revert to CI-safe settings:

Browser = await _playwright.Chromium.LaunchAsync(new BrowserTypeLaunchOptions
{
    Headless = true,
    Args = new[]
    {
        "--disable-dev-shm-usage",
        "--no-sandbox"
    }
});

CI must always run headless.

CLI Tests

CLI tests are faster and more precise.

They behave more like unit/integration hybrids (intragrationtests if you like):

Validate both happy + unhappy paths
Assert command output
Assert YAML written to disk
Avoid UI overhead

Run them with:

cd Tests
dotnet test

Example:

[Fact]
public async Task servers_tree_cli_workflow_test()
{
    await File.WriteAllTextAsync(Path.Combine(fs.Root, "config.yaml"), "");

    var (output, yaml) = await ExecuteAsync("servers", "add", "node01");
    Assert.Equal("Server 'node01' added.\n", output);
    Assert.Contains("name: node01", yaml);

    (output, yaml) = await ExecuteAsync("systems", "add", "host01");
    Assert.Equal("System 'host01' added.\n", output);

    (output, yaml) = await ExecuteAsync("systems", "add", "host02");
    Assert.Equal("System 'host02' added.\n", output);

    (output, yaml) = await ExecuteAsync("systems", "add", "host03");
    Assert.Equal("System 'host03' added.\n", output);

    (output, yaml) = await ExecuteAsync(
        "systems", "set", "host01",
        "--runs-on", "node01"
    );
    Assert.Equal("System 'host01' updated.\n", output);

    (output, yaml) = await ExecuteAsync("services", "add", "immich");
    Assert.Equal("Service 'immich' added.\n", output);

    (output, yaml) = await ExecuteAsync("servers", "tree", "node01");
    Assert.Equal("""
                 node01
                 ├── System: host01
                 │   ├── Service: immich
                 │   └── Service: paperless
                 ├── System: host02
                 └── System: host03

                 """, output);
}

CLI Testing Best Practices

Assert exact output where meaningful
Validate file side effects
Test invalid arguments and failure modes
Avoid brittle whitespace assertions unless intentional
Keep tests deterministic
Avoid shared filesystem state

General Testing Standards

1. Behaviour First

Test what the user observes — not how we implement it.

2. Prefer Integration Over Micro-Mocking

We value confidence over isolation purity.

3. Fast Feedback

CLI tests should be fast
E2E tests should be minimal but meaningful

4. Tests Are Documentation

A good test explains:

What the feature does
How it’s expected to behave
What regressions look like

5. Stability > Coverage %

High-value tests matter more than coverage numbers.

6. No Flaky Tests

If a test is flaky:

Fix it immediately
Or remove it
Flaky tests erode trust

Definition of Done

A feature is complete when:

Behaviour is implemented
Tests exist
Tests pass locally
Tests pass in CI
Edge cases are covered
No debug flags remain enabled

We optimise for:

Confidence
Refactorability
Clarity
Long-term maintainability

Tests are a first-class citizen of this project.

testing-guidelines.md 6.7 KB Permalänk Historik Rå