Testland
Browse all skills & agents

golden-file-conventions

Reference catalog for snapshot / golden file management - naming conventions, directory layout, when to add / update / remove a baseline, sanitization (timestamps, IDs, PII), per-OS / per-runtime variant strategy, and review workflow for snapshot diffs in PRs. Use when designing a snapshot-testing convention or auditing an existing one for drift.

Install with skills.sh (any agent)

npx skills add testland/qa --skill golden-file-conventions
View source

golden-file-conventions

Terminology note: "golden file" / "golden master" are practitioner-emergent terms popularized by the Working Effectively with Legacy Code tradition. ISTQB has no canonical entry - the closest formal term is "snapshot test." This catalog uses both interchangeably; assume "golden file" and "snapshot" mean the same thing in the rest of the body.

A reference catalog for how to manage snapshot / golden files. It supplies the conventions that active snapshot-management workflows follow when updating / pruning golden files.

When to use

  • A team is starting snapshot-testing on a project.
  • A team has an existing snapshot suite and reviews are slow because of unclear conventions.
  • A developer is unsure when to update a baseline vs. when to fix the regression.
  • A PR review keeps stalling on "is this snapshot diff intentional?"

Naming conventions

Per-test snapshot file

Most snapshot frameworks (Jest, Vitest, pytest-snapshot, RSpec Snapshot) use a path adjacent to the test file:

src/
  components/
    Button.tsx
    Button.test.tsx
    __snapshots__/
      Button.test.tsx.snap

Convention: one snapshot file per test file, named <test-file-name>.snap. Do not split snapshots across multiple files per test.

Per-test name within a snapshot file

Inside a .snap file, each snapshot is keyed by <describe> > <it> chain:

exports[`Button renders with primary variant 1`] = `<button class="primary">...</button>`;

The trailing 1 is the snapshot index when one test takes multiple snapshots - keep these to a minimum (≤3 per test); beyond that, split the test.

Per-OS / per-browser variants (visual snapshots)

For visual / screenshot-based snapshots, the name carries the platform suffix (per playwright-snapshots, in the qa-visual-regression plugin):

Button-primary-1-chromium-linux.png
Button-primary-1-firefox-linux.png
Button-primary-1-webkit-darwin.png

OS / browser suffixes are load-bearing - anti-aliasing and font metrics differ. Don't strip them.

Directory layout

LayoutWhen to use
Adjacent (__snapshots__/ next to test)Default. Reviewer sees the diff in the same PR view as the test.
Centralized (tests/__fixtures__/)Cross-test fixtures (golden inputs reused by many tests).
External (s3://snapshots-bucket/)Visual snapshots that are large; CI uploads / downloads. Common with Percy, Chromatic, Playwright + S3.

Default to adjacent. Centralized only when fixtures are reused. External only when artifact size makes adjacent impractical.

When to add a baseline

Add a snapshot when:

  • A new component / page / output ships and its rendered shape is load-bearing.
  • A bug fix changes a specific output that lacked coverage; the snapshot is the regression-prevention asset.
  • A redesign locked the new design system; refresh all baselines in one PR.

Don't add a snapshot for:

  • Output that's expected to change frequently (e.g. timestamps, randomized IDs). Snapshot the shape, not the volatile values - see Sanitization below.
  • Internal-only rendering paths the user never sees.
  • Test scaffolding output (e.g. test runner banners).

Sanitization (the load-bearing rule)

A snapshot that contains volatile values (timestamps, UUIDs, random IDs, current dates) breaks every run. Sanitize before snapshotting:

Volatile fieldSanitization pattern
TimestampsReplace with a fixed string [TIMESTAMP] or freeze the clock (vi.useFakeTimers()).
UUIDsReplace with [UUID] or seed a deterministic generator.
Auto-increment IDsReplace with [ID] or use a sequence-controlled fixture.
File paths (/var/folders/...)Replace with [PATH] or normalize via project root.
Memory addresses (object refs)Avoid in serialized output; use a custom serializer.
User-data tokensStrip before snapshotting; tokens shouldn't be in the test surface anyway.

Most frameworks support custom serializers / matchers - use them. Jest's expect.any(Date) matcher pattern is canonical:

expect(result).toMatchSnapshot({
  createdAt: expect.any(Date),
  uuid: expect.any(String),
});

The serializer normalizes volatile fields before comparison, so the snapshot shows Any<Date> rather than a specific timestamp.

Update vs. fix decision tree

When a snapshot diff appears in a PR:

Is the diff explained by code changes in the same PR?
├── No  → REGRESSION; fix the code, do not update the snapshot.
└── Yes → Did the diff align with the intent (described in the PR title)?
    ├── No  → REGRESSION (cascade from an unrelated change); investigate before updating.
    └── Yes → Is the diff isolated to the components the PR is supposed to change?
        ├── No  → INVESTIGATE: a CSS / token / shared-component change affected unrelated snapshots.
        └── Yes → UPDATE: run `--update-snapshots` and commit.

The most common review failure is rubber-stamping snapshot updates - accepting a 47-component diff because the PR title says "Refactor Button". A snapshot-diff classifier can implement this decision tree.

Severity tiering

Every snapshot has an implicit severity:

TierBehaviorExamples
CriticalBlocks merge on diff; requires explicit reviewer acceptance.Production-shipped pages; payment flows; auth.
StandardBlocks merge on diff; author can self-approve with a clear PR description.Internal admin tooling; non-shipping experiments.
AdvisorySurfaces diff but doesn't block.Unstable areas under active redesign; new baselines during ramp-up.

Promote Advisory → Standard after ~2 weeks of stability. Promote Standard → Critical for security-sensitive surfaces.

Pruning rules

Remove a snapshot when:

  • The associated test is deleted.
  • The component / page is removed from the product.
  • The snapshot has produced more false positives than real regressions over a 3-month window - it's noise, not signal.

The "test deleted but snapshot remained" cleanup can be automated.

Anti-patterns

Anti-patternWhy it failsFix
Updating snapshots in a separate "snapshot refresh" PRReviewer can't see the code change that justifies the diff.Always update snapshots in the same PR as the source change.
--update-snapshots in PR CI as the defaultSnapshots become tautologies; never catch a regression.Update snapshots only in interactive runs; PR CI fails on diff.
Snapshotting raw HTML for componentsBrittle to attribute-order changes from tooling upgrades.Snapshot the React / Vue / Svelte component tree (e.g. react-test-renderer), not raw HTML; OR use a normalizer.
One mega-snapshot per pageA 5kb diff is uninterpretable; reviewers approve to move on.Per-component snapshots; smaller surface = faster review.
Storing snapshots externally without checksumsA drift in S3 vs. the test code makes "what changed?" hard.Include checksums in the test code; verify on each run.
Snapshots of error messages with stack tracesStack traces include line numbers that drift with every refactor.Snapshot the error type + message only; strip the trace.
Cross-OS shared snapshotsAnti-aliasing / font / line-ending differences flake the test.Per-OS snapshot suffixes (see naming above).

Review workflow

  1. PR opens with snapshot diff. Reviewer reads the PR title / description first to understand intent.
  2. Reviewer checks each diff cell against the decision tree above.
  3. For unintended diffs: comment with the specific cell + ask for code investigation; do not approve.
  4. For intended diffs: approve; the snapshot becomes the new baseline.
  5. For ambiguous diffs: request a second reviewer; treat as Critical-tier even if labeled Standard.

References

  • ISO/IEC/IEEE 29119 series - formal test-document conventions (cite by stable ID).
  • playwright-snapshots - visual-snapshot-specific naming and per-OS suffix conventions.
  • visual-baseline-conventions - broader visual-coverage conventions; this skill is the text/object-snapshot equivalent.

Related skills

bogus-data

Authors .NET test fixtures using the Bogus library - fluent typed `Faker` builders with `.RuleFor` per property, generation via `Generate()` / `GenerateBetween(min, max)` / `GenerateLazy()`, and `UseSeed()` for reproducibility. Provides the Bogus equivalent of Python's Faker / Ruby's FactoryBot. Use when the project is C# / F# / VB.NET and the team needs typed fixture creation.

boundary-value-generator

Generates boundary-value test cases from typed input specifications - for each input field, produces the canonical 6-point set (one below, at, and above the lower bound; one below, at, and above the upper bound) plus equivalence-class representatives. Emits cases as parameterized test inputs (pytest @parametrize / Jest test.each / xUnit InlineData / etc.). Use when a function or endpoint has numeric / string-length / collection-size constraints and the team needs systematic edge-case coverage.

e2e-test-narrative-builder

Assembles a multi-step end-to-end user-journey test from a list of high-level user intents - translates each intent ("user signs up", "user adds product to cart", "user completes checkout with promo code") into the corresponding test-runner step (Playwright / Cypress / Selenium / Karate), wires shared state across steps via test fixtures, and emits the resulting test as a single Scenario in the project's E2E framework. Use when scaffolding an E2E test that exercises a complete user flow rather than a single page.

factory-bot-data

Authors Ruby FactoryBot factories with traits, associations, sequences, and the three build strategies (build / create / build_stubbed); integrates with RSpec / Minitest test suites; pairs with Faker for randomized field values. Use when the project is Ruby / Rails and needs structured fixture creation with referential integrity.

faker-data

Authors test-data factories using Faker: the Python `faker` library, the `@faker-js/faker` JS port, and the `faker-ruby` gem. Owns the library mechanics end to end: install per language, the provider catalogue (person / internet / location / date / finance / lorem), locale selection and multi-locale mode, and seed-based determinism for reproducible runs. Scope is generating fresh values for tests that start from nothing, not replacing values inside an existing dataset that already holds real records, which raises referential-integrity and re-identification concerns this skill does not address. Prefer this skill when the codebase already uses the Faker family or when cross-language consistency across Python, JS, and Ruby matters; use mimesis-data only when deeper Python locale coverage is the primary requirement. Use when authoring fixtures or factories that need realistic-looking field values.

malicious-payload-bank

Reference catalog of curated adversarial input payloads keyed by attack class - SQL injection, XSS, SSRF, path traversal, command injection, XXE, prototype pollution, regex DoS, Unicode confusables, header injection - plus per-context guidance for which payloads apply (URL parameter / form input / JSON body / file upload). Use when authoring negative-test cases for input validation, fuzz targets, or a security-focused test suite that needs to exercise the OWASP Top 10 attack surface.

mimesis-data

Authors Python test fixtures using mimesis - a fast, type-hinted, locale-aware test-data generator with 46 locales - covering Person / Address / Internet / Datetime providers and the Schema/Field pattern for typed-dict generation. Pairs with factory_boy when referential integrity is needed. Use when the project is Python and the team values speed, type hints, or strong locale coverage over Faker's larger ecosystem.

mountebank-imposters

Authors Mountebank imposters (multi-protocol mock servers - HTTP, HTTPS, TCP, SMTP, LDAP, gRPC, WebSockets, GraphQL, and more) by POSTing JSON definitions to the Mountebank control API on port 2525, configures stubs with predicates and responses, and uses record-playback proxy mode to capture upstream traffic. Use when the project needs a multi-protocol mock server beyond HTTP-only tools like WireMock or MSW.

msw-handlers

Authors Mock Service Worker (MSW) request handlers for both browser and Node.js test environments using the `http.get` / `http.post` / `HttpResponse.json` API, wires them via `setupWorker` (browser) or `setupServer` (Node), and manages the test lifecycle (`server.listen` / `resetHandlers` / `close`). Use when the project uses JavaScript / TypeScript and needs to mock fetch / XHR at the network layer for both Vitest / Jest unit tests and Cypress / Playwright integration tests.

negative-test-generator

Generates negative / error-path test cases that mirror happy-path tests - for each happy-path test, produces companions exercising input validation rejection, missing required fields, type mismatches, authorization failures, rate-limit errors, and adversarial payloads from the malicious-payload-bank. Emits cases as parameterized tests in the project's runner format. Use when a feature has happy-path coverage but the rejection / error / unauthorized paths are untested.

pairwise-test-case-generator

Generates parameterized test inputs combining boundary-value, equivalence-class, and pairwise-combinatorial cases from a typed multi-input specification - produces the cross-product of cases up to a configurable strength (1-wise / 2-wise / N-wise) using all-pairs reduction so the test surface stays tractable. Emits cases in the project's test-runner-native parametrize format. Use when a function or endpoint takes 3+ inputs whose interactions matter and full Cartesian product would explode.

seed-data-curator

Builds a reproducible E2E seed dataset for the project's test environments - picks a representative user / org / data-product cross-section, generates the rows via the project's chosen factory library (FactoryBot / mimesis / Bogus / Faker + factory_boy), persists the dataset as a checked-in fixture (SQL dump / JSON / per-engine seed file), and wires it into the test bootstrap. Use when starting E2E coverage on a project that has no seed strategy, or when an existing seed has drifted.

synthetic-data-tool-selector

Chooses between the four mainstream synthetic test-data generators - Faker (JavaScript), FactoryBot (Ruby), mimesis (Python), Bogus (.NET) - picks the right tool by language and use case (raw value generation vs. typed factory orchestration), shows side-by-side equivalents for the same fixture across all four, and emits the language-appropriate code. Use when starting test-data work on a project and the team wants the "which tool should I use" decision documented.

synthetic-pii-generator

Generates realistic-but-fake personally identifiable information (PII) - emails, phone numbers, SSNs / national IDs, addresses, names, credit-card numbers (test BIN ranges), date-of-birth - for non-production environments. Wraps Faker / mimesis with PII-aware constraints so generated values match real format expectations (Luhn-valid card numbers, region-valid phone formats, ITIN/SSN format) without ever generating real-person data. Use when seeding test environments, building demo data, or replacing real PII in copied datasets.

test-data-patterns

Pure reference catalog of the cross-language object-construction patterns for test data - Test Data Builder (Pryce/Freeman), Factory (with traits and associations), Object Mother, Fixture composition (per-test / per-describe / shared), Snapshot (defers to `golden-file-conventions` for the operational details), and Production-Data Anonymisation. Distinct from per-language data wrappers (`factory-bot-data` Ruby, `faker-data` JS, `mimesis-data` Python, `bogus-data` .NET) which document tool-specific configuration; this catalog is the architecture-tier reference for choosing **which pattern** before reaching for the tool. Use when choosing a test-data construction pattern for a new suite, or auditing an existing suite whose fixtures have drifted into shared mutable state.

wiremock-stubs

Authors WireMock stub mappings for HTTP service mocking - `stubFor` with verb/path/header matchers + `willReturn` response shaping, lifecycle via `WireMockServer` (start / stop) or JUnit `WireMockExtension`, request verification via `verify()`, and dynamic-port allocation for parallel tests. Use when the project is JVM-based and tests need to mock HTTP dependencies (third-party APIs, internal microservices) at the network layer.