contract-test-scaffolder

A scaffolder that takes a contract artifact and emits the test stubs the consumer or provider runs to enforce that contract. Generates the tests; engineer fills the values - does not invent contract behavior the artifact does not declare.

When invoked

Required inputs: Contract artifact (OpenAPI YAML/JSON / GraphQL SDL / .proto / Pact JSON), Direction (consumer | provider), Target framework (Pact-JS/JVM/Python/Go/Ruby; Jest / pytest / Go test / JUnit for the runner). Optional: list of operations (defaults to all in the artifact).

Step 1 - Detect contract type

[[ "$ART" == *.yaml || "$ART" == *.json ]] && jq -e '.openapi' "$ART" >/dev/null && echo "openapi"
grep -q 'type Query' "$ART" 2>/dev/null && echo "graphql-sdl"
[[ "$ART" == *.proto ]] && echo "protobuf"
jq -e '.consumer.name and .provider.name and .interactions' "$ART" 2>/dev/null && echo "pact"

Step 2 - Pick the scaffolding strategy

Default for OpenAPI consumers is Pact; schemathesis is generated as complementary (Pact = consumer-driven examples; schemathesis = property-based exhaustion). Different failure modes, not a replacement.

Step 3 - Scaffold per strategy

Pact consumer (OpenAPI / GraphQL → Pact-JS). Per Pact's "contract by example" principle (https://docs.pact.io/), each interaction "describes a single concrete request/response pair" - the scaffold emits one interaction per documented response plus a INPUT-NEEDED for the example payload (the agent never invents example values absent from the spec).

// tests/contracts/cart-service.consumer.spec.ts
import { Pact } from '@pact-foundation/pact';
import { CartClient } from '../../src/cart-client';

const provider = new Pact({ consumer: 'web-app', provider: 'cart-service', port: 1234 });

describe('CartService consumer contract', () => {
  beforeAll(() => provider.setup());
  afterAll(() => provider.finalize());
  afterEach(() => provider.verify());

  test('POST /api/cart/items — adds an item (201)', async () => {
    await provider.addInteraction({
      state: 'cart is empty',                  // INPUT-NEEDED: align with provider state file
      uponReceiving: 'a request to add SKU-001',
      withRequest: {
        method: 'POST', path: '/api/cart/items',
        body: { sku: 'SKU-001', qty: 1 },      // INPUT-NEEDED: replace per AddItemRequest schema
      },
      willRespondWith: {
        status: 201,
        body: { sku: 'SKU-001', qty: 1, addedAt: '2026-05-08T10:00:00Z' },  // INPUT-NEEDED: per CartLineItem schema
      },
    });
    const response = await new CartClient(provider.mockService.baseUrl).addItem('SKU-001', 1);
    expect(response.sku).toBe('SKU-001');
  });
  // One scaffolded test per documented response code (200 / 201 / 400 / 401 / 404 / 409 / …).
});

If OpenAPI examples: is absent, payloads are emitted as INPUT-NEEDED with the JSON Schema fragment quoted as a comment.

schemathesis runner (OpenAPI / GraphQL fuzzing). Per https://schemathesis.readthedocs.io/en/stable/, schemathesis "generates property-based tests from your OpenAPI or GraphQL schema and exercises the edge cases that break your API" via Hypothesis:

# scripts/contract-fuzz.sh
uvx schemathesis run --base-url "${SCHEMATHESIS_BASE_URL:-http://localhost:8080}" --checks all ./openapi.yaml

Protobuf compatibility runner. Per buf docs, buf breaking compares current .proto against a baseline:

- name: Lock baseline (run once on main)
  run: buf build -o proto-baseline.bin
- name: Detect breaking changes vs baseline
  run: buf breaking --against proto-baseline.bin

Pact provider verification. For a service owner with consumer pacts in pacts/:

// tests/contracts/cart-service.provider.spec.ts
import { Verifier } from '@pact-foundation/pact';
test('verifies all consumer pacts', () => new Verifier({
  providerBaseUrl: process.env.PROVIDER_URL || 'http://localhost:8080',
  pactUrls: ['./pacts/web-app-cart-service.json'],
  stateHandlers: { 'cart is empty': async () => { /* INPUT-NEEDED: reset fixture */ } },
}).verifyProvider());

Step 4 - Hand-off block

Every scaffolded file ends with a HAND-OFF comment block instructing the engineer to (1) replace every INPUT-NEEDED with a spec-derived value (no inventing), (2) run the test locally to confirm the mock is reachable, (3) hand failing CI gate output to contract-drift-investigator, and (4) publish the pact via the team's distribution convention (Pact Broker or pacts/).

Refuse-to-proceed rules

Refuses to: invent example payloads (emits INPUT-NEEDED with schema fragment); generate Pact tests for a service the team neither calls nor owns; auto-publish to a Pact Broker (CI / release concern); generate schemathesis runs against production base URLs (hard-codes localhost); mix consumer + provider scaffolds in one file.

Anti-patterns

Limitations

• Per-language scaffolds limited to documented Pact bindings. JS, JVM, Python, Go, Ruby are primary; Rust / PHP fall back to a generic scaffold.
• Stateful APIs require human input. state cannot be inferred from OpenAPI alone.
• GraphQL subscriptions are aspirational. Pact-graphql supports queries / mutations cleanly; subscriptions emit a INPUT-NEEDED comment.
• No pact-file merging. New operations emit a separate file; merging is a Pact Broker / pact-merge concern.
• Protobuf scaffolding skips the gRPC test client itself - that's the per-language stub generator's job.

Hand-off targets

• Drift investigation when a contract gate fails → contract-drift-investigator.
• OpenAPI diff between two spec versions → openapi-contract-diff.
• GraphQL schema regression → graphql-schema-regression.
• Hand-authored Pact tests → pact-contract-testing.
• Protobuf compatibility → protobuf-compat-checking.
• CI contract gate → contract-compatibility-gate.

References

• Pact (consumer-driven contracts, "contract by example"): https://docs.pact.io/
• OpenAPI 3.1: https://spec.openapis.org/oas/v3.1.0
• Schemathesis (OpenAPI 2.0 / 3.0 / 3.1 / 3.2 + GraphQL June-2018+): https://schemathesis.readthedocs.io/en/stable/
• buf breaking: https://buf.build/docs/breaking/overview