Testland
Browse all skills & agents

payment-webhook-replay

Workflow-driven skill that builds payment webhook replay + recovery tests. Covers the idempotency contract (every webhook handler must handle redelivery without side effects), the replay simulators (Stripe CLI `stripe trigger`, Adyen Customer Area resend, PayPal Webhook Simulator, Braintree webhook test), the signature-verification gauntlet (HMAC-SHA256 per gateway, expired-timestamps rejection), and the partial-failure recovery scenarios. Use when designing webhook robustness tests.

Install with skills.sh (any agent)

npx skills add testland/qa --skill payment-webhook-replay
View source

payment-webhook-replay

Overview

Payment webhooks are the source of truth for asynchronous state transitions (settlement, refund completion, dispute state). Every webhook handler must be:

  1. Signature-verified - reject spoofed payloads.
  2. Idempotent - redelivery doesn't double-charge.
  3. Order-tolerant - out-of-order delivery is normal.
  4. Replay-safe - months-old replay doesn't break.

This skill produces the test suite for these properties.

When to use

  • New webhook handler for any payment gateway.
  • After a webhook-related incident (missed events, double processing).
  • Migrating between gateways or webhook formats.

How to use

  1. Confirm the handler reads the raw request body, not a parsed-then-restringified copy - signature verification fails on the restringified bytes otherwise.
  2. Write the signature-verification gauntlet for the gateway (Step 1): reject unsigned, reject wrong-secret, accept valid, reject expired-timestamp.
  3. Add the idempotency dedup test plus the event-ID handler (Step 2).
  4. Wire a replay simulator for the gateway (Step 3), then run the Worked example end to end against staging.
  5. Add the harder recovery scenarios - out-of-order delivery, mid-handler crash, archive replay, per-gateway suite layout - from references/advanced-recovery-scenarios.md.

Step 1 - Signature-verification gauntlet

Per docs.stripe.com/webhooks/signatures (opens in new window), the Stripe gauntlet is four cases - unsigned, wrong-secret, valid, and expired-timestamp:

const payload = JSON.stringify({ type: 'payment_intent.succeeded' });

test('rejects unsigned payload', async () => {
  const res = await fetch('/webhooks/stripe', { method: 'POST', body: payload });
  expect(res.status).toBe(401);  // No Stripe-Signature header
});

test('rejects wrong-secret signature', async () => {
  const wrongSig = stripe.webhooks.generateTestHeaderString({ payload, secret: 'wrong-secret' });
  const res = await fetch('/webhooks/stripe', {
    method: 'POST', body: payload, headers: { 'stripe-signature': wrongSig },
  });
  expect(res.status).toBe(401);
});

test('accepts valid signature', async () => {
  const sig = stripe.webhooks.generateTestHeaderString({
    payload, secret: process.env.STRIPE_WEBHOOK_SECRET!,
  });
  const res = await fetch('/webhooks/stripe', {
    method: 'POST', body: payload, headers: { 'stripe-signature': sig },
  });
  expect(res.status).toBe(200);
});

test('rejects expired timestamp', async () => {
  const oldSig = stripe.webhooks.generateTestHeaderString({
    payload,
    secret: process.env.STRIPE_WEBHOOK_SECRET!,
    timestamp: Math.floor(Date.now()/1000) - 3600,  // 1 hour ago
  });
  const res = await fetch('/webhooks/stripe', {
    method: 'POST', body: payload, headers: { 'stripe-signature': oldSig },
  });
  expect(res.status).toBe(401);
});

The signature scheme differs per gateway; run the same four-case gauntlet against each:

GatewaySignature schemeReference
StripeHMAC-SHA256 over the payload; the signature carries a timestamp, so old timestamps rejectwebhooks/signatures (opens in new window)
AdyenHMAC-SHA256 over the canonical string; validated per-event, not per-requestverify-hmac-signatures (opens in new window)
PayPalSHA256-with-RSA; verify via the PayPal verification endpoint or SDK helperwebhooks/rest (opens in new window)
Braintreeparser validates bt_signature against the merchant's public keyBraintree webhook parser

Step 2 - Idempotency

Redelivery must not double-process. Dedup on the gateway-issued event ID:

test('redelivered webhook handled idempotently', async () => {
  const payload = makeWebhookPayload({ type: 'payment_intent.succeeded' });
  const sig = signPayload(payload);

  const before = await db.payment_records.count();
  await postWebhook(payload, sig);
  const after1 = await db.payment_records.count();
  await postWebhook(payload, sig);  // Redelivery
  const after2 = await db.payment_records.count();

  expect(after1 - before).toBe(1);
  expect(after2).toBe(after1);
});

The handler looks up by event ID and acks duplicates without re-doing the work:

async function handleEvent(event) {
  const existing = await db.events.findOne({ event_id: event.id });
  if (existing) {
    return 200;  // Already handled; safe to ack
  }
  await processEvent(event);
  await db.events.create({ event_id: event.id, processed_at: new Date() });
  return 200;
}

Step 3 - Replay simulators

Each gateway ships a way to (re)send a real event at the handler:

GatewayReplayNotes
Stripestripe trigger payment_intent.succeeded; stripe events resend evt_test_12345CLI (stripe-cli (opens in new window))
AdyenCustomer Area transaction "Resend webhook"Re-sends with the original signature - useful for idempotency tests
PayPalDashboard simulator; REST equivalent via the APIsimulate-event (opens in new window)
Braintreegateway.webhookTesting.sampleNotification(kind, id)Generates a test signature for any event kind (parse/node (opens in new window))

Worked example

Replay one gateway's event (Stripe) end to end and assert both core properties - the signature is accepted and a redelivery is idempotent.

Drive a real event at the handler with the CLI:

stripe trigger payment_intent.succeeded    # synthetic event to the forward URL
stripe events resend evt_test_12345        # replay a captured event (30-day window)

Then assert the guarantees in-process:

test('replayed Stripe event: signature accepted, idempotent on resend', async () => {
  const payload = makeWebhookPayload({ type: 'payment_intent.succeeded' });
  const sig = stripe.webhooks.generateTestHeaderString({
    payload: JSON.stringify(payload),
    secret: process.env.STRIPE_WEBHOOK_SECRET!,
  });

  const first = await postWebhook(payload, sig);
  expect(first.status).toBe(200);                   // valid signature accepted

  const resend = await postWebhook(payload, sig);   // same event redelivered
  expect(resend.status).toBe(200);

  const rows = await db.payment_records.count({ event_id: payload.id });
  expect(rows).toBe(1);                             // processed exactly once
});

That is the minimum end-to-end proof for one gateway: a real replayed event passes signature verification and processes exactly once. Extend it to out-of-order, crash-recovery, and archive-replay scenarios in references/advanced-recovery-scenarios.md.

Anti-patterns

Anti-patternWhy it failsFix
Skip signature verificationSpoofed webhook payloadsAlways verify
Skip idempotencyReplay double-processesEvent-ID dedup table
Trust HTTP 200 == processedServer may have crashedAtomic commit + event-ID record
Hardcoded webhook secrets in testsLeaked via test snapshotsEnv vars
No "future-dated" webhook testClock skew + redeliveryTest +5min and -5min
Single-platform tests onlyPer-gateway quirksPer-gateway test directory
body-parser ate the raw bytesSignature verification fails on parsed-then-restringifiedRaw-body middleware
No partial-failure / retry testCrash mid-handler corrupts stateAtomic transaction with event-ID

Limitations

  • Replay simulator availability varies. Stripe CLI is most developer-friendly; others require dashboard or API.
  • Real production replay is slower than test mode; SLA windows differ.
  • Bank-initiated webhooks (chargebacks) aren't always test-mode-triggerable.
  • Signature algorithms differ per gateway. Stripe HMAC-SHA256 with timestamp; Adyen HMAC-SHA256 with canonical string; PayPal SHA256-with-RSA.

References

Webhook order-tolerance, partial-failure, and archive-replay scenarios

View source (opens in new window)

Webhook order-tolerance, partial-failure, and archive-replay scenarios

Deep reference for payment-webhook-replay SKILL.md. Consult once the core signature + idempotency + replay-simulator surface is in place and the suite needs the harder recovery scenarios: out-of-order delivery, mid-handler crash recovery, replay of old events from the gateway archive, and the per-gateway suite layout.

Order-tolerance tests

Webhooks can arrive out of order:

test('out-of-order event delivery handled', async () => {
  const completedEvent = makeEvent({ type: 'payment_intent.succeeded' });
  const creatingEvent = makeEvent({ type: 'payment_intent.created' });

  // Deliver completed BEFORE created
  await postWebhook(completedEvent);
  await postWebhook(creatingEvent);

  // Final state should still be correct
  const record = await db.payments.findOne({ intent_id: completedEvent.data.id });
  expect(record.status).toBe('succeeded');
});

The handler must use versioned events or state-machine gates to handle this:

# Don't blindly overwrite state
def handle_event(event):
    record = db.payments.get(event.intent_id)
    new_state = event.data.status
    if state_transition_allowed(record.status, new_state):
        record.status = new_state
        record.save()
    # else: stale event, ignore

Partial-failure scenarios

What happens when the handler crashes mid-processing?

test('crash mid-processing → retry succeeds', async () => {
  let crashOnce = true;
  const handler = makeHandler({
    onProcessEvent: () => {
      if (crashOnce) {
        crashOnce = false;
        throw new Error('simulated crash');
      }
    },
  });

  await expect(handler.process(event)).rejects.toThrow();  // First attempt crashes
  await handler.process(event);  // Retry succeeds; idempotent

  const record = await db.payments.findOne({ event_id: event.id });
  expect(record).toBeTruthy();
});

Handlers should commit state changes atomically - either the processing succeeds and the event is marked handled, or both roll back.

Replay-from-archive

Production sometimes loses webhooks (network outage, deploy issue). Per gateway docs, all support some form of replay:

GatewayReplay windowMethod
Stripe30 daysstripe events resend <event_id>
AdyenUnlimited (Customer Area)Manual or notification-resend API
PayPal30 daysWebhook resend endpoint
BraintreeUnlimited (Control Panel)Manual or webhookTesting.sampleNotification

Test:

test('handler accepts replay from 7-day-old event', async () => {
  const oldEvent = makeEvent({ created: Math.floor(Date.now()/1000) - 7*86400 });
  const result = await handler.process(oldEvent);
  expect(result).toBe(200);
});

Suite layout

One directory per gateway so per-gateway quirks stay isolated:

tests/payment/webhooks/
  stripe/
    signature.test.ts
    idempotency.test.ts
    order-tolerance.test.ts
    replay.test.ts
  adyen/
    ... (same structure)
  paypal/
    ...
  braintree/
    ...
  fixtures/
    payloads/

Related skills

3ds-test-flow-reference

Cross-gateway, protocol-level reference for 3-D Secure (3DS 2.x) test coverage. Covers the EMVCo frictionless / challenge / not-applicable flow paths, SCA under EU PSD2, and the per-PAN test cards for Stripe, Adyen, and Braintree. The gateway-specific wrappers (adyen-test-mode / stripe-test-cards-and-webhooks / braintree-test-cards) compose this skill for single-gateway work, so use one of those for a single-gateway query. Use this skill when designing multi-gateway 3DS test coverage, auditing a 3DS redirect round-trip, or investigating a challenge-flow regression that is not gateway-specific.

adyen-test-mode

Wraps Adyen test-mode patterns: test-API-key initialization, the canonical Adyen test cards (Visa 4111 1111 1111 1111; 5454 5454 5454 5454 3DS frictionless; 4917 6100 0000 0000 3DS 2 challenge), the per-flow result codes (Authorised / Refused / Pending / RedirectShopper / Error), the HMAC-SHA256 webhook validation, and notifications-vs-API duality (Adyen separates synchronous API calls from asynchronous notifications). Use when testing Adyen-integrated code.

braintree-test-cards

Wraps Braintree (PayPal-owned) sandbox testing patterns: sandbox merchant credentials, Drop-in / Hosted Fields client-side patterns, the Transaction lifecycle (submitted_for_settlement → settled), Braintree's distinctive test-card behaviours (specific PANs trigger specific errors), and the webhook verification (Braintree Webhook Parser). Use when testing Braintree-integrated code.

chargeback-flow-test-author

Workflow-driven skill that builds the chargeback / dispute test suite: canonical reason codes (Visa 10.4 fraud, 13.1 services-not-provided; Mastercard MCC 4855), per-gateway dispute APIs (Stripe Disputes, Adyen Chargeback notifications, PayPal Disputes), the evidence-submission flow + window, and disposition outcomes (won / lost / accepted). Use when designing dispute coverage; for refunds use refund-test-matrix-builder, for webhook redelivery + idempotency use payment-webhook-replay, and for the lifecycle state model use payment-flow-states-reference.

payment-flow-states-reference

Pure-reference catalog of payment lifecycle state machines across Stripe, Adyen, PayPal, and Braintree: canonical states (created / requires_action / processing / succeeded / requires_capture / canceled / failed), authorisation vs capture, asynchronous webhook states, and refund / dispute / chargeback transitions. Use when designing tests for payment flows or auditing state-handling code; this is the state model, not a builder - to author suites on it use refund-test-matrix-builder (refunds), chargeback-flow-test-author (disputes), or payment-webhook-replay (webhook replay).

paypal-sandbox

Wraps PayPal Sandbox testing patterns: sandbox account creation (Business + Personal accounts in developer.paypal.com), the Orders v2 API (create / capture / refund), webhook event simulator (developer.paypal.com webhook simulator), sandbox-account-specific test cards, and the OAuth2 client-credentials flow for sandbox. Use when testing PayPal-integrated code.

pci-dss-scope-reference

Pure-reference catalog of PCI DSS v4.0 scope reduction techniques + the testable scope boundaries. Covers the SAQ levels (A through D, picked by how cardholder data flows), the PAN-storage prohibitions (only first-6 + last-4 retained; nothing else cleartext), the tokenization + hosted-fields scope-reduction patterns (Stripe Elements / Adyen Drop-in / Braintree Hosted Fields keep PAN off your servers), Network-Segmentation as PCI scope-reduction, and the testable behaviours the scope boundary creates. This is the catalog of WHY the boundary matters and what it makes testable - not a checker that verifies a given integration against the standard. Use when designing or auditing the PCI scope of a payment integration.

refund-test-matrix-builder

Workflow-driven skill that builds a refund test matrix from a payment-flow inventory: refund variants (full / partial / multiple-partials / over-refund / refund-on-disputed / refund-on-already-refunded), per-gateway nuances (Stripe, Adyen, PayPal, Braintree refund APIs), and timing variants (immediate / next-day / declined-by-bank), one test case per cell. Use for refund coverage on a new integration; for chargeback / dispute coverage use chargeback-flow-test-author, for webhook redelivery + idempotency use payment-webhook-replay, and for the state model use payment-flow-states-reference.

stripe-subscription-billing-test-author

Builds test suites for Stripe recurring-billing flows: trial-to-paid conversion, proration on plan upgrade and downgrade, dunning on failed renewal, cancel and reactivation, and the full subscription webhook event matrix (invoice.payment_failed, customer.subscription.updated, customer.subscription.deleted, invoice.paid). Uses Stripe Billing test clocks (POST /v1/test_helpers/test_clocks) to time-travel through billing cycles without calendar delay. Distinct from stripe-test-cards-and-webhooks (one-time PaymentIntents) and payment-webhook-replay (idempotency + replay robustness). Does not cover single-event CLI replay or handler idempotency testing (see payment-webhook-replay for those). Use when authoring tests for subscription or recurring-billing integrations.

stripe-test-cards-and-webhooks

Wraps Stripe API testing patterns: test-mode initialization, the canonical test cards (4242 success; 4000 0000 0000 0002 declined; 4000 0027 6000 3184 3DS challenge per 3ds-test-flow-reference), the Stripe CLI webhook flow (`stripe listen --forward-to`), the Stripe CLI fixture commands (`stripe trigger payment_intent.succeeded`), and the webhook signature verification (Stripe-Signature header + HMAC-SHA256). Use when testing Stripe-integrated code.