payment-webhook-replay
Workflow-driven skill that builds payment webhook replay + recovery tests. Covers the idempotency contract (every webhook handler must handle redelivery without side effects), the replay simulators (Stripe CLI `stripe trigger`, Adyen Customer Area resend, PayPal Webhook Simulator, Braintree webhook test), the signature-verification gauntlet (HMAC-SHA256 per gateway, expired-timestamps rejection), and the partial-failure recovery scenarios. Use when designing webhook robustness tests.
Install with skills.sh (any agent)
npx skills add testland/qa --skill payment-webhook-replaypayment-webhook-replay
Overview
Payment webhooks are the source of truth for asynchronous state transitions (settlement, refund completion, dispute state). Every webhook handler must be:
This skill produces the test suite for these properties.
When to use
How to use
Step 1 - Signature-verification gauntlet
Per docs.stripe.com/webhooks/signatures (opens in new window), the Stripe gauntlet is four cases - unsigned, wrong-secret, valid, and expired-timestamp:
const payload = JSON.stringify({ type: 'payment_intent.succeeded' });
test('rejects unsigned payload', async () => {
const res = await fetch('/webhooks/stripe', { method: 'POST', body: payload });
expect(res.status).toBe(401); // No Stripe-Signature header
});
test('rejects wrong-secret signature', async () => {
const wrongSig = stripe.webhooks.generateTestHeaderString({ payload, secret: 'wrong-secret' });
const res = await fetch('/webhooks/stripe', {
method: 'POST', body: payload, headers: { 'stripe-signature': wrongSig },
});
expect(res.status).toBe(401);
});
test('accepts valid signature', async () => {
const sig = stripe.webhooks.generateTestHeaderString({
payload, secret: process.env.STRIPE_WEBHOOK_SECRET!,
});
const res = await fetch('/webhooks/stripe', {
method: 'POST', body: payload, headers: { 'stripe-signature': sig },
});
expect(res.status).toBe(200);
});
test('rejects expired timestamp', async () => {
const oldSig = stripe.webhooks.generateTestHeaderString({
payload,
secret: process.env.STRIPE_WEBHOOK_SECRET!,
timestamp: Math.floor(Date.now()/1000) - 3600, // 1 hour ago
});
const res = await fetch('/webhooks/stripe', {
method: 'POST', body: payload, headers: { 'stripe-signature': oldSig },
});
expect(res.status).toBe(401);
});The signature scheme differs per gateway; run the same four-case gauntlet against each:
| Gateway | Signature scheme | Reference |
|---|---|---|
| Stripe | HMAC-SHA256 over the payload; the signature carries a timestamp, so old timestamps reject | webhooks/signatures (opens in new window) |
| Adyen | HMAC-SHA256 over the canonical string; validated per-event, not per-request | verify-hmac-signatures (opens in new window) |
| PayPal | SHA256-with-RSA; verify via the PayPal verification endpoint or SDK helper | webhooks/rest (opens in new window) |
| Braintree | parser validates bt_signature against the merchant's public key | Braintree webhook parser |
Step 2 - Idempotency
Redelivery must not double-process. Dedup on the gateway-issued event ID:
test('redelivered webhook handled idempotently', async () => {
const payload = makeWebhookPayload({ type: 'payment_intent.succeeded' });
const sig = signPayload(payload);
const before = await db.payment_records.count();
await postWebhook(payload, sig);
const after1 = await db.payment_records.count();
await postWebhook(payload, sig); // Redelivery
const after2 = await db.payment_records.count();
expect(after1 - before).toBe(1);
expect(after2).toBe(after1);
});The handler looks up by event ID and acks duplicates without re-doing the work:
async function handleEvent(event) {
const existing = await db.events.findOne({ event_id: event.id });
if (existing) {
return 200; // Already handled; safe to ack
}
await processEvent(event);
await db.events.create({ event_id: event.id, processed_at: new Date() });
return 200;
}Step 3 - Replay simulators
Each gateway ships a way to (re)send a real event at the handler:
| Gateway | Replay | Notes |
|---|---|---|
| Stripe | stripe trigger payment_intent.succeeded; stripe events resend evt_test_12345 | CLI (stripe-cli (opens in new window)) |
| Adyen | Customer Area transaction "Resend webhook" | Re-sends with the original signature - useful for idempotency tests |
| PayPal | Dashboard simulator; REST equivalent via the API | simulate-event (opens in new window) |
| Braintree | gateway.webhookTesting.sampleNotification(kind, id) | Generates a test signature for any event kind (parse/node (opens in new window)) |
Worked example
Replay one gateway's event (Stripe) end to end and assert both core properties - the signature is accepted and a redelivery is idempotent.
Drive a real event at the handler with the CLI:
stripe trigger payment_intent.succeeded # synthetic event to the forward URL
stripe events resend evt_test_12345 # replay a captured event (30-day window)Then assert the guarantees in-process:
test('replayed Stripe event: signature accepted, idempotent on resend', async () => {
const payload = makeWebhookPayload({ type: 'payment_intent.succeeded' });
const sig = stripe.webhooks.generateTestHeaderString({
payload: JSON.stringify(payload),
secret: process.env.STRIPE_WEBHOOK_SECRET!,
});
const first = await postWebhook(payload, sig);
expect(first.status).toBe(200); // valid signature accepted
const resend = await postWebhook(payload, sig); // same event redelivered
expect(resend.status).toBe(200);
const rows = await db.payment_records.count({ event_id: payload.id });
expect(rows).toBe(1); // processed exactly once
});That is the minimum end-to-end proof for one gateway: a real replayed event passes signature verification and processes exactly once. Extend it to out-of-order, crash-recovery, and archive-replay scenarios in references/advanced-recovery-scenarios.md.
Anti-patterns
| Anti-pattern | Why it fails | Fix |
|---|---|---|
| Skip signature verification | Spoofed webhook payloads | Always verify |
| Skip idempotency | Replay double-processes | Event-ID dedup table |
| Trust HTTP 200 == processed | Server may have crashed | Atomic commit + event-ID record |
| Hardcoded webhook secrets in tests | Leaked via test snapshots | Env vars |
| No "future-dated" webhook test | Clock skew + redelivery | Test +5min and -5min |
| Single-platform tests only | Per-gateway quirks | Per-gateway test directory |
body-parser ate the raw bytes | Signature verification fails on parsed-then-restringified | Raw-body middleware |
| No partial-failure / retry test | Crash mid-handler corrupts state | Atomic transaction with event-ID |
Limitations
References
Webhook order-tolerance, partial-failure, and archive-replay scenarios
View source (opens in new window)Webhook order-tolerance, partial-failure, and archive-replay scenarios
Deep reference for payment-webhook-replay SKILL.md. Consult once the core signature + idempotency + replay-simulator surface is in place and the suite needs the harder recovery scenarios: out-of-order delivery, mid-handler crash recovery, replay of old events from the gateway archive, and the per-gateway suite layout.
Order-tolerance tests
Webhooks can arrive out of order:
test('out-of-order event delivery handled', async () => {
const completedEvent = makeEvent({ type: 'payment_intent.succeeded' });
const creatingEvent = makeEvent({ type: 'payment_intent.created' });
// Deliver completed BEFORE created
await postWebhook(completedEvent);
await postWebhook(creatingEvent);
// Final state should still be correct
const record = await db.payments.findOne({ intent_id: completedEvent.data.id });
expect(record.status).toBe('succeeded');
});The handler must use versioned events or state-machine gates to handle this:
# Don't blindly overwrite state
def handle_event(event):
record = db.payments.get(event.intent_id)
new_state = event.data.status
if state_transition_allowed(record.status, new_state):
record.status = new_state
record.save()
# else: stale event, ignorePartial-failure scenarios
What happens when the handler crashes mid-processing?
test('crash mid-processing → retry succeeds', async () => {
let crashOnce = true;
const handler = makeHandler({
onProcessEvent: () => {
if (crashOnce) {
crashOnce = false;
throw new Error('simulated crash');
}
},
});
await expect(handler.process(event)).rejects.toThrow(); // First attempt crashes
await handler.process(event); // Retry succeeds; idempotent
const record = await db.payments.findOne({ event_id: event.id });
expect(record).toBeTruthy();
});Handlers should commit state changes atomically - either the processing succeeds and the event is marked handled, or both roll back.
Replay-from-archive
Production sometimes loses webhooks (network outage, deploy issue). Per gateway docs, all support some form of replay:
| Gateway | Replay window | Method |
|---|---|---|
| Stripe | 30 days | stripe events resend <event_id> |
| Adyen | Unlimited (Customer Area) | Manual or notification-resend API |
| PayPal | 30 days | Webhook resend endpoint |
| Braintree | Unlimited (Control Panel) | Manual or webhookTesting.sampleNotification |
Test:
test('handler accepts replay from 7-day-old event', async () => {
const oldEvent = makeEvent({ created: Math.floor(Date.now()/1000) - 7*86400 });
const result = await handler.process(oldEvent);
expect(result).toBe(200);
});Suite layout
One directory per gateway so per-gateway quirks stay isolated:
tests/payment/webhooks/
stripe/
signature.test.ts
idempotency.test.ts
order-tolerance.test.ts
replay.test.ts
adyen/
... (same structure)
paypal/
...
braintree/
...
fixtures/
payloads/Related skills
3ds-test-flow-reference
Cross-gateway, protocol-level reference for 3-D Secure (3DS 2.x) test coverage. Covers the EMVCo frictionless / challenge / not-applicable flow paths, SCA under EU PSD2, and the per-PAN test cards for Stripe, Adyen, and Braintree. The gateway-specific wrappers (adyen-test-mode / stripe-test-cards-and-webhooks / braintree-test-cards) compose this skill for single-gateway work, so use one of those for a single-gateway query. Use this skill when designing multi-gateway 3DS test coverage, auditing a 3DS redirect round-trip, or investigating a challenge-flow regression that is not gateway-specific.
adyen-test-mode
Wraps Adyen test-mode patterns: test-API-key initialization, the canonical Adyen test cards (Visa 4111 1111 1111 1111; 5454 5454 5454 5454 3DS frictionless; 4917 6100 0000 0000 3DS 2 challenge), the per-flow result codes (Authorised / Refused / Pending / RedirectShopper / Error), the HMAC-SHA256 webhook validation, and notifications-vs-API duality (Adyen separates synchronous API calls from asynchronous notifications). Use when testing Adyen-integrated code.
braintree-test-cards
Wraps Braintree (PayPal-owned) sandbox testing patterns: sandbox merchant credentials, Drop-in / Hosted Fields client-side patterns, the Transaction lifecycle (submitted_for_settlement → settled), Braintree's distinctive test-card behaviours (specific PANs trigger specific errors), and the webhook verification (Braintree Webhook Parser). Use when testing Braintree-integrated code.
chargeback-flow-test-author
Workflow-driven skill that builds the chargeback / dispute test suite: canonical reason codes (Visa 10.4 fraud, 13.1 services-not-provided; Mastercard MCC 4855), per-gateway dispute APIs (Stripe Disputes, Adyen Chargeback notifications, PayPal Disputes), the evidence-submission flow + window, and disposition outcomes (won / lost / accepted). Use when designing dispute coverage; for refunds use refund-test-matrix-builder, for webhook redelivery + idempotency use payment-webhook-replay, and for the lifecycle state model use payment-flow-states-reference.
payment-flow-states-reference
Pure-reference catalog of payment lifecycle state machines across Stripe, Adyen, PayPal, and Braintree: canonical states (created / requires_action / processing / succeeded / requires_capture / canceled / failed), authorisation vs capture, asynchronous webhook states, and refund / dispute / chargeback transitions. Use when designing tests for payment flows or auditing state-handling code; this is the state model, not a builder - to author suites on it use refund-test-matrix-builder (refunds), chargeback-flow-test-author (disputes), or payment-webhook-replay (webhook replay).
paypal-sandbox
Wraps PayPal Sandbox testing patterns: sandbox account creation (Business + Personal accounts in developer.paypal.com), the Orders v2 API (create / capture / refund), webhook event simulator (developer.paypal.com webhook simulator), sandbox-account-specific test cards, and the OAuth2 client-credentials flow for sandbox. Use when testing PayPal-integrated code.
pci-dss-scope-reference
Pure-reference catalog of PCI DSS v4.0 scope reduction techniques + the testable scope boundaries. Covers the SAQ levels (A through D, picked by how cardholder data flows), the PAN-storage prohibitions (only first-6 + last-4 retained; nothing else cleartext), the tokenization + hosted-fields scope-reduction patterns (Stripe Elements / Adyen Drop-in / Braintree Hosted Fields keep PAN off your servers), Network-Segmentation as PCI scope-reduction, and the testable behaviours the scope boundary creates. This is the catalog of WHY the boundary matters and what it makes testable - not a checker that verifies a given integration against the standard. Use when designing or auditing the PCI scope of a payment integration.
refund-test-matrix-builder
Workflow-driven skill that builds a refund test matrix from a payment-flow inventory: refund variants (full / partial / multiple-partials / over-refund / refund-on-disputed / refund-on-already-refunded), per-gateway nuances (Stripe, Adyen, PayPal, Braintree refund APIs), and timing variants (immediate / next-day / declined-by-bank), one test case per cell. Use for refund coverage on a new integration; for chargeback / dispute coverage use chargeback-flow-test-author, for webhook redelivery + idempotency use payment-webhook-replay, and for the state model use payment-flow-states-reference.
stripe-subscription-billing-test-author
Builds test suites for Stripe recurring-billing flows: trial-to-paid conversion, proration on plan upgrade and downgrade, dunning on failed renewal, cancel and reactivation, and the full subscription webhook event matrix (invoice.payment_failed, customer.subscription.updated, customer.subscription.deleted, invoice.paid). Uses Stripe Billing test clocks (POST /v1/test_helpers/test_clocks) to time-travel through billing cycles without calendar delay. Distinct from stripe-test-cards-and-webhooks (one-time PaymentIntents) and payment-webhook-replay (idempotency + replay robustness). Does not cover single-event CLI replay or handler idempotency testing (see payment-webhook-replay for those). Use when authoring tests for subscription or recurring-billing integrations.
stripe-test-cards-and-webhooks
Wraps Stripe API testing patterns: test-mode initialization, the canonical test cards (4242 success; 4000 0000 0000 0002 declined; 4000 0027 6000 3184 3DS challenge per 3ds-test-flow-reference), the Stripe CLI webhook flow (`stripe listen --forward-to`), the Stripe CLI fixture commands (`stripe trigger payment_intent.succeeded`), and the webhook signature verification (Stripe-Signature header + HMAC-SHA256). Use when testing Stripe-integrated code.