Testland
Browse all skills & agents

idempotency-test-author

Build-an-X for idempotency tests in any async/job/API context - idempotency-key handling (per Stripe / AWS prescriptive guidance pattern), retry-safe semantics (exactly-once vs at-least-once vs at-most-once), side-effect commutativity verification, fingerprint-based dedup, idempotency-window tuning. Use when authoring tests for any system where the same input could be processed twice (SQS Standard at-least-once, RabbitMQ requeue, retry-on-error logic, webhook redelivery, browser double-click, mobile-network retry).

idempotency-test-author

Overview

Idempotency = the property that processing the same input twice produces the same effect as processing it once. Without it, every async system silently corrupts data on retry.

Per AWS Prescriptive Guidance (docs.aws.amazon.com/prescriptive-guidance/latest/build-idempotent-applications):

Idempotency is essential for:

  • At-least-once delivery systems (SQS Standard, BullMQ retry, Sidekiq retry, webhook redelivery)
  • Retry-on-error patterns (HTTP 5xx → retry)
  • Network-failure recovery (mobile clients re-submit on timeout)
  • User actions that can replay (browser back-button, refresh)

Per Stripe's idempotency docs (stripe.com/docs/api/idempotent_requests), the canonical pattern is idempotency keys: the client passes a unique key per logical operation; the server records key → response and returns the cached response on duplicates.

When to use

  • Authoring tests for any handler that mutates state on input.
  • Reviewing PRs that add a new POST/PATCH endpoint or job processor.
  • The system is on at-least-once delivery (SQS Standard, RabbitMQ requeue) - idempotency tests are mandatory, not optional.
  • Webhook receivers that must handle vendor-side redelivery.

Step 1 - Classify delivery semantics

SemanticsExamplesTest requirement
Exactly-onceSQS FIFO, Kafka with EOSIdempotency tests are nice-to-have
At-least-onceSQS Standard, RabbitMQ requeue, BullMQ retry, webhook redeliveryIdempotency tests MANDATORY
At-most-onceUDP, fire-and-forgetIdempotency irrelevant; data-loss tests instead

Most production systems are at-least-once (or are at-least-once in failure modes). Default to mandatory tests.

Step 2 - Idempotency-key pattern

The canonical pattern (per Stripe):

  1. Client generates a unique key per logical operation (UUID).
  2. Client passes the key with every retry of that operation.
  3. Server stores (key, request_hash, response) on first request.
  4. Server returns the cached response on duplicate keys.
from typing import Tuple

class IdempotentEndpoint:
    def __init__(self, store):
        self.store = store

    def post_charge(self, idempotency_key: str, charge_data: dict) -> Tuple[int, dict]:
        cached = self.store.get(idempotency_key)
        if cached:
            # Duplicate request: return cached response
            return cached["status"], cached["body"]

        # First request: process + store
        result = process_charge(charge_data)
        self.store.set(idempotency_key, {"status": 200, "body": result})
        return 200, result

Test pattern:

def test_duplicate_idempotency_key_returns_cached_response(endpoint, store):
    key = "client-uuid-123"
    charge = {"amount": 100, "currency": "USD"}

    status1, body1 = endpoint.post_charge(key, charge)
    status2, body2 = endpoint.post_charge(key, charge)

    assert (status1, body1) == (status2, body2)

    # And only one charge was actually executed:
    assert charge_processor.execute.call_count == 1

Step 3 - Hash mismatch on key reuse

If a client reuses an idempotency key with a DIFFERENT body, the server must reject (per Stripe spec):

def test_idempotency_key_with_different_body_rejected(endpoint):
    key = "client-uuid-456"
    endpoint.post_charge(key, {"amount": 100})

    with pytest.raises(IdempotencyConflictError):
        endpoint.post_charge(key, {"amount": 200})   # same key, different body

This catches client bugs (key not properly scoped to one logical operation).

Step 4 - Side-effect commutativity for non-key designs

Some systems can't add idempotency keys (legacy webhook receivers, existing APIs). For these, design idempotent side effects:

# NON-idempotent (counter increment):
def credit_account(account_id, amount):
    db.execute("UPDATE accounts SET balance = balance + %s WHERE id = %s",
               (amount, account_id))

# IDEMPOTENT (use a transaction-id / fingerprint):
def credit_account(account_id, amount, txn_id):
    cursor = db.execute(
        "INSERT INTO transactions(txn_id, account_id, amount) VALUES (%s, %s, %s) "
        "ON CONFLICT (txn_id) DO NOTHING RETURNING id",
        (txn_id, account_id, amount)
    )
    if cursor.rowcount == 0:
        return  # duplicate; skip
    db.execute("UPDATE accounts SET balance = balance + %s WHERE id = %s",
               (amount, account_id))

Test pattern:

def test_credit_idempotent_via_txn_id():
    credit_account(account_id=1, amount=100, txn_id="t-1")
    credit_account(account_id=1, amount=100, txn_id="t-1")   # duplicate
    assert get_balance(1) == 100   # not 200

Step 5 - Idempotency-window tuning

Idempotency keys consume storage; choose a TTL based on the maximum expected retry window. Common choices:

SystemRecommended TTL
Stripe API24 hours (per Stripe docs)
Internal HTTP retries1 hour
SQS at-least-once consumersMatch SQS message retention (default 4 days)
Webhook receivers7 days (vendors retry over multi-day windows)

Test pattern:

def test_idempotency_key_expires(endpoint, freezer):
    freezer.move_to("2026-05-06 00:00:00")
    endpoint.post_charge("key-1", charge)

    freezer.move_to("2026-05-07 00:01:00")   # 24h + 1min later
    # Key has expired; same key now treated as new request:
    endpoint.post_charge("key-1", charge)
    assert charge_processor.execute.call_count == 2

Step 6 - Race-condition test (concurrent duplicate)

The hardest case: two requests with the same idempotency key arrive simultaneously. Without atomic store + check, both can pass the "is this duplicate?" check.

Test pattern:

def test_concurrent_duplicate_processed_only_once(endpoint, charge_processor):
    key = "race-key"
    charge = {"amount": 100}

    with ThreadPoolExecutor(max_workers=2) as executor:
        f1 = executor.submit(endpoint.post_charge, key, charge)
        f2 = executor.submit(endpoint.post_charge, key, charge)
        r1, r2 = f1.result(), f2.result()

    assert r1 == r2
    assert charge_processor.execute.call_count == 1   # NOT 2

The implementation must use atomic CAS (e.g., DB unique constraint on idempotency_key column with ON CONFLICT DO NOTHING, or Redis SETNX).

Step 7 - End-to-end test recipe per handler

For each at-least-once handler:

  1. ✅ Single-request happy path test (baseline)
  2. ✅ Duplicate-key test (Step 2)
  3. ✅ Different-body-same-key rejection test (Step 3)
  4. ✅ TTL expiry test (Step 5)
  5. ✅ Concurrent-duplicate race test (Step 6)
  6. ✅ For event-driven systems: test that re-delivery from the broker (SQS message redelivery, RabbitMQ requeue) doesn't double-process

Anti-patterns

Anti-patternWhy it failsFix
Test only the first requestMisses every retry/duplicate scenarioAlways include Step 2 + 6
Idempotency check via SELECT-then-INSERTRace between SELECT and INSERT; concurrent duplicates both passAtomic CAS (Step 6)
Forget TTL on idempotency-key storeStorage grows unbounded; eventual outageSet TTL per system (Step 5)
Counter-based side effects without txn_id dedupIdempotency-broken even with idempotency keys aboveRefactor to commutative ops (Step 4)
Skip concurrent testMost race conditions only surface under loadAlways include Step 6

Limitations

  • This is a build-an-X workflow, not a tool wrapper. Tests use the application's existing test framework (pytest, JUnit, RSpec, etc.).
  • Idempotency-key TTL trades storage for safety; tune per business need.
  • Some operations are inherently non-idempotent (e.g., "send a one-time SMS"); document and accept the risk, or design a delivery-receipt protocol.
  • Distributed-transaction idempotency is out of scope; see saga-transaction-tests in qa-saga-cqrs.

References

  • aws-idem - AWS Prescriptive Guidance on building idempotent applications
  • stripe.com/docs/api/idempotent_requests - Stripe's idempotency-key pattern (the de facto industry standard)
  • sqs-patterns - Standard SQS is at-least-once; idempotency tests are mandatory
  • rabbitmq-patterns - requeue + redelivery semantics need idempotent consumers
  • cron-job-test-author - cron jobs need idempotency for safe overlap recovery
  • webhook-delivery-tester - cross-plugin: webhook receivers need idempotency