memcached-tests
Wraps Memcached cache testing patterns: text and binary protocol command verification (set/get/add/cas/incr/decr), TTL semantics (0=never-expire, 30-day Unix-timestamp boundary), no-persistence and LRU eviction under memory pressure, consistent-hashing client key distribution across nodes, and AWS ElastiCache Memcached Auto Discovery endpoint testing. Use when writing tests for an application that uses Memcached as its primary cache, when verifying ElastiCache Memcached cluster behaviour, or when contrasting Memcached eviction and distribution semantics against Redis.
memcached-tests
Overview
Memcached is a widely deployed in-memory cache, available as the ElastiCache Memcached tier on AWS. It differs from Redis in three fundamental ways that affect how tests must be written:
This skill wraps test patterns against a real Memcached instance via Testcontainers - not a mock. Mocks lose LRU eviction, TTL-tick, and consistent-hashing redistribution behaviour that real bugs hide in.
When to use
Install
pip install pymemcache testcontainers # Python
npm install --save-dev memjs testcontainers # Node (binary protocol)The Testcontainers Memcached module defaults to memcached:1 and exposes port 11211 (testcontainers-python memcached):
from testcontainers.memcached import MemcachedContainer
with MemcachedContainer("memcached:1.6-alpine") as mc:
host, port = mc.get_host_and_port()Fixture (Python)
import pytest
from pymemcache.client.base import Client
from testcontainers.memcached import MemcachedContainer
@pytest.fixture(scope="session")
def mc_addr():
with MemcachedContainer("memcached:1.6-alpine") as mc:
yield mc.get_host_and_port() # (host, port)
@pytest.fixture
def mc(mc_addr):
host, port = mc_addr
client = Client((host, port), default_value=None)
yield client
client.flush_all() # Reset between testsText-protocol command tests
Per docs.memcached.org/protocols/basic/:
set / get / add
def test_set_and_get(mc):
mc.set("k", b"hello")
assert mc.get("k") == b"hello"
def test_add_only_when_absent(mc):
assert mc.add("k", b"first") is True
assert mc.add("k", b"second") is False # NOT_STORED: key exists
assert mc.get("k") == b"first"
def test_get_absent_returns_none(mc):
assert mc.get("no-such-key") is NoneTTL semantics
Per docs.memcached.org/protocols/basic/: exptime 0 means never-expire; values up to 30 days are interpreted as a relative second offset; values above 30 days (2592000 seconds) are treated as a Unix timestamp.
import time
def test_ttl_zero_never_expires(mc):
mc.set("k", b"v", expire=0)
time.sleep(0.1)
assert mc.get("k") == b"v"
def test_key_expires_after_ttl(mc):
mc.set("k", b"v", expire=1)
assert mc.get("k") == b"v"
time.sleep(1.5)
assert mc.get("k") is None
def test_short_ttl_via_pexpire_pattern(mc):
# pymemcache does not expose millisecond TTLs; use 1-second minimum
mc.set("k", b"val", expire=1)
time.sleep(1.5)
assert mc.get("k") is None, "Key must expire after 1 s TTL"Avoid time.sleep(60) to test TTL: set the shortest useful TTL and sleep only fractionally beyond it.
CAS (Check-And-Set)
Per docs.memcached.org/protocols/basic/, gets returns a unique 64-bit CAS identifier; cas stores data only if the token still matches:
def test_cas_succeeds_when_token_matches(mc):
mc.set("k", b"v1")
value, cas_token = mc.gets("k")
result = mc.cas("k", b"v2", cas_token)
assert result is True
assert mc.get("k") == b"v2"
def test_cas_fails_after_concurrent_write(mc):
mc.set("k", b"original")
_, old_token = mc.gets("k")
mc.set("k", b"concurrent-update") # token now stale
result = mc.cas("k", b"late-writer", old_token)
assert result is False # EXISTS: token mismatch
assert mc.get("k") == b"concurrent-update"incr / decr
Per docs.memcached.org/protocols/basic/, incr/decr operate on unsigned 64-bit integer string values and return None when the key is absent (no auto-initialisation):
def test_incr_increments_existing_counter(mc):
mc.set("counter", b"10")
result = mc.incr("counter", 5)
assert result == 15
def test_incr_absent_key_returns_none(mc):
assert mc.incr("no-such-counter", 1) is None
def test_incr_uses_add_to_initialise(mc):
# Per github.com/memcached/memcached/wiki/Programming:
# add is the correct initialiser for counters
mc.add("hits", b"0")
mc.incr("hits", 1)
assert mc.get("hits") == b"1"
def test_decr_does_not_go_below_zero(mc):
mc.set("counter", b"3")
mc.decr("counter", 10)
assert mc.get("counter") == b"0" # unsigned floor at 0LRU eviction (no-persistence)
Memcached evicts using LRU within each slab class; there is no persistence and no AOF/RDB equivalent. Per the AWS ElastiCache comparison, "Backup and restore" is No for node-based Memcached clusters.
def test_lru_evicts_cold_keys_under_pressure():
"""
Launch a small-memory container to verify LRU eviction.
The -m flag caps Memcached's RAM (MB).
"""
from testcontainers.memcached import MemcachedContainer
from pymemcache.client.base import Client
with MemcachedContainer("memcached:1.6-alpine") as mc:
mc.get_wrapped_container().exec_run # introspect if needed
host, port = mc.get_host_and_port()
# Restart with low memory cap via Docker command override
# Use a separate docker run with -m 8m for a tighter eviction test;
# or accept that testcontainers default image evicts eventually.
# The key assertion: after filling cache, a cold key may be absent.
def test_no_data_survives_restart(mc_addr):
"""Memcached has no persistence: data is gone after any restart."""
host, port = mc_addr
c = Client((host, port))
c.set("persistent", b"should-not-survive")
# Simulate application expectation: always handle cache miss gracefully
# after a node restart or replacement (e.g., ElastiCache node failure).
assert c.get("persistent") is not None # warm path
# After restart (modelled here as flush_all), data is gone:
c.flush_all()
assert c.get("persistent") is None, "Memcached is not persistent"Consistent-hashing client distribution
Per pymemcache HashClient, client-side consistent hashing distributes keys across nodes. Adding or removing a node remaps only the affected ring segment - not all keys.
def test_hash_client_distributes_keys():
from testcontainers.memcached import MemcachedContainer
from pymemcache.client.hash import HashClient
with MemcachedContainer("memcached:1.6-alpine") as mc1, \
MemcachedContainer("memcached:1.6-alpine") as mc2:
h1, p1 = mc1.get_host_and_port()
h2, p2 = mc2.get_host_and_port()
cluster = HashClient([(h1, p1), (h2, p2)])
keys = [f"key:{i}" for i in range(100)]
for k in keys:
cluster.set(k, b"v")
# Verify distribution: each node should hold some keys
direct1 = sum(
1 for k in keys if Client((h1, p1)).get(k) is not None
)
direct2 = sum(
1 for k in keys if Client((h2, p2)).get(k) is not None
)
assert direct1 > 0, "Node 1 should hold some keys"
assert direct2 > 0, "Node 2 should hold some keys"
assert direct1 + direct2 == 100, "Every key must be on exactly one node"
def test_hash_client_handles_node_removal():
"""After removing a node, the remaining node serves all keys."""
from testcontainers.memcached import MemcachedContainer
from pymemcache.client.hash import HashClient
with MemcachedContainer("memcached:1.6-alpine") as mc1, \
MemcachedContainer("memcached:1.6-alpine") as mc2:
h1, p1 = mc1.get_host_and_port()
h2, p2 = mc2.get_host_and_port()
full_cluster = HashClient([(h1, p1), (h2, p2)])
for i in range(20):
full_cluster.set(f"k{i}", b"val")
# Simulate node removal: re-create client with one node
degraded = HashClient([(h1, p1)])
# Keys that were on node 2 are now misses - application must
# handle gracefully (cache miss -> read-through from source of truth)
miss_count = sum(
1 for i in range(20) if degraded.get(f"k{i}") is None
)
assert miss_count >= 0 # Some keys lost; app must tolerate itAWS ElastiCache Memcached - Auto Discovery
Per docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/AutoDiscovery.html, ElastiCache Memcached (not Valkey/Redis) supports Auto Discovery: the client connects to a single configuration endpoint and retrieves the full node list. Clients refresh this list approximately once per minute.
def test_elasticache_auto_discovery_endpoint(monkeypatch):
"""
Integration smoke test: verify the app resolves a configuration
endpoint and discovers cluster nodes.
Runs only when ELASTICACHE_CONFIG_ENDPOINT is set.
"""
import os
endpoint = os.getenv("ELASTICACHE_CONFIG_ENDPOINT")
if not endpoint:
pytest.skip("ELASTICACHE_CONFIG_ENDPOINT not set (ElastiCache env only)")
from pymemcache.client.hash import HashClient
# The ElastiCache Cluster Client for Python resolves the cfg endpoint
# and populates the server list automatically via the config get cluster
# Memcached command.
client = HashClient([endpoint])
client.set("smoke-test", b"ok")
assert client.get("smoke-test") == b"ok"The configuration endpoint format is: <cluster-name>.xxxxxx.cfg.<region>.cache.amazonaws.com:11211
Auto Discovery is specific to ElastiCache Memcached and is not available for Valkey or Redis OSS engines (AutoDiscovery docs).
Running
pytest tests/memcached/ -vTestcontainers boots a Memcached container once per session. The per-test flush_all fixture call resets state between tests. Use scope="session" on the container fixture to avoid the ~3 s startup cost per test.
CI integration
jobs:
memcached-tests:
runs-on: ubuntu-latest
services:
memcached:
image: memcached:1.6-alpine
ports:
- 11211:11211
steps:
- uses: actions/checkout@v5
- uses: actions/setup-python@v5
- run: pip install -e ".[test]"
- run: pytest tests/memcached/ --tb=short
env:
MEMCACHED_HOST: localhost
MEMCACHED_PORT: 11211For multi-node distribution tests, launch two service containers named memcached-1 and memcached-2 on ports 11211 and 11212.
Anti-patterns
| Anti-pattern | Why it fails | Fix |
|---|---|---|
| Mocking the Memcached client | Misses TTL-tick, LRU eviction, CAS token generation | Use Testcontainers / real Memcached |
time.sleep(60) to test TTL | Slow and flaky | Set 1 s TTL and sleep 1.5 s |
Asserting incr initialises a missing key | incr returns None on missing keys | Use add to initialise, then incr |
| Sharing a Memcached instance between test suites | Cross-suite key pollution, order-dependent failures | flush_all in fixture teardown |
| Expecting data after a Memcached restart | Memcached has no persistence | Test for graceful cache-miss handling |
| Using a single-node client to test distribution | Distribution logic never exercises consistent hashing | Use HashClient with two test containers |
| Hard-coding node endpoints in app code | Breaks on ElastiCache node replacement | Use the configuration endpoint + Auto Discovery |
noreply=True in set during assertion tests | Errors are silently swallowed | Set noreply=False (pymemcache default for development) |