varnish-test-vtc-syntax

Wraps the varnishtest CLI + VTC (Varnish Test Case) syntax for testing VCL configurations. Covers the VTC test-file format (varnishtest scripts with server { ... } + client { ... } + varnish v1 -vcl+backend { ... } blocks), the grace-mode + saint-mode behaviours (stale-while-revalidate + stale-if-error equivalents in VCL), the PURGE method handler pattern (vcl_purge subroutine + ACL guards), and surrogate-key invalidation via xkey vmod. Use when authoring or testing Varnish-based caching layers.

Install with skills.sh (any agent)

npx skills add testland/qa --skill varnish-test-vtc-syntax

View source

varnish-test-vtc-syntax

Overview

varnishtest (CLI) executes .vtc (Varnish Test Case) files that spin up a Varnish instance + a mock backend + a mock client in-process. Per varnish-cache.org/docs (opens in new window), VTC is the canonical way to test VCL - Varnish's configuration language - without a real network.

When to use

Authoring or modifying VCL.
Testing PURGE handlers, grace/saint mode, surrogate-key logic.
Regression-testing VCL after Varnish version bump.
CI gate before a VCL deploy.

Authoring

Install

Varnish ships varnishtest in the standard distribution:

apt install varnish        # Debian/Ubuntu
brew install varnish       # macOS
varnishtest -v             # verify

Anatomy of a VTC

varnishtest "basic cache hit test"

server s1 {
  rxreq
  txresp -hdr "Cache-Control: max-age=60" -body "hello"
} -start

varnish v1 -vcl+backend {
  // VCL goes here
} -start

client c1 {
  txreq -url "/"
  rxresp
  expect resp.status == 200
  expect resp.body == "hello"

  txreq -url "/"
  rxresp
  expect resp.http.x-cache == "HIT"   // assumes vcl_deliver sets this
} -run

varnish v1 -expect cache_hit == 1

Per Varnish docs, the VTC file has four block types:

Block	Purpose
`server sN`	Mock origin
`varnish vN`	Varnish instance with VCL
`client cN`	Send requests, assert responses
`barrier`, `delay`, `shell`	Synchronisation + setup

Testing PURGE

The canonical pattern from Varnish docs:

acl purge {
  "localhost";
  "10.0.0.0"/8;
}

sub vcl_recv {
  if (req.method == "PURGE") {
    if (!client.ip ~ purge) {
      return (synth(403, "Not allowed"));
    }
    return (purge);
  }
}

sub vcl_purge {
  return (synth(200, "Purged"));
}

The VTC:

client c1 {
  txreq -url "/foo"           // populate cache
  rxresp
  expect resp.status == 200

  txreq -url "/foo" -method "PURGE"
  rxresp
  expect resp.status == 200

  // Next request should hit origin again
  txreq -url "/foo"
  rxresp
  expect resp.http.x-cache == "MISS"
} -run

Grace mode (stale-while-revalidate equivalent)

Per Varnish docs and per stale-while-revalidate-reference:

sub vcl_backend_response {
  set beresp.grace = 1h;       // serve stale for 1h while async-refreshing
}

sub vcl_deliver {
  if (obj.ttl < 0s) {
    set resp.http.x-cache = "GRACE";
  } else if (obj.hits == 0) {
    set resp.http.x-cache = "MISS";
  } else {
    set resp.http.x-cache = "HIT";
  }
}

VTC for grace mode:

varnishtest "grace serves stale while refreshing"

server s1 {
  rxreq
  txresp -hdr "Cache-Control: max-age=1"

  rxreq
  delay 2                        // simulate slow refresh
  txresp -hdr "Cache-Control: max-age=1"
} -start

varnish v1 -vcl+backend {
  sub vcl_backend_response {
    set beresp.grace = 60s;
  }
} -start

client c1 {
  txreq -url "/"
  rxresp
  expect resp.status == 200

  delay 1.5                      // past TTL

  txreq -url "/"
  rxresp
  expect resp.status == 200
  expect resp.http.x-cache == "GRACE"
} -run

Surrogate-key (xkey vmod)

import xkey;

sub vcl_backend_response {
  set beresp.http.xkey = "user-1 posts-feed";   // tag the object
}

sub vcl_recv {
  if (req.method == "PURGE" && req.http.xkey-purge) {
    set req.http.X-Purges = xkey.softpurge(req.http.xkey-purge);
    return (synth(200));
  }
}

Running

varnishtest -v cache-tests.vtc
# Or: varnishtest -v tests/*.vtc

# Verbose with full Varnish log output:
varnishtest -vvv cache-tests.vtc

Exit code 0 = pass; non-zero = fail.

Parallel runs

varnishtest -j 4 tests/*.vtc

Each VTC runs in an isolated Varnish instance; parallel-safe.

Parsing results

varnishtest output:

**** v1   vsl:  0     SLT_End
**** v1   vsl_dispatch_complete
*    top  TEST cache-tests.vtc passed (1.34)

passed or FAILED. The failure output shows the failing expect line and the actual value.

For CI: parse passed / FAILED count in the summary.

CI integration

jobs:
  vcl-tests:
    runs-on: ubuntu-latest
    container: varnish:7
    steps:
      - uses: actions/checkout@v5
      - name: Run VCL tests
        run: varnishtest -v tests/vcl/*.vtc

Anti-patterns

Anti-pattern	Why it fails	Fix
Testing VCL by hitting a real Varnish in dev	Slow; environment-dependent	Use varnishtest - in-process
`delay 60` for TTL test	Tests slow; flaky	Set short TTL (max-age=1, delay 1.5)
Missing `-expect cache_hit == N`	Pass relies on observer-affects-system	Always assert cache counters
Untested PURGE ACL	Open PURGE = cache-flush DoS	Test 403 from external IP
No grace-mode test	Production grace surprises	Verify x-cache GRACE on stale fetch
One mega-VTC	Failures opaque	One concern per file
Skip `varnishlog` inspection	Hard to debug failures	Use `-vvv` in CI for failure logs
Hand-roll surrogate-key without xkey vmod	Manual ban-list grows; slow regex matches	Use xkey vmod

Limitations

VTC is Varnish-specific. Doesn't transfer to nginx / Apache cache testing.
Mock client/server is in-process. Doesn't catch network- layer issues (TLS handshake, slow-client, hostname resolution).
xkey vmod requires Varnish Plus or self-compile. Open-source Varnish has purge.soft but not surrogate keys natively.
Doesn't test CDN-edge behaviour. Varnish is one tier; full multi-tier tests need cdn-cache-purge-tests patterns.

References

Varnish documentation: varnish-cache.org/docs/ (opens in new window).
varnishtest reference: github.com/varnishcache/varnish-cache/.../reference/varnishtest.rst (opens in new window).
VTC syntax: github.com/varnishcache/varnish-cache/.../reference/vtc.rst (opens in new window).
Companion catalogs: cache-coherence-patterns-reference, stale-while-revalidate-reference, cache-stampede-reference.
Sibling tools: redis-cache-tests, cdn-cache-purge-tests, browser-cache-control-tests.

Related skills

browser-cache-control-tests

Wraps browser-side Cache-Control testing with Playwright (Cypress for legacy stacks): asserting response Cache-Control headers from Network events, ETag round-trips (If-None-Match → 304), service-worker strategies (Workbox cacheFirst / networkFirst / staleWhileRevalidate), and reload semantics (normal vs hard). Covers MDN Cache-Control + RFC 9111. Use when auditing browser-tier caching in E2E tests. For CDN-edge purge use cdn-cache-purge-tests; for the reverse-proxy tier use varnish-test-vtc-syntax; for an app-tier store use redis-cache-tests; SWR directive semantics live in stale-while-revalidate-reference.

cache-coherence-patterns-reference

Pure-reference catalog of cache-coherence patterns across the request path. Defines the five-tier cache stack (browser → CDN → reverse-proxy → application → data store), the per-tier cache-writing patterns (cache-aside, write-through, write-back, write-around, refresh-ahead), and the canonical invalidation strategies (TTL-only, event-driven purge, surrogate keys, version-tagged URLs, soft purge), plus an anti-pattern table and a worked multi-tenant coherence-test example. Deep detail - the RFC 9111 Cache-Control / Vary / ETag directive tables and the cross-tier coherence + per-tier test surface - lives in references/. Use for pattern selection, Cache-Control header design, and coherence audits; use a cache-key-collision check when the question is whether two requests in an existing system collide on a concrete key scheme. Consumed by redis-cache-tests, cdn-cache-purge-tests, varnish-test-vtc-syntax, browser-cache-control-tests, and the cache-key-collision check.

cache-key-discriminator-audit

Audits whether a cache key carries every discriminator the cached response actually depends on, so two requests that must not share a slot cannot collide. Ranks identity discriminators (tenant, user, authorization scope, plan tier) above presentation ones (locale, region, currency, feature flag), maps each missing discriminator to the data-exposure or wrong-content consequence it causes, classifies each key-and-value pair into a critical / high / medium severity band (including the Python lru_cache-on-an-instance-method trap), and writes the fix as a namespaced key builder plus the matching HTTP Vary header. Use when a cache key is being designed or changed, when a per-user or per-tenant response is about to be stored in a shared cache or CDN, or when investigating a report that one user or tenant saw another's data.

cache-stampede-reference

Pure-reference catalog of cache-stampede (thundering-herd) phenomena and mitigations: parallel misses on key expiry trigger simultaneous recomputation (often congestion-collapse), countered by three families - locking, external recomputation near-expiry, and probabilistic early expiration via XFetch (`(time() - delta * beta * log(rand(0,1))) >= expiry`). Use when designing cache-refresh strategy or diagnosing a stampede incident. This is the single failure-mode pattern; for the broader multi-tier pattern catalog use cache-coherence-patterns-reference, for the stale-while-revalidate / stale-if-error extensions use stale-while-revalidate-reference; a reference consumed by redis-cache-tests, not a runnable test.

cdn-cache-purge-tests

Wraps CDN cache-purge testing patterns for Cloudflare (POST /zones/{zone_id}/purge_cache, single-file / everything / cache-tags / hostname / prefix), Fastly (POST purge-by-key / purge-all, surrogate-keys via Surrogate-Key header), and CloudFront (CreateInvalidation API + paths). Covers end-to-end test patterns (write origin → trigger purge → assert edge serves fresh), purge-propagation delay testing (typically 1-30s globally), surrogate-key + cache-tag patterns for group-purge, and Cache-Status header verification (cf-cache-status: HIT/MISS/BYPASS). Use when designing or auditing CDN cache-invalidation workflows.

memcached-tests

Wraps Memcached cache testing against a real container: an inline set/get/expire/no-persistence worked example, with the exhaustive protocol-command tests (set/get/add/cas/incr/decr, TTL 0=never-expire / 30-day Unix-timestamp boundary) and the LRU-eviction, consistent-hashing distribution, ElastiCache Auto Discovery, and CI-wiring deep-dives in references/. Use when writing tests for an application that uses Memcached as its primary cache, when verifying ElastiCache Memcached cluster behaviour, or when contrasting Memcached eviction and distribution semantics against Redis.

redis-cache-tests

Wraps Redis cache testing: EXPIRE / PEXPIRE / TTL verification (Redis 7+ NX/XX/GT/LT flags), cache-aside write-then-invalidate (write source → DEL key → assert fresh read), eviction under memory pressure (maxmemory + allkeys-lru), pub/sub invalidation across nodes, and tenant key-namespacing. Use when Redis is the app's primary cache. For a Memcached app-tier store use memcached-tests; for the browser/HTTP tier use browser-cache-control-tests; a runnable test, not the cache-stampede-reference or cache-coherence-patterns-reference catalogs.

stale-while-revalidate-reference

Pure-reference catalog of RFC 5861's stale-while-revalidate + stale-if-error Cache-Control extensions. Defines stale-while-revalidate=N (caches MAY serve a stale response while asynchronously revalidating, up to N seconds after expiry) and stale-if-error=N (caches MAY serve stale on 5xx upstream errors). Distinguishes from RFC 9111's must-revalidate (forbids serving stale) and from manual cache-aside refresh (synchronous). Covers the interaction with the freshness lifetime (max-age) and the cache-stampede-mitigation properties. Use when designing the cache-refresh boundary or auditing existing Cache-Control headers.