serverless-integration-test-builder
Workflow-driven skill that builds the integration-test suite for a serverless application from its IaC definition (SAM template / serverless.yml / Wrangler config / Vercel functions / Netlify functions). Walks through: identifying the function inventory + event sources, picking the right local-emulator per function (sam local / Miniflare / netlify dev / vercel dev / serverless-offline), generating test events per event source, asserting on cold-start + timeout budgets, and emitting the test directory + CI config. Use when introducing integration tests to a serverless project.
Install with skills.sh (any agent)
npx skills add testland/qa --skill serverless-integration-test-builderserverless-integration-test-builder
Overview
Serverless integration tests are awkward because the test surface is multi-tool: each platform has its own local emulator, event-source binding model, and cold-start characteristic. This skill produces a coherent test directory from the IaC definition.
The output: a test directory layout + per-function test files
When to use
Step 1 - Inventory functions from IaC
Per platform:
| Platform | IaC source | Where functions live |
|---|---|---|
| AWS SAM | template.yaml AWS::Serverless::Function | Resources.<name> |
| Serverless Framework | serverless.yml functions: | each entry |
| Cloudflare Workers | wrangler.toml + script bindings | [[durable_objects.bindings]], scripts |
| Vercel | pages/api/ + app/api/ + middleware | filesystem-derived |
| Netlify | netlify/functions/, netlify/edge-functions/ | filesystem-derived |
# Example: SAM
yq '.Resources | to_entries | map(select(.value.Type == "AWS::Serverless::Function")) | .[].key' template.yaml
# Example: Serverless Framework
yq '.functions | keys' serverless.ymlOutput: a function inventory:
functions:
- name: get-user
platform: aws-sam
runtime: nodejs20.x
handler: src/get-user.handler
timeout: 10
memory: 512
events:
- type: api
path: /users/{id}
method: GET
- name: process-upload
platform: aws-sam
runtime: python3.12
timeout: 300
events:
- type: s3
bucket: uploads
events: ['s3:ObjectCreated:*']Step 2 - Pick the emulator per function
Default: pick one emulator per platform detected in Step 1's inventory - the per-platform skill below is the canonical driver for that runtime. Use multiple emulators only when the project mixes platforms (e.g., Vercel app
| Platform | Test path | Skill |
|---|---|---|
| AWS Lambda (Node/Python/etc.) | sam local invoke or direct handler | aws-sam-local-testing |
| AWS Lambda (.NET) | Amazon.Lambda.TestUtilities | lambda-test-tools-net |
| Cloudflare Workers | vitest-pool-workers / Miniflare | cloudflare-workers-miniflare |
| Vercel Edge | @edge-runtime/jest-environment | vercel-edge-runtime-testing |
| Netlify Functions | netlify dev + direct handler | netlify-functions-tests |
| Serverless Framework | serverless-offline / direct | serverless-framework-test-plugin |
Step 3 - Generate test events per event source
| Event source | Tool |
|---|---|
| API Gateway request | sam local generate-event apigateway aws-proxy |
| S3 object created | sam local generate-event s3 put |
| SQS message | sam local generate-event sqs receive-message |
| DynamoDB Streams | sam local generate-event dynamodb update |
| EventBridge | sam local generate-event events |
Cloudflare Workers fetch | new Request('https://...') directly |
| Vercel/Netlify | construct Request object |
Commit events to tests/fixtures/events/. Reference in tests.
Step 4 - Emit test scaffold per function
tests/
integration/
get-user.test.ts
process-upload.test.ts
fixtures/
events/
api-get-users-id.json
s3-object-created.json
payloads/
user-1.json
conftest.py # Or jest setup
README.md # How to run + matrixPer-function test:
// tests/integration/get-user.test.ts
import { handler } from '../../src/get-user';
import event from '../fixtures/events/api-get-users-id.json';
describe('get-user', () => {
test('returns 200 for valid id', async () => {
const response = await handler({ ...event, pathParameters: { id: '1' } } as any, {} as any);
expect(response.statusCode).toBe(200);
});
test('returns 404 for missing id', async () => {
const response = await handler({ ...event, pathParameters: { id: 'missing' } } as any, {} as any);
expect(response.statusCode).toBe(404);
});
test('completes within timeout budget', async () => {
// Per lambda-timeout-budget-reference: timeout=10s; want headroom
const start = Date.now();
await handler({ ...event, pathParameters: { id: '1' } } as any, {} as any);
const elapsed = Date.now() - start;
expect(elapsed).toBeLessThan(5000); // 50% of timeout
});
});Step 5 - Cold-start + timeout budget assertions
For each function, derive a budget from cold-start-budget-reference
Typical cold starts per runtime (per AWS / Cloudflare / Vercel docs; bigger packages and memory classes skew higher):
| Runtime | Typical cold start |
|---|---|
| AWS Lambda Node.js (256MB) | 200-700ms |
| AWS Lambda Python (256MB) | 250-800ms |
| AWS Lambda Java 11 (512MB, no SnapStart) | 1.5-6s |
| AWS Lambda Java 11 (512MB, SnapStart (opens in new window)) | 100-300ms |
| AWS Lambda .NET (1GB) | 1-3s |
| AWS Lambda Go (256MB) | 100-300ms |
| AWS Lambda Rust (256MB) | 50-200ms |
| Cloudflare Workers (opens in new window) | 0-5ms (V8 isolate spawn) |
| Vercel Edge Runtime | 5-30ms |
| Vercel Node.js Functions | 200-500ms (small) to 2-3s (large) |
| Netlify Functions | 300ms-2s |
The upstream integration - not the function's own timeout - is usually the operational ceiling (per docs.aws.amazon.com/lambda (opens in new window); Lambda's hard maximum is 900s):
| Integration | Timeout ceiling |
|---|---|
| API Gateway (REST + HTTP API) | 29s hard - function timeout must be < 29s |
| Application Load Balancer | 4s default; configurable to 4000s |
| CloudFront Lambda@Edge | 5s viewer functions; 30s origin functions |
| SQS event source | Per-queue visibility timeout (default 30s); function timeout must be below it |
| DynamoDB Streams | 6-hour batch window; per-batch function limit |
| EventBridge (async) | Async with retry on timeout - requires idempotency |
| Step Functions | Per-task timeout, default 60s |
| Cloudflare Workers | 10ms CPU (free) / 50ms CPU (paid); 30s wall-clock |
| Vercel Edge Functions | 30s wall-clock max |
Set the assertion threshold at roughly 50% of the binding ceiling so a regression trips the test before it trips production:
budgets:
get-user:
cold_start_p99_ms: 800 # AWS Lambda Node@512MB typical
warm_p99_ms: 200
timeout_s: 10
integration_timeout: api-gateway-29s # API GW < 29s
process-upload:
cold_start_p99_ms: 1500
warm_p99_ms: 2000
timeout_s: 300
integration_timeout: s3-event-async # No upstream limitThese become assertion thresholds in staging-smoke tests (run against deployed functions, not local).
Step 6 - CI integration
# .github/workflows/serverless-tests.yml
jobs:
local-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v4
- uses: aws-actions/setup-sam@v2
- run: sam build --use-container
- run: npm ci && npx jest tests/integration/
staging-deploy-and-smoke:
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
needs: local-tests
steps:
- uses: actions/checkout@v5
- run: sam deploy --stack-name staging --parameter-overrides Stage=staging
- run: npx jest tests/staging-smoke/Two-tier: local-fast tests on every PR; deploy + smoke on main.
Step 7 - Document coverage matrix
# Serverless Integration Test Matrix
## Functions covered
| Function | Local test | Cold-start test | Timeout test | Event sources |
|---|---|---|---|---|
| get-user | ✅ tests/integration/get-user.test.ts | staging only | ✅ <5s | API GW GET /users/{id} |
| process-upload | ✅ | staging only | ✅ <150s of 300s | S3 ObjectCreated |
| ... | | | | |
## Gaps
- `legacy-export` - no integration tests; on backlog.
- Cold-start tests not feasible locally for any function (per `cold-start-budget-reference`).
## How to add a new function
1. Add to IaC.
2. Generate event fixture.
3. Add test file following the template.
4. Update this matrix.Anti-patterns
| Anti-pattern | Why it fails | Fix |
|---|---|---|
| One emulator for all functions | Mixed-platform tests are awkward; some emulators miss platform-specific behaviour | Per-platform emulator |
| No staging deploy + smoke tests | Local-pass + prod-fail gap | Two-tier CI |
| Hand-rolled event payloads | Schema drift; missed fields | Use sam local generate-event or commit fixtures |
| Skip cold-start budget assertions | p99 spikes go unnoticed | Per-function budget in staging smoke |
| Test directory mixed with prod code | tests/ dir not in deploy artifact | Keep separate |
| No README documenting matrix | Future maintainers can't add functions consistently | Step 7 README |
| Inflate jest test timeouts to mask slow handlers | Hides performance regression | Match jest timeout to function timeout |
| Mock AWS SDK in all tests | Tests pass; IAM permission bugs hide | Use LocalStack or real test account |
Output
This skill produces:
References
Related skills
aws-sam-local-testing
Wraps AWS SAM (Serverless Application Model) Local CLI for testing Lambda functions locally: `sam local invoke` (single invocation with event payload), `sam local start-api` (local API Gateway emulator), `sam local start-lambda` (local Lambda invoke endpoint for AWS SDK clients), and event-payload generation (`sam local generate-event`). Use when testing Lambda + API Gateway + integrated AWS services locally.
azure-functions-tests
Runs Azure Functions locally using Azure Functions Core Tools v4 (`func start`), Azurite storage emulation, and framework-native unit tests for handler code (.NET isolated worker model, Node.js v4, Python v2). Covers HTTP, queue, and timer trigger testing, admin-endpoint invocation for non-HTTP triggers, and binding verification via local.settings.json. Use when testing Azure Functions before deployment, reproducing trigger behaviour without live Azure services, or gating function handler logic in CI.
cloudflare-workers-miniflare
Wraps Miniflare 3 (the official Cloudflare Workers simulator) and Wrangler dev for testing Workers locally. Covers Miniflare's getMiniflare() programmatic API (workerd-backed simulation matching prod), the wrangler dev local-mode (live-reload during dev), KV / Durable Objects / R2 / D1 bindings emulation, and Vitest + @cloudflare/vitest-pool-workers for in-process tests. Use when testing Cloudflare Workers code locally.
cold-start-budget-reference
Pure-reference catalog of cold-start budgets across serverless runtimes. Covers AWS Lambda's three-phase cold start (Init: download+unzip+runtime-bootstrap; Init code: imports + module load; Invoke: handler execution), Cloudflare Workers' isolate model (sub-millisecond cold starts via V8 isolates per developers.cloudflare.com), Vercel Edge Runtime, Lambda SnapStart for JVM (snapshot-restore for Java), and provisioned-concurrency trade-offs. Includes per-runtime typical cold-start ranges and the testable behaviours each model creates. Use when designing latency budgets, choosing a runtime, or auditing cold-start variance in production.
lambda-test-tools-net
Wraps Amazon.Lambda.TestTool (the canonical .NET Lambda local-testing toolkit from github.com/aws/aws-lambda-dotnet) for invoking Lambda handlers from xUnit / NUnit tests with simulated AWS Lambda contexts (ILambdaContext, ILambdaSerializer). Covers handler-direct invocation, mock context fixtures, the dotnet-lambda CLI, and integration with the .NET LambdaSerializer for JSON. Use when testing AWS Lambda functions written in C#/.NET.
lambda-timeout-budget-reference
Pure-reference catalog of AWS Lambda timeout + billing semantics. Covers Lambda's hard 15-minute (900s) wall-clock limit, the timeout-vs-deadline relationship (Lambda Context.getRemainingTimeInMillis), per-invocation billing (rounded to 1ms; per-invocation + duration × memory), the memory-vs-CPU relationship (CPU scales linearly with memory), the integration-timeout cascade (API Gateway 29s → Lambda 15min; SQS visibility-timeout vs Lambda timeout), and per-runtime nuances. Use when designing a Lambda's timeout config, debugging timeout-vs-billing surprises, or sizing memory for compute-bound workloads.
netlify-functions-tests
Wraps Netlify Functions testing patterns: Netlify Dev (`netlify dev`) for local routing emulation, the @netlify/functions handler API testing pattern, Netlify Edge Functions (Deno runtime) vs Background Functions (Lambda under the hood) distinction, and scheduled-function (cron) test patterns. Use when testing Netlify Functions or Edge Functions.
serverless-framework-test-plugin
Wraps the Serverless Framework (serverless.com) test ecosystem: serverless-offline (local HTTP emulator), serverless-jest-plugin / serverless-mocha-plugin (per-runtime test runners), and the `serverless invoke local` CLI for one-off invocations. Use when testing Lambda functions deployed via the Serverless Framework.
vercel-edge-runtime-testing
Wraps Vercel Edge Runtime testing patterns: the @edge-runtime/jest-environment + edge-runtime CLI for executing Web-Standard APIs (Request / Response / fetch) in jest tests, the `vercel dev` local emulator for full route testing, and the Edge vs Node Function divergence (no fs, no Buffer; Request / Response only). Covers the 30s Edge function timeout per vercel.com/docs. Use when testing Vercel Edge Functions or middleware.