push-notification-test-author
Build-an-X for push notification tests (push notifications, web push, FCM / APNs push messages) across Web Push (RFC 8030 / VAPID), Apple Push Notification Service (APNs), and Firebase Cloud Messaging (FCM) - covers subscription handshake, payload encryption, badge / sound / click-action assertions, expired-subscription cleanup, silent-vs-alert, and topic-vs-targeted routing; also carries the in-app notification test workflow (WebSocket / SSE / Firebase-listener delivery, read-unread state, multi-session fan-out, offline-then-reconnect) in references/in-app.md. Use when authoring tests for any push or in-app notification flow.
Install with skills.sh (any agent)
npx skills add testland/qa --skill push-notification-test-authorpush-notification-test-author
Overview
Three push platforms dominate:
| Platform | Standard / Provider |
|---|---|
| Web Push | IETF RFC 8030 + VAPID (RFC 8292) |
| APNs (iOS / iPadOS / macOS) | Apple proprietary HTTP/2 |
| FCM (Android + cross-platform) | Google Firebase |
Each platform has distinct test patterns; Step 1 picks the isolation level and Steps 2 - 4 cover the per-platform test approach.
When to use
How to use
Step 1 - Choose the test isolation level
| Level | Example | Tradeoffs |
|---|---|---|
| Mock the SDK | Patch FCM/APNs/Web-Push library send method | Fast; misses provider-side behavior |
| Sandbox / emulator | APNs sandbox, FCM emulator (limited), web-push test browser | Realistic; slower |
| End-to-end with test devices | Real device farm | Highest fidelity; expensive + flaky |
Default: mock the SDK - fast, deterministic, covers payload-shape + error-path logic which is most of what regressions hit. Use sandbox/emulator when verifying provider-side behavior (encryption, rate-limit, real 410 handling); use real device farms only for grouped-notification / channel UX work.
Step 2 - Web Push tests
Per IETF RFC 8030 (Web Push Protocol): the user-agent subscribes via pushManager.subscribe(), the app server sends an encrypted (RFC 8291) + VAPID-signed (RFC 8292) push to the returned endpoint, and the service worker's push event calls self.registration.showNotification(). Status code 410 Gone means the subscription is invalid (user revoked or expired); the app must remove it from storage.
Mock webPush.sendNotification and assert payload shape plus the 410 cleanup path; add a service-worker harness test that dispatches a synthetic PushEvent. Full Node.js + service-worker recipes: references/platform-test-patterns.md.
Step 3 - APNs tests
Apple Push Notification Service has two environments per developer.apple.com/documentation/usernotifications (opens in new window):
| Environment | Use |
|---|---|
api.sandbox.push.apple.com | Development; uses development APNs certificate |
api.push.apple.com | Production |
Tests typically run against sandbox + use a development APNs certificate or Auth Key (.p8 file). Mock apns_client.send, assert the aps payload shape (alert title / body, sound), and cover the HTTP 410 (Unregistered) token-cleanup path. Python recipes: references/platform-test-patterns.md.
Step 4 - FCM tests
Firebase Cloud Messaging supports HTTP v1 API + legacy HTTP API. Use HTTP v1 for new code (legacy deprecated). Mock admin.messaging().send, assert the cross-platform message shape (token, notification, data, android, apns), and test the messaging/registration-token-not-registered cleanup path (same as APNs Step 3). Node.js recipe: references/platform-test-patterns.md.
Step 5 - Silent vs alert push
Tests should distinguish the two and assert correct payload:
it('uses content-available for background sync', () => {
const payload = buildSilentSyncPush();
expect(payload.aps['content-available']).toBe(1);
expect(payload.aps.alert).toBeUndefined(); // no UI for silent
});Step 6 - Click-action / deep-link tests
The push payload includes a click-action / URL that opens a specific app screen. Test that the right deep-link is in the payload:
def test_order_push_deep_links_to_order_screen():
payload = build_order_push(order_id=123)
assert payload["data"]["click_action"] == "/orders/123"End-to-end click-action tests require device automation (Espresso / XCUITest); cross-ref appium-testing (in the qa-mobile plugin).
Step 7 - Topic vs targeted routing
FCM supports topic subscriptions (broadcast to all subscribers of a topic) vs targeted (single device token). Tests for topic routing:
it('subscribes user to order-updates topic', async () => {
const subSpy = jest.spyOn(admin.messaging(), 'subscribeToTopic')
.mockResolvedValue({ successCount: 1, failureCount: 0, errors: [] });
await subscribeToOrderUpdates('user-device-token', userId);
expect(subSpy).toHaveBeenCalledWith(['user-device-token'], `user-${userId}`);
});Step 8 - End-to-end test recipe
For each push channel:
Worked example
An e-commerce app sends a Web Push "order shipped" notification and must clean up revoked subscriptions.
Anti-patterns
| Anti-pattern | Why it fails | Fix |
|---|---|---|
| Test only happy path | Miss expired-token cleanup; storage grows; spam to dead devices | Step 2-4 410 / 404 / unregistered tests |
| Hardcode VAPID public key in tests + checked into repo | Key rotation breaks tests | Inject via env var |
| Send to real production APNs in tests | Real users get test push | Sandbox environment (Step 3) |
| Skip silent-vs-alert distinction | Apple throttles silent push; missing flag → notifications dropped | content-available test (Step 5) |
| Skip click-action test | Deep links break silently after refactors | Step 6 |
Limitations
References
Firebase listener tests (RTDB + Firestore)
View source (opens in new window)Firebase listener tests (RTDB + Firestore)
Provider-specific variants of the in-app notification workflow. The core WebSocket / SSE and notification-store tests live in in-app.md (opens in new window); this file holds the Firebase Realtime Database and Firestore listener patterns plus the offline write-queue behavior they share. Run both against the Firebase Local Emulator Suite (opens in new window) so listener tests never hit production.
Realtime Database (onValue)
Per the RTDB read/write docs (opens in new window), onValue() fires once immediately with current data and again on every subsequent change at that location and below. RTDB uses a persistent WebSocket internally.
import { initializeApp } from 'firebase/app';
import { getDatabase, ref, onValue, set, off, connectDatabaseEmulator } from 'firebase/database';
const app = initializeApp({ projectId: 'test-project', databaseURL: 'http://127.0.0.1:9000?ns=test' });
const db = getDatabase(app);
connectDatabaseEmulator(db, '127.0.0.1', 9000);
describe('in-app notification - RTDB listener', () => {
const notifRef = ref(db, 'users/u-1/notifications/n-1');
afterEach(() => off(notifRef));
it('delivers notification to listener when record is written', (done) => {
onValue(notifRef, (snapshot) => {
if (!snapshot.exists()) return;
expect(snapshot.val().type).toBe('ORDER_SHIPPED');
done();
});
set(notifRef, { type: 'ORDER_SHIPPED', read: false, ts: Date.now() });
});
it('reflects read-state update when notification is marked read', (done) => {
const updates = [];
onValue(notifRef, (snapshot) => {
if (!snapshot.exists()) return;
updates.push(snapshot.val().read);
if (updates.length === 2) {
expect(updates[0]).toBe(false);
expect(updates[1]).toBe(true);
done();
}
});
set(notifRef, { type: 'ORDER_SHIPPED', read: false, ts: Date.now() }).then(() =>
set(notifRef, { type: 'ORDER_SHIPPED', read: true, ts: Date.now() })
);
});
});Firestore (onSnapshot)
Per the Firestore listen docs (opens in new window), onSnapshot() fires immediately with the current document and again on each change. The snapshot carries metadata.hasPendingWrites (true when local changes are not yet backend-confirmed) and metadata.fromCache (true when served from the local cache). Offline-then-reconnect tests assert fromCache transitions.
import { getFirestore, doc, onSnapshot, setDoc, connectFirestoreEmulator } from 'firebase/firestore';
const firestoreDb = getFirestore(app);
connectFirestoreEmulator(firestoreDb, '127.0.0.1', 8080);
it('delivers live notification and clears pending-writes flag', (done) => {
const notifDoc = doc(firestoreDb, 'notifications', 'n-99');
const states = [];
const unsub = onSnapshot(notifDoc, { includeMetadataChanges: true }, (snap) => {
if (!snap.exists()) return;
states.push({ pending: snap.metadata.hasPendingWrites, fromCache: snap.metadata.fromCache });
if (states.length >= 2 && !snap.metadata.hasPendingWrites && !snap.metadata.fromCache) {
expect(states[0].pending).toBe(true);
expect(states[states.length - 1].pending).toBe(false);
unsub();
done();
}
});
setDoc(notifDoc, { type: 'INVOICE_READY', read: false });
});Offline write queue and reconnect
The RTDB SDK queues writes locally when offline and delivers them after reconnect, per the offline capabilities docs (opens in new window). Connection state is exposed at /.info/connected (a boolean updated on every connection state change; individual client state only, not global). In integration tests, assert /.info/connected transitions from false to true on reconnect to confirm the client re-established its listener subscriptions before asserting notification delivery.
In-app notification test authoring
View source (opens in new window)In-app notification test authoring
In-app notifications are the channel that email, SMS, push, and webhook tests do not cover: real-time messages delivered to a connected client inside the application, typically via a persistent transport. Four transport stacks are common:
| Transport | Standard / provider | Primary use |
|---|---|---|
| WebSocket | IETF RFC 6455 | Bidirectional; chat, live feeds, collaboration |
| SSE | WHATWG HTML Living Standard (EventSource) | Server-to-client only; activity feeds, progress |
| Firebase RTDB listeners | Firebase Realtime Database | JSON tree synced to all clients |
| Firestore onSnapshot | Cloud Firestore | Document / collection live listeners |
This reference walks a common test workflow and then provides per-transport patterns. Transport-level protocol tests (frame parsing, flow control, SSE reconnect timing) belong to qa-realtime-protocols; this reference tests the notification feature layer that runs on top.
When to use
Step 1 - Choose the isolation level
| Level | Example | Trade-offs |
|---|---|---|
| Mock the transport | Simulate WebSocket messages via a test double | Fast, deterministic; misses server fan-out logic |
| Local server | ws / socket.io test server in the same process | Covers serialization and handler logic |
| Firebase emulator | Firebase Local Emulator Suite | Covers RTDB / Firestore rules + listener behavior |
| Full integration | Real backend + test user tokens | Highest fidelity; slowest |
Default: mock the transport for unit tests of the notification handler; use the Firebase emulator for RTDB / Firestore delivery tests; reserve full integration for fan-out and ordering scenarios.
Step 2 - WebSocket delivery tests
Per RFC 6455 Section 4 (opens in new window), the opening handshake upgrades HTTP to a persistent bidirectional channel (101 Switching Protocols). The server sends a notification as a text or binary frame (opcodes 0x1 / 0x2 per RFC 6455 Section 5.2). Tests mock at the frame-receive boundary so the notification handler is exercised without a live server.
Test pattern (Node.js / Jest with ws):
const WebSocket = require('ws');
const { jest } = require('@jest/globals');
describe('in-app notification handler - WebSocket', () => {
let server;
let wss;
beforeEach((done) => {
wss = new WebSocket.Server({ port: 0 }, done);
});
afterEach((done) => {
wss.close(done);
});
it('delivers notification payload to the client handler', (done) => {
wss.once('connection', (ws) => {
ws.send(JSON.stringify({ type: 'NEW_MESSAGE', id: 'n-1', body: 'Hello' }));
});
const client = new WebSocket(`ws://localhost:${wss.options.port}`);
client.on('message', (data) => {
const msg = JSON.parse(data);
expect(msg.type).toBe('NEW_MESSAGE');
expect(msg.id).toBe('n-1');
client.close();
done();
});
});
it('marks notification unread on receipt', (done) => {
wss.once('connection', (ws) => {
ws.send(JSON.stringify({ type: 'NEW_MESSAGE', id: 'n-2', body: 'Hi' }));
});
const client = new WebSocket(`ws://localhost:${wss.options.port}`);
const notificationStore = createNotificationStore(); // app module under test
client.on('message', (data) => {
notificationStore.receive(JSON.parse(data));
expect(notificationStore.unreadCount()).toBe(1);
client.close();
done();
});
});
});Per RFC 6455 Section 7.4.1 (opens in new window), the close code 1000 signals normal closure; 1001 means the endpoint is going away. Tests for reconnect logic should simulate 1001 or 1006 (abnormal closure) and assert that the client attempts reconnection and re-subscribes to notification channels.
Step 3 - SSE delivery tests
Per the WHATWG Server-Sent Events spec (opens in new window), EventSource dispatches events as MessageEvent objects carrying data and lastEventId, and on disconnect the user agent auto-reconnects with a Last-Event-ID header so the server resumes from the last acknowledged event. Tests should assert this recovery path.
Test pattern (Node.js with eventsource + express):
const EventSource = require('eventsource');
const express = require('express');
describe('in-app notification handler - SSE', () => {
let app;
let httpServer;
let sentEvents = [];
beforeAll((done) => {
app = express();
app.get('/notifications/stream', (req, res) => {
res.set({ 'Content-Type': 'text/event-stream', 'Cache-Control': 'no-cache' });
sentEvents.forEach(({ id, data }) => {
res.write(`id: ${id}\ndata: ${JSON.stringify(data)}\n\n`);
});
});
httpServer = app.listen(0, done);
});
afterAll((done) => httpServer.close(done));
it('dispatches notification event to the handler', (done) => {
sentEvents = [{ id: 'e-1', data: { type: 'PAYMENT_RECEIVED', amount: 50 } }];
const port = httpServer.address().port;
const es = new EventSource(`http://localhost:${port}/notifications/stream`);
es.onmessage = (event) => {
const payload = JSON.parse(event.data);
expect(payload.type).toBe('PAYMENT_RECEIVED');
expect(event.lastEventId).toBe('e-1');
es.close();
done();
};
});
});An SSE event's retry field (milliseconds) sets the client reconnection delay; assert the client honors a server-supplied value in integration tests.
Step 4 - Firebase Realtime Database listener tests
onValue() fires once immediately with current data and again on every change at that location and below (RTDB read/write docs (opens in new window)). Run listener tests against the Firebase Local Emulator Suite (opens in new window), not production. Emulator setup plus the write and read-state test patterns are in firebase-listener-tests.md (opens in new window).
Step 5 - Firestore onSnapshot tests
onSnapshot() fires immediately with the current document and again on each change, carrying metadata.hasPendingWrites and metadata.fromCache; assert fromCache transitions for offline-then-reconnect delivery (Firestore listen docs (opens in new window)). The includeMetadataChanges test pattern is in firebase-listener-tests.md (opens in new window).
Step 6 - Notification center read/unread state
In-app notification centers track aggregate unread counts and per-notification read state. Test the state machine independently of the transport:
describe('notification store', () => {
it('increments unread count when a new notification arrives', () => {
const store = createNotificationStore();
store.receive({ id: 'n-1', type: 'COMMENT', read: false });
expect(store.unreadCount()).toBe(1);
});
it('decrements unread count when notification is marked read', () => {
const store = createNotificationStore();
store.receive({ id: 'n-1', type: 'COMMENT', read: false });
store.markRead('n-1');
expect(store.unreadCount()).toBe(0);
});
it('markAllRead resets unread count to zero', () => {
const store = createNotificationStore();
['n-1', 'n-2', 'n-3'].forEach((id) =>
store.receive({ id, type: 'COMMENT', read: false })
);
store.markAllRead();
expect(store.unreadCount()).toBe(0);
});
});Step 7 - Fan-out to multiple sessions
Fan-out (one server event reaching N simultaneously connected clients) is a distinct failure mode from single-client delivery. Test with multiple concurrent WebSocket or SSE clients:
it('delivers the same notification to all connected sessions', (done) => {
const PORT = wss.options.port;
const received = [];
const SESSIONS = 3;
const clients = Array.from({ length: SESSIONS }, () => new WebSocket(`ws://localhost:${PORT}`));
clients.forEach((ws) => {
ws.on('message', (data) => {
received.push(JSON.parse(data));
if (received.length === SESSIONS) {
const ids = received.map((m) => m.id);
expect(new Set(ids).size).toBe(1); // same notification id
expect(ids.length).toBe(SESSIONS); // all sessions received it
clients.forEach((c) => c.close());
done();
}
});
});
// wait for all clients to connect, then broadcast
let connected = 0;
wss.on('connection', () => {
connected += 1;
if (connected === SESSIONS) broadcastNotification({ id: 'n-fan', type: 'ALERT' });
});
});Step 8 - Offline-then-reconnect delivery
Tests for RTDB and Firestore leverage the emulator's network simulation; for WebSocket/SSE, simulate by closing the connection before events are sent:
it('delivers queued notifications after reconnect', (done) => {
let reconnected = false;
let client = new WebSocket(`ws://localhost:${PORT}`);
client.once('open', () => {
// simulate disconnect by closing the connection abruptly
client.terminate();
// reconnect after a short delay
client = new WebSocket(`ws://localhost:${PORT}`);
reconnected = true;
client.on('message', (data) => {
expect(reconnected).toBe(true);
const msg = JSON.parse(data);
expect(msg.type).toBe('QUEUED_NOTIFICATION');
client.close();
done();
});
});
});For RTDB/Firestore, the SDK's offline write queue and the /.info/connected reconnect assertion are covered in firebase-listener-tests.md (opens in new window).
Step 9 - Ordering assertions
In-app notification streams must deliver events in consistent order, especially when multiple events are emitted in quick succession. Per the WHATWG SSE spec (opens in new window), MessageEvent.lastEventId tracks the sequence position and is replayed as the Last-Event-ID header on reconnect, enabling gap detection:
it('delivers notifications in emission order', (done) => {
const received = [];
wss.once('connection', (ws) => {
['n-1', 'n-2', 'n-3'].forEach((id) =>
ws.send(JSON.stringify({ id, type: 'ACTIVITY', seq: parseInt(id.split('-')[1]) }))
);
});
const client = new WebSocket(`ws://localhost:${PORT}`);
client.on('message', (data) => {
received.push(JSON.parse(data));
if (received.length === 3) {
const seqs = received.map((m) => m.seq);
expect(seqs).toEqual([1, 2, 3]);
client.close();
done();
}
});
});Step 10 - Test recipe checklist
For each in-app notification transport in the codebase:
Anti-patterns
| Anti-pattern | Why it fails | Fix |
|---|---|---|
| Test only single-session delivery | Fan-out bugs (missed broadcasts, duplicates) go undetected | Step 7 multi-client test |
| Assert notification text in transport test | Couples UI copy to protocol test; brittle on copy changes | Assert type + id fields; test UI text separately |
| Fire-and-forget RTDB write without awaiting listener | Race between write and listener callback | Use onValue callback as the assertion gate (Steps 4-5) |
Skip fromCache / hasPendingWrites assertions | Offline delivery bugs appear only in production | Firestore includeMetadataChanges: true (Step 5) |
| Mock at the application store instead of the transport boundary | Transport serialization bugs (JSON parse errors, binary frame issues) go untested | Mock at the WebSocket/SSE message event boundary (Steps 2-3) |
| Reconnect test that only asserts the connection reopened | Does not verify listener re-subscription or queued-event delivery | Assert notification receipt after reconnect (Step 8) |
Limitations
References
Push platform test patterns
View source (opens in new window)Push platform test patterns
Full per-platform test-pattern code for push-notification-test-author (opens in new window). Each recipe mocks at the SDK boundary (the default isolation level); swap for sandbox / emulator when verifying provider-side behavior.
Web Push (Node.js with web-push)
The RFC 8030 flow and the 410 Gone cleanup requirement are covered in Step 2 of the skill (opens in new window); the recipe below mocks webPush.sendNotification at the SDK boundary.
const webPush = require('web-push');
const { jest } = require('@jest/globals');
describe('push notification', () => {
beforeAll(() => {
webPush.setVapidDetails(
'mailto:test@example.com',
VAPID_PUBLIC_KEY,
VAPID_PRIVATE_KEY,
);
});
it('sends order-status notification with correct payload', async () => {
const sendSpy = jest.spyOn(webPush, 'sendNotification').mockResolvedValue({
statusCode: 201,
});
await pushOrderUpdate(testSubscription, { orderId: 123, status: 'shipped' });
expect(sendSpy).toHaveBeenCalledWith(
testSubscription,
expect.stringContaining('"orderId":123'),
expect.any(Object),
);
});
it('removes expired subscription on 410 response', async () => {
jest.spyOn(webPush, 'sendNotification').mockRejectedValue({ statusCode: 410 });
await pushOrderUpdate(testSubscription, { orderId: 123 });
const stored = await Subscription.findOne({ endpoint: testSubscription.endpoint });
expect(stored).toBeNull();
});
});Service-worker side test
In a service-worker test harness (sw-toolbox-test or workbox-cli's testing utilities):
self.addEventListener('push', event => {
event.waitUntil(
self.registration.showNotification('Order update', {
body: event.data.json().status,
icon: '/icons/order.png',
data: { orderId: event.data.json().orderId },
})
);
});
// Test: simulate push event
const event = { data: { json: () => ({ orderId: 123, status: 'shipped' }) } };
self.dispatchEvent(new PushEvent('push', event));
expect(self.registration.showNotification).toHaveBeenCalled();APNs (Python with httpx + apns2)
Run against api.sandbox.push.apple.com with a development APNs certificate or Auth Key (.p8 file).
import pytest
from unittest.mock import patch
from my_app.notifications import send_apns
def test_apns_payload_shape():
with patch("my_app.notifications.apns_client.send") as mock_send:
mock_send.return_value = {"status": 200}
send_apns(device_token="abc123", title="Order shipped", body="Your order is on the way")
sent_payload = mock_send.call_args.kwargs["payload"]
assert sent_payload["aps"]["alert"]["title"] == "Order shipped"
assert sent_payload["aps"]["alert"]["body"] == "Your order is on the way"
assert sent_payload["aps"]["sound"] == "default"Invalid-token handling (HTTP 410 from APNs):
def test_apns_410_removes_token():
with patch("my_app.notifications.apns_client.send") as mock_send:
mock_send.return_value = {"status": 410, "reason": "Unregistered"}
send_apns_with_cleanup(device_token="abc123", ...)
token = DeviceToken.objects.filter(token="abc123").first()
assert token is NoneFCM (Node.js with firebase-admin)
Use the HTTP v1 API for new code (legacy HTTP API is deprecated). Invalid-token responses include messaging/registration-token-not-registered; test the cleanup path the same way as APNs.
const admin = require('firebase-admin');
const { jest } = require('@jest/globals');
it('sends FCM message with correct shape', async () => {
const sendSpy = jest.spyOn(admin.messaging(), 'send').mockResolvedValue('msg-id-123');
await sendFcmOrderUpdate('device-token', { orderId: 123 });
expect(sendSpy).toHaveBeenCalledWith(
expect.objectContaining({
token: 'device-token',
notification: expect.objectContaining({ title: 'Order Update' }),
data: expect.objectContaining({ orderId: '123' }),
android: expect.objectContaining({ priority: 'high' }),
apns: expect.any(Object), // FCM cross-platform routing
}),
);
});Related skills
mailpit-testing
The email-testing home: configures and runs Mailpit - modern dev-mailbox server for SMTP testing with built-in REST API for assertions; default SMTP `1025` + Web UI `8025`; Chaos mode (configurable SMTP errors for resilience testing), message tagging, search filters. Carries the end-to-end email-flow workflow (multipart body, link-rewrite resolution, unsubscribe per RFC 8058, bounce + complaint webhooks) in references/email-flows.md and the legacy MailHog capture patterns + migration path in references/mailhog-legacy.md. Use when developing or testing email-sending code locally / in CI - SMTP capture, full-flow assertions, or migrating an existing MailHog deployment.
sms-test-author
Build-an-X for SMS-flow tests - uses Twilio Magic Numbers (`+15005550006` valid recipient, `+15005550001` invalid number, `+15005550002` cannot route, `+15005550003` international restriction, etc.) and Test Credentials for safe assertion-only Twilio interactions; covers segment-counting (GSM-7 vs UCS-2 encoding); rate-limit + opt-out keyword (STOP / HELP / UNSUBSCRIBE) handling; alphanumeric sender vs short-code vs 10DLC differences. Use when authoring tests for any Twilio-backed SMS flow.
webhook-delivery-tester
The single webhook-testing home, sender AND receiver: build-an-X for webhook delivery + receiver tests per Standard Webhooks (standardwebhooks.com) - HMAC-SHA256 signature verification, retry semantics with exponential backoff + jitter, replay-window check via timestamp tolerance, ordering guarantees, dead-letter handling for permanent failures, content-type + body-encoding fidelity - plus inbound capture-and-replay hardening (runtime-signed fixtures, tampered-payload and future-timestamp rejection, key-rotation acceptance, sanitized production captures) in references/inbound-replay.md. Use when authoring tests for webhook senders OR receivers in any system (Stripe / Twilio / SendGrid / GitHub / GitLab outbound webhooks; SaaS app inbound webhooks), including payment and realtime integrations.