push-notification-test-author
Build-an-X for push notification tests (push notifications, web push, FCM / APNs push messages) across Web Push (RFC 8030 / VAPID), Apple Push Notification Service (APNs), and Firebase Cloud Messaging (FCM) - covers subscription handshake, payload encryption, badge / sound / click-action assertions, expired-subscription cleanup, silent-vs-alert, and topic-vs-targeted routing. Use when authoring tests for any push notification flow.
Install with skills.sh (any agent)
npx skills add testland/qa --skill push-notification-test-authorpush-notification-test-author
Overview
Three push platforms dominate:
| Platform | Standard / Provider |
|---|---|
| Web Push | IETF RFC 8030 + VAPID (RFC 8292) |
| APNs (iOS / iPadOS / macOS) | Apple proprietary HTTP/2 |
| FCM (Android + cross-platform) | Google Firebase |
Each platform has distinct test patterns; Step 1 picks the isolation level and Steps 2 - 4 cover the per-platform test approach.
When to use
How to use
Step 1 - Choose the test isolation level
| Level | Example | Tradeoffs |
|---|---|---|
| Mock the SDK | Patch FCM/APNs/Web-Push library send method | Fast; misses provider-side behavior |
| Sandbox / emulator | APNs sandbox, FCM emulator (limited), web-push test browser | Realistic; slower |
| End-to-end with test devices | Real device farm | Highest fidelity; expensive + flaky |
Default: mock the SDK - fast, deterministic, covers payload-shape + error-path logic which is most of what regressions hit. Use sandbox/emulator when verifying provider-side behavior (encryption, rate-limit, real 410 handling); use real device farms only for grouped-notification / channel UX work.
Step 2 - Web Push tests
Per IETF RFC 8030 (Web Push Protocol): the user-agent subscribes via pushManager.subscribe(), the app server sends an encrypted (RFC 8291) + VAPID-signed (RFC 8292) push to the returned endpoint, and the service worker's push event calls self.registration.showNotification(). Status code 410 Gone means the subscription is invalid (user revoked or expired); the app must remove it from storage.
Mock webPush.sendNotification and assert payload shape plus the 410 cleanup path; add a service-worker harness test that dispatches a synthetic PushEvent. Full Node.js + service-worker recipes: references/platform-test-patterns.md.
Step 3 - APNs tests
Apple Push Notification Service has two environments per developer.apple.com/documentation/usernotifications (opens in new window):
| Environment | Use |
|---|---|
api.sandbox.push.apple.com | Development; uses development APNs certificate |
api.push.apple.com | Production |
Tests typically run against sandbox + use a development APNs certificate or Auth Key (.p8 file). Mock apns_client.send, assert the aps payload shape (alert title / body, sound), and cover the HTTP 410 (Unregistered) token-cleanup path. Python recipes: references/platform-test-patterns.md.
Step 4 - FCM tests
Firebase Cloud Messaging supports HTTP v1 API + legacy HTTP API. Use HTTP v1 for new code (legacy deprecated). Mock admin.messaging().send, assert the cross-platform message shape (token, notification, data, android, apns), and test the messaging/registration-token-not-registered cleanup path (same as APNs Step 3). Node.js recipe: references/platform-test-patterns.md.
Step 5 - Silent vs alert push
Tests should distinguish the two and assert correct payload:
it('uses content-available for background sync', () => {
const payload = buildSilentSyncPush();
expect(payload.aps['content-available']).toBe(1);
expect(payload.aps.alert).toBeUndefined(); // no UI for silent
});Step 6 - Click-action / deep-link tests
The push payload includes a click-action / URL that opens a specific app screen. Test that the right deep-link is in the payload:
def test_order_push_deep_links_to_order_screen():
payload = build_order_push(order_id=123)
assert payload["data"]["click_action"] == "/orders/123"End-to-end click-action tests require device automation (Espresso / XCUITest); cross-ref appium-testing (in the qa-mobile plugin).
Step 7 - Topic vs targeted routing
FCM supports topic subscriptions (broadcast to all subscribers of a topic) vs targeted (single device token). Tests for topic routing:
it('subscribes user to order-updates topic', async () => {
const subSpy = jest.spyOn(admin.messaging(), 'subscribeToTopic')
.mockResolvedValue({ successCount: 1, failureCount: 0, errors: [] });
await subscribeToOrderUpdates('user-device-token', userId);
expect(subSpy).toHaveBeenCalledWith(['user-device-token'], `user-${userId}`);
});Step 8 - End-to-end test recipe
For each push channel:
Worked example
An e-commerce app sends a Web Push "order shipped" notification and must clean up revoked subscriptions.
Anti-patterns
| Anti-pattern | Why it fails | Fix |
|---|---|---|
| Test only happy path | Miss expired-token cleanup; storage grows; spam to dead devices | Step 2-4 410 / 404 / unregistered tests |
| Hardcode VAPID public key in tests + checked into repo | Key rotation breaks tests | Inject via env var |
| Send to real production APNs in tests | Real users get test push | Sandbox environment (Step 3) |
| Skip silent-vs-alert distinction | Apple throttles silent push; missing flag → notifications dropped | content-available test (Step 5) |
| Skip click-action test | Deep links break silently after refactors | Step 6 |
Limitations
References
Push platform test patterns
View source (opens in new window)Push platform test patterns
Full per-platform test-pattern code for push-notification-test-author (opens in new window). Each recipe mocks at the SDK boundary (the default isolation level); swap for sandbox / emulator when verifying provider-side behavior.
Web Push (Node.js with web-push)
The RFC 8030 flow and the 410 Gone cleanup requirement are covered in Step 2 of the skill (opens in new window); the recipe below mocks webPush.sendNotification at the SDK boundary.
const webPush = require('web-push');
const { jest } = require('@jest/globals');
describe('push notification', () => {
beforeAll(() => {
webPush.setVapidDetails(
'mailto:test@example.com',
VAPID_PUBLIC_KEY,
VAPID_PRIVATE_KEY,
);
});
it('sends order-status notification with correct payload', async () => {
const sendSpy = jest.spyOn(webPush, 'sendNotification').mockResolvedValue({
statusCode: 201,
});
await pushOrderUpdate(testSubscription, { orderId: 123, status: 'shipped' });
expect(sendSpy).toHaveBeenCalledWith(
testSubscription,
expect.stringContaining('"orderId":123'),
expect.any(Object),
);
});
it('removes expired subscription on 410 response', async () => {
jest.spyOn(webPush, 'sendNotification').mockRejectedValue({ statusCode: 410 });
await pushOrderUpdate(testSubscription, { orderId: 123 });
const stored = await Subscription.findOne({ endpoint: testSubscription.endpoint });
expect(stored).toBeNull();
});
});Service-worker side test
In a service-worker test harness (sw-toolbox-test or workbox-cli's testing utilities):
self.addEventListener('push', event => {
event.waitUntil(
self.registration.showNotification('Order update', {
body: event.data.json().status,
icon: '/icons/order.png',
data: { orderId: event.data.json().orderId },
})
);
});
// Test: simulate push event
const event = { data: { json: () => ({ orderId: 123, status: 'shipped' }) } };
self.dispatchEvent(new PushEvent('push', event));
expect(self.registration.showNotification).toHaveBeenCalled();APNs (Python with httpx + apns2)
Run against api.sandbox.push.apple.com with a development APNs certificate or Auth Key (.p8 file).
import pytest
from unittest.mock import patch
from my_app.notifications import send_apns
def test_apns_payload_shape():
with patch("my_app.notifications.apns_client.send") as mock_send:
mock_send.return_value = {"status": 200}
send_apns(device_token="abc123", title="Order shipped", body="Your order is on the way")
sent_payload = mock_send.call_args.kwargs["payload"]
assert sent_payload["aps"]["alert"]["title"] == "Order shipped"
assert sent_payload["aps"]["alert"]["body"] == "Your order is on the way"
assert sent_payload["aps"]["sound"] == "default"Invalid-token handling (HTTP 410 from APNs):
def test_apns_410_removes_token():
with patch("my_app.notifications.apns_client.send") as mock_send:
mock_send.return_value = {"status": 410, "reason": "Unregistered"}
send_apns_with_cleanup(device_token="abc123", ...)
token = DeviceToken.objects.filter(token="abc123").first()
assert token is NoneFCM (Node.js with firebase-admin)
Use the HTTP v1 API for new code (legacy HTTP API is deprecated). Invalid-token responses include messaging/registration-token-not-registered; test the cleanup path the same way as APNs.
const admin = require('firebase-admin');
const { jest } = require('@jest/globals');
it('sends FCM message with correct shape', async () => {
const sendSpy = jest.spyOn(admin.messaging(), 'send').mockResolvedValue('msg-id-123');
await sendFcmOrderUpdate('device-token', { orderId: 123 });
expect(sendSpy).toHaveBeenCalledWith(
expect.objectContaining({
token: 'device-token',
notification: expect.objectContaining({ title: 'Order Update' }),
data: expect.objectContaining({ orderId: '123' }),
android: expect.objectContaining({ priority: 'high' }),
apns: expect.any(Object), // FCM cross-platform routing
}),
);
});Related skills
email-flow-test-author
Build-an-X for end-to-end email-flow tests - trigger → SMTP capture (via Mailpit / MailHog) → header assertions (DKIM/SPF/DMARC when relayed via real MTA) → body assertions (HTML + plain-text alternative) → link-rewrite + tracking-pixel handling → unsubscribe-link verification → bounce + complaint testing in non-prod (via Mailtrap-style services). Use when authoring tests for any transactional or marketing email flow regardless of the SMTP capture tool.
in-app-notification-test-author
Build-an-X workflow for testing real-time in-app notifications delivered over WebSocket (RFC 6455) or Server-Sent Events (WHATWG SSE spec), Firebase Realtime Database / Firestore listeners, and notification center read/unread state - covers fan-out to multiple sessions, offline-then-reconnect delivery, and ordering guarantees. Distinct from email, SMS, push, and webhook channels. Use when authoring tests for any notification that appears inside a connected web or mobile app UI without leaving the application.
mailhog-testing
Captures and asserts SMTP email in tests with MailHog, the Go-based dev mailbox (SMTP sink on `1025`, web UI + JSON API on `8025`, single Go binary or Docker), reading captured mail via APIv2 (`/api/v2/messages`, `/api/v2/search`) and injecting failures with the Jim chaos monkey. Use when a project already runs MailHog to test password-reset, verification, or notification emails; for new projects prefer Mailpit (richer API, active maintenance) - migration path in references.
mailpit-testing
Configures and runs Mailpit - modern dev-mailbox server for SMTP testing with built-in REST API for assertions; default SMTP `1025` + Web UI `8025`; ships single static binary or multi-architecture Docker images; features Chaos mode (configurable SMTP errors for resilience testing), message tagging (manual + auto via filters and plus-addressing), search filters. Use when the user develops email-sending code locally / in CI and needs SMTP capture with programmatic test assertions, or when migrating from MailHog (which Mailpit succeeds).
sms-test-author
Build-an-X for SMS-flow tests - uses Twilio Magic Numbers (`+15005550006` valid recipient, `+15005550001` invalid number, `+15005550002` cannot route, `+15005550003` international restriction, etc.) and Test Credentials for safe assertion-only Twilio interactions; covers segment-counting (GSM-7 vs UCS-2 encoding); rate-limit + opt-out keyword (STOP / HELP / UNSUBSCRIBE) handling; alphanumeric sender vs short-code vs 10DLC differences. Use when authoring tests for any Twilio-backed SMS flow.
webhook-delivery-tester
Build-an-X for webhook delivery + receiver tests per Standard Webhooks (standardwebhooks.com) - HMAC-SHA256 signature verification, retry semantics with exponential backoff + jitter, replay-window check via timestamp tolerance, ordering guarantees, dead-letter handling for permanent failures, content-type + body-encoding fidelity. Use when authoring tests for webhook senders OR receivers in any system (Stripe / Twilio / SendGrid / GitHub / GitLab outbound webhooks; SaaS app inbound webhooks).