Testland
Browse all skills & agents

mqtt-tests

Test MQTT v5.0 with Mosquitto broker in CI + paho-mqtt clients - QoS 0 / 1 / 2 delivery semantics, retained messages, Last Will and Testament (LWT), shared subscriptions ($share/group/topic), $SYS topic introspection. Critical for IoT, embedded, and M2M systems where wire-level guarantees matter. Also carries STOMP-over-WebSocket + AMQP 0-9-1 broker testing (Spring / RabbitMQ frame, ack-mode, and exchange/binding tests via Testcontainers) in references/stomp-amqp.md. Use when a product speaks MQTT, STOMP, or AMQP on the wire and QoS 1 / 2 redelivery, retained-message state, LWT, ack-mode, or broker-topology behavior needs a broker-backed test - including smoke-testing a new broker auth / ACL / persistence config.

Install with skills.sh (any agent)

npx skills add testland/qa --skill mqtt-tests
View source

mqtt-tests

This skill covers the v5.0 surfaces tests must exercise: QoS 0 / 1 / 2 delivery semantics, retained messages, Last Will and Testament (LWT), shared subscriptions, and $SYS topic introspection - all per the MQTT v5.0 spec (opens in new window).

When to use

  • IoT / sensor / M2M product where MQTT is the wire protocol.
  • Pre-deploy gate: QoS 1 + 2 redelivery semantics correct, retained-message + LWT setup right.
  • Smoke test new broker config (auth, ACL, persistence).

How to use

  1. Stand up an eclipse-mosquitto:2 broker as a CI service on port 1883 with a persistence-enabled config (Step 1).
  2. Wire a paho-mqtt v5 client with callback_api_version=VERSION2 (Step 2).
  3. Exercise the QoS matrix - assert QoS 1 redelivers buffered messages to a reconnecting clean_start=False session (Step 3).
  4. Verify a retained message reaches a late subscriber, then clear it with an empty retained payload (Step 4).
  5. Cover the advanced broker behaviors - LWT on abnormal disconnect, shared-subscription round-robin, and $SYS diagnostics (references/lwt-shared-subs-sys.md).
  6. Use a unique client_id per test and clean up retained state between runs (Anti-patterns).

Step 1 - Run Mosquitto broker in CI

# GitHub Actions service
services:
  mosquitto:
    image: eclipse-mosquitto:2
    ports:
      - 1883:1883
    volumes:
      - ./tests/mosquitto.conf:/mosquitto/config/mosquitto.conf

tests/mosquitto.conf:

listener 1883
allow_anonymous true
persistence true
persistence_location /mosquitto/data/
log_dest stdout

Step 2 - paho-mqtt client setup (Python)

pip install paho-mqtt
import paho.mqtt.client as mqtt

def test_connect_v5():
    client = mqtt.Client(
        client_id="test-1",
        protocol=mqtt.MQTTv5,
        callback_api_version=mqtt.CallbackAPIVersion.VERSION2,
    )
    client.connect("localhost", 1883)
    client.loop_start()
    # ... exercise ...
    client.disconnect()
    client.loop_stop()

Step 3 - QoS matrix tests

Per the MQTT v5.0 spec (opens in new window):

QoSGuaranteeUse
0At most once (best effort, may be lost)High-frequency sensor where loss is acceptable
1At least once (may duplicate)Most application messaging
2Exactly once (slowest, full PUBREC/PUBREL/PUBCOMP handshake)Billing, payments, anything dedup-required
def test_qos1_redelivers_after_disconnect():
    received = []
    sub = mqtt.Client(client_id="sub", protocol=mqtt.MQTTv5,
                       clean_start=False, callback_api_version=mqtt.CallbackAPIVersion.VERSION2)
    sub.on_message = lambda c, u, msg: received.append(msg.payload)
    sub.connect("localhost", 1883)
    sub.subscribe("sensors/temp", qos=1)
    sub.loop_start()
    time.sleep(0.5)

    # Disconnect subscriber, publish messages, reconnect
    sub.disconnect()
    sub.loop_stop()

    pub = mqtt.Client(client_id="pub", protocol=mqtt.MQTTv5,
                       callback_api_version=mqtt.CallbackAPIVersion.VERSION2)
    pub.connect("localhost", 1883)
    pub.publish("sensors/temp", "22.5", qos=1).wait_for_publish()
    pub.disconnect()

    # Reconnect; broker delivers buffered QoS 1 messages
    sub.reconnect()
    sub.loop_start()
    time.sleep(2.0)
    sub.disconnect()
    sub.loop_stop()

    assert b"22.5" in received

clean_start=False is required for the broker to retain offline session state; without it, QoS 1 messages are lost.

Step 4 - Retained message test

Per the MQTT v5.0 spec (opens in new window), "servers store and distribute the most recent message on a topic to new subscribers, enabling state sharing without republishing."

def test_retained_message_delivered_to_late_subscriber():
    # Publisher sets retain=True
    pub = mqtt.Client(client_id="pub", protocol=mqtt.MQTTv5,
                       callback_api_version=mqtt.CallbackAPIVersion.VERSION2)
    pub.connect("localhost", 1883)
    pub.publish("device/status", "online", qos=1, retain=True).wait_for_publish()
    pub.disconnect()

    # New subscriber connects 5s later - still receives retained msg
    time.sleep(5)
    received = []
    sub = mqtt.Client(client_id="sub-late", protocol=mqtt.MQTTv5,
                       callback_api_version=mqtt.CallbackAPIVersion.VERSION2)
    sub.on_message = lambda c, u, msg: received.append(msg.payload)
    sub.connect("localhost", 1883)
    sub.subscribe("device/status", qos=1)
    sub.loop_start()
    time.sleep(1)
    sub.disconnect()
    sub.loop_stop()

    assert received == [b"online"]

To clear: publish empty payload with retain=True.

LWT, shared subscriptions, and $SYS introspection

See references/lwt-shared-subs-sys.md for Last Will and Testament on abnormal disconnect, shared-subscription round-robin ($share/<group>/<topic>), and $SYS broker-diagnostics tests.

Worked example

A sensor gateway publishes temperature to sensors/temp at QoS 1. QA needs to confirm a subscriber that drops offline still receives messages published during the outage.

  1. Connect a subscriber with clean_start=False and subscribe to sensors/temp at QoS 1 (Step 3).
  2. Disconnect the subscriber to simulate an outage.
  3. From a separate publisher, publish("sensors/temp", "22.5", qos=1).wait_for_publish().
  4. Reconnect the subscriber with reconnect() and pump the loop for ~2s.
  5. Assert b"22.5" in received - the broker delivered the buffered QoS 1 message on reconnect.

Result: offline-session persistence is verified end to end. Had the test used clean_start=True, the broker would have discarded the session and the message would have been lost - the exact regression this test guards.

Anti-patterns

Anti-patternWhy it failsFix
Test only QoS 0 ("works on my machine")QoS 1/2 redelivery bugs shipStep 3 covers matrix
Use clean_start=True then expect persistenceBroker discards session; QoS 1 buffer lostclean_start=False (Step 3)
Skip LWT testStale "online" status persists when clients crashLWT reference
Hardcode same client_id across testsBroker disconnects existing on connect; flakeUnique client_id per test
Forget retained-message cleanupSubsequent test runs see stale statePublish empty retained payload between tests

Limitations

  • MQTT v3.1.1 vs v5.0 differs on Will Properties + reason codes; pin client + broker version per test.
  • Mosquitto auth + ACL are not tested here; enable in a separate test suite if security requirements demand.
  • Some advanced v5 features (request/response, topic alias) require newer paho-mqtt; verify minimum version per the paho-mqtt reference (opens in new window).

References

  • MQTT v5.0 spec (opens in new window) - QoS, retained, LWT, shared subscriptions, $SYS
  • STOMP over WebSocket + AMQP 0-9-1 broker testing: references/stomp-amqp.md
  • websocket-tests - alternative for browser-side bidirectional
  • webhook-delivery-tester (qa-notifications) - HTTP alternative for at-least-once delivery

MQTT LWT, shared subscriptions, and $SYS introspection

View source (opens in new window)

MQTT LWT, shared subscriptions, and $SYS introspection

Advanced broker behaviors beyond core QoS and retained-message delivery: Last Will and Testament, shared subscriptions, and $SYS diagnostics.

Last Will and Testament (LWT)

Per the MQTT v5.0 spec (opens in new window), "When clients disconnect abnormally, servers automatically publish predetermined messages to notify other clients of unavailability."

def test_lwt_published_on_abnormal_disconnect():
    # Subscriber listens for status updates
    received = []
    monitor = mqtt.Client(client_id="monitor", protocol=mqtt.MQTTv5,
                           callback_api_version=mqtt.CallbackAPIVersion.VERSION2)
    monitor.on_message = lambda c, u, msg: received.append(msg.payload)
    monitor.connect("localhost", 1883)
    monitor.subscribe("device/+/status", qos=1)
    monitor.loop_start()

    # Device sets LWT then crashes
    device = mqtt.Client(client_id="device-1", protocol=mqtt.MQTTv5,
                          callback_api_version=mqtt.CallbackAPIVersion.VERSION2)
    device.will_set("device/1/status", "offline", qos=1, retain=False)
    device.connect("localhost", 1883)
    device.publish("device/1/status", "online", qos=1, retain=True).wait_for_publish()
    # Simulate crash (no clean disconnect)
    device._sock.close()

    time.sleep(3)  # broker keepalive timeout
    monitor.disconnect()
    monitor.loop_stop()

    assert b"offline" in received

Shared subscriptions ($share/...)

Per the MQTT v5.0 spec (opens in new window), shared subscriptions distribute messages among group members rather than broadcasting:

$share/<groupname>/<topic-filter>
def test_shared_subscription_round_robin():
    received_a = []
    received_b = []
    sub_a = mqtt.Client(client_id="sub-a", protocol=mqtt.MQTTv5,
                         callback_api_version=mqtt.CallbackAPIVersion.VERSION2)
    sub_b = mqtt.Client(client_id="sub-b", protocol=mqtt.MQTTv5,
                         callback_api_version=mqtt.CallbackAPIVersion.VERSION2)
    sub_a.on_message = lambda c, u, msg: received_a.append(msg.payload)
    sub_b.on_message = lambda c, u, msg: received_b.append(msg.payload)

    sub_a.connect("localhost", 1883); sub_a.subscribe("$share/workers/jobs", qos=1); sub_a.loop_start()
    sub_b.connect("localhost", 1883); sub_b.subscribe("$share/workers/jobs", qos=1); sub_b.loop_start()
    time.sleep(0.5)

    pub = mqtt.Client(client_id="pub", protocol=mqtt.MQTTv5,
                       callback_api_version=mqtt.CallbackAPIVersion.VERSION2)
    pub.connect("localhost", 1883)
    for i in range(10):
        pub.publish("jobs", f"job-{i}", qos=1).wait_for_publish()
    pub.disconnect()
    time.sleep(1)

    sub_a.disconnect(); sub_a.loop_stop()
    sub_b.disconnect(); sub_b.loop_stop()

    # Each got some, neither got all
    assert 0 < len(received_a) < 10
    assert 0 < len(received_b) < 10
    assert len(received_a) + len(received_b) == 10

$SYS topic introspection

Per the MQTT v5.0 spec (opens in new window), $SYS/... reserved topics provide broker diagnostics. Useful for monitoring tests:

def test_broker_reports_connected_clients():
    received = []
    monitor = mqtt.Client(client_id="monitor", protocol=mqtt.MQTTv5,
                           callback_api_version=mqtt.CallbackAPIVersion.VERSION2)
    monitor.on_message = lambda c, u, msg: received.append((msg.topic, msg.payload))
    monitor.connect("localhost", 1883)
    monitor.subscribe("$SYS/broker/clients/connected", qos=0)
    monitor.loop_start()
    time.sleep(15)  # $SYS update interval default = 10s
    monitor.disconnect()
    monitor.loop_stop()

    assert any(b for _, b in received if int(b) >= 1)

$SYS/... topic set varies per broker - Mosquitto + EMQX + HiveMQ each publish slightly different metrics.

STOMP over WebSocket + AMQP 0-9-1 testing

View source (opens in new window)

STOMP over WebSocket + AMQP 0-9-1 testing

This skill covers two complementary enterprise messaging protocols that travel together on Spring and RabbitMQ stacks: STOMP over WebSocket (used by Spring @MessageMapping endpoints and browser clients) and AMQP 0-9-1 (used by RabbitMQ producer/consumer code). Both require protocol-level test coverage that HTTP and WebSocket-only tools miss.

STOMP frame semantics are defined in the STOMP 1.2 specification (opens in new window). AMQP 0-9-1 exchange/queue/binding and acknowledgement behaviour is documented in RabbitMQ's AMQP concepts guide (opens in new window).

Nearest neighbors and differentiation:

  • The host SKILL.md (mqtt-tests) - covers QoS 0/1/2 for IoT/M2M; does not cover STOMP frames, exchange routing, or Java AMQP client patterns.
  • websocket-tests - covers raw WebSocket frames; does not know STOMP frame types, ack modes, or AMQP broker topology.
  • webhook-delivery-tester (qa-notifications) - covers HTTP at-least-once delivery; different transport and no broker involved.

When to use

  • Spring @MessageMapping / @SubscribeMapping endpoints need pre-deploy frame-level validation.
  • RabbitMQ exchange/queue/binding topology must be verified before release.
  • Consumer ack modes (auto, client, client-individual) or prefetch limits need explicit test coverage.
  • A Testcontainers RabbitMQ fixture is needed in Java test suites to avoid a shared broker dependency.

Step 1 - Start RabbitMQ with Testcontainers (Java)

The Testcontainers RabbitMQ module spins up an isolated broker per test class. Per java.testcontainers.org/modules/rabbitmq/ (opens in new window), add the dependency:

<!-- Maven -->
<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>testcontainers-rabbitmq</artifactId>
    <version>2.0.5</version>
    <scope>test</scope>
</dependency>
@Testcontainers
class BrokerTest {

    @Container
    static RabbitMQContainer rabbit =
        new RabbitMQContainer("rabbitmq:3.13-management");

    @BeforeAll
    static void enablePlugins() {
        // Web STOMP is not enabled by default in the base image;
        // enable for STOMP-over-WebSocket tests
        rabbit.execInContainer("rabbitmq-plugins", "enable",
            "rabbitmq_stomp", "rabbitmq_web_stomp");
    }
}

The management image exposes: AMQP on 5672, management UI on 15672, and STOMP on 61613 (after plugin is enabled). The container maps these to random host ports retrieved via rabbit.getMappedPort(5672) etc.

Step 2 - STOMP frame handshake test

Per the STOMP 1.2 spec (opens in new window), a session opens with a CONNECT frame (or STOMP frame; both are valid in 1.2) that MUST carry accept-version and host headers. The server replies with a CONNECTED frame carrying version.

// Uses the stompclient Java library or Spring's StompSession
StompSession session = stompClient
    .connectAsync("ws://localhost:" + rabbit.getMappedPort(15674) + "/ws",
        new StompSessionHandlerAdapter() {})
    .get(5, TimeUnit.SECONDS);

assertThat(session.isConnected()).isTrue();
session.disconnect();

For raw frame assertions use a TCP/WebSocket client and read the CONNECTED frame: a missing version header means the broker rejected the accept-version negotiation.

Heart-beat is negotiated via heart-beat:<outgoing-ms>,<incoming-ms> in the CONNECT frame. Per the spec: "if <cx> is 0 (the client cannot send heart-beats) or <sy> is 0 (the server does not want to receive heart-beats) then there will be none; otherwise, there will be heart-beats every MAX(<cx>,<sy>) milliseconds."

Step 3 - STOMP SUBSCRIBE / SEND / ACK frame tests

Per the STOMP 1.2 spec (opens in new window), the SUBSCRIBE frame requires id (unique subscription identifier) and destination, with an optional ack header.

Ack mode table

Ack modeSpec guaranteeWhen to use
auto (default)Broker treats each delivered frame as acknowledged; no client ACK neededFire-and-forget; high-throughput sensors
clientClient sends ACK; each ACK is cumulative - acknowledges all prior messages on the subscriptionBatch processing where ordering matters
client-individualEach ACK or NACK applies only to the single message identified by the frame's id header - no cumulative effectIndependent per-message processing; DLQ workflows
BlockingQueue<String> received = new LinkedBlockingQueue<>();

session.subscribe("/queue/orders", new StompFrameHandler() {
    @Override
    public Type getPayloadType(StompHeaders headers) { return String.class; }

    @Override
    public void handleFrame(StompHeaders headers, Object payload) {
        received.add((String) payload);
        // ACK required when ack=client or ack=client-individual
        session.acknowledge(headers.getMessageId(), true);
    }
});

session.send("/queue/orders", "order-42");

String msg = received.poll(3, TimeUnit.SECONDS);
assertThat(msg).isEqualTo("order-42");

To assert client-individual behavior, subscribe with ack: client-individual, send two messages, ACK the second before the first, and assert the first is redelivered - confirming non-cumulative semantics per the spec.

Step 4 - RabbitMQ Web STOMP (WebSocket port 15674)

Per rabbitmq.com/docs/web-stomp (opens in new window), enabling the plugin: rabbitmq-plugins enable rabbitmq_web_stomp. The plugin "listens on all interfaces on port 15674" at path /ws. Browser clients connect as:

const ws = new WebSocket('ws://127.0.0.1:15674/ws');
const client = Stomp.over(ws);
client.connect('guest', 'guest', onConnect, onError, '/');

For server-side integration tests use the same STOMP TCP port (61613) via the rabbitmq_stomp plugin, which per rabbitmq.com/docs/stomp (opens in new window) "ships in the core distribution and handles STOMP 1.0 through 1.2." RabbitMQ STOMP destination prefixes:

PrefixMeaning
/queue/<name>STOMP-managed durable queue
/topic/<routing-key>Topic exchange pub/sub
/exchange/<name>/<routing-key>Named exchange with routing key
/amq/queue/<name>Queue created outside the STOMP adapter
/temp-queue/<name>Auto-delete reply queue

Step 5 - AMQP 0-9-1 exchange, queue, and binding tests

Per the RabbitMQ AMQP concepts guide (opens in new window), "messages are published to exchanges, which distribute message copies to queues using rules called bindings." The four exchange types:

TypeRouting rule
directExact match on routing key per rabbitmq.com/docs/exchanges (opens in new window)
fanoutCopies to all bound queues; routing key ignored
topic* matches one dot-segment; # matches zero or more
headersRoutes on message attribute map instead of routing key
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
factory.setPort(rabbit.getMappedPort(5672));
factory.setUsername("guest");
factory.setPassword("guest");

try (Connection conn = factory.newConnection();
     Channel ch = conn.createChannel()) {

    // Declare a durable direct exchange and a durable queue
    ch.exchangeDeclare("orders.direct", "direct", /*durable=*/true);
    ch.queueDeclare("orders.eu", /*durable=*/true,
        /*exclusive=*/false, /*autoDelete=*/false, null);
    ch.queueBind("orders.eu", "orders.direct", "eu");

    // Publish with delivery-mode=2 (persistent)
    AMQP.BasicProperties props = new AMQP.BasicProperties.Builder()
        .deliveryMode(2)
        .contentType("application/json")
        .build();
    ch.basicPublish("orders.direct", "eu", props,
        "{\"id\":1}".getBytes(StandardCharsets.UTF_8));

    // Consume and assert
    GetResponse resp = ch.basicGet("orders.eu", /*autoAck=*/false);
    assertThat(resp).isNotNull();
    assertThat(new String(resp.getBody())).contains("\"id\":1");
    ch.basicAck(resp.getEnvelope().getDeliveryTag(), /*multiple=*/false);
}

The Java client API is documented at rabbitmq.com/client-libraries/java-api-guide (opens in new window).

Step 6 - Consumer ack mode and prefetch tests

Per rabbitmq.com/docs/confirms (opens in new window):

  • basic.ack is used for positive acknowledgements.
  • basic.nack is the RabbitMQ extension for negative acknowledgements (supports multiple flag; basic.reject does not).
  • basic.reject rejects a single message; set requeue=false to send the message to a Dead Letter Exchange instead of back to the queue.
  • Delivery tags are "monotonically growing positive integers scoped per channel" and "deliveries must be acknowledged on the same channel they were received on."
// Prefetch = 1: broker sends at most 1 unacked message at a time
ch.basicQos(1);

boolean autoAck = false;
ch.basicConsume("orders.eu", autoAck, "consumer-tag",
    new DefaultConsumer(ch) {
        @Override
        public void handleDelivery(String tag, Envelope env,
                                   AMQP.BasicProperties props,
                                   byte[] body) throws IOException {
            try {
                process(body);
                ch.basicAck(env.getDeliveryTag(), /*multiple=*/false);
            } catch (Exception e) {
                // requeue=false routes to DLX
                ch.basicNack(env.getDeliveryTag(),
                    /*multiple=*/false, /*requeue=*/false);
            }
        }
    });

Per the confirms doc, "basic.qos sets the max number of unacknowledged deliveries permitted on a channel; a value of zero means no limit." Setting basicQos(1) is the recommended pattern for fair dispatch in round-robin consumer pools.

Step 7 - Publisher confirms test

Per rabbitmq.com/docs/publishers (opens in new window), publisher confirms "provide a mechanism for application developers to keep track of what messages have been successfully accepted by RabbitMQ." Enable on the channel then await confirmation:

ch.confirmSelect();

ch.basicPublish("orders.direct", "eu", null,
    "ping".getBytes(StandardCharsets.UTF_8));

boolean acked = ch.waitForConfirms(5000 /*ms*/);
assertThat(acked).isTrue();

For throughput tests, use streaming confirms (asynchronous) rather than waitForConfirms per message; the publishers doc warns that "waiting for confirmation after each message causes a very significant negative effect on throughput."

Example: end-to-end STOMP publish / AMQP consume

Send a message via STOMP (as a browser or Spring client would) and receive it via AMQP (as a backend service would), asserting the message survives the bridge:

// STOMP sender (port 61613 TCP or 15674 WS)
session.send("/exchange/orders.direct/eu",
    new StompHeaders(), "order-99".getBytes());

// AMQP receiver - same queue that exchange routes to
Thread.sleep(200); // allow broker routing
GetResponse r = ch.basicGet("orders.eu", true /*autoAck*/);
assertThat(r).isNotNull();
assertThat(new String(r.getBody())).isEqualTo("order-99");

This test catches misconfigured exchange-queue bindings that unit tests on the STOMP layer alone would not reveal.

Anti-patterns

Anti-patternWhy it failsFix
Test with ack=auto onlyclient / client-individual redelivery bugs ship silentlyStep 3 covers ack mode matrix
Declare non-durable queues in integration testsBroker restart drops queue; CI becomes flakyUse durable=true in queueDeclare
Use bare waitForConfirms() with no timeoutHangs CI on unroutable messagesPass a timeout ms value
Share one channel across threadsPer the Java API guide, channels are not thread-safeOne channel per thread
Forget ch.basicQos in round-robin consumerOne slow consumer starves; others idleSet basicQos(1) before basicConsume
Assert only STOMP without checking AMQP bindingBinding misconfiguration is invisible to STOMP layerUse the bridge test in Step 7

Limitations

  • AMQP 1.0 (a separate protocol) is out of scope; RabbitMQ 4.3+ supports it but under a different plugin. The skill covers AMQP 0-9-1 only.
  • Spring's @MessageMapping layer (SockJS + STOMP) adds session management on top of raw STOMP; test it via StompClient in Spring's spring-messaging test support, not raw TCP frames.
  • Testcontainers RabbitMQContainer uses the official rabbitmq Docker image; CI must have Docker available. Use @Container (static) for test-class scope rather than per-test to keep suite time reasonable.
  • TLS/mTLS and vhost access control are not tested here; add a separate security suite if your deployment uses them.

References