Messaging
Part 6 of 6 · WebSockets & MQTTMQTT Brokers, Auth & Bridging — ACL, Wildcards, Last Will & Bridge Patterns
The broker authenticates, ACLs by topic, stores retained and sessions, and publishes Last Will on unclean disconnect. Namespace tenants; never grant # to untrusted clients. Bridge with prefix rewrite so topics cannot loop. MQTT QoS 2 is still not Kafka exactly-once.
- 1Gist
- 2Maps
- 3Q&A
- 4Sandbox
Voice readout needs Web Speech Synthesis in this browser.
How you keep a multi-tenant broker from leaking
Prefer
Prefix ACL + LWT retained presence + prefix-rewriting bridges
Bind token tenant_id to a topic prefix. Last-will marks devices offline without a heartbeat storm. Bridges rewrite prefixes or hop headers so a message cannot ping-pong.
- mTLS for devices; password/JWT for apps — often combine.
- Session and retained replication is a failover test, not a checkbox.
- Shared subscriptions need their own ACL: who may join the group.
Alternative
Open #, no LWT, naive broker-to-broker mirror
Looks convenient. Untrusted clients subscribe to firmware topics. Ghost 'online' users linger. A bridge without rewrite reflects forever and melts WAN.
- Single Mosquitto box is an SPOF — plan cluster or cloud HA.
- Oversized persistent sessions are a disk incident — set expiry.
- QoS 2 on the MQTT hop does not make the Kafka sink exactly-once.
CONNECT to a loop-free bridge
Interviews want ACL, then LWT, then why the WAN mirror exploded.
- 1
CONNECT + will
Authenticate. Register LWT topic/payload/QoS/retain. - 2
ACL the PUB/SUB
Prefix per tenant. Deny # for untrusted clients. Publish never uses wildcards anyway. - 3
Route, retain, session
Broker is the live system of record. App is a client. - 4
Unclean disconnect
Broker publishes the will. Retained will → last-known presence. - 5
Bridge
Rewrite prefix to the peer or to Kafka. Choose QoS per hop. Never equate to Kafka EOS.
Overview
Operate MQTT beyond hello-world: broker roles, authentication/authorization, Last Will and Testament, wildcard safety, and bridge patterns into other brokers or Kafka — without re-teaching Kafka internals.
Popular brokers: EMQX, HiveMQ, Mosquitto, VerneMQ, cloud IoT cores. Interview angle: the broker is the system of record for live topic routing.
Broker responsibilities
- Accept CONNECT
- Authenticate
- Authorize PUB/SUB per topic
- Route matching messages
- Store retained + persistent session state
- Run clusters for HA
Your application is often just another client — publish commands, subscribe to telemetry.
AuthN / AuthZ
AuthN — username/password, JWT, mTLS client certs, OAuth hooks (broker plugins).
AuthZ (ACL) — which clientIds may publish/subscribe which topic filters.
Principle: never allow # for untrusted clients. Namespace by tenant: tenant/TENANT_ID/# with ACL binding id to token claims. Prefer MQTT 5 enhanced auth where available.
mTLS is strong for devices; password/JWT is easier for apps. Production often combines (device cert + app token).
Wildcards and ACL pitfalls
Subscribing to devices/+/cmd may be OK; devices/# may leak firmware topics. Publish ACL should deny wildcards (invalid anyway) and deny another tenant's prefix.
Shared subscriptions ($share/group/topic) need separate ACL thinking — who can join the group. Competing consumers are a job-queue lite, not Kafka partitions.
Last Will and Testament (LWT)
On CONNECT, the client registers a will topic/payload/QoS/retain. If the client disconnects ungracefully, the broker publishes the will — classic presence: client/alive → offline. Graceful DISCONNECT suppresses the will. Combine with a retained will payload for last-known presence.
Clearing retained (including a retained will) is still: publish retained empty payload to that topic. Depth for retain vs session queues: MQTT essentials.
Bridging patterns
- Broker→broker — site A topics mirrored to site B (WAN IoT).
- MQTT→Kafka — durable analytics/replay; map topic→topic or topic→record key; QoS chosen per hop. Cross-link Kafka delivery for acks/EOS — do not equate to MQTT QoS 2.
- Kafka→MQTT — push processed alerts back to devices.
- MQTT↔WS gateway — browsers talk WS; devices talk MQTT; gateway translates. Edge scaling still looks like WebSocket fan-out.
Bridge loops: a topic reflected forever. Fix with prefix rewrite or hop headers.
Flow
- 1
1 Device CONNECT plus will
- next2 AuthN and ACL
- 2
2 AuthN and ACL
- next3 Topic route retain LWT
- 3
3 Topic route retain LWT
- next4 Deliver to matching clients
- 4
4 Deliver to matching clients
- next5 Bridge with prefix rewrite
- 5
5 Bridge with prefix rewrite
- next6 High-value Kafka bridge
- 6
6 High-value Kafka bridge
Lesson map
MQTT Brokers, Auth & Bridging — ACL, Wildcards, Last Will & Bridge Patterns
The broker authenticates, ACLs by topic, stores retained and sessions, and publishes Last Will on unclean disconnect. Namespace tenants; never grant # to untrusted clients. Bridge with prefix rewrite so topics cannot loop. MQTT QoS 2 is still not Kafka exactly-once.
Architecture. Architecture
Select a node to see why it exists, or an edge to see the protocol, direction, effect, and consequence.
Mermaid export
flowchart TB dev["1 Device CONNECT plus will"] auth["2 AuthN and ACL"] route["3 Topic route retain LWT"] del["4 Deliver to matching clients"] dev -->|1 Device CONNECT plus will| auth auth -->|2 AuthN and ACL to 3 Topic route retain LWT| route route -->|3 Topic route retain LWT| del
Comparative failure modes
| Miss | Symptom |
|---|---|
Open ACL # | Data exfiltration |
| No LWT | Ghost "online" users |
| Bridge loops | Message ping-pong; WAN melt |
| Oversized persistent sessions | Disk pressure — set session expiry |
| Single Mosquitto box | SPOF — plan cluster or cloud HA |
| Untested failover | Session/retained replication lag; sticky device reconnects surprise you |
Sandbox: ACL + LWT (Python)
Unclean disconnect publishes a retained offline will. Graceful disconnect would not.
Press Run. Snippets must be self-contained — no network, files, or native modules.
Same idea (TypeScript)
Press Run. Snippets must be self-contained — no network, files, or native modules.
Pitfalls
Site A bridges factory/# to site B. Site B bridges factory/# back. What do you see on the WAN in 60 seconds? Draw the prefix rewrite that stops it. Then a device loses power: which CONNECT fields made the dashboard flip to offline without a heartbeat?
Interview Q&A
What is LWT?
Answer
A broker-published message on unclean disconnect, configured at CONNECT (topic, payload, QoS, retain). Graceful DISCONNECT suppresses it. Retained will payloads give last-known presence.
How to clear retained?
Answer
Publish a retained empty payload to that exact topic. Same rule as in MQTT essentials.
ACL for multi-tenant?
Answer
Bind the token's tenant_id to a topic prefix such as tenant/TENANT_ID/. Deny #. Deny publish to another tenant's prefix even if the filter looks clever.
Bridge loop symptom?
Answer
The same payload ping-pongs until you cut the link. Fix with topic prefix rewrite or hop headers so a bridged message is not eligible to bridge back.
mTLS vs password?
Answer
mTLS is strong for devices (cert per device, revocation). Password/JWT is easier for apps and humans. Often combine: devices mTLS, services JWT, both still ACL'd.
Why bridge to Kafka?
Answer
Replay, analytics, stream processing. MQTT is live fan-out; Kafka is a durable log. See Kafka delivery for durability semantics — do not duplicate ISR or transactions here.
Shared subscription use?
Answer
Scale competing consumers on one topic (job queue lite). ACL who may join $share/group/.... Not Kafka partition assignment — no sticky key order.
Cluster gotcha?
Answer
Session and retained replication lag. Test failover with sticky device reconnects. A single Mosquitto process is an SPOF.
Is the app a broker?
Answer
Usually no. The app CONNECTs like any other client, publishes commands, and subscribes to telemetry. Building an in-app topic router duplicates what EMQX/HiveMQ already do.
MQTT to WebSocket gateway?
Answer
Browsers talk WS (or MQTT-over-WS); devices talk MQTT. The gateway translates. You inherit both WS scale and broker ACL. Depth: choice matrix.
What QoS on a Kafka bridge?
Answer
Choose per hop. Losing a metric sample can be QoS 0 into MQTT and acks=1 into Kafka. A door-unlock command is QoS 1 plus an idempotency key on the consumer. QoS 2 still is not EOS.
Where does this cluster go next?
Answer
next is null. Loop to the hub via related. Kafka pages stay in their own series — we only named the analogy.