Add Multipath TCP (MPTCP) support for listeners, bridges, libmosquitto and clients - #3682
Open
moroznah wants to merge 4 commits into
Open
Add Multipath TCP (MPTCP) support for listeners, bridges, libmosquitto and clients#3682moroznah wants to merge 4 commits into
moroznah wants to merge 4 commits into
Conversation
Add a new MOSQ_OPT_MPTCP integer option to mosquitto_int_option(). When set to 1, libmosquitto creates its connection to the broker using IPPROTO_MPTCP instead of IPPROTO_TCP, on Linux with kernel 5.6 or later. If the running kernel does not support MPTCP, the connection silently falls back to plain TCP. On non-Linux platforms the option returns MOSQ_ERR_NOT_SUPPORTED. This adds a net__socket_stream() helper that encapsulates the MPTCP-or-fallback socket creation, so the broker can reuse it for listeners. Signed-off-by: Igor Sakulin <fxmoroz@gmail.com>
Add a new boolean listener option 'mptcp'. When set to true, the listener socket is created with IPPROTO_MPTCP so that MPTCP-capable clients can connect using Multipath TCP. Plain TCP clients are unaffected and continue to work as normal. This is only available on Linux with kernel 5.6 or later. If the running kernel does not support MPTCP, the listener falls back to plain TCP and a warning is logged. On non-Linux platforms the option is ignored with a warning. Like socket_domain, the option is not reloaded on reload signal since listener sockets are not recreated. Signed-off-by: Igor Sakulin <fxmoroz@gmail.com>
Add a --mptcp command line option to mosquitto_pub, mosquitto_sub and mosquitto_rr which makes the clients connect to the broker using Multipath TCP, via the new MOSQ_OPT_MPTCP library option. This is only available on Linux with kernel 5.6 or later; if the running kernel does not support MPTCP the connection falls back to plain TCP. On other platforms the option produces an error. Signed-off-by: Igor Sakulin <fxmoroz@gmail.com>
Add a new boolean bridge option 'bridge_mptcp'. When set to true, the bridge connects to the remote broker using Multipath TCP via the MOSQ_OPT_MPTCP support in the client code, including for round-robin primary retry connections. This is only available on Linux with kernel 5.6 or later. If the running kernel does not support MPTCP, the connection falls back to plain TCP. On non-Linux platforms the option is ignored with a warning. Signed-off-by: Igor Sakulin <fxmoroz@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add Multipath TCP (MPTCP) support
Summary
This PR adds opt-in Multipath TCP (RFC 8684) support across mosquitto, in four
logical commits:
MOSQ_OPT_MPTCPformosquitto_int_option(); sharednet__socket_stream()helpermptcp true|false--mptcpflag formosquitto_pub,mosquitto_sub,mosquitto_rrbridge_mptcp true|falseMPTCP lets a single connection use several network paths at once (e.g. WiFi +
cellular, or two NICs), giving seamless failover between paths and potential
throughput aggregation without any change to the MQTT layer. This is
particularly useful for MQTT deployments on mobile/vehicular/IIoT clients with
redundant uplinks, and for broker-to-broker bridges over unreliable WANs.
Everything is opt-in and off by default. No behaviour changes unless the
new options are explicitly enabled. No new build dependencies.
Usage
Broker listener:
Bridge:
libmosquitto:
Clients:
Platform support and fallback semantics
net.mptcp.enabled=1). All code paths areguarded with
#if defined(__linux__), so Windows/macOS/BSD builds arebyte-for-byte unaffected at the socket layer.
MOSQ_OPT_MPTCPreturnsMOSQ_ERR_NOT_SUPPORTED;the
mptcp/bridge_mptcpconfig options are accepted but ignored with alogged warning; the client
--mptcpflag produces an error.IPPROTO_MPTCPfails withEINVAL/EPROTONOSUPPORT/ENOPROTOOPTand wefall back to plain TCP. The broker detects the fallback (see design
notes) and logs a warning; the library falls back silently.
mptcp truelistener remains fully compatible with plain-TCP clients:an MPTCP listening socket accepts both MPTCP and regular TCP connections,
so enabling the option cannot lock out existing clients.
Design notes
Why substitute the protocol at
socket()time instead of ingetaddrinfo()hints. Passing
IPPROTO_MPTCPviaai_protocolis rejected by many libcversions (getaddrinfo validates the protocol against its own tables). The
established pattern — the same one used by curl and iperf3 — is to resolve
addresses normally and substitute
IPPROTO_MPTCPwhen creating the socket,falling back to the resolved
ai_protocolif the kernel refuses. That is whatthe new
net__socket_stream(domain, type, protocol, use_mptcp)helper inlib/net_mosq.cdoes; it is the single place holding the fallback-errnologic, and is shared by the client connect path (both the synchronous path and
the
WITH_ADNSasync path innet__try_connect_step2()) and the broker'slistener setup.
Fallback vs. hard failure. A hard failure on kernels without MPTCP would
make
--mptcp/mptcp trueundeployable in mixed fleets and would breakbrokers on kernel downgrades. Silent fallback alone would mask
misconfiguration on the broker side, so the compromise is: clients fall back
silently (documented), while the broker verifies the created listener socket
with
getsockopt(SOL_SOCKET, SO_PROTOCOL)and logsWarning: MPTCP is not supported by the kernel, falling back to TCP.if thesocket is not actually MPTCP. The check costs one getsockopt per listener at
startup, only when
mptcp trueis set.IPPROTO_MPTCPavailability at build time. glibc only exposesIPPROTO_MPTCPfrom 2.32. Since the value (262) is part of the kernel ABI andcannot change,
net_mosq.hdefines it when missing, guarded by#if defined(__linux__) && !defined(IPPROTO_MPTCP).<netinet/in.h>isincluded first so the
#ifndeftest is evaluated against the systemdefinition and never conflicts with it. This keeps the feature buildable on
older distros whose kernels do support MPTCP.
Internal API change.
net__try_connect()gains a trailingbool use_mptcpparameter. This is an internal symbol (not exported inlinker.version), so there is no ABI impact on libmosquitto. Its only twocallers are updated:
net__socket_connect()passesmosq->mptcp, and thebridge round-robin primary-retry probe in
src/bridge.cpassescontext->mptcp. Choosing a parameter over a global keeps the function usablefrom both broker and lib contexts.
Bridge wiring.
bridge_mptcpis stored onstruct mosquitto__bridgeandcopied onto the bridge's connection context in
bridge__new()(
new_context->mptcp = bridge->mptcp). Because the broker's bridge contextsreuse the lib's
struct mosquittoand connect throughnet__socket_connect(),both the normal connect path and the primary-retry path use MPTCP
consistently with no bridge-specific socket code.
Public option numbering/ABI.
MOSQ_OPT_MPTCP = 18extendsenum mosq_opt_tat the end; existing values are untouched. Themptcpflagis a new
boolinstruct mosquitto(internal, not part of the public ABI),placed alongside
tcp_nodelay. Old applications running against the newlibrary see no behavioural difference.
Interaction with existing socket options.
TCP_NODELAY,SO_KEEPALIVE/TCP_KEEPIDLEetc. are supported by the kernel's MPTCP socketoption layer on recent kernels; on older kernels where a particular option is
not yet supported on MPTCP sockets, the existing code paths already only log
a warning on
setsockopt()failure, so behaviour degrades gracefully.Reload semantics. Like
socket_domain, the listenermptcpoption isskipped on config reload, because listener sockets are not recreated on
SIGHUP.bridge_mptcpfollows normal bridge option handling (bridges arerecreated on reload).
Scope deliberately excluded. No MPTCP-specific tuning is exposed (path
manager, subflow limits,
MPTCP_INFO): these are administered system-wide viaip mptcp/ sysctls and would bloat the config surface. WebSockets listenersare unaffected in this PR (the option applies to the TCP socket under any
listener type on the broker side, since it is applied at
socket()creationfor the listener). TLS works unchanged on top of MPTCP since it is purely
above the transport.
Documentation
man/mosquitto.conf.5.xml:mptcp(listener section) andbridge_mptcp(bridge section) entries.
man/common/option-mptcp.xml+ references frommosquitto_pub.1.xml,mosquitto_sub.1.xml,mosquitto_rr.1.xml.include/mosquitto/libmosquitto_options.h:MOSQ_OPT_MPTCPAPI docs.mosquitto.conf: commented examples formptcpandbridge_mptcp.--helpoutput updated.ChangeLog.txt intentionally not modified to avoid conflicts — happy to add
entries if preferred.
Testing performed
On Linux 6.18 (
net.mptcp.enabled=1), Makefile build with-Wall -Wextra -Wconversion, clean:mptcp true—ss -Mlnshows the listeningsocket as MPTCP.
mosquitto_sub --mptcp+mosquitto_pub --mptcp—straceconfirmssocket(AF_INET, SOCK_STREAM, IPPROTO_MPTCP),ss -Mnshows the established connection as MPTCP on both ends, message delivered.
mosquitto_pubagainst themptcp truelistener —works unchanged.
net.mptcp.enabled=0, client--mptcpfallsback (
strace:IPPROTO_MPTCP→ENOPROTOOPT→IPPROTO_TCP, publishsucceeds); broker logs the fallback warning and serves TCP.
mptcp truelistener, A withbridge_mptcp true—ss -Mnshows the bridge connection as MPTCP;messages flow across the bridge.
mptcp notabool→ properError: Invalid 'mptcp' valueat startup; options on non-listener/bridgecontext rejected by the existing
REQUIRE_*macros.Checklist
Signed-off-by, DCO), email matches Eclipse accountdevelop