fix: replace socketcan with minimal native addon + fs streams - #401
Conversation
Replace the socketcan npm package (which uses uv_poll_t) with a minimal N-API addon that opens a PF_CAN socket and returns the raw fd. The fd is then read via fs.createReadStream (libuv threadpool) and written via fs.write, eliminating the uv_poll_t silent-stall failure mode that causes intermittent N2K data loss under GC pressure or event loop starvation. The new CanChannel class is a drop-in replacement for socketcan's channel API (onMessage, onStopped, start, stop, send), keeping changes minimal across canbus.ts, simpleCan.ts, candumpjs.ts, and cansendjs.ts. Refs: SignalK/signalk-server#1626, SignalK/signalk-server#2140
Testing on RPi4 with CAN hat1. Clone and build on the Pi: cd ~
git clone -b replace-socketcan https://github.com/dirkwa/canboatjs.git canboatjs-test
cd canboatjs-test
npm install
npm run buildThe 2. Quick smoke test with candumpjs: node dist/bin/candumpjs.js --format candump can0You should see live N2K frames scrolling. 3. Full Signal K server test: Point your Signal K server's canboatjs dependency at the local build. In your Signal K server directory: npm install ~/canboatjs-testThen restart the server and verify:
4. Stress test (the whole point): node --expose-gc -e "
setInterval(() => { global.gc(); }, 100);
setTimeout(() => { console.log('GC stress test done'); process.exit(0); }, 300000);
" &
node dist/bin/candumpjs.js --format candump can0This runs aggressive GC every 100ms for 5 minutes while candumpjs reads frames. With the old socketcan, this would stall. With the new fs.createReadStream approach, it should be rock solid. |
|
This sounds like a bug in socketcan. Has this been reported upstream? |
|
I don’t think direct replacement is necessarily the right strategy. Would it make sense to extract this to a separate module and use the strategy the esbuild has for loading prebuilt binaries (all platforms’ specific modules as optional deps and use the one that gets installed)? Not an insignificant amount of work and a maintenance burden for sure. Naturally the best would be to have this fixed upstream. Seems like it is easily reproducible? |
|
I definitely like the idea of moving this to a new package. @tkurki the long term plan is a change to nodejs which will make so no native code is needed at all. |
|
Upstream fix in socketcan: Separate module with prebuilt binaries (esbuild strategy): Longer term: Happy to move this into a separate package if you feel strongly about it, but given the addon is tiny, Linux-only, and hopefully temporary until Node.js core support lands, inlining it seemed like the lowest-maintenance path rather than maintaining a soon deprecated repo. |
|
Note - This code surfaced a pre existing race condition. Full test setup: |
…ds (#416) The fs.createReadStream-based read path introduced in #401 used a libuv threadpool worker doing a blocking read() on the CAN socket. On Linux, close() on a fd does NOT interrupt a read() blocked on that fd in another thread, and shutdown() returns EOPNOTSUPP on PF_CAN/SOCK_RAW (.shutdown = sock_no_shutdown in net/can/raw.c through v6.18). As a result process.exit() hung indefinitely waiting to join the blocked worker — see SignalK/signalk-server#2618. PR #415's shutdown() approach silently fails for the same kernel reason. Switch to the original architecture: open both read and write sockets in non-blocking mode, register a uv_poll_t watcher on the read fd, and drain all available frames via a non-blocking native read when the watcher fires. Neither direction touches the libuv threadpool, so process.exit() terminates cleanly even on a quiet bus. This also restores the original rationale for #405 — the event-loop starvation under load from issue #1626 was the *write* path blocking threadpool workers (fixed by the native non-blocking writeCanFrame), not the read path. The threadpool-read in #401 was layered on top without being needed. Verified on vcan0: exit completes immediately, frame send/receive round-trips at the same per-burst capacity as master (~222 frames before kernel SO_RCVBUF saturates, identical to pre-existing behavior). Closes #415 (its diagnosis is correct but the shutdown() fix doesn't work on Linux CAN sockets).
Summary
Replaces the
socketcannpm package with a minimal N-API native addon (~50 lines C++) that opens a PF_CAN socket and returns the raw fd. The fd is then read viafs.createReadStream(libuv threadpool) and written viafs.write, completely eliminating theuv_poll_tsilent-stall failure mode.Problem
Users on SocketCAN hardware (SailorHat, Pican-M, Waveshare CAN hats) experience intermittent silent N2K data stalls requiring server restart. The root cause is in the
socketcannpm package:uv_poll_tto deliver CAN frames from the kernel to Node.jsrecv()again — silent, permanent stallcandump can0keeps working because it has its own separate fdrawcanhas the identical problem — also usesuv_poll_init_socketSolution
The native addon does only what cannot be done in pure JS:
socket(PF_CAN, SOCK_RAW, CAN_RAW)— create the socketbind()— bind to the CAN interfaceFrom there,
fs.createReadStreamreads CAN frames on libuv's threadpool (blocking reads on worker threads, not the event loop), andfs.writehandles the send path. Nouv_poll_tanywhere.A
CanChannelwrapper class provides the exact same API as socketcan's channel (onMessage,onStopped,start(),stop(),send()), keeping changes minimal across consumers.Changes
native/canSocket.cpp— minimal N-API addonlib/canSocket.ts—CanChanneldrop-in wrapper using fs streamsbinding.gyp— node-gyp build configlib/canbus.ts,lib/simpleCan.ts,lib/bin/candumpjs.ts,lib/bin/cansendjs.ts— switched fromsocketcantoCanChannelpackage.json— removedsocketcanfrom optionalDependencies, addednode-addon-api,gypfile: trueTesting
Why not net.Socket?
net.Socket({ fd })was the original plan, but Node.js'suv_guess_handle()returnsUV_UNKNOWN_HANDLEfor CAN socket fds.fs.createReadStreamuses the threadpool instead — equally reliable, no poll handles involved.Refs: SignalK/signalk-server#1626, SignalK/signalk-server#2140