Synapse

Brokerless IoT ingestion engine with spatial anomaly detection

System Overview

Synapse is an experimental edge computing architecture that replaces centralized message brokers, such as MQTT or Kafka, with a symmetric peer-to-peer mesh topology. Built on ZeroMQ, the system allows identical edge nodes to ingest telemetry and perform data validation locally without relying on a central cloud server. Data integrity checks are executed through spatial corroboration using the Uber H3 indexing system.

Key Capabilities

• Symmetric Mesh Topology: Communication is handled via ZeroMQ PUB and SUB sockets, eliminating central single points of failure.
• Spatial Anomaly Detection: Nodes validate sensor data against peers in the same physical H3 cell using the robust Median Absolute Deviation algorithm.
• Zero-Touch Provisioning: Dynamic local discovery is implemented via mDNS and DNS Service Discovery, allowing network formation without static configuration files.
• Active Connection Management: Bounded peering limits optimize socket utilization with dynamic failover protocols and self-peering filtration.
• Spatial Relaying: Telemetry is propagated beyond direct communication ranges. Intermediate nodes act as relays utilizing path vectors to prevent routing loops and duplicate transmissions.
• Cryptographic Security: Transport channels are encrypted using ZeroMQ CURVE. Nodes authorize peers dynamically through Over-The-Air Activation over REST HTTP endpoints.
• Payload Compression: Bandwidth is optimized via a pluggable engine supporting zlib, lz4, and zstd. Spatial topic headers remain in clear-text to allow native ZeroMQ prefix matching.
• Local Time-Series Persistence: Historical telemetry from the local node and corroborated peers is archived in a local SQLite database, featuring automated window-function pruning.
• Secure Local Dashboards: Each node serves a localized web interface, protected by a dedicated Caddy reverse proxy sidecar that terminates TLS locally.

Quickstart Configuration

The repository includes a Makefile for standard execution. Python 3.11 or higher is required.

Clone the repository:

git clone https://github.com/onyks-os/Synapse.git
cd Synapse

Start the local simulated network:

make start

Access the localized dashboard by navigating to https://localhost:8441.

Stop the network:

make stop

Kubernetes Deployment

Production deployments on edge clusters utilize the provided Helm chart. The chart implements a DaemonSet architecture, ensuring exactly one Synapse instance and its associated Caddy sidecar are scheduled per physical node.

Install the chart:

helm install synapse-node ./charts/synapse

Configuration overrides, such as proxy settings and SQLite persistence paths, are managed via the values.yaml file.

Performance Benchmarks

The repository includes a native benchmarking tool to evaluate mesh convergence and throughput scaling.

Run the incremental stress test:

make benchmark

Recorded metrics for a 50-node cluster demonstrate mesh convergence in 2.9 seconds with an aggregate throughput of approximately 2470 messages per second. Applying an active connection limit of 8 reduces socket overhead by 84 percent and decreases convergence time to 2.39 seconds.

Chaos Engineering

System resilience is validated through an integrated Chaos Monkey utility. The tool forces node evictions and injects anomalous data payloads to trigger spatial corroboration limits.

Execute the chaos suite locally:

make chaos

Network degradation, including induced latency and packet loss, is simulated via Pumba interfacing directly with the Linux kernel Traffic Control subsystem.

License

This project is distributed under the MIT License.