BloomFLv2

BloomFLv2 is a lightweight, demo-friendly decentralized federated learning platform for edge-style anomaly detection.

This version is tuned for:

• CSV-driven multi-device simulation
• Deep Residual MLP with SiLU activations
• FP16 optimized inference & training paths
• Zero-config mDNS node discovery
• ECIES & PSK encrypted gossip protocol
• Provenance-aware metrics and dashboards
• Sensor + camera feature fusion

Instead of requiring a heavy real-time multi-camera training setup, BloomFLv2 can read device CSV files, simulate multiple edge nodes, train local models per node, exchange model updates through gossip, and visualize convergence, metrics, and node behavior in a modern dashboard.

What It Does

BloomFLv2 includes:

• FastAPI backend for metrics, simulation control, node history, config, and data inspection
• Next.js frontend dashboard for simulation, gossip visualization, metrics, convergence, and node-level analysis
• lightweight sensor_anomaly training path for local/dev systems
• CSV generator for seeded multi-device demo data
• camera-augmented anomaly features for OV7670-style capture workflows
• support for richer inference pages such as YOLO and FastSAM

Core Idea

Each CSV file is treated like one edge device.

For a light simulation run:

• each node reads its own CSV shard
• local features are extracted from sensor windows
• optional camera-derived features are fused into the same training sample
• every node trains locally
• nodes exchange model updates through gossip
• the dashboard shows progress, node state, gossip activity, and convergence

This makes it practical to demonstrate federated learning behavior even when you only have one real hardware unit.

Project Structure

Key folders:

• api/ — FastAPI backend routes and services
• frontend/ — Next.js dashboard
• src/bloomfl/ — core training, node, transport, monitoring, and models
• simulation/ — local multi-node simulation runner
• scripts/ — setup and seed-data utilities
• data/sensor_devices/ — CSV input files for the light simulation profile
• data/external/uci_occupancy_detection/ — bundled UCI occupancy source files used to prepare seeded CSVs

Advanced Features

• Deep Residual Classifier: Uses a residual block architecture with SiLU (Sigmoid Linear Unit) activations and LayerNorm for stable convergence on edge data.
• Mixed Precision (FP16): Support for reduced precision computations to minimize memory footprint and increase throughput on edge hardware.
• Adaptive Discovery (mDNS): Automatic node discovery using Multicast DNS, enabling peer-to-peer connectivity without static IP configurations.
• End-to-End Encryption: Secure gossip engine utilizing ECIES for key exchange and PSK-based symmetric encryption for model weight transfers.
• Gossip Impact Analytics: Quantifies the "Gossip Benefit" by comparing local validation ∆ before and after peer synchronization.
• Convergence Forecasting: Real-time linear regression on log-loss to predict the remaining rounds until target accuracy/loss is achieved.
• Anomaly Timeline: Detailed provenance tracking for sensor data, marking 'seeded' vs 'live' samples and providing ground-truth visualization.
• Rich AI Surfaces: Integration with YOLOv12n and FastSAM for multi-modal anomaly confirmation.

CSV Data Format

The light simulation expects one CSV per device under data/sensor_devices/.

Minimum useful columns:

timestamp,device_id,temperature,humidity,pir,rcwl,anomaly_flag

Extended camera-aware columns supported by this repo:

timestamp,device_id,temperature,humidity,pir,rcwl,anomaly_flag,anomaly_label,camera_brightness,camera_motion_score,camera_person_score,camera_capture_fresh,camera_frame_path

Notes:

• older sensor-only CSV files still work
• missing camera fields fall back safely
• camera_frame_path is relative to the sensor CSV directory
• one CSV file generally represents one simulated device

Quick Start On Windows

From:

C:\d_drive\projects\bloomFL

Run:

.\setup.cmd
.\run-dev.cmd

This will:

• create a local Python virtual environment in .venv
• install Python dependencies
• install frontend dependencies
• create .env from .env.example if needed
• launch backend and frontend in separate PowerShell windows

Default URLs:

• backend: http://localhost:8000
• backend docs: http://localhost:8000/docs
• frontend: http://localhost:3000

Manual Setup

If you prefer to run things manually:

python -m venv .venv
.\.venv\Scripts\python.exe -m pip install --upgrade pip
.\.venv\Scripts\python.exe -m pip install -r requirements.txt -r api\requirements.txt -e .
npm install --prefix frontend

Start backend:

.\.venv\Scripts\python.exe -m uvicorn api.main:app --host 0.0.0.0 --port 8000

Start frontend in a second terminal:

cd frontend
$env:NEXT_PUBLIC_API_URL = "http://localhost:8000"
npm run dev

Recommended Demo Workflow

1. Generate or provide CSV data

Option A: use your own hardware-generated CSV files

• place them in data/sensor_devices/
• keep one file per device, for example:
  • device-001.csv
  • device-002.csv
  • device-003.csv

Option B: generate seeded demo data inside the app

• open http://localhost:3000/data
• use the generator controls
• generate multi-device sensor + camera-style demo data

2. Run the lightweight simulation

• open http://localhost:3000/simulation
• choose:
  • profile: light
  • nodes: 3 to 5
  • rounds: 10 to 30
  • transport: tcp
  • base port: 50100

3. Explore the results

Useful pages:

• /dashboard — overall health, quality, gossip impact
• /simulation — run control and status
• /gossip — animated peer-to-peer transfer view
• /metrics — metrics over time
• /convergence — convergence and forecast view
• /nodes — per-node state table
• /nodes/[nodeId] — detailed node history
• /data — seeded data generator and anomaly timeline

Included Seed Dataset

This repo includes source files from the UCI Occupancy Detection dataset under:

• data/external/uci_occupancy_detection/datatraining.txt
• data/external/uci_occupancy_detection/datatest.txt
• data/external/uci_occupancy_detection/datatest2.txt

You can prepare CSV device files from that dataset with:

.\.venv\Scripts\python.exe scripts\prepare_uci_sensor_seed_data.py --num-devices 5 --rows-per-device 800

Setup Scripts

Windows helpers included in this repo:

• setup.cmd / setup.ps1
• run-dev.cmd / run-dev.ps1

Helpful variants:

.\run-dev.cmd -ApiPort 8001 -FrontendPort 3001

.\run-dev.ps1 -Reload

.\run-dev.ps1 -InstallIfNeeded

Light Simulation Notes

The light profile is intentionally optimized for local machines.

It uses:

• CSV-backed sensor anomaly training
• repo-local simulation paths under .sim/
• short rounds
• lightweight model updates
• loopback networking for local node communication

This is the best mode for:

• demos
• professor presentations
• proof-of-concept runs
• laptops without strong GPU support

Camera Logic

The current lightweight CSV path supports camera-derived features and labels such as:

• camera_confirmed_intrusion
• camera_occupancy_mismatch
• camera_motion_hotspot
• camera_heat_signature
• environmental_spike

This is useful when you have a separate collector program that already writes combined sensor and camera data into CSV files.

Backend And Frontend Verification

Common checks:

.\.venv\Scripts\python.exe -m pytest tests\test_second_pass.py -q

cd frontend
npm run build

Known Practical Usage Pattern

For many demos, the most reliable workflow is:

• use one real ESP32-based hardware collector as proof of concept
• save real sensor/camera output into CSV
• create additional seeded CSV files to represent more devices
• run BloomFLv2 on those CSVs
• show federated learning and gossip behavior in the dashboard

That keeps the demo honest, reproducible, and much lighter than trying to run a fully live multi-device deployment on one machine.

Troubleshooting

setup.cmd stops during dependency install

Run the manual install commands from the setup section above. The Python and frontend dependencies can be installed separately.

Ports are already in use

Launch with different ports:

.\run-dev.cmd -ApiPort 8001 -FrontendPort 3001

The simulation says not converged

That can be valid. A completed run may finish without converging under the configured threshold. The dashboard now distinguishes between:

• still tracking
• not enough evaluation data
• finished but not converged
• converged at a specific round

The machine cannot handle the heavy simulation

Use the light profile. That is the intended profile for local/dev/demo use.

Tech Stack

• Python
• FastAPI
• Pydantic
• PyTorch
• Next.js
• React
• Recharts
• Flower-compatible local training interfaces
• custom gossip-based peer exchange

License

MIT