Issue with Human Detection Using WiFiDenSpose (ESP32 Setup)

[jackgupta079-hash]

I am working on the WiFiDenSpose human detection system, and I am facing some confusion regarding detection and skeleton modeling. Below is my setup and questions:

My Current Setup
I have configured 2 ESP32 nodes (Node 1 and Node 2).
Both nodes are connected to the same WiFi router.
My PC is also connected to the same router.

Execution

I run the following command successfully:

RUST_LOG=debug cargo run --release --bin sensing-server -- --http-port 8080 --source esp32 --udp-port 5005

The ESP32 logs are displayed correctly.
I can see values like:
Heart Rate (HR)
Breathing Rate (BR)

UI and WebSocket
The UI opens successfully at:
http://localhost:8080/ui/observatory.html
WebSocket connection is also successful:
ws://localhost:8765/ws/sensing

Problem

Even though everything is running:

No human movement is detected.
No skeleton model is generated.

My Questions

1. Can WiFi-only sensing detect humans and generate a skeleton model?

I am currently using only WiFi (ESP32), without any additional sensors.

2. Is additional hardware required?

Do I need sensors like:
mmWave radar
LiDAR
for accurate human detection or skeleton tracking?

3.About the ML Model:

Does your system use a Machine Learning model or just RF sensing?
If an ML model is used:
How does it work in your system?
Is it already deployed in your implementation?

4. Model Replication:

How can I replicate your ML model in my setup?
Steps needed for:
Data collection
Training the model
Deployment in the current system

5. Detection Capability:

Is the current setup only capable of simple presence detection (like motion or breathing)?
Or can it achieve full human pose/skeleton tracking?

Additional Info

I have attached screenshots/logs showing:

ESP32 output (HR, BR values)
UI and WebSocket connection

[anitaihua]

This project is just deceiving people, can you believe it? It's just an incomplete conjecture

[ruvnet]

Good questions — your setup is working correctly (ESP32 logs, HR/BR values, WebSocket connection all confirmed). Here's what's happening:

1. Can WiFi-only sensing detect humans and generate skeletons?

Yes, but with important caveats. The current system uses WiFi CSI (Channel State Information) heuristics — not a pre-trained ML model — for person detection. It derives:

• Presence/absence — reliable with calibration
• Motion level (still/walking/active) — reliable
• Person count (1-3) — works but sensitive to room geometry
• Breathing rate — requires person to be still, 6-30 BPM band via FFT
• Heart rate — requires quiet environment, extracted from phase variance
• Skeleton keypoints — these are synthesized from CSI signal features (variance, motion band, dominant frequency), not detected by a neural network

No additional hardware (mmWave, LiDAR) is needed. The ESP32 nodes provide the CSI data over WiFi.

2. Why no movement detection in your setup?

Most likely the adaptive model hasn't been trained for your room. The default heuristic thresholds are generic. Calibrate it:

# Record empty room (~60 seconds)
curl -X POST "http://localhost:8080/api/v1/record/start?label=empty"
# wait 60s
curl -X POST "http://localhost:8080/api/v1/record/stop"

# Record you sitting still (~60 seconds)
curl -X POST "http://localhost:8080/api/v1/record/start?label=still"
# wait 60s
curl -X POST "http://localhost:8080/api/v1/record/stop"

# Record you walking (~60 seconds)
curl -X POST "http://localhost:8080/api/v1/record/start?label=walking"
# wait 60s
curl -X POST "http://localhost:8080/api/v1/record/stop"

# Train the classifier for your room
curl -X POST http://localhost:8080/api/v1/adaptive/train

After training, the system should start detecting presence and motion.

3. About the ML model

The current detection pipeline is heuristic-based (signal processing + thresholds), not a trained ML model. The adaptive training endpoint trains a lightweight logistic regression classifier on 15 CSI features specific to your room.

For actual neural network pose estimation, the wifi-densepose-train crate supports training on MM-Fi and Wi-Pose datasets (see ADR-015, ADR-016), but this requires a GPU training pipeline and is not deployed in the default sensing server.

4. Node placement tips

With 2 nodes:

• Place them at different heights (e.g., desk height and above head)
• Keep them 2-4 meters apart
• Avoid placing both in direct line-of-sight to the router — you want multipath diversity
• Make sure both nodes are provisioned with different --node-id and --tdm-slot values

5. What the screenshots show

Your Observatory UI is receiving data (good) — the HR/BR values are from the vital signs FFT pipeline. The skeleton/movement tabs need the adaptive model trained for your room first.

[jackgupta079-hash]

This project is just deceiving people, can you believe it? It's just an incomplete conjecture

how can you say please tell me

[jackgupta079-hash]

Good questions — your setup is working correctly (ESP32 logs, HR/BR values, WebSocket connection all confirmed). Here's what's happening:

1. Can WiFi-only sensing detect humans and generate skeletons?

Yes, but with important caveats. The current system uses WiFi CSI (Channel State Information) heuristics — not a pre-trained ML model — for person detection. It derives:

• Presence/absence — reliable with calibration
• Motion level (still/walking/active) — reliable
• Person count (1-3) — works but sensitive to room geometry
• Breathing rate — requires person to be still, 6-30 BPM band via FFT
• Heart rate — requires quiet environment, extracted from phase variance
• Skeleton keypoints — these are synthesized from CSI signal features (variance, motion band, dominant frequency), not detected by a neural network

No additional hardware (mmWave, LiDAR) is needed. The ESP32 nodes provide the CSI data over WiFi.

2. Why no movement detection in your setup?

Most likely the adaptive model hasn't been trained for your room. The default heuristic thresholds are generic. Calibrate it:

Record empty room (~60 seconds)

curl -X POST "http://localhost:8080/api/v1/record/start?label=empty"

wait 60s

curl -X POST "http://localhost:8080/api/v1/record/stop"

Record you sitting still (~60 seconds)

curl -X POST "http://localhost:8080/api/v1/record/start?label=still"

wait 60s

curl -X POST "http://localhost:8080/api/v1/record/stop"

Record you walking (~60 seconds)

curl -X POST "http://localhost:8080/api/v1/record/start?label=walking"

wait 60s

curl -X POST "http://localhost:8080/api/v1/record/stop"

Train the classifier for your room

curl -X POST http://localhost:8080/api/v1/adaptive/train
After training, the system should start detecting presence and motion.

3. About the ML model

The current detection pipeline is heuristic-based (signal processing + thresholds), not a trained ML model. The adaptive training endpoint trains a lightweight logistic regression classifier on 15 CSI features specific to your room.

For actual neural network pose estimation, the wifi-densepose-train crate supports training on MM-Fi and Wi-Pose datasets (see ADR-015, ADR-016), but this requires a GPU training pipeline and is not deployed in the default sensing server.

4. Node placement tips

With 2 nodes:

• Place them at different heights (e.g., desk height and above head)
• Keep them 2-4 meters apart
• Avoid placing both in direct line-of-sight to the router — you want multipath diversity
• Make sure both nodes are provisioned with different --node-id and --tdm-slot values

5. What the screenshots show

Your Observatory UI is receiving data (good) — the HR/BR values are from the vital signs FFT pipeline. The skeleton/movement tabs need the adaptive model trained for your room first.

How can a WiFi signal detect heart rate, breathing rate, and even generate a human skeleton model? WiFi is just an RF signal, while mmWave and LiDAR sensors are typically used for human detection and skeleton modeling. I also reverse-engineered the firmware of Ruview, where vital information is displayed, but I’m not sure how this vital data is derived from WiFi signals.

[anitaihua]

This project is just deceiving people, can you believe it? It's just an incomplete conjecture

how can you say please tell me

The internet is full of promotional videos, and no one can truly build a successful video, and all the videos are in 3D. Can you believe it?

[ruvnet]

Follow-up: Recent improvements relevant to your setup

@jackgupta079-hash — two recent PRs directly improve what you'll see with your 2-node setup:

1. PR fix(sensing-server): person count hysteresis to stop flickering #295 — Added real hysteresis to person count estimation. Previously the count could fluctuate rapidly between ticks; now it uses a smoothing window that requires consistent signal changes before updating the count.

2. PR fix(sensing-server): detect ESP32 offline after 5s timeout #300 — ESP32 nodes are now marked offline after 5 seconds of no frames (was previously too aggressive). This means the UI won't flicker between "1 node" and "0 nodes" if a single UDP packet is dropped.

Re: skeleton generation

To set expectations clearly: the current pipeline does not generate full skeleton models from WiFi alone. What you get from WiFi CSI is:

• Presence/absence detection
• Motion classification (still/walking/active)
• Person count (1-3, improved with PR fix(sensing-server): person count hysteresis to stop flickering #295)
• Breathing rate (when still)

Full pose/skeleton estimation requires either a trained ML model (not yet shipped as a pre-trained checkpoint) or additional sensor modalities (camera, mmWave). The 3D skeleton you see in the UI is a visualization driven by CSI heuristics, not a true DensePose reconstruction.

We're actively working on making this clearer in the onboarding flow.

[ruvnet]

It sounds like the server is receiving CSI data (HR/BR values visible) but the skeleton visualization isn't mapping correctly.

A few things to check:

• The skeleton overlay requires the adaptive model to be trained for your environment — run curl -X POST http://localhost:3000/api/v1/adaptive/train
• With only 2 nodes, spatial resolution is limited — 3+ nodes are recommended for skeleton modeling
• Make sure you're viewing the sensing dashboard (not the demo page)
• See Known Issues & Setup Troubleshooting Guide #301 for the full troubleshooting guide

Closing as a setup/support question. Reopen if you're still stuck after trying the above.