By "GPLv3" do you mean "GPL-3.0-or-later"? https://www.gnu.org/licenses/gpl-faq.html#VersionThreeOrLate...

Ive worked on some sonification projects that used signals from xbox kinect lidar, piezos, and other sensors. Co-author on paper i wrote developed a "strummable" theremin that divided physical space with invisible "strings" of various tunings. We preferred running synthesis on PC when possible and just outputting midi and OSC, as DSP on ESP32 has limits for what can be achieved in under 5-10ms. If the goal is hardware audio output, you may need to look into dedicated DSP chips and an audio shield for better DAC- but for prototyping can easily bang a square wave through any of esp32 pins

I wonder if somebody could make an open hardware version of the Leap Motion with this technique (though I'm not sure how accurate/repeatable the sensing is - Leap Motion could detect with an accuracy of < 0.7mm)

I'm sure the kids will love this! Wi-Fi Theremin sounds great.

Great project and great idea, thank you for sharing !

I don't know if it's useful but one technique I have used in sonification during the experimentation phase is to skip the real time aspect, capture all the available "channels" and generate all the possible permutations of what is mapped where.

Then you can listen to the outputs, see what sounds good, and then test it in real time to check if the musicality is actually a result of the physical interaction and not an artifact or a product of noise.

ESPectre takes a different architectural approach that might address some of the challenges you encountered:

1. Instead of STA/AP mode on a single ESP32, ESPectre uses the natural traffic between your existing router and an ESP32-S3 in station mode. To ensure a stable, continuous CSI packet rate, I implemented a traffic generator that sends ICMP pings to the gateway at a configurable rate (default: 20 pps). This provides bidirectional traffic (request + reply) that reliably triggers CSI generation, giving you predictable packet timing without relying on ambient network traffic or special sniffing modes.

2. Rather than applying filters directly to raw CSI, ESPectre uses Moving Variance Segmentation (MVS) on unfiltered spatial turbulence (std dev of subcarrier amplitudes).

3. The filters are applied to features, not to the segmentation signal itself. This preserves motion sensitivity while cleaning up the feature data

I found that having a stable transmitter (the router) combined with controlled traffic generation provides more consistent multipath patterns and predictable CSI timing, which makes the segmentation more reliable.

That's a fair point, and as a math graduate, I absolutely agree that ML is fundamentally applied math.

When I say 'No ML,' I mean there is no training phase, no labeled data needed, and no neural network model used to infer the rules.

The distinction here is that all the logic is based purely on signal processing algorithms.

Thanks for raising the point!

Probably one per room because all the examples at https://github.com/francescopace/espectre/blob/main/CALIBRAT... are about a single room.

Is the author reads this, how does the system cope with multiple rooms in the same house, maybe a two or three storeys house?

Tommysense creates a sensing mesh between devices, while ESPectre uses your existing Wi-Fi router as the transmitter. As a result, ESPectre needs only one device per area but requires a compatible router with solid 2.4 GHz coverage. The overall goal is similar, but ESPectre is open-source!

It's neat that Tommysense works on top of esphome... I'm currently using Bermuda BLE trilateration, but it doesn't quite work, especially in a multy-story living space (e.g. a townhouse). So I already have a bunch of esphome Bluetooth proxies all over the building.

But no source and "lifetime license if you join our discord" is kinda not my jam.

Thanks for sharing the paper; I'll certainly be taking a look at that research!

Currently, ESPectre performs only binary motion detection (IDLE/MOTION) based on simple statistical thresholding.

It cannot ignore cats or prioritize size over speed directly on the device, but ESPectre's architecture is designed to enable this kind of advanced classification externally.

It collects a rich set of pre-processed features (spatial turbulence, entropy, etc.) and transmits them via MQTT.

Any external server (like a Home Assistant add-on or a dedicated Python script) can use these features as the input for a trained ML model to perform classification (e.g., Cat vs. Human vs. Fall detection vs. Gesture detection).

Regardin Ruckus Router / Beamforming: for CSI sensing, stability is generally more important than raw power. I recommend starting by disabling beamforming or reducing the power output if you experience poor motion sensitivity, as the stability of the ESP32 receiver is often the bottleneck.

Are you using two separate S3s as a dedicated Transmitter/Receiver pair, or are both transmitting data simultaneously?

Sure, the ESP32 will connect to whichever mesh node provides the best 2.4 GHz signal

- It monitors CSI from that specific node (the one it's associated with)

- If the ESP32 roams to a different mesh node, it will start monitoring CSI from the new node

The system doesn't care about the router's internal mesh topology, it just needs a stable connection to receive CSI data from the associated access point.

The project’s open-source nature acts as an ethical safeguard, and I am explicitly not pursuing any identity recognition features, just movement detection.

But you are absolutely right that, in theory, misuse of this technology could reveal certain behavioral patterns that might lead to identification.

However, it can also be extremely useful for safety purposes, for example, detecting people during a house fire or an earthquake.

Hahaha, I love your brain!