Low power detection sensor module for nRF52 & ESP32

[rbomze]

Hello again! This pull request replaces PR #8766. Presented by @Columbo818 and myself, it enables nRF52- and ESP32-based nodes to run the DetectionSensor module in low or even shutdown power modes.

The code takes effect when the following conditions are met:

• The device is nRF52840-based or ESP32-based*
• The nodes role is set to Sensor
• Power-saving mode is enabled
• And, of course, the Detection Sensor module is enabled with a valid GPIO pin

Update - The following variables were renamed in this PR due to their misleading names:

• state_broadcast_secs -> state_broadcast_interval
• minimum_broadcast_secs -> message_rate_limit

Its behavior is configurable via the Minimum Sleep Time, the State Broadcast Interval and Minimum Broadcast Message Rate Limit [or similar in the future].

• Minimum sleep time (min_wake_secs) determines how long the device stays awake after a detection (and also after cold boot).
• Message Rate Limit (message_rate_limit) determines the minimal time between reporting detection events.
• State Broadcast Interval (state_broadcast_interval), determines the interval between wake-ups, if desired.

The nRF52 knows two low power states. A low power mode (down to 14uA) and a shutdown (0.5uA), while still sensing a pin for a desired state (LOW/HIGH) which leads to a bootup.
If a State Broadcast Interval is set it the device enters a low power loop polling the pin. With unset interval it first enters a low power delay to comply with the Rate Limit and later goes into shutdown offering a power consumption lower than the self-discharge any battery.
This is necessary as in shutdown there is no RTC and thus it is impossible to keep track of time to compare with a timestamp of the last message send (for Rate Limit).

ESP32 based devices enter a deepSleep(..) while the (RTC-)GPIO is monitored via EXT0.

*Only ESP32 with EXT0 like ESP32, ESPS32S2/3 are supported. ESP32C3/6 lack this feature and thus are not supported. They fall back to normal operation, if power saving and detection sensor are enabled. Except DIY devices there are no commercially available devices without EXT0. Using EXT1 would interfere with the User Button and it would be impossible to choose a desired wake up level (LOW/HIGH), as EXT1 applies to all specified PIN concurrently, including the User Button, which has a default pull-up/-down.

These power consumption measurements were done with a Fluke 77IV directly attached to the battery, with internal pull-ups enabled:

Heltec T114 V2 (nRF52):

• In “ON” state: 9-10 mA
• In the low-power loop: 0.01-0.02 mA
• In shutdown state with sensing pin: 0.01 mA
  (More precise measurements are not possible with my hardware)

Heltec WiFi LoRa 32 V2 (ESP32):

• In "ON" state: >100 mA
• in deep sleep with enabled EXT0: 1,77 mA
• in deep sleep (=soft shutdown): 1,77 mA

Haruki’s Meshtastic Experiments confirm the high power consumption of the Heltec V2 with a measured value of 2.68mA.
Additional measurements of the ESP32 devices by the community are welcome.

In a nutshell: this PR makes it possible to run a detection sensor node at a remote location for months or even years on a small battery.

There is one thing that should be mentioned in the documentation and applies to all telemetry, metrics or sensing modules, which do allow low power mode. When activated concurrently and using power saving, race conditions with doSleep(..) occur. This code is designed to work quite well with other modules active, as a GPIO event will always be detected during sleep, but the parameters should be chosen the following:

• min_wake_secs: set to a high value, so the other modules have time to run, send their data and trigger deepSleep(..) themself.
• state_broadcast_interval: Best to unset the value (=0). State message are quite senseless when other modules report on a regular basis anyway. A wake up by timeout without the value set does not trigger a state broadcast.
• message_rate_limit: As you please. Except state_broadcast_interval is set on an nRF52. Then it's mandatory to set it to a value lower than the other sensors interval.

🤝 Attestations
[✅] I have tested that my proposed changes behave as described.
[✅] I have tested that my proposed changes do not cause any obvious regressions on the following devices:
[✅] Heltec WiFi LoRa 32 (V2)
[✅] Heltec T114

[phaseloop]

@rbomze Just FIY - I think NRF52 power consumption (in any mode) can be greatly reduced on many boards by enabling on-chip voltage regulators to work in DC-DC mode instead of LDO mode. AFAIK T114 board has LC filter onboard that allows to run REG1 in DC-DC mode (REG0 is not supported in that design)

Worth trying - I'm planning to research that but no earlier than in a month as I other work in progress for NRF52.

Also - I learned about it only today - many ESP32 models have "ultra low power CPU core" that may be a nice research area for stuff like Meshtastic.

The ULP coprocessor and RTC memory
remain powered up during the Deep-sleep mode. Hence, the developer can store a program for the ULP
coprocessor in the RTC slow memory to access RTC GPIO, RTC peripheral devices, RTC timers and internal
sensors in Deep-sleep mode.

https://esp32.com/viewtopic.php?t=37705

[phaseloop]

For some reason SD is initialized only on Bluetooth enable sequence. I will be fixing that in #8793 but maybe a quick workaround would be to init soft device early on boot to have one common API across firmware to avoid need to checking that constantly.

[phaseloop]

Will it improve power usage if for example we sent also telemetry message (if needed of course given time periods elapsed) in the minimum (5s-10s) wakeup windows instead of keeping device up for whole minute just for the telemetry timer to trigger the same?

[rbomze]

I question if device telemetry data (i mean the radio metrics) is any worth on a device which is sleeping most of the time, because no useful radio statistic can be acquired during that time.
When speaking about telemetry data in general (temperature sensors, etc), sure. It does help a lot. See my updated text on the PR description.
What would make a lot of sense is to have battery voltage information in every telemetry data, a single byte would be enough and the receiver can track the nodes battery state, instead clogging the airtime with an additional message.

[phaseloop]

Radio statistics of course not - but either environment statistics or device battery is useful. I know battery stats are part of the detection message but it can not be used with monitoring software looking for generic telemetry frames.

On device boot we already send NodeInfo - if the boot reason is wakeup and node is sensor, maybe it would be worth to send telemetry packet too instead of wasting battery to just wait for arbitrary number of seconds. This could be configurable.

[rbomze]

There are 2 great use cases for the detection sensor:

• Alerting, like sensing a door/window using a reed switch or PIR
• In combination with another sensor for environmental metrics as a temper detector. Imagine you have a grey box measuring eg. temperature and humidity and you want to know if someone opens it up, rips it from its mounting, etc.

[phaseloop]

I understand, one question: If I want to monitor some remote site with alarm detection and device is waking up periodically (in addition to GPIO alarm trigger) i need to keep it up at least for 70s in order to be able to see what battery I have left there, right?

[rbomze]

If you want to wait for the device metrics to be sent (which include the battery voltage) it would be, but not anymore. I added the battery voltage to the state broadcast.
The state broadcast was broken anyway. Instead of the state it reported hasDetectionEvent(), so it always reported "Friendlyname status: 0".
Now you can set it to 24h or 1 week and get "Friendlyname status: high, 56%"

[rbomze]

So essentially we can set min_wake_secs to 5s, state_broadcast_secs to 24h and minimum_broadcast_secs to 10min and we get a nice alarm sensor.
The potentially possible 0.5uA are crazy low. Even if it would be 2uA, a tiny 100mA battery would last 5.7 years, if only monitoring.
The Nordic datasheet clearly states it uses 0.4uA when off and additional 100nA sensing one pin. So 0.5uA in total. What spoils the party is potentially the LDO in front of the nRF52 and other components.
The RAK 'Nordic nRF52840 BLE Core Module', seemingly on a baseboard, was measured by Haruki with 2uA in shutdown.

[phaseloop]

Cool, thanks for explanation! I would suggest renaming "state_broadcast_secs" to "state_broadcast_interval". Same with minimum broadcast :)

[rbomze]

Cool, thanks for explanation! I would suggest renaming "state_broadcast_secs" to "state_broadcast_interval". Same with minimum broadcast :)

I just renamed:
state_broadcast_secs -> state_broadcast_interval
minimum_broadcast_secs -> message_rate_limit
This all is now much more readable. I will PR to change the descriptions of the Apps and the Website too.
minimum_broadcast_secs was plain misleading or even wrong btw., because it also rate limited the detection messages.

[phaseloop]

I did not use detection module - but I think at this exact point LoRa radio and other subsystems are already down. Does minimum broadcast secs parameter makes sense in such device condition or should it be enforced higher in doDeepSleep() before other subsystems are shut down?

[rbomze]

They are down - but what when we wake up 3 seconds later by GPIO and don't know the time on the nRF52?

Getting this whole thing working, especially with an nRF52 which has disabled RTC in shutdown, measuring the sleeptime of the ESP32, making, both, the beloved shutdown state of the nRF52 and low power mode possible made my head twist for a couple of days. Since today everything should work smoothly.

[phaseloop]

ok, I get it - I understood this as minimum wakeup time but this is different setting. Overall this is a good idea and work :)

[rbomze]

It's all a bit confusing. Even the word 'minimum' itself - it should be throttling.

[phaseloop]

After second thought - maybe it's better way to not use softdevice functions at all (given SD is active only in certain scenarios) and just use raw registers. As SoftDevice has impact on RAM (and probably power usage) my initial comment about enabling softdevice early on boot is not valid.

[rbomze]

During development i learned that direct access to NRF_POWER->GPREGRET is definitly unsafe when sd is in operation. I encountered various sporadic crashes in development until i learned that.
At first i used GPREGRET2, later i realized using GPREGRET is not in conflict with any code, as we skip the bootloader during warmstarts with NVIC_SystemReset() anyway.
Regarding performance - the good thing is that access happens rarely and never in a loop (except during exit of the lpLoop(sleepTime)). In total 1 read and 1 write(+clr) in my code and 1 time during the shutdown (setting DFU-magic to skip the bootloader on next start). This can be even combined in an if/else, so we only get 1 read and 1 write(+clr).

[rbomze]

I know the ULP some bit. about 4 years ago i wrote a program in this limited assembly language to monitor a pin attached to a simple 433mhz on/off receiver.
I was able to decode things and launch the application on the ESP32 with the decoded payload passed to it, when the conditions met. Quite cool stuff. I should put that on Github.
It was the reason back then to get an oscilloscope (a logic analyzer would have been better just for that task).

[phaseloop]

During development i learned that direct access to NRF_POWER->GPREGRET is definitly unsafe when sd is in operation. I encountered various sporadic crashes in development until i learned that.

Good to know - I'll research that, thanks. I will check whether is is generic NRF52 issue or this firmware issue. Maybe in the end it will be still a good idea to enable SD at boot and just keep it there all the time. I'm having similar concerns with implementing PowerHAL layer where depending on usage sometimes softdevice is on and sometimes off.

In my case accessing triggering initBrownout() when sd was off was crashing the CPU core.

[rbomze]

During development i learned that direct access to NRF_POWER->GPREGRET is definitly unsafe when sd is in operation. I encountered various sporadic crashes in development until i learned that.

Good to know - I'll research that, thanks. I will check whether is is generic NRF52 issue or this firmware issue. Maybe in the end it will be still a good idea to enable SD at boot and just keep it there all the time. I'm having similar concerns with implementing PowerHAL layer where depending on usage sometimes softdevice is on and sometimes off.

In my case accessing triggering initBrownout() when sd was off was crashing the CPU core.

I think i read in Nordics papers or their forum, that sd_power_*() functions are the only safe way to access these registers when SD is in operation.

[rbomze]

I have the perfect solution for the register. Macros:

#define SAFE_GPREGRET_SET(reg_idx, value)                                 \
    do {                                                                  \
        volatile uint32_t *reg_ptr;                                       \
        if (reg_idx == 0) {                                               \
            reg_ptr = &(NRF_POWER->GPREGRET);                             \
        } else if (reg_idx == 1) {                                        \
            reg_ptr = &(NRF_POWER->GPREGRET2);                            \
        } else {                                                          \
            break;                                                        \
        }                                                                 \
        if (!(sd_power_gpregret_clr(reg_idx, 0xFF) == NRF_SUCCESS &&      \
              sd_power_gpregret_set(reg_idx, (value)) == NRF_SUCCESS)) {  \
            *reg_ptr = (value);                                           \
        }                                                                 \
    } while (0)

#define SAFE_GPREGRET_GET(reg_idx, out_var)                               \
    do {                                                                  \
        volatile uint32_t *reg_ptr;                                       \
        if (reg_idx == 0) {                                               \
            reg_ptr = &(NRF_POWER->GPREGRET);                             \
        } else if (reg_idx == 1) {                                        \
            reg_ptr = &(NRF_POWER->GPREGRET2);                            \
        } else {                                                          \
            break;                                                        \
        }                                                                 \
        if (sd_power_gpregret_get(reg_idx, &(out_var)) != NRF_SUCCESS) {  \
            (out_var) = *reg_ptr;                                         \
        }                                                                 \
    } while (0)

usage:

SAFE_GPREGRET_SET(0, 0xB1);  // GPREGRET
SAFE_GPREGRET_SET(1, 0xA5);  // GPREGRET2

uint32_t val;
SAFE_GPREGRET_GET(1, val);   // Reads from GPREGRET2

Looks a bit bloated at first, but is still efficient. Unfortunately the registered are called GPREGRET and GPREGRET2 (as they added GPREGRET2 later), btw. it is available on most nRF5x, only the nRF51-Series seem to lack it. So if we want to use both registers in the future this distinction is necessary.

[rbomze]

I want to test the code thoroughly in the next days in conjunction with additional sensors attached.
Yet it was just tested with the detection sensor solely. In theory the state machine should work... but nothing beats testing.
I have a RP2040 board in my shelf - i checked the code and it should work very similar like the lpLoop() of the nRF52 while waiting for the 'Minimum Broadcast' to pass. So it should work very well too.

[rbomze]

During development i learned that direct access to NRF_POWER->GPREGRET is definitly unsafe when sd is in operation. I encountered various sporadic crashes in development until i learned that.

Good to know - I'll research that, thanks. I will check whether is is generic NRF52 issue or this firmware issue. Maybe in the end it will be still a good idea to enable SD at boot and just keep it there all the time. I'm having similar concerns with implementing PowerHAL layer where depending on usage sometimes softdevice is on and sometimes off.
In my case accessing triggering initBrownout() when sd was off was crashing the CPU core.

I think i read in Nordics papers or their forum, that sd_power_*() functions are the only safe way to access these registers when SD is in operation.

I had a little chat with ChatGPT about the registers and softdevice. Verdicts:

1. SoftDevice owns certain POWER peripheral registers
2. SoftDevice enforces register access control. Direct write access may generate a SOFTDEVICE_ASSERT and reset.
3. The SoftDevice expects all POWER peripheral modifications to go through its APIs. Direct writes can race with SoftDevice operations.
4. Nordic’s specification explicitly forbids direct access: "Any register marked as SoftDevice‐controlled must only be accessed through SoftDevice API calls."

I remember that when i started to rewrite all the accesses to sd_power_*()-functions the last one, saving the GPREGRET silently failed and the register remained unchanged.
When i started to check for the return value i realized that the functions fail when SD is already shutdown, therefor these fallbacks are necessary. My proposition - don't really care about Softdevice and use the wrapper macros instead.

[phaseloop]

Yeah, looks good - you can also save flash size by making it a function to be called instead of preprocessor pasting it each time.

Also I suppose we will be forced to keep SD enabled at boot because filesystem driver is buggy:

https://github.com/adafruit/Adafruit_nRF52_Arduino/blob/4a2d8dd5be9686b6580ed2249cae43972922572f/libraries/InternalFileSytem/src/flash/flash_nrf5x.c#L137

While async wait is checking for sd being enabled - flash erase operation is not. I just guess no one just ever tested meshtastic NRF52 without bluetooth compiled in or fully disabled....

I will be enabling SD at boot anyway in my PR - so maybe we can just simplify all RF52 codebase and use sd_* functions and don't care about wrappers. What do you think?

[phaseloop]

@rbomze I did a research about soft device - unfortunately this is hard-baked in Bluetooth library and enabling it externally causes conflicts. So we need to use your idea about safe functions - maybe it's better to use dedicated method sd_softdevice_is_enabled

[rbomze]

@rbomze I did a research about soft device - unfortunately this is hard-baked in Bluetooth library and enabling it externally causes conflicts. So we need to use your idea about safe functions - maybe it's better to use dedicated method sd_softdevice_is_enabled

Calling sd_softdevice_is_enabled() and later an if/else with another call or direct setting is much more binary than simple call and run the fallback.
Thanks to you i reentered the rabbit hole of writing small code. And one sentence i read today was: 'The compiler is smarter than all of us.' 😆

Here's some macro even in case there is no spoon... eh - Bluetooth. Skipping the conditional branch.

#ifndef MESHTASTIC_EXCLUDE_BLUETOOTH
    #define SAFE_GPREGRET_SET(value)                                      \
        do {                                                              \
            if (!(sd_power_gpregret_clr(0, 0xFF) == NRF_SUCCESS &&       \
                  sd_power_gpregret_set(0, (value)) == NRF_SUCCESS)) {   \
                NRF_POWER->GPREGRET = (value);                            \
            }                                                             \
        } while (0)

    #define SAFE_GPREGRET2_SET(value)                                     \
        do {                                                              \
            if (!(sd_power_gpregret_clr(1, 0xFF) == NRF_SUCCESS &&       \
                  sd_power_gpregret_set(1, (value)) == NRF_SUCCESS)) {   \
                NRF_POWER->GPREGRET2 = (value);                           \
            }                                                             \
        } while (0)

    #define SAFE_GPREGRET_GET(out_var)                                    \
        do {                                                              \
            if (sd_power_gpregret_get(0, &(out_var)) != NRF_SUCCESS) {   \
                (out_var) = NRF_POWER->GPREGRET;                          \
            }                                                             \
        } while (0)

        do {                                                              \
            if (sd_power_gpregret_get(1, &(out_var)) != NRF_SUCCESS) {   \
                (out_var) = NRF_POWER->GPREGRET2;                         \
            }                                                             \
        } while (0)
#else
    /* direct access only */
    #define SAFE_GPREGRET_SET(value)      do { NRF_POWER->GPREGRET  = (value); } while (0)
    #define SAFE_GPREGRET2_SET(value)     do { NRF_POWER->GPREGRET2 = (value); } while (0)
    #define SAFE_GPREGRET_GET(out_var)    do { (out_var) = NRF_POWER->GPREGRET;  } while (0)
    #define SAFE_GPREGRET2_GET(out_var)   do { (out_var) = NRF_POWER->GPREGRET2; } while (0)
#endif

called by

SAFE_GPREGRET_SET(0xB1);
SAFE_GPREGRET2_SET(0x42);
uint32_t reason;
SAFE_GPREGRET_GET(reason);

We won't use GPREGRET2 anyway so soon.

[phaseloop]

The only suspicion I have is that calling sd_power* when SD is not enabled may actually crash CPU sometimes without returning an error. This is what happens when you run initBrownout() on boot a d before bluetooth starts :(

[rbomze]

@rbomze I did a research about soft device - unfortunately this is hard-baked in Bluetooth library and enabling it externally causes conflicts. So we need to use your idea about safe functions - maybe it's better to use dedicated method sd_softdevice_is_enabled

sd_softdevice_enable(...) gets called in line #319 of bluefruit.cpp by Bluefruit.begin(); by NRF52Bluetooth::setup(). And never disabled, even after shutting down Bluetooth in our code.
And yeah - Softdevice is Nordics rom/library for Bluetooth, and convenient, safe, functions.

[phaseloop]

Yeah, sadly Adafruit lib does not try to check if SD is already enabled. Nordic claims that using NVIC_* methods is unsafe if SD is enabled because on soft reboot some registers and memory parts are left initialized and enabling SD again on boot may fail / crash device.

Which may explain some bug reports in this repo.

[rbomze]

Yeah, sadly Adafruit lib does not try to check if SD is already enabled. Nordic claims that using NVIC_* methods is unsafe if SD is enabled because on soft reboot some registers and memory parts are left initialized and enabling SD again on boot may fail / crash device.

Which may explain some bug reports in this repo.

AdafruitBluefruit does this: VERIFY_STATUS( sd_softdevice_enable(&clock_cfg, nrf_error_cb), false );
and this very line will return some error code. What do you need sd_*() for again (before Bluetooth)?? .. i am reading your PR