Tensorflow speech sample on Microbit V2

[espruino-discuss]

Posted at 2020-11-28 by Abhigkar

I recently got my Microbit V2 that is very much supported by Espruino. I noticed that Tensor Flow is already activated in microbit v2 board file. Thai means it supports TF by default.

Since mBit v2 has inbuild microphone I wonder how easy or complex is to implement/run well known TF-Lite Yes-No demo or Edge Impulse Voice Activated Microbit demo.

Edge impulse internally using tensor flow lib which looks same as in Espruino TF

PS: You can also export the trained models to compatible Arduino or C++ library.

By the way this link is broken here Espruino GitHub and the this is the correct URL

[espruino-discuss]

Posted at 2020-11-29 by Abhigkar

New Updates:

I am so excited to have MbitV2 in my hand so could not resist to try and run existing tensorflow micro_speech model on microbit.

I started to look an existing Espruino approach from this post to generate espruino compatible tflite model. I encountered so many issues while convert existing model(hello world/sin wave), creating new model(speach), and even building espruino for mBitV2.

1 - Convert existing hello_world model

This code is no longer working with the new notebook.

import base64;
print("var model=atob(\""+base64.b64encode(tflite_model)+"\");")

but this works

import base64;
print(base64.b64encode(tflite_model))

After fixing the base64 stuff, i tried to run the same example in emulator but it neither gives any error nor expected outcome.

Original example vs Updated example

2- "Prompt not detected - upload failed" error

I am getting upload error when trying to upload this code direct to RAM.

Is this related to Low RAM or Flash??? How can I fix this?

Connected to Espruino MICROBIT2
>Execution Interrupted
New interpreter error:
Execution Interrupted
Execution Interrupted
Execution Interrupted
Execution Interrupted
Execution Interrupted
Execution Interrupted
Execution Interrupted
Execution Interrupted
Execution Interrupted
Execution Interrupted
Execution Interrupted
Execution Interrupted
Execution Interrupted
Execution Interrupted
Execution Interrupted
Execution Interrupted
Execution Interrupted
Prompt not detected - upload failed. Trying to recover...

Microbit is flashed this hex

>process.env
={
  VERSION: "2v08",
  GIT_COMMIT: "9a1dd6eb",
  BOARD: "MICROBIT2",
  FLASH: 524288, STORAGE: 40960, RAM: 65536,
  SERIAL: "d80c74c5-75c326d4",
  CONSOLE: "Bluetooth",
  MODULES: "Flash,Storage,hea" ... "opixel,tensorflow",
  EXPTR: 536882360 }
> 

3- Build error.

To get more space on Microbit I tried to remove 'NEOPIXEL' module from the board file and build make clean && BOARD=MICROBIT2 make.

But I got errors right after elf file is created

here is the truncated log

......
...
...
/mnt/c/Users/ABHINAV.000/Documents/esp/Espruino/targetlibs/nrf5x_12/components/drivers_nrf/hal/nrf_gpio.h:648:22: warning: inlining failed in call to 'nrf_gpio_pin_write': call is unlikely and code size would grow [-Winline]
 __STATIC_INLINE void nrf_gpio_pin_write(uint32_t pin_number, uint32_t value)
                      ^
targets/nrf5x/jshardware.c:740:9: note: called from here
         nrf_gpio_pin_write(MB_LED_COL2, i&1);
         ^
/mnt/c/Users/ABHINAV.000/Documents/esp/Espruino/targetlibs/nrf5x_12/components/drivers_nrf/hal/nrf_gpiote.h:322:22: warning: inlining failed in call to 'nrf_gpiote_event_clear': call is unlikely and code size would grow [-Winline]
 __STATIC_INLINE void nrf_gpiote_event_clear(nrf_gpiote_events_t event)
                      ^
targetlibs/nrf5x_12/components/drivers_nrf/gpiote/nrf_drv_gpiote.c:251:5: note: called from here
     nrf_gpiote_event_clear(NRF_GPIOTE_EVENTS_PORT);
     ^
targets/nrf5x/jshardware.c: In function 'dly.isra.0':
/mnt/c/Users/ABHINAV.000/Documents/esp/Espruino/targetlibs/nrf5x_12/components/drivers_nrf/delay/nrf_delay.h:162:22: warning: inlining failed in call to 'nrf_delay_us': call is unlikely and code size would grow [-Winline]
 __STATIC_INLINE void nrf_delay_us(uint32_t number_of_us)
                      ^
targets/nrf5x/jshardware.c:961:3: note: called from here
   nrf_delay_us((uint32_t)microsec);
   ^
src/jsutils.c: In function 'jsAssertFail':
/mnt/c/Users/ABHINAV.000/Documents/esp/Espruino/targetlibs/nrf5x_12/components/toolchain/cmsis/include/core_cm4.h:1790:22: warning: inlining failed in call to 'NVIC_SystemReset': call is unlikely and code size would grow [-Winline]
 __STATIC_INLINE void NVIC_SystemReset(void)
                      ^
src/jsutils.c:374:3: note: called from here
   NVIC_SystemReset();
   ^
/mnt/c/Users/ABHINAV.000/Documents/esp/Espruino/targetlibs/nrf5x_12/components/toolchain/cmsis/include/core_cm4.h:1790:22: warning: inlining failed in call to 'NVIC_SystemReset': call is unlikely and code size would grow [-Winline]
 __STATIC_INLINE void NVIC_SystemReset(void)
                      ^
src/jsutils.c:374:3: note: called from here
   NVIC_SystemReset();
   ^
python scripts/check_elf_size.py MICROBIT2 espruino_2v08.93_microbit2.elf
Testing espruino_2v08.93_microbit2.elf for MICROBIT2
STORAGE: 442368 -> 483328
FS DATA: 465824 -> 465840 (16 bytes)
CODE: 126976 -> 465840 (338612 bytes)
==========================
 CODE AND STORAGE OVERLAP
   by 23472 bytes
==========================
make/common/NRF5X.make:440: recipe for target 'espruino_2v08.93_microbit2.app_hex' failed
make: *** [espruino_2v08.93_microbit2.app_hex] Error 1

---

Posted at 2020-11-30 by @gfwilliams

Hi,

Thanks for the info about the link... Just fixed.

Sorry to break this to you but Tensorflow speech was the first thing I tried. The micro:bit isn't fast enough to work with their example code - the code would need re-writing to do less processing.

What is the error from: print("var model=atob(\""+base64.b64encode(tflite_model)+"\");")?

This seems like valid Python to me, so I'd surprised it would suddenly start failing?

Not sure what's going on with the upload - is it possible you had other code running in the background on the micro:bit? But either way that code looks like it wouldn't work - it records 6000 samples of wave data, and I'm not sure you're going to be able to feed 6000 samples into Tensorflow without running out of RAM.

Re, the build : You're building without RELEASE=1 - you need this to get everything in memory.

The CODE AND STORAGE OVERLAP error is another error related to not having enough flash memory. The code will fit (just) but there's then no room to save any code to flash, so it errors.

---

Posted at 2020-12-01 by Abhigkar

Thank @gfwilliams for your response. It is very unfortunate that mBit v2 still not so powerful to run Esprunio and TF both. I am sure there must me some performances optimisation there to fit in.