Avoiding FPGA - possible simplification

[aaronjauf]

Hello,
very nice project and thank you for sharing!
Did you already try indoor navigation using the IMU array and are do you expect 100 Hz to be enough for a e.g. vacuum cleaner rover?
I was wondering if the FPGA can be eliminated: My idea is to use a raspberry pi pico and interfacing the I2C interfaces of the IMUs using the PIO/DMA. The clock line can be shared, allowing for 27 IMUs (since the pico has 28 gpio lines), if all IMUs are set to the same I2C address, otherwise 54 IMUs. What's your opinion on this?

Best regards
Aaron

[will127534]

So originally I did think about using RP2040 but because of the lack of GPIO (I want to do 32 IMU) and some bad timing (RP2350 released a couple weeks after I got the board back), I end up going with a FPGA, but it is not that expensive though.

Issue with I2C is speed and complexity compare to SPI bus, you are limited to 1Mhz while SPI in this project is running at 20Mhz.

Overall if I have a second revision in the future I'll go with RP2350 + SPI to the IMUs, I think that is the ultimate solution.

[nippoo]

Hey, thanks for the great project! I'm interested in porting this to an MCU - do you have any other thoughts? (I'd happily send over a PR, though likely makes sense to keep two separate projects as it's a bit of a redesign...) I'd like to run some slightly more advanced algorithms: interference suppression (to remove resonances from closely mounted IMUs), an outlier filter (to discard any out-of-tolerance IMUs), and a Kalman filter for position estimation (as well as something to calculate and cancel out the earth's precession!). So I'm considering either the RP2350, or something a little bit more powerful where the SPI I/O can run on one core and the more sophisticated algorithms can run on a fixed 100Hz loop on the other core...

As an aside: why did you choose the ICM-42688-P? It has the same stated random walk (2.8mdps/sqrt(Hz)) as the ST LSM6DSV IMU, which is half the price, so I'm curious if you've tested/know it to be better or if there's any other reason!

[aaronjauf]

I believe the reference work uses the ICM ( https://www.youtube.com/watch?v=7fgcxDRti5s ). As for the MCU, I believe the RP2040 would be sufficient (since the RP2350 is not yet available). One GPIO for MOSI, one for SCK, two for CS (16 IMUs + 16 IMUs), 16 for MISO. So the idea is to read out 16 IMUs in parallel, then the next 16. This can be done using PIO and DMA.

[will127534]

The reason at least for me to use these ICM IMUs is because they have a very unique feature - You can send a 30~40Khz clock through the pins and clock its internal ASIC, which guarantees synchronization among all the sensor and MCU and you don't have to deal with traditional FSYNC (Or any external sync signal) that only gives you the time delta between last measurements and your external event and you will need to do linear interpolation.

Sidenote here: So after this project bring-up phase and having more familiar with this IMU I've added it to my camera boards and uses FSYNC function with Vsync output from the camera sensors, so now I do have both experience (CLKIN vs FSYNC). I'm so glad that I've got this IMU and uses the CLKIN in this IMU array project, otherwise it is going to be a pain to do synchronization.

As for the algorithm, I haven't finish writing the report yet but I'm able to replicate reference 6 (Aligning the Forces—Eliminating the Misalignments in IMU Arrays - by John-Olof Nilsson et. al) to calibrate the IMU array and run the calibration with RP2040 @ 240Mhz with 500Hz IMU data output. Yes I did use both CPU cores with one deliciated to IMU + calibration/array fusion processing and the other for WiFi/GPS ... etc, but I'm still surprised with the performance I can get with RP2040. (The calibration is even running in floating point lol).

[will127534]

The thing that you need to keep in mind is synchronization: Not just IMU array itself but with other systems too.
So the pin count may be good enough on paper but having a few more to do clock adjustment will be much better. At least this is what I think right now.

Feel free to just grab the files here and starts your own repo though, git isn't good for HW design anyways lol. Also, if you have any good papers on the filtering stuff please do post some links here : )

[will127534]

Sony is using ST also, see reference 2 for details. (H. Kamata, M. Kimishima, T. Sawada, Y. Suga, H. Takeda, K. Yamashita, and S. Mitani, MEMS Gyro Array Employing Array Signal Processing for Interference and Outlier Suppression, IEEE Inertial 2020, Hiroshima, Japan)

[DSIsaac]

Hello，
I had tried IMU array with 2 imu, but it
don't solve the problem of IMU YAW axis drift. So how did you solve this problem？Can I solve this problem by increase the quantity of IMU?

[NightHawk32]

Hello， I had tried IMU array with 2 imu, but it don't solve the problem of IMU YAW axis drift. So how did you solve this problem？Can I solve this problem by increase the quantity of IMU?

Hey, author of
Purely Inertial Navigation with a Low-Cost MEMS Sensor Array by Lukas Blocher et. al
https://ieeexplore.ieee.org/document/9430468
here. To deal with the yaw drift is not an easy task. You will gain a decrease of noise of sqrt(n) with n IMUs. But since you lack the reference it is hard to deal with the drifting offsets. On x and y you can use gravity as a reference, for the Z-axis this would require an additional sensor. But when choosing a high quality sensor you can keep up the yaw angle over 10min or longer and still stay within a 1deg range of precision.
What are your requirements exactly?

In my experience calibrating offsets, sensitivity and misalignment of all IMUs within the array is crucial.

[gmaxwell]

Can yaw drift be addressed by having the array geometry such that there is a hierarchy that can be zeroed against gravity? E.g. in some orientation some sensor is aligned to gravity, so it can be zeroed, another pair is turned partway between gravity and x and another is turned the opposite way. If they were summed and drift free, they'd read gravity.. and so on.

Assuming the orientation of the senors are known, I think you can setup an array where the solution for the offsets isn't rank deficient but it's not immediately obvious to me if this requires knowing the true direction of gravity in the array coordinate space, which would be somewhat tautological.

[nippoo]

Hi @will127534, thanks for your reply - seems like there are about 3 threads going on in parallel right now. 😅

All the papers I've read have already been linked in this thread. The production version uses a CY8C6247BZI MCU, it seems (with 8 I2C/SPI peripherals). The linked video has some interesting insights - at 20:06 they imply the IMUs aren't synchronised and the FPGA is presumably handling synchronisation. I assume there's a good reason they've done this, but perhaps it's just a tradeoff of a bit of development time vs being able to double the number of IMUs at the same cost.

On another ST thread someone suggested you could start the IMU measurement by simultaneously toggling the CS pin low. I wonder if this would work, and why they've chosen that MCU - with 16 sensors you could read them in 2 sets of 8, sequentially: https://community.st.com/t5/mems-sensors/how-to-configure-and-synchronize-lsm6dso-to-raspeberry-pi/m-p/189524

That said, linear interpolation isn't massively complex either. I need to do a bit of digging to figure out how easy it is to get reliable timestamps from the data...

[nippoo]

@gmaxwell to your issue, even if your gyro is in a fixed location/orientation, the problem is still intractable: yaw drift in the plane perpendicular to gravity is impossible to calculate unless you have an external reference.

This does bring up a more interesting issue generally though: how reliable/consistent is the drift on each gyro? Is it random noise or could you calibrate / bin the IMUs to find the best one?

Looking at the graph from the paper linked above, it looks like some individual IMUs perform better than the calculated average of all 16 (there are a couple of purple lines below the blue average trace at the bottom right of the graph). Rather than going to all the effort of combining sensors etc, could you just instead leave them all on for a day, and select the best one, and discard all other results?

[will127534]

@nippoo I'm pretty much free in terms of where discussion be for my git reops/projects, typically it will be better to start a discussion but most people just asking questions in the issue anyways. Either way works for me, it is not as crazy long as some other issue I've seen on Github.

For the synchronisation I was reading ST's datasheet but I'm very much confused about how the heck do people use it without synchronisation signals, I did not see even the FSYNC functions exists in those IMUs. If I must guess maybe they are using the Data Ready (DRDY) and timestamp it in the FPGA against reference clock, and align all the IMUs with linear interpolation?

I'm not saying that I'm against linear interpolation, it just that if there is a easier way to synchronisation why bother. I think most people looking at this repo probably overestimate the work this project has done, the FPGA code here is merely a data aggregator, while the synchronisation is handled by a relatively niche function in the ICM-42688P that not a lot of people is aware of.

Start the IMU measurement by simultaneously toggling the CS pin won't work because their internal clock is still not aligned among the IMUs, and IMU sampling is based on its internal clock. You might able to start measurements at the same time but as time goes on it will still drift because IMUs can't align/adjust its internal clock just using the CS pin.

Finally, IMU sampling is not a one-shot procedure like how some ADC could be, because the sample have to pass through digital filter which needs a few samples to get started.

[will127534]

By the way I have some analyzer code in the analyzer branch with the data I've collected, I've also added some code/HW on the data processer board.

[will127534]

https://www.willwhang.dev/Project-IMU-Array/

[dishilov]

@will127534 Thank you for the dev blog writeup.