EEPROM byte/bit-wise writing without erasing possible?

[espruino-discuss2]

Posted at 2019-11-05 by ChristianW

From this page http://www.espruino.com/AT24 I understand you can write whatever number of bytes to an EEPROM.
What I understood about EEPROMs that before writing a certain area (page) needs to be erased first (resulting in all FF) before it is written.
But what happens if I write only a byte?

The reason for my question is this: I want to store an odometer value in an EEPROM and avoid writing too often reducing wear.
What I know from the automotive world is that they are using a certain area all set to FF and (redundantly) clearing single bits in a certain pattern without erasing anything.

Is it possible with Espruino to do the same:
a) Erase a certain EEPROM area (set it to all FF) only once during initialization.
b) Later during lifetime only write single bytes in a way that only zero bits are cleared without doing an erase again?

Something like:

// init (once)
eeprom.write(address,data,true);

// run (every N kilometers / hours)
eeprom.write(address,data,false);

[espruino-discuss2]

Posted at 2019-11-05 by @allObjects

Afaik it is not the writing 'per se', but the flipping of the bits that wears the silicon down.

Depending on the architecture/construction/type of the FLASH EEPROMs, you can write just bytes, but you have to erase by pages.

For your solution, with these assumptions:

• your odometer reading fits into a block of 4 (or 8) bytes
• you have a byte addressable and byte writeable EEPROM
• your EEPROM has 256 bytes (for the example only)
• your EEPROM has 256 bytes pages (for the example only, and for the first part, it does not even matter)

I suggest to:

• divide your memory space into two regions:

  • 1. space management - 8 bytes - 0..7 - bit by bit = 64 spaces - 64-bits
  • 2. odometer readings - 248 bytes - 8..255 - 62 blocks of 4 bytes for each reading

You see that you have 2 more space management bits than you need to manage the 62 blocks for the odometer readings, which you will use for a different purpose - explained later.

To write an odometer reading you proceed as follows:

Find the the first non-zero (1) bit in the first 8 bytes starting with the 3rd bit . Byte and bit position adjusted by some constant tells you what block of 4 bytes you can use next.

Since you already read the space management byte, you flip the found (1) to a zero and overwrite the the byte with the effect that only the flipped byte changes state only (and the bits 'before' that one do actually not matter). Then you write the four bytes with the odometer reading value.

For the very first time / block, byte is 0, bit pos is 2, and 1st byte of odometer reading writing block is 8 - all values 0 based.

After writing six (6) odometer readings, the byte will be 1, bit will be 0, and 1st block byte is 56.

When you cannot find a 1 bit anymore, you know that your EEPROM 'is full', and for that reason you clear the 2nd bit of the first space management byte... because you may have more than one 256 bytes EEPROM for storage.

Applying this to real world EEPROMs, you just scale this up:

• each page contains its own space management area(s) and its data storage area(s).
• each page has a bit telling when it is full
• each page has a bit telling it can be eased (because some retrieval mechanism has read / consumed / freed the page)
• with that you have a circular memory buffer / fifo...

I know that flash EEPROMs are very cheap and have immense capacity... but thy also have the issue of wear and the need for delays on write, and need erasing. Therefore, I would look into alternatives, such as FRAMs and MRAMS: solid state memories that use polarized magnetized zones for storing bits. They have no wear and need no delays on write (you find them in some forum posts... )... and there are actually microcontroller chips on the market that do have FRAM/MRAM technology as RAM... so power can be turned of any time with no need to save the RAM somewhere and to have the buffered power left to do so (very simple setup for power supply, brown out and resume logic).

On the other hand - being realistic - the wear of 100K writes combined with the size and the price should not be that a problem: just keep somewhere the information how many cycles you have run thru and then toss it out.

Btw, sent you a message (see 'yellish' letter below your gravatar image in the top right corner of the page...

[espruino-discuss2]

Posted at 2019-11-05 by ChristianW

Hi @allObjects, thanks for your extensive reply.
Actually what you wrote would be my fallback solution in case the intended one was not possible.

The question was: Could a write done without an erase? Meaning: only turning off single bits one by one, because that's what my car odometer internally does.

Starting with a huge bitfield like FF FF FF FF FF FF FF FF (whatever size + redundant copies) after N kilometers one bit is deleted: FF FF FF FF FF FF FF FE, after another N the next bit: FF FF FF FF FF FF FE FE (and the same on the redundant copy or copies).
The exact algorithm is not yet known to me and maybe there are cases when parts are actually erased and re-written again, but it seems that clearing only single bits step by step reduces wear.
This is somewhat confirmed in this discussion here: https://www.microchip.com/forums/m550855.aspx

And it was the reason for my question.

[espruino-discuss2]

Posted at 2019-11-05 by @allObjects

And pictoral:

`
:Space managment bytes         :Data storage bytes                     :
:1st      2nd     n-th last    :1st blck          n-block      last blk:
:76543210 765 \\ 43210 76543210:                                       :
.--------.--- // -----.--------.--------.--------.--- // -----.--------.
|11111111|111 \\ 11111|11111111|ffffffff|ffffffff|fff \\ fffff|ffffffff|
'--x-----'--- // -----'-------x'--------'--------'--- // -----'--------'
 ^^|                          | ^                              ^
 |||                          | |    'points' to last block    |
 |||                          '-)------------------------------'
 ||| 'points' to first block    |
 ||'----------------------------'
 ||
 |'---: 1/0 page to keep / to clear (for reuse)
 '----: 1/0 page has space / is full
`

Ic. - Make's sense since you are interested only in the miles / kms you completed.

My approach is a general one: while thinking about the things beyond the completed miles/kms, like time and other items you might want to take notes for later graphing - speed, direction, altitude, coordinates, temperatures (ambient, batteries, motor, charge states,...), the algorithm stays the same... only the data block grows in size and every block includes a time stamp. Adding a time stamp lets you optimize the memory usage, because you write back only on 'significant' changes...

The reason to use the bits in the byte the reverse order - from MSB to LSB - is to simplify the code: starting with a compare byte that has the MSB set and then shift right while set bit still in compare byte and (&&) 'and' (&) is false. When the while loop ends and the compare byte still has 'the' bit then that's the space.

[espruino-discuss2]

Posted at 2019-11-06 by Robin

Tue 2019.11.05

fr #1 'I want to store an odometer value in an EEPROM and avoid writing too often reducing wear.'

How often is the bike actually storing data and what is the expected memory storage lifetime?

IMO I think this is being over thought. (although an interesting thought project)

AT24C series are less than ~1 USD ea
Even with the aggressive lifetime suggestion of 10,000 hrs see link below, that's an awful load of sore saddle time. My 0.02 worth. . . .

We had a write duration discussion six months ago. Did you catch this?

http://forum.espruino.com/comments/14699417/
10 hrs/day page write usage equivalent of three years lifetime.

This is an easy to use chip AT24C

http://ww1.microchip.com/downloads/en/devicedoc/doc0180.pdf

[espruino-discuss2]

Posted at 2019-11-06 by ChristianW

Back to the original question:
Is bitwise deletion possible?

After studying the datasheet of the 24c32 that is coming with my RTC module the answer is: No

It only supports byte-write or page-write meaning 32 bytes in one of the 256 pages.

Regarding space management:
Since it's an odometer and values should only go up, it should be quite simple.

What about this simpler method:
(the example shows 4 pages with 1 byte each, but can be scaled to 256 pages of 32 bytes easily)

1. initialization: write odometer value to first page, set rest to 0: [ 1, 0, 0, 0 ]
2. current element: is the one with the highest number (the last one written)
3. write new value: a) find current element, move one page forward (wrap around the ends like a ring buffer), write new value to that element: [1, 2, 0, 0]

Next iterations: [1, 2, 3, 0] [1, 2, 3, 4], [5, 2, 3, 4] etc...

This way wear is spread across all pages evenly, there is no need of any extra space management structure.
Since the highest value marks the current element there can be a payload with history data just after the actual odometer value:
[ [1, p], [2, p], [3, p], ...]

Also writing can be reduced to something like: once every N minutes or K kilometers, whichever comes first.
If there is a power fail, only the last N minutes or K kilometers are lost.

The datasheet for 24C32 states 1,000,000 write cycles endurance - so without any levelling and writing every minute or km that would be 1000000 km (which is more than enough) or 694 days, nearly 2 years of non-stop riding.
Spreading this over 256 pages should multiply these values by 256 and keep me on the safe side...

[espruino-discuss2]

Posted at 2019-11-06 by @allObjects

Fixed size block writes require simple algorithms only. It gets a bit more nifty with variable sise block writes. Of course, you can also start with the anchor to find the last written block when each block includes (at the beginning) some type/length information. For any fast access though you would go for a binary-search-like algorithm, and with minimal information at the beginning of each page (page size a choice), information about page full and stradling information (information about begin in previous page of 'record' that stradles the pages) - 2 or more).

I do not see the need for skewing (ring buffer 'spiraling'), since it is not a rewrite. Even if you have to use page write, after the page is full, it is not written anymore. When reaching memory full, all pages would have about the sam amount of writes / wear (with fixed size block recordings).

If memory is read out with freeing space before full, information about ring buffer begin is required any and would have to be written somewhere. Using a bit on each page can avoid that. Freeing (w/ Erasing) would then have to happen on full with inactive, delete-able and active records.

I conclude that one can't get away with just writing application data. There is always some meta / managment data required to make it decently operable.

[espruino-discuss2]

Posted at 2019-11-06 by Robin

Wed 2019.11.06

'Is bitwise deletion possible?'

My take on it was only an erase, set to a 1 is part of the endurance value. Writing a zero to clear the bit, even using a byte-write didn't count.

'The datasheet for 24C32 states 1,000,000 write cycles endurance'

I read that too and wondered if anyone could clear up my understanding of a write cycle. I took it to mean an individual bit for each cycle, but could it mean a full page write is just one write cycle?

https://en.wikipedia.org/wiki/EEPROM
The web has many conflicting interpretations of how this specification tid-bit should be understood
( re-reading the .pdf to see if I am able to uncover that mystery )

If the latter is the case, then for:

fr #6 'Also writing can be reduced to something like: once every N minutes or K kilometers'

wouldn't just performing a page write (erase all then write once) be the least destructive and best compromise?


EDIT: (getting close)

https://searchstorage.techtarget.com/definition/write-cycle