Efficiently moving things around in zig-zag Graphics buffer visualized w/ 24/32 bpp displays / neopixel strings

[espruino-discuss2]

Posted at 2019-02-11 by @allObjects

Recently played around with neopixels that I control w/ an Espruino Graphics object (see x by y w/ 24 bpp Graphics buffer visualized with neopixel string).

var c=`
// Get some speed for odd graphics buffer manipulation, such as for running
// text... moving rainbows,... (odd zig-zag... and 'pull out' into other buffers)
// On PICO w/ dev=false profiled on 20 cols x 7 rows w/ 133 of 140 neopixels
// and col counts: ccc = 1 | 9 | 18 --> 0.326 | 0.357 | 0.388 [ms]
// Development Example - set dev = true (in line 35):
//       copy left - destination same as or different from source
// 
//
// 7 x 4 24bpp graphics buffer in zigo-zag driving a 28 neopixels string
//
//        00       01       02       03       04       05       06
//       .-----------------.        .-----------------.
// 00    |00 01 02|03 04 05|06 07 08|09 10 11|12 13 14|15 16 17|18 19 20 --.
//       |                 |        |                 |                    |
// 01 .--|41 40 39|38 37 36|35 34 33|32 31 30|29 28 27|26 25 24|23 22 21 <-'
//    |  |                 | <----- |                 |
// 02 '->|42 43 44|45 46 47|48 49 50|51 52 53|54 55 56|57 58 59|60 61 62 --.
//       |                 |        |                 |                    |
// 03    |83 82 81|80 79 78|77 76 75|74 73 72|71 70 69|68 67 66|65 64 63 <-'
//       '-----------------'        '-----------------'
`;

Especially in a x-y-panel arrangement of neopixels, the Espruino Graphics object is for many things superior to manipulating a buffer directly. Graphics object is even able to handle the zig-zag storage arrangement which allows minimal hardware effort to place wire a neopixels string onto a panel (no 'fly-back' wire).

Drawback of x-y-panel arrangement though creates some challenges when trying to move things around, such as running text, walking rainbow, etc. Of course, redrawing with different origin works to some extent; but besides the slow-down, jumpiness and not being able to begin text outside (at negative x or column) of the display buffer (loosing space), redrawing is not an option.

This pushed me to venture into integrated / inline C... and I really like it... could have used that earlier when fooling around with displays to speed up operations on scattered memory locations.

Speed is what I'm looking for... I copy 133 of 140 neopixels in less than 0.4[ms]. This is not only helpful on Espruino boards - to get to better neopixel refresh rates, but on all the ...strapped ESP-8266 that have to go to tend to WiFi 'at once in a while'... and application processing has to fit into the breaks in between; otherwise, as we know, ESP8266 crashes (with infinite reboot loop). Times on ESP8266 may even look better, because the processor runs faster, I recall... I don't know whether the serial flash messes with the speed... but it's worth a try... Who is a taker for that test?

A nice extra was that I need to pass at least 8 parameters... 4 are supported... Solution is to put all numeric and pointer args into an Uint32Array and have just one single argument to pass... With that mastered, just anything that would be needed for context can be passed from JS to C.

Having to use a parameter array makes calling the C function directly in the application code quite verbose and cumbersome. Therefore a wrapper around the call provides many goodies:

• Destination buffer can be same as source buffer, but does not have to (for in-place, only left and up copy is handled so far).
• Prepare an argument block and reuse it on the same buffer(s) with just changing the copying parameters, which are column and row counts. All the other setting stay the same for the same buffer(s).
• Do some checking of argument validity, such as that buffers must be flat strings (implemented), stay within buffer boundaries (not implemented yet).
• Hiding the pixel to byte translating numbers
• Hiding the E.getAddressOf(...) for the buffer pointers
• Have a place to pass back from C / to stick the return value (and error values negatively) to: args[0]

Next is to make it work for non-zig-zag and safe for in-place copying / shifting for all directions an (reasonable) combinations there of... (that wii switches the copy begins and ends around).

Here the code:

...sorry, textarea size on forum initial post exceeded... post is cropped... see next... post #2.

[espruino-discuss2]

Posted at 2019-02-11 by @allObjects

Here the code:

// cTrial9.js
//
var c=`
// c trials... to get some speed in odd memory manipulation,
// expecially for graphics buffers to move things around...
// such as running text... (odd zig-zag... and pull out)
// On PICO w/ dev=false profiled on 20 cols x 7 rows w/
// col counts: ccc = 1 | 9 | 18 --> 0.326 | 0.357 | 0.388 [ms]

// Development Example - set dev = true (in line 29):
//       copy left - destination same as or different from source
// 
//
// 7 x 4 24bpp graphics buffer in zigo-zag driving a 28 neopixels string
//
//        00       01       02       03       04       05       06
//       .-----------------.        .-----------------.
// 00    |00 01 02|03 04 05|06 07 08|09 10 11|12 13 14|15 16 17|18 19 20 --.
//       |                 |        |                 |                    |
// 01 .--|41 40 39|38 37 36|35 34 33|32 31 30|29 28 27|26 25 24|23 22 21 <-'
//    |  |                 | <----- |                 |
// 02 '->|42 43 44|45 46 47|48 49 50|51 52 53|54 55 56|57 58 59|60 61 62 --.
//       |                 |        |                 |                    |
// 03    |83 82 81|80 79 78|77 76 75|74 73 72|71 70 69|68 67 66|65 64 63 <-'
//       '-----------------'        '-----------------'
//
`;
// define destination
var dev = false
  , dcc = (dev) ? 7 : 20
  , drc = (dev) ? 4 :  7
  , db, dg = Graphics.createArrayBuffer(dcc,drc,24
            ,{zigzag:true,color_order:"rgb"} );

var lon = false
  , log = function() { console.log.apply(console,arguments); };

// copy zig-zag-ed neopixles to left - args array w/ [0] = result
var czxLArgs; // 'globalized' argument array

function czxL( // copy zig-zag pixels left
      // helper / wrapper to call compiled C: cdC.czxL(args)
      ccc // =0: arg setup, >0: copy col count in pixels
    , crc // copy row count in pixels
    , dst // destination Uint8ArrayBuffer
    , dco // dst col offset in pixels (0..cols-1)
    , dcc // dst col count in pixels
    , src // source Uint8ArrayBuffer
    , sco // src col offset in pixels (0..cols-1)
    , scc // src col count in pixels
    , bpp // bytes per pixel, if absent|falsy: default=3
    , chk // check for validity (not yet implemented)
    ) {
  var args = new Uint32Array(9)
    , addr = E.getAddressOf(args,1)
    , tSiz = (typeof bpp == "undefined") ? 3 : bpp;
  args[0] = 0; // will be return value
  args[1] = ccc*tSiz;
  args[2] = crc;
  args[3] = E.getAddressOf(dst,1);
  args[4] = dco*tSiz;
  args[5] = dcc*tSiz;
  args[6] = E.getAddressOf(src,1);
  args[7] = sco*tSiz;
  args[8] = scc*tSiz;
  if (args[2] && args[6]) { // check for flat strs | avoid disaster
    if (ccc>0) { // doit, otherwise just args setup for reuse
      var t0 = getTime();
      args[0] = cdC.bciLeft(addr);
      var t = getTime() - t0;
      log("t =",("      "+Math.round(t*1000000)/1000).substr(-6),"[ms]");
    }
  } else {
    args[0] = -1; // src or dst not flat string (args[2] or args[3] is 0)
  }
  return args;
}

var cdC = E.compiledC(`
// int bciLeft(int)
int bciLeft(unsigned int *args) {
  unsigned int   ccc = *(args+1);
  unsigned int   crc = *(args+2);
  char          *dst = (char*)*(args+3);
  unsigned int   dbo = *(args+4);
  unsigned int   dcc = *(args+5);
  char          *src = (char*)*(args+6);
  unsigned int   sbo = *(args+7);
  unsigned int   scc = *(args+8);
  char            *d;
  char            *s;
  unsigned int     c;
  unsigned int     r = 0; // row
  int            cnt = 0;
  while (r < crc) {   // row < row count
    d = dst + dbo; s = src + sbo; c = ccc;
    while (c-- > 0) *(d++) = *(s++); // copy in even/zig rows - inc'g
    dst += dcc; src += scc; r++; cnt += ccc;
    if (r < crc) {
      dst += dcc - 1; src += scc - 1; d = dst - dbo; s = src - sbo; c = ccc;
      while (c-- > 0) *(d--) = *(s--); // copy in odd/zag rows - dec'g
      dst++; src++; r++; cnt += ccc;
  } }
  return cnt;
}
`);

function init() {
  db = dg.buffer; // 'globalize' dst buf
  for (var idx=0; idx<db.length; idx++) {  // prime buf w/......values
    db[idx] = idx % 256; }                 // ...values 0..256,0..256,...
  if (dev) dmpd();
  czxLArgs = czxL( // 'copy zig-zag-ed pixel Left'
      (dev) ? 2 : 19  // ccc - copy col count in pixels | 0 = args build only
    , (dev) ? 4 :  7   // crc - copy row count
    ,         db       // dst - destination buffer
    ,         0        // dco - dst col offset
    , (dev) ? 7 : 20   // dcc - dst col count
    ,         db       // src - source buffer
    , (dev) ? 3 :  1   // sco - src col offset
    , (dev) ? 7 : 20   // scc - src col count
    );
  if (dev) dmpd();
  log("czxLArgs =",czxLArgs);
}

function onInit() {
  init();
}

function dmpd() { log("dst buf:"); dmp(db,dcc,3); } // dump destination
function dmp(buf,bcc,xSiz) { // dump buf to console for debug
  var x = 0, r = 0, s = "", i, j;
  for (i=0;i<bcc;i++) { s+=" "+("  "+i).substr(-3);
    for (j=1;j<xSiz;j++) s+="   "; } log("rows \\ cols",s); s = "";       
  while (x<buf.length) {
    for (i=0;i<bcc;i++) {
      s+=("  "+buf[x++]).substr(-3)+("  "+buf[x++]).substr(-3)+("  "+buf[x++]).substr(-3)+" ";
    } log((" "+(r++)).substr(-2),"-  '->",s,"--."); s = "";
    if (x<buf.length) {
      x+=(bcc*xSiz)-1;
      for (i=0;i<bcc;i++) {
        s+=("  "+buf[x--]).substr(-3)+("  "+buf[x--]).substr(-3)+("  "+buf[x--]).substr(-3)+" ";
      } log((" "+(r++)).substr(-2),"-  .--",s,"<-'"); s = "";
      x+=(bcc*xSiz)+1;
    }
  } log("-- .");
}

c = "";
setTimeout(onInit,500); // while dev'g; comment before upload for save()

The console output showing before and after of the development example copying colums 3 and 4 in 7 x 4 to columns 0 and 1 (output sligtly modified/condensed to fit forum's width and not to wrap):

 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v01 (c) 2018 G.Williams
>
dst buf:
rows\cols 0         1         2         3         4         5         6
 0 '-> 0  1  2   3  4  5   6  7  8   9 10 11  12 13 14  15 16 17  18 19 20--.
 1 .--41 40 39  38 37 36  35 34 33  32 31 30  29 28 27  26 25 24  23 22 21<-'
 2 '->42 43 44  45 46 47  48 49 50  51 52 53  54 55 56  57 58 59  60 61 62--.
 3 .--83 82 81  80 79 78  77 76 75  74 73 72  71 70 69  68 67 66  65 64 63<-'
-- .
t =  0.327 [ms]
dst buf:
rows\cols 0         1         2         3         4         5         6
 0 '-> 9 10 11  12 13 14   6  7  8   9 10 11  12 13 14  15 16 17  18 19 20--.
 1 .--32 31 30  29 28 27  35 34 33  32 31 30  29 28 27  26 25 24  23 22 21<-'
 2 '->51 52 53  54 55 56  48 49 50  51 52 53  54 55 56  57 58 59  60 61 62--.
 3 .--74 73 72  71 70 69  77 76 75  74 73 72  71 70 69  68 67 66  65 64 63<-'
-- .
czxLArgs = new Uint32Array([24, 6, 4, 536875436, 0, 21, 536875436, 9, 21])
>

[gfwilliams]

Posted at 2019-02-11 by @gfwilliams

Wow, glad the inline C is working well for you! Unfortunately it's not something that'll work on ESP32/ESP8266 though unless someone's willing to host their own compiler service.

How does this compare to Graphics.scroll? I know that isn't desperately fast but then it's not having to handle that many pixels.

Just to add another potential option here when it comes to low-level Graphics manipulation: Create your own Graphics that has a non zig-zag byte ordering, and then use drawImage to draw that Graphics onto the neopixel one just before you output it.

[espruino-discuss2]

Posted at 2019-02-11 by @allObjects

Some of that that is in the path... for rolling text: copying from a fix written buffer with a leading and trailing blank into a transfer buffer that is then shipped out to neopixel string. No redrawing, just copying into a buffer...

My neopixel display IS hardware zig-zag... and for rolling text, with more than one byte in height, there is no way to not do non-contiguous copying...

Vaguely remember coming across the graphic scroll... Did not compare that yet. Even if I trow above code out in favor of the graphic scroll - which I may - but not for all things, it was a great learning experience...

I guess I need to study more conciously the merge requests / changes between the releases.

[espruino-discuss2]

Posted at 2019-02-13 by @allObjects

@gordon, I know that Graphics class is natively implemented and exposes through a wrapper the buffer directly as value and with and height through methods. I plan to write as prototypic extension as module that can copy, cut, paste - implemented in it's core in C. For that I need t know how the Graphics object was setup (w, h, bpp and options)... and to get to that, I would need to know how I get to with just the pointer (pointing to the meta or header of the object).

Therefore:

How Graphics instance's 'top layer' is structured? - Where would I look for that information?

[gfwilliams]

Posted at 2019-02-13 by @gfwilliams

The info you want is in https://github.com/espruino/Espruino/tree/master/libs/graphics

and specifically: https://github.com/espruino/Espruino/blob/master/libs/graphics/graphics.h#L58

The Graphics instance has a child called "\xffgfx" which contains the structure. It's not guaranteed to be a flat string though.

Trying to support all the different encoding styles is going to take you a while though!

[espruino-discuss2]

Posted at 2019-02-13 by @allObjects

...indeed... kind of a cartesian product says hell(o)!...

To keep complexity limited, I can see multiple, different extensions and the application has to load only what is needed... it could even be instance based rather than prototypical, even though latter I prefer (for encapsulation reasons and simplicity of use). For starters I will go for support of single neopixel string w/ zig-zag and not-zig-zag.

A) Is there reason not to group fontAbc and modMmmN, since they are build-time conditioned? - https://github.com/espruino/Espruino/blob/master/libs/graphics/graphics.h#L64L68 and https://github.com/espruino/Espruino/blob/master/libs/graphics/graphics.h#L70L72 , respective
B) What is STM32F4's alignment in structs? - byte for things <= byte, 2 bytes for things of 2 bytes, 4 bytes of 4 bytes

I assume SAVE_ON_FLASH flag is at runtime accessible somewhere...

[gfwilliams]

Posted at 2019-02-13 by @gfwilliams

A) Is there reason not to group fontAbc and modMmmN

I'm not sure I understand?

B) What is STM32F4's alignment in structs?

4 byte normally I think, but the PACKED_FLAGS after it means it's 1 byte aligned

I assume SAVE_ON_FLASH flag is at runtime accessible somewhere...

I'm afraid not - all you could do is check for some function (drawCircle?) that doesn't exist with SAVE_ON_FLASH. These things were never really meant to be accessed from JS-land so there hasn't been any thought put into making it easy

[espruino-discuss2]

Posted at 2019-02-13 by @allObjects

...instead of grouped I should have said bundled... (adjacent lines in the source).

for reasons of having unconditional structure accessible as one, contiguous block, incl. cursor pos - (https://github.com/espruino/Espruino/blob/master/libs/graphics/graphics.h#L69, though may not be of interest.

[gfwilliams]

Posted at 2019-02-14 by @gfwilliams

The modified items aren't flags - they're actual X/Y coordinates of the modified area - so I'm not sure it makes sense to put them together? Or you mean to move cursorX/Y to earlier in the struct, so they are always in the same place?

I guess for what you're doing you're unlikely to have to look any further than bgColor though?

[espruino-discuss2]

Posted at 2019-02-14 by @allObjects

you mean to move cursorX/Y to earlier in the struct, so they are always in the same place?

exactly.

In other words: 'all' that is always there comes first, after that the optional 'things'.

And in deed, the modified extent and font alignment/rotation information do not have to be in teh same #ifndef...

Btw, I was surprised that this space/byte/variable saving business to fit onto devices w/ low flash is that fine granular... a lot of work... but it makes sense (not the work but the way it is done).

[gfwilliams]

Posted at 2019-02-18 by @gfwilliams

Ahh - then I'll try and get those positions modified.

The space saving may not be entirely required I guess but it's easy for things to get out of control. Also the Graphics lib uses a slightly odd way of storing that data (in a String, copied into a flat memory area for each gfx call) so the less data there is, the faster it'll run.