fs.writeFileSync() not working with serial link

[espruino-discuss3]

Posted at 2017-10-02 by miCatotonic

Thanks to Wilberforce I was able to write a constant string to flash memory. While a constant string is nice, my long term goal is to serve a single page app from flash or a SD card with a file size of 1, 2MB or more. My next baby step was to write a single file of ~800 bytes to flash. Unfortunately after dropping my baud all the way down to 9600 to send a file in binary form to my ESP32, I still lose arbitrary blocks of data. It might be 1 to 20 bytes, but it means I am a long way from sending 200-500 KB file to the my ESP32 flash or SD card. Here's my code(mangled from over a week of hacking) and sample file. The need to write data from a serial link is not only for saving server files, but also for being able to save production data from other devices.

var formatFlashDrive = false;
var writeFile = true;

Serial2.setup(9600, { tx: D17, rx: D16 });
var fileWriteStarted = false;
var fileName = 'index.html';
var fs = require("fs");
var intervalID, existingBlinkRate = 0, timerCount = 0;
Serial2.print("Are you there?");

if(formatFlashDrive){
  console.log('Formatting FS - only need to do once');   
  try {
    console.log("Flash Files: ",fs.readdirSync());
    } catch (e) { 
      console.log('Uncaught Error: Unable to mount media : NO_FILESYSTEM'.e);
       E.flashFatFS({addr : int=0x300000, sectors : int=256, format : bool=false }); 
  }
}
if(!writeFile){console.log("Ready to write to fs");}
if(!writeFile){
  console.log("Reading index.html from flash");
  try{
    console.log(fs.readFileSync("index.html")); // prints "Hello World!!!"
    Serial2.write(fs.readFileSync("index.html"));
  }catch(e){
    console.log("File Read Error = ",e); 
    Serial2.write('File Read Error');
  }
  //flashLED(10);
}
Serial2.on('data',function(data){
  if(!writeFile)return;
  if(!fileWriteStarted){ 
    setTimeout(function(){
      console.log('Reading ',fileName);
      Serial2.write(fs.readFileSync("index.html"));
    },2000);
    fileWriteStarted = true;
    console.log('Writing data');
    fs.writeFileSync(fileName,data);
    return;
  }
  fs.appendFileSync(fileName, data);
  console.log('Appending data');
});

---

Attachments:

• ws.html

[espruino-discuss3]

Posted at 2017-10-03 by Wilberforce

Rather than using serial, you can use http to update the flash.

You can do an http request and then write the output to fs, you could host the file on your local computer, it out the file in github or gist.

---

Posted at 2017-10-04 by miCatotonic

@wilberforce
Thanks for your input. While this might solve my short term problem, it will not solve my long term goal. In my production environment I will need to save, serve and update dozens of non static (10-100KB) files on top of several large static files totaling 1.5-3MB for my SPA . The non static files will need to come in serially, being that one of my connections to the environment will be RS485. I mention RS485 because of the limitation of flow control, if I don't have a way of taking data at at least 19200 baud, hopefully higher, I will need to rethink the Espruino implementation idea.

---

Posted at 2017-10-04 by @gfwilliams

When you do Serial.setup you should be able to specify a cts pin. I'm not 100% sure it works on ESP32, but it should: http://www.espruino.com/Reference#l_Serial_setup

Otherwise, it might make sense to set up some kind of serial protocol with a command and acknowledgement?

---

Posted at 2017-10-04 by miCatotonic

@gfwilliams
Thanks for responding.
I do not know of a way to implement a cts function with RS485, which is a two wire bi-polar uni-directional interface. I can only guess about interrupts, buffer sizes, buffer overwrites,being a problem. But being that with ESP-32 is new to Espruino and not many others are experiencing a similar problem, I will have to wait or find and pay someone knowledgeable about the ESP-32 and Espruino to fix this.

---

Posted at 2017-10-05 by @gfwilliams

Ahh - with RS485 you'd have trouble with CTS flow control. I'd suggest implementing a command & response sort of thing then... What you're doing won't work that well on any Espruino device: you get data from the serial port a few bytes at a time, and each time you're opening and closing a file... it's just taking too much time.

If you don't want to do some command/response thing you could use E.openFile to open the file and keep it open, which would help - and could also write in blocks of 512 bytes - which would match the block size in FAT

---

Posted at 2017-10-05 by Wilberforce

It's the esp32 port so the blocksize is 4096.

---

Posted at 2017-10-05 by miCatotonic

@gfwilliams and @wilberforce thank you for your help.
My plan for large data captures was to implement request/response protocol, but as initially stated, I cannot capture <800 bytes at 9600 baud reliably. My existing infrastructure requires 19200 baud and I would like to go higher.

---

Posted at 2017-10-05 by miCatotonic

@wilberforce
Have you checked your github account lately.