bytes lost during SSH --> UART high volume data transfer

[jubeormk1]

Problem

On transferring data at high speed from the SSH client to the serial port, I have observed that there is data missing. I was expecting that the TCP and SSH stacks would implement back pressure to control the flow of data to the maximum baud rate of the serial communication. Instead we are losing some chars somehow. I have tested it wouth test_sshstamp and the main at 3ed66bb.

Setup

1. Prepare a long txt. Use a book or generate a random file.
2. download, and run SSH-Stamp

git clone -b traits_refactor https://github.com/brainstorm/ssh-stamp.git test_sshstamp
cd test_sshstamp && cargo run-esp32c6 -- p ~/devs/prog_esp32c6

3. Connect to the indicated WiFi access point with the PSK, Setup keys, etc

4. Configure the serial port with the default SSH-Stamp baud rate and start logging the data from the serial port:

stty -F ~/devs/com_esp32c6 115200 -echo raw && cat ~/devs/com_esp32c6 > ~/out/uart_read_115200.txt

4. Send the text file file via ssh:

ssh ssh-stamp-key < ~/resources/Frankenstein.rcnl.txt

5. On completion, compare the original and received file:

ls -l ~/out/uart_read_115200.txt ~/resources/Frankenstein.rcnl.txt
diff ~/out/uart_read_115200.txt ~/resources/Frankenstein.rcnl.txt
# etc

Additional tests @115200bps

I have tried feeding the reference text at different rates (lines per second). As an example:

Broken pipe
$ cat ~/resources/Frankenstein.rcnl.txt | ./feed_at_hz.sh 50 | tee /dev/tty | ssh ssh-stamp-key

and got the next results:

• Flooded out produces significant and variable data lost (uart_read_115200*.txt)
• Sending lines at 30 Hz and under produces correct transmissions (diff shows no difference in the files)
• Sending lines at 40 Hz and over produces characters lost in growing proportion

Data

Size of the files:

444K May 17 12:02 ~/out/uart_read_1000hz.txt
491K May 17 12:01 ~/out/uart_read_100hz.txt
492K May 17 11:58 ~/out/uart_read_10hz.txt
424K May 17 08:21 ~/out/uart_read_115200.1.txt
423K May 17 08:22 ~/out/uart_read_115200.2.txt
421K May 17 08:24 ~/out/uart_read_115200.3.txt
422K May 17 08:26 ~/out/uart_read_115200.4.txt
424K May 17 10:40 ~/out/uart_read_115200.5.prev_version_3ed66bb093.txt
424K May 17 10:28 ~/out/uart_read_115200.txt
488K May 17 12:05 ~/out/uart_read_200hz.txt
492K May 17 12:16 ~/out/uart_read_20hz.txt
492K May 24 09:40 ~/out/uart_read_30hz.txt
492K May 24 09:33 ~/out/uart_read_40hz.txt
446K May 17 12:04 ~/out/uart_read_500hz.txt
492K May 24 09:25 ~/out/uart_read_50hz.txt
492K Mar 24 02:01 ~/resources/Frankenstein.rcnl.txt

Script feed_at_hz.sh:

#!/usr/bin/env bash
# Usage: cat data.txt | ./feed_at_hz.sh <frequency_hz>

FREQ="${1:?Usage: $0 <frequency_hz>}"
INTERVAL=$(awk "BEGIN { printf \"%.6f\", 1/$FREQ }")

while IFS= read -r line; do
    printf '%s\n' "$line"
    sleep "$INTERVAL"
done