Skip to content

Commit 77988c7

Browse files
committed
Add incremtntal transfer
Closes: #16
1 parent 52e49e1 commit 77988c7

File tree

4 files changed

+573
-39
lines changed

4 files changed

+573
-39
lines changed

README.md

Lines changed: 32 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -15,9 +15,6 @@ Fastsync targets the following use case:
1515
* **Compression is handled externally.** Fastsync does not compress the stream.
1616
If the data volume benefits from compression, then compress the files ahead
1717
of time with e.g. `lz4`, `brotli`, or `zstd`.
18-
* **A full transfer is necessary.** Fastsync always sends all files. If some
19-
files are already present at the receiving side, or similar data is already
20-
present, `rsync` might be a better fit.
2118

2219
## Building
2320

@@ -49,6 +46,38 @@ On the receiving end, suppose we download with 32 TCP connections:
4946
cd /some/path
5047
fastsync recv 100.71.154.83:4440 32
5148

49+
File modification timestamps are preserved during all transfers.
50+
51+
## Incremental transfers
52+
53+
Fastsync supports incremental transfers with the `--incremental` flag. When enabled, fastsync will:
54+
55+
1. Compare files by name, size, and modification timestamp
56+
2. Skip files that already exist at the destination with matching size and timestamp
57+
3. Transfer only files that are missing or have different size/timestamp
58+
59+
Both sender and receiver must use the `--incremental` flag:
60+
61+
# Sender
62+
fastsync send 100.71.154.83:4440 --incremental file.tar.gz
63+
64+
# Receiver
65+
fastsync recv 100.71.154.83:4440 32 --incremental
66+
67+
## Testing
68+
69+
To run all tests:
70+
71+
cargo test
72+
73+
To run only unit tests:
74+
75+
cargo test --lib
76+
77+
To run integration tests:
78+
79+
cd tests && ./test_incremental.sh
80+
5281
## Known issues
5382

5483
* It's too spammy.

0 commit comments

Comments
 (0)