Adding --no-content-size to the cli

[bimbashrestha]

#1926

Ran

#!/bin/bash
while true
do
echo "line" >> /tmp/log.txt
done

and

./zstd --compress --no-content-size --stdout /tmp/log.txt > /tmp/log.txt.zst

[terrelln]

Please add a test to playTests.sh. You can use zstd -l or zstd -lv to check if the frame content size is written into the frame.

[Cyan4973]

Consider adding a line about this new command in programs/zstd.1.md.

[terrelln]

QQ about the use case

When using --no-content-size, but the input file size is known, do we still want to adjust our compression parameters for the input file size? Should we mention that behavior in the man page?

[Cyan4973]

I think the only explicit use case is about compressing a file which is being appended to at the same time, thus changing its size during the compression process.

Setting the size to UNKNOWN disable any parameter adaptation, and make the behavior similar to what providing the input through a pipe would have done.

In contrast, just disabling the content size field, using ZSTD_CCtx_setParameter(cctx, ZSTD_c_contentSizeFlag, 0); would have given a chance to adjust the compression parameters to the first detected size of the file. This could make a small difference for smaller files.

My own preference would be for the second case, though I have no "strong" reason for that, just some "gut feeling", essentially because it feels closer to the expressed command : "no-content-size", rather than "no-content-size-and-assume-large-file". But anyway, I suspect the difference between these 2 options is pretty small and won't matter much.

[terrelln]

I'm in favor of keeping parameter adjustment as well.

[bimbashrestha]

Ah okay then! Haden't thought about changing parameters honestly but that makes sense to me. I'll change it to unset ZSTD_c_contentSizeFlag instead of setting filesize to unknown.

[terrelln]

LGTM once Travis is passing

[bimbashrestha]

I thought I noticed Travis failing but it's okay now. Guess it was flakey:/

[terrelln]

I thought I noticed Travis failing but it's okay now. Guess it was flakey:/

Yeah, I re-ran it and it passed. Seemed to be some infra flakiness, because it was a super weird issue.