[VL] shuffle: TypeAwareCompress(tac) for column-wise data compression like (U)INT64

[guowangy]

What changes are proposed in this pull request?

Introduces TypeAwareCompress (TAC) — a column-wise compression layer for shuffle that selects
an algorithm based on each buffer's data type, applied per-buffer alongside the existing LZ4/ZSTD
codec path.

For INT64/UINT64 columns the values are often clustered in a small range, making
Frame-of-Reference + Bit-Packing (FFOR) significantly more effective than generic byte-level
compression. TAC exploits this by encoding 8-byte integer buffers with a 4-lane FFOR codec before
the standard codec sees them.

Here is the performance data on TPCH/TPCDS:

	Total Latency	Shuffle Write Size
TPCH-6T	-15%	-32%
TPCDS-6T	-6%	-14%

New files

Path	Purpose
cpp/core/utils/tac/ffor.hpp	Header-only 4-lane FFOR codec for uint64_t
cpp/core/utils/tac/FForCodec.{h,cc}	Arrow-Result wrapper around ffor.hpp
cpp/core/utils/tac/TypeAwareCompressCodec.{h,cc}	Type dispatch; self-describing wire format (codec ID + element width embedded in header, so decompression needs no type hint)
cpp/velox/shuffle/VeloxTypeAwareCompress.h	Maps Velox TypeKind → TacDataType (BIGINT → kUInt64)

Shuffle integration

• Payload.cc/h: BlockPayload::fromBuffers accepts an optional bufferTypes vector. Per-buffer:
  if TypeAwareCompressCodec::support(type) is true, use TAC; otherwise fall back to LZ4/ZSTD.
  A new wire marker kTypeAwareBuffer = -3 is added; decompression in readCompressedBuffer is
  self-describing. If TAC compressed size ≥ original, falls back to kUncompressedBuffer.
• Options.h: adds enableTypeAwareCompress (default false) to LocalPartitionWriterOptions.
• VeloxHashShuffleWriter: populates bufferTypes from the schema when TAC is enabled.
• GlutenConfig.scala: new config spark.gluten.sql.columnar.shuffle.typeAwareCompress.enabled (default false).
• ColumnarShuffleWriter / LocalPartitionWriterJniWrapper: forward the new option to native.

Disabled by default — no behaviour changes for existing deployments.

How was this patch tested?

cpp/core/tests/FForCodecTest.cc covers:

• Round-trip correctness for random, all-zero, monotonic, and near-max value patterns
• maxCompressedLength boundary checks
• Invalid input size rejection

cpp/velox/tests/VeloxShuffleWriterTest.cc: extended to exercise the TAC path end-to-end through
VeloxHashShuffleWriter.

Was this patch authored or co-authored using generative AI tooling?

Co-authored-by: Claude Sonnet 4.6

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[FelixYBW]

We have enabled this in Gazelle. @marin-ma do you still remember why we didn't introduce it in Gluten?

[github-actions]

Run Gluten Clickhouse CI on x86

[marin-ma]

@FelixYBW We used the compression and arrow ipc payload API in gazelle, and added the FastPFor compression for integer column types and it's also used as the default compression method for integers in gazelle https://github.com/fast-pack/FastPFOR

Because adding a new compression algorithm for the Arrow API requires extra patches, we removed it mainly due to maintenance concerns. Using the default LZ4 algorithm did not result in significant performance regression.

[guowangy]

@marin-ma We have also investigated FastPFor library, but it only supports int32 type. This version is aimed to optimize int64 data type, which can archive better compression ratio and speed than int32 types.