we use this pattern when writing to parquet in our our different pipeline stages:
future: dd.Scalar = ddf.to_parquet(
Path(outfile),
version='2.6',
allow_truncated_timestamps=True,
write_index=False,
compute=False,
)
Do we need to be using version with a static version number? Do we need truncated timestamps?
we use this pattern when writing to parquet in our our different pipeline stages:
Do we need to be using
versionwith a static version number? Do we need truncated timestamps?