data: port German cross-dataset dedup with reproducible provenance

[AuraIis]

Scope

Focused port of scripts/data/dedup_de_fresh.py from fix/gate-regression-stop; no unrelated hardening-branch changes are included.

Preserved behaviour

• exact dedup on normalized SHA-1
• MinHash-LSH near-dedup using word shingles
• cross-dataset only: fresh documents are checked against the reference index but are not inserted back into it
• existing top-level retention fields remain available: fresh_seen, kept, dropped_exact, dropped_near, ref_docs, drop_pct

Reproducibility additions

• CLI-configurable min_chars, num_perm, threshold and shingle size
• ordered SHA-256 reference manifest with host/container basename matching
• cheap size validation by default and optional full --verify-ref-hashes
• manifest hash, thresholds, input paths/sizes and malformed-record count in the report
• declared datasketch data dependency
• unit/integration coverage for manifest drift and the retained report contract

The server-generated reference SHA-256 values can be added to the concrete run manifest after they finish; this PR defines and tests the format without inventing those values.