haveibeenpwned-downloader (Java)

A Java port of haveibeenpwned-downloader, the .NET CLI tool that downloads every Pwned Passwords hash range from Have I Been Pwned so the data can be used offline, without depending on the k-anonymity API.

This is a feature-complete equivalent of the .NET tool: same arguments, same defaults, same output formats, same retry behaviour, same statistics, same exit codes.

Requirements

• JDK 21 or newer (uses virtual threads)
• Maven, to build

There are no runtime dependencies. Everything is built on the JDK: java.net.http.HttpClient for transport, a hand-written argument parser, and a hand-written ANSI progress renderer.

Build

mvn package

That produces target/haveibeenpwned-downloader.jar, an executable jar:

java -jar target/haveibeenpwned-downloader.jar --help

To run it as a plain command, drop this alias somewhere on your PATH:

alias haveibeenpwned-downloader='java -jar /path/to/haveibeenpwned-downloader.jar'

Usage examples

Download all SHA-1 hashes to a single file called pwnedpasswords.txt:

java -jar haveibeenpwned-downloader.jar pwnedpasswords

Download all SHA-1 hashes to individual files in a directory called hashes:

java -jar haveibeenpwned-downloader.jar hashes -s false

Download all NTLM hashes to a single file called pwnedpasswords_ntlm.txt:

java -jar haveibeenpwned-downloader.jar -n pwnedpasswords_ntlm

Use 64 parallel requests and overwrite an existing file:

java -jar haveibeenpwned-downloader.jar pwnedpasswords -o -p 64

Give up on a prefix after five retries:

java -jar haveibeenpwned-downloader.jar pwnedpasswords --max-retries 5

Parameters

Parameter	Default	Description
[outputFile]	pwnedpasswords	Output name. Becomes <name>.txt for single file output, or a directory called <name>
-s/--single	true	Single .txt file, or one .txt file per prefix in a directory
-p/--parallelism	8 × processor count	Concurrent requests. Values below 2 fall back to the default
-o/--overwrite	false	Overwrite existing output
-n/--ntlm	false	Fetch NTLM hashes instead of SHA-1
--max-retries	unlimited	Retries per prefix. Omit for unlimited, 0 to disable. Delay grows 2s per attempt, capped at 10s
-h/--help		Print help
--version		Print version

Flags accept an explicit boolean, so both -s and -s false work, as do --single=false and the bundled form -on.

Exit codes

Code	Meaning
0	Success
-1	Download failed
-2	Cancelled with Ctrl+C
-99	Bad arguments, or an unhandled startup failure

On POSIX shells these surface as 0, 255, 254 and 157 respectively, exactly as they do for the .NET tool.

How it maps onto the .NET tool

.NET	Java
Spectre.Console.Cli CommandApp	ArgumentParser + HelpText
PwnedPasswordsDownloader.Settings	Settings
AddHttpClient("PwnedPasswords")	PwnedPasswordsClient
Statistics	Statistics
ExecuteWithRetriesAsync	Retry
Channel<Task<DownloadedRange>> (bounded)	ArrayBlockingQueue<Future<DownloadedRange>> + producer thread
Parallel.ForEachAsync	virtual-thread executor bounded by a Semaphore
AnsiConsole.Progress()	ProgressDisplay + ConsoleOutput
CancellationTokenSource / Console.CancelKeyPress	CancellationToken / SignalHandling
GetHashRange	HashRanges.prefix

Concurrency

Single file mode downloads up to parallelism ranges at once but appends them strictly in prefix order, which is what the .NET bounded channel of tasks achieves. Directory mode downloads each prefix straight to its own file with at most parallelism requests in flight.

.NET sets EnableMultipleHttp2Connections. The JDK's HttpClient multiplexes every request for an origin over a single HTTP/2 connection, so this port uses a small round-robin pool of clients (one per 64 requested parallel requests, capped at 8) to reach comparable concurrency.

Cancellation

Ctrl+C is intercepted via sun.misc.Signal so the process can shut down on its own terms and return -2, mirroring args.Cancel = true in the .NET tool. If that API is unavailable the port falls back to a shutdown hook, which still cancels the download but cannot control the exit code.

Deliberate deviations

Two places where this port does not match the .NET tool byte for byte. Both are called out here rather than hidden:

1. No UTF-8 BOM between ranges. In single file mode the .NET tool builds each range with new StreamWriter(memoryStream, Encoding.UTF8, leaveOpen: true). Encoding.UTF8 carries a byte-order-mark preamble, and StreamWriter writes that preamble on first write to a stream positioned at zero — which is the case for every fresh per-range buffer. That would put a BOM (EF BB BF) in front of all 1,048,576 range sections in the output file. This port writes plain UTF-8 with no BOM anywhere. I could not verify the .NET behaviour empirically here (no .NET SDK on the build machine), so treat this as the one item worth checking if exact byte parity matters to you.

2. No summary line when the run is skipped. If the output already exists and -o was not passed, the .NET tool prints the "already exists" message and then still prints Finished downloading all hash ranges in 0ms (NaN hashes per second) along with a Cloudflare summary of zeroes, because the early return happens inside the progress callback rather than out of the command. This port prints the "already exists" message and the summary in the same order, so the observable output matches. If you would rather suppress the trailing summary, return early from Downloader.execute when prepareOutput returns false.

3. Error messages include the cause chain. The .NET tool prints {exception.GetType().Name}: {exception.Message}. Doing exactly that in Java produces a useless IOException: closed for every mid-transfer failure, because the JDK's response body stream reports all of them with that one message and puts the real reason in the cause. Retry and failure messages therefore render the chain, e.g. IOException: closed <- IOException: fixed content-length: 100000, bytes received: 39 <- EOFException: EOF reached while reading. Capped at five levels.

Beyond those: line endings use the platform separator (System.lineSeparator()), matching .NET's StreamWriter.WriteLine, and the progress bar is a close visual approximation of the Spectre.Console column set rather than a pixel-identical copy. When stdout is not a terminal the live progress line is suppressed and only the summary lines are printed.

Tests

mvn test

The suite covers prefix generation, argument parsing, the retry schedule, statistics, cancellation, and end-to-end downloads in both output modes against an in-process stand-in for the range API (FakeRangeServer), including gzip responses, NTLM mode, overwrite guards, Cloudflare cache accounting, retry-then-succeed, and permanent failure handling.