Skip to content

docs: state that proxies cover downloads only - #128

Merged
LeyckerS merged 1 commit into
LeyckerS:mainfrom
basisworks:docs/proxy-scope-112
Aug 2, 2026
Merged

docs: state that proxies cover downloads only#128
LeyckerS merged 1 commit into
LeyckerS:mainfrom
basisworks:docs/proxy-scope-112

Conversation

@basisworks

Copy link
Copy Markdown
Contributor

Closes #112.

Documentation only — no code touched, so the verification suite is unaffected.

What I checked before writing anything

The issue's claim holds, and it is a little broader than "the browser goes direct":

  • _PROXY_POOL.next() is called in exactly one place in the shipping code —
    moon_download.py, inside download_file, where it builds dl_session. Every
    other reference is load/close_all in moon_cli.py and moon_engine.py, or a
    monkeypatch in the tests.
  • moon_extract.py contains no proxy=, proxies= or proxy_auth argument at all.
    The fuckingfast curl_cffi session is constructed with impersonate, timeout and
    max_clients and nothing else.
  • dn_launch_kwargs() builds the Chrome launch args and never adds --proxy-server;
    there is no --proxy-server anywhere in the tree.

So both extraction paths connect directly, not just the datanodes browser. I wrote it
that way rather than repeating the issue's wording, since the fuckingfast side is the
one a reader is most likely to assume is covered — it makes no window, so it is easy to
picture as "just an HTTP request the pool would wrap."

What changed

File Change
README.md scope on the proxies.txt row, plus a short paragraph under the Optional files table
docs/CONFIGURATION.md scope on the row, plus a What proxies cover subsection with the mechanism
docs/CLI.md scope on the --proxies row, plus a paragraph after the warning list
docs/FAQ.md new Do proxies hide me from the file host? entry

The issue named the first three. I added docs/FAQ.md because "Are proxies required?"
sits directly above it and answers only the format question, which is the exact place a
reader forms the assumption the issue is about. Happy to drop that hunk if you would
rather keep the FAQ to its current shape.

Two things I deliberately kept in, because they are what turns a note into something a
user can act on:

  • the design reason — routing a real Chrome through a rotating pool makes the Turnstile
    challenge harder to pass, not easier, and the bandwidth is in the download. Stated so
    the scope reads as a decision rather than an oversight.
  • the symptom — pages open, downloads fail. That is the shape of the report in docs: nothing says proxies cover downloads only — the browser always goes direct #112, and
    a user who has read it once will diagnose an unusable proxy list in seconds instead of
    concluding the proxy system is broken.

No existing sentence changed meaning; the format lines and the warning list are exactly
as they were.

The pool is read in one place, download_file in moon_download.py, to build
the download session. Nothing in moon_extract.py takes a proxy: the datanodes
Chrome instance is launched with no --proxy-server and the fuckingfast
curl_cffi session is constructed without one, so both connect directly.

Every place proxies.txt was documented described the file format or the
load-time warnings, never the scope. Someone configuring a proxy list had no
way to learn that the half of the run which identifies them to the host is
not covered.

README.md, docs/CONFIGURATION.md and docs/CLI.md now state the scope where
the file is documented. docs/FAQ.md gets it too, under the question a user
actually asks, since 'Are proxies required?' was the one place that invited
the wrong assumption directly.

Closes LeyckerS#112

@LeyckerS LeyckerS left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merging. This is the standard for a documentation PR here.

You checked, and I checked your checking. Every factual claim holds:

  • _PROXY_POOL.next() appears at exactly one call site outside the tests — moon_download.py:469.
  • grep -n "proxy-server\|proxy=" moon_extract.py returns nothing. The datanodes Chrome really is launched without one.
  • _CurlSession(impersonate=FF_IMPERSONATE, timeout=30, max_clients=64) — no proxy argument, so the fuckingfast session is direct too.

The second and third are yours, not the issue's. #112 said "the browser goes direct"; you established that the curl_cffi path does as well, which makes it true of both providers rather than just the visible one. That is the kind of widening that makes a docs change worth more than the issue asked for.

The framing is what I would have struggled to write. This line, in particular:

Rotation is bandwidth cover, not identity cover

That is the whole issue in six words, and it is the sentence that stops the next person filing this as a bug. Likewise putting the symptom in all four documents — pages open, downloads fail, that is the design working — because that is the exact observation that produced #112 in the first place.

Four files, consistent wording, no code touched, and you said so rather than leaving me to check. Closes #112.

One thing for whoever gets to it next, not for you: docs/FAQ.md still says "The status bar shows how many were loaded" a few lines above your addition. That is only true after a run has started — before then the chip reads "no proxy" regardless. It is #115, and your section makes the surrounding text slightly more misleading than it was, through no fault of yours.

Thanks, and welcome.

@LeyckerS
LeyckerS merged commit d34a3fb into LeyckerS:main Aug 2, 2026
1 check passed
LeyckerS added a commit that referenced this pull request Aug 2, 2026
Fourteenth outside contributor.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs: nothing says proxies cover downloads only — the browser always goes direct

2 participants