diff --git a/README.md b/README.md index 8851b61..008fe5c 100644 --- a/README.md +++ b/README.md @@ -319,7 +319,13 @@ Place next to the scripts: | File | Purpose | |:--|:--| -| `proxies.txt` | Proxy list — `ip:port:user:pass` or `http://user:pass@ip:port` | +| `proxies.txt` | Proxy list — `ip:port:user:pass` or `http://user:pass@ip:port`. **Downloads only** — see below | + +Proxies wrap the **download** session and nothing else. Link extraction always connects +directly, from your own address: the shared Chrome instance datanodes needs for its +Turnstile challenge, and the `curl_cffi` session fuckingfast uses, are never routed +through the pool. An unreachable proxy list therefore stalls downloads while pages keep +opening normally — that is the design, not a broken proxy. --- diff --git a/docs/CLI.md b/docs/CLI.md index 8ba0efd..b2d30bd 100644 --- a/docs/CLI.md +++ b/docs/CLI.md @@ -20,7 +20,7 @@ are ignored. | `--browsers` | integer | `8` | Number of parallel extraction workers. Despite the name, it does not start that many browsers: datanodes uses one shared Chrome instance. | | `--streams` | integer | `24` | Maximum number of concurrent download streams. | | `--retries` | integer | `3` | Maximum extraction attempts per URL. Network retries inside one download are separate. | -| `--proxies` | path | `proxies.txt` | Proxy-list file to load. A missing file, or one that yields no usable proxies, prints a warning — except the implicit default (`proxies.txt` when `--proxies` is omitted), whose absence stays silent as the normal no-proxy state. | +| `--proxies` | path | `proxies.txt` | Proxy-list file to load. Applies to **downloads only** — extraction always goes direct, see below. A missing file, or one that yields no usable proxies, prints a warning — except the implicit default (`proxies.txt` when `--proxies` is omitted), whose absence stays silent as the normal no-proxy state. | | `--version` | flag | — | Print the version and exit. Works even without `--urls`/`--output`. | `--urls` and `--output` are required. The parser accepts integer values for @@ -37,6 +37,16 @@ missing or empty": - the implicit default path missing: no warning - whenever any proxies load or lines get skipped: `[proxies] {n} loaded, {s} skipped` +`--proxies` covers the download half of a run and no more. The pool is read once, in +`download_file`, to build the download session; the extraction layer never receives one, +so the datanodes browser and the fuckingfast `curl_cffi` session both connect from your +own address on every run. `[proxies] 20 loaded` means the bytes are covered — it does not +mean the page loads were. + +If a run shows pages opening while every download fails, check the proxies before +suspecting the extractor: that split is exactly what an unusable proxy list looks like +here. + ## Examples ### fuckingfast.co batch diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index 57f1409..edffb2f 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -133,9 +133,25 @@ Place them next to the scripts: | File | Purpose | |:--|:--| -| `proxies.txt` | proxy list — `ip:port:user:pass` or `http://user:pass@ip:port` | +| `proxies.txt` | proxy list — `ip:port:user:pass` or `http://user:pass@ip:port`. **downloads only**, see below | | `settings.json` | written by the GUI: settings, pasted links, language | +### What proxies cover + +The pool is consulted in exactly one place — `download_file` in `moon_download.py`, where +`_PROXY_POOL.next()` picks the entry the download session is built from. Nothing in +`moon_extract.py` takes a proxy: the Chrome instance datanodes needs for its Turnstile +challenge is launched with no `--proxy-server`, and the `curl_cffi` session fuckingfast +uses is constructed without one. Both connect directly. + +So a run with `proxies.txt` loaded has the file host seeing your own address for every +page load and your proxy's for the bytes. Rotation is bandwidth cover, not identity +cover — if you need the extraction half proxied too, that is not something this file can +do for you today. + +The practical symptom: put unreachable proxies in the list and pages still open, while +downloads fail. That is the design working, not the proxy system failing. + ## Output files | File | Contents | diff --git a/docs/FAQ.md b/docs/FAQ.md index a93ce3a..3a58ad4 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -67,6 +67,17 @@ http://user:pass@ip:port The status bar shows how many were loaded. +## Do proxies hide me from the file host? + +Only for the download. The pool wraps the download session and nothing else, so link +extraction — the datanodes Chrome window that answers the Turnstile challenge, and the +fuckingfast `curl_cffi` session — connects directly from your own address every time. + +That is deliberate: pushing a real Chrome through a rotating pool makes the challenge +harder to pass, not easier, and the bandwidth that proxies are actually there for is all +in the download. But it means `PROXY 20` in the footer covers half the run, and the half +it does not cover is the half that identifies you to the host. + ## Can I skip the captcha entirely? With a datanodes **premium API key**, yes — extraction becomes one JSON GET, no browser,