Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -319,7 +319,13 @@ Place next to the scripts:

| File | Purpose |
|:--|:--|
| `proxies.txt` | Proxy list — `ip:port:user:pass` or `http://user:pass@ip:port` |
| `proxies.txt` | Proxy list — `ip:port:user:pass` or `http://user:pass@ip:port`. **Downloads only** — see below |

Proxies wrap the **download** session and nothing else. Link extraction always connects
directly, from your own address: the shared Chrome instance datanodes needs for its
Turnstile challenge, and the `curl_cffi` session fuckingfast uses, are never routed
through the pool. An unreachable proxy list therefore stalls downloads while pages keep
opening normally — that is the design, not a broken proxy.

---

Expand Down
12 changes: 11 additions & 1 deletion docs/CLI.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ are ignored.
| `--browsers` | integer | `8` | Number of parallel extraction workers. Despite the name, it does not start that many browsers: datanodes uses one shared Chrome instance. |
| `--streams` | integer | `24` | Maximum number of concurrent download streams. |
| `--retries` | integer | `3` | Maximum extraction attempts per URL. Network retries inside one download are separate. |
| `--proxies` | path | `proxies.txt` | Proxy-list file to load. A missing file, or one that yields no usable proxies, prints a warning — except the implicit default (`proxies.txt` when `--proxies` is omitted), whose absence stays silent as the normal no-proxy state. |
| `--proxies` | path | `proxies.txt` | Proxy-list file to load. Applies to **downloads only** — extraction always goes direct, see below. A missing file, or one that yields no usable proxies, prints a warning — except the implicit default (`proxies.txt` when `--proxies` is omitted), whose absence stays silent as the normal no-proxy state. |
| `--version` | flag | — | Print the version and exit. Works even without `--urls`/`--output`. |

`--urls` and `--output` are required. The parser accepts integer values for
Expand All @@ -37,6 +37,16 @@ missing or empty":
- the implicit default path missing: no warning
- whenever any proxies load or lines get skipped: `[proxies] {n} loaded, {s} skipped`

`--proxies` covers the download half of a run and no more. The pool is read once, in
`download_file`, to build the download session; the extraction layer never receives one,
so the datanodes browser and the fuckingfast `curl_cffi` session both connect from your
own address on every run. `[proxies] 20 loaded` means the bytes are covered — it does not
mean the page loads were.

If a run shows pages opening while every download fails, check the proxies before
suspecting the extractor: that split is exactly what an unusable proxy list looks like
here.

## Examples

### fuckingfast.co batch
Expand Down
18 changes: 17 additions & 1 deletion docs/CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,9 +133,25 @@ Place them next to the scripts:

| File | Purpose |
|:--|:--|
| `proxies.txt` | proxy list — `ip:port:user:pass` or `http://user:pass@ip:port` |
| `proxies.txt` | proxy list — `ip:port:user:pass` or `http://user:pass@ip:port`. **downloads only**, see below |
| `settings.json` | written by the GUI: settings, pasted links, language |

### What proxies cover

The pool is consulted in exactly one place — `download_file` in `moon_download.py`, where
`_PROXY_POOL.next()` picks the entry the download session is built from. Nothing in
`moon_extract.py` takes a proxy: the Chrome instance datanodes needs for its Turnstile
challenge is launched with no `--proxy-server`, and the `curl_cffi` session fuckingfast
uses is constructed without one. Both connect directly.

So a run with `proxies.txt` loaded has the file host seeing your own address for every
page load and your proxy's for the bytes. Rotation is bandwidth cover, not identity
cover — if you need the extraction half proxied too, that is not something this file can
do for you today.

The practical symptom: put unreachable proxies in the list and pages still open, while
downloads fail. That is the design working, not the proxy system failing.

## Output files

| File | Contents |
Expand Down
11 changes: 11 additions & 0 deletions docs/FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,17 @@ http://user:pass@ip:port

The status bar shows how many were loaded.

## Do proxies hide me from the file host?

Only for the download. The pool wraps the download session and nothing else, so link
extraction — the datanodes Chrome window that answers the Turnstile challenge, and the
fuckingfast `curl_cffi` session — connects directly from your own address every time.

That is deliberate: pushing a real Chrome through a rotating pool makes the challenge
harder to pass, not easier, and the bandwidth that proxies are actually there for is all
in the download. But it means `PROXY 20` in the footer covers half the run, and the half
it does not cover is the half that identifies you to the host.

## Can I skip the captcha entirely?

With a datanodes **premium API key**, yes — extraction becomes one JSON GET, no browser,
Expand Down