Skip to content

Commit 80ea11f

Browse files
committed
docs: add chinese and portuguese full documentation
1 parent 7ed7abc commit 80ea11f

File tree

160 files changed

+55154
-7465
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

160 files changed

+55154
-7465
lines changed

docs/en/deep-dive/architecture/browser-domain.md

Lines changed: 102 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -10,23 +10,17 @@ The Browser domain represents the highest level of Pydoll's automation hierarchy
1010
The Browser domain sits at the intersection of process management, protocol communication, and resource coordination. It orchestrates multiple specialized components to provide a unified interface for browser automation:
1111

1212
```mermaid
13-
graph TB
14-
subgraph "Browser Domain"
15-
Browser[Browser Instance]
16-
Browser --> ConnectionHandler[Connection Handler]
17-
Browser --> ProcessManager[Process Manager]
18-
Browser --> ProxyManager[Proxy Manager]
19-
Browser --> TempDirManager[Temp Directory Manager]
20-
Browser --> TabRegistry[Tab Registry]
21-
Browser --> ContextAuth[Context Proxy Auth]
22-
end
13+
graph LR
14+
Browser[Browser Instance]
15+
Browser --> ProcessManager[Process Manager]
16+
Browser --> ProxyManager[Proxy Manager]
17+
Browser --> TempDirManager[Temp Directory Manager]
18+
Browser --> TabRegistry[Tab Registry]
19+
Browser --> ConnectionHandler[Connection Handler]
2320
24-
ConnectionHandler <--> |WebSocket| CDP[Chrome DevTools Protocol]
2521
ProcessManager --> |Manages| BrowserProcess[Browser Process]
26-
TabRegistry --> Tab1[Tab Instance 1]
27-
TabRegistry --> Tab2[Tab Instance 2]
28-
TabRegistry --> Tab3[Tab Instance N]
29-
22+
ConnectionHandler <--> |WebSocket| CDP[Chrome DevTools Protocol]
23+
TabRegistry --> |Manages| Tabs[Tab Instances]
3024
CDP <--> BrowserProcess
3125
```
3226

@@ -143,43 +137,96 @@ class Browser:
143137
- **Memory efficiency**: Prevents duplicate Tab instances for same target
144138
- **Event routing**: Ensures events route to correct Tab instance
145139

146-
### Proxy and Context Authentication
140+
### Proxy Authentication Architecture
141+
142+
Pydoll implements **automatic proxy authentication** via the Fetch domain to avoid exposing credentials in CDP commands. The implementation uses **two distinct mechanisms** depending on proxy scope:
147143

148-
Pydoll implements **automatic proxy authentication** via the Fetch domain to avoid exposing credentials in CDP commands:
144+
#### Mechanism 1: Browser-Level Proxy Auth (Global Proxy)
145+
146+
When a proxy is configured via `ChromiumOptions` (applies to all tabs in the default context):
149147

150148
```python
151-
class Browser:
152-
def __init__(self, ...):
153-
# Stores proxy credentials per context
154-
self._context_proxy_auth: dict[str, tuple[str, str]] = {}
149+
# In Browser.start() -> _configure_proxy()
150+
async def _configure_proxy(self, private_proxy, proxy_credentials):
151+
# Enable Fetch AT BROWSER LEVEL
152+
await self.enable_fetch_events(handle_auth_requests=True)
155153

156-
async def create_browser_context(self, proxy_server, ...):
157-
# Extract credentials from proxy URL
158-
sanitized_proxy, (user, pwd) = self._sanitize_proxy_and_extract_auth(proxy_server)
159-
160-
# Send sanitized proxy to CDP (no credentials exposed)
161-
response = await self._execute_command(
162-
TargetCommands.create_browser_context(proxy_server=sanitized_proxy)
163-
)
164-
context_id = response['result']['browserContextId']
165-
166-
# Store credentials for later authentication
167-
if user and pwd:
168-
self._context_proxy_auth[context_id] = (user, pwd)
169-
170-
return context_id
154+
# Register callbacks AT BROWSER LEVEL (affects ALL tabs)
155+
await self.on(FetchEvent.REQUEST_PAUSED, self._continue_request_callback, temporary=True)
156+
await self.on(FetchEvent.AUTH_REQUIRED,
157+
partial(self._continue_request_with_auth_callback,
158+
proxy_username=credentials[0],
159+
proxy_password=credentials[1]),
160+
temporary=True)
161+
```
162+
163+
**Scope:** Browser-wide WebSocket connection → affects **all tabs in default context**
164+
165+
#### Mechanism 2: Tab-Level Proxy Auth (Per-Context Proxy)
166+
167+
When a proxy is configured per-context via `create_browser_context(proxy_server=...)`:
168+
169+
```python
170+
# Store credentials per context
171+
async def create_browser_context(self, proxy_server, ...):
172+
sanitized_proxy, extracted_auth = self._sanitize_proxy_and_extract_auth(proxy_server)
173+
174+
response = await self._execute_command(
175+
TargetCommands.create_browser_context(proxy_server=sanitized_proxy)
176+
)
177+
context_id = response['result']['browserContextId']
178+
179+
if extracted_auth:
180+
self._context_proxy_auth[context_id] = extracted_auth # Store per context
181+
182+
return context_id
183+
184+
# Setup auth for EACH tab in that context
185+
async def _setup_context_proxy_auth_for_tab(self, tab, browser_context_id):
186+
creds = self._context_proxy_auth.get(browser_context_id)
187+
if not creds:
188+
return
189+
190+
# Enable Fetch ON THE TAB (tab-level WebSocket)
191+
await tab.enable_fetch_events(handle_auth=True)
192+
193+
# Register callbacks ON THE TAB (affects only this tab)
194+
await tab.on(FetchEvent.REQUEST_PAUSED,
195+
partial(self._tab_continue_request_callback, tab=tab),
196+
temporary=True)
197+
await tab.on(FetchEvent.AUTH_REQUIRED,
198+
partial(self._tab_continue_request_with_auth_callback,
199+
tab=tab,
200+
proxy_username=creds[0],
201+
proxy_password=creds[1]),
202+
temporary=True)
171203
```
172204

173-
When a Tab is created in a context with stored credentials, Pydoll automatically:
205+
**Scope:** Tab-level WebSocket connection → affects **only that specific tab**
206+
207+
#### Why Two Mechanisms?
208+
209+
| Aspect | Browser-Level | Tab-Level |
210+
|--------|---------------|-----------|
211+
| **Trigger** | Proxy in `ChromiumOptions` | Proxy in `create_browser_context()` |
212+
| **WebSocket** | Browser-level connection | Tab-level connection |
213+
| **Scope** | All tabs in default context | Only tabs in that context |
214+
| **Efficiency** | One listener for all tabs | One listener per tab |
215+
| **Isolation** | No context separation | Each context has different credentials |
216+
217+
**Design rationale for tab-level auth:**
174218

175-
1. Enables Fetch domain on the Tab
176-
2. Registers `Fetch.requestPaused` handler (continues normal requests)
177-
3. Registers `Fetch.authRequired` handler (provides credentials, then disables Fetch)
219+
- **Context isolation**: Each context can have a **different proxy** with **different credentials**
220+
- **CDP limitation**: Fetch domain cannot be scoped to a specific context at browser level
221+
- **Tradeoff**: Slightly less efficient (one listener per tab), but necessary for per-context proxy support
178222

179223
This architecture ensures **credentials never appear in CDP logs** and authentication is handled transparently.
180224

181225
!!! warning "Fetch Domain Side Effects"
182-
Enabling Fetch for proxy auth temporarily pauses **all requests** in that Tab until the auth callback fires. This is a CDP limitation - Fetch enables global request interception. After authentication completes, Fetch is disabled to minimize overhead.
226+
- **Browser-level Fetch**: Temporarily pauses **all requests across all tabs** in the default context until auth completes
227+
- **Tab-level Fetch**: Temporarily pauses **all requests in that specific tab** until auth completes
228+
229+
This is a CDP limitation - Fetch enables request interception. After authentication completes, Fetch is disabled to minimize overhead.
183230

184231
## Initialization and Lifecycle
185232

@@ -579,16 +626,27 @@ tab = await browser.new_tab()
579626

580627
**Alternative explored:** Auto-close initial tab in `new_tab()`. Rejected because it's surprising behavior (implicit side effects).
581628

582-
### Context-Specific Proxy Auth: Fetch Domain Overhead
629+
### Proxy Authentication: Two-Level Architecture Tradeoff
583630

584-
Pydoll's proxy authentication uses Fetch domain to hide credentials:
631+
Pydoll's proxy authentication uses two different Fetch domain strategies:
585632

633+
**Browser-Level (Global Proxy):**
586634
- **Security benefit**: Credentials never logged in CDP traces
587-
- **Performance cost**: Fetch pauses **all requests** until auth completes
635+
- **Performance cost**: Fetch pauses **all requests across all tabs** until auth completes
636+
- **Efficiency**: Single listener for all tabs in default context
588637
- **Mitigation**: Fetch is disabled after first auth, minimizing overhead
589638

639+
**Tab-Level (Per-Context Proxy):**
640+
- **Security benefit**: Credentials never logged in CDP traces
641+
- **Performance cost**: Fetch pauses **all requests in that tab** until auth completes
642+
- **Efficiency**: Separate listener per tab (less efficient, but necessary for isolation)
643+
- **Isolation benefit**: Each context can have different proxy credentials
644+
- **Mitigation**: Fetch is disabled after first auth per tab
645+
590646
**Why not use Browser.setProxyAuth?** This CDP command doesn't exist. Fetch is the only mechanism for programmatic auth.
591647

648+
**Why tab-level for contexts?** CDP's Fetch domain cannot be scoped to a specific BrowserContext. Since each context can have a different proxy with different credentials, Pydoll must handle auth at the tab level to respect context boundaries.
649+
592650
### Port Randomization Strategy
593651

594652
Random CDP ports (9223-9322) prevent collisions when running parallel browser instances:

docs/en/deep-dive/architecture/browser-requests-architecture.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -170,13 +170,13 @@ A common question: if JavaScript can execute the request, why use network events
170170

171171
| Information Source | JavaScript (Fetch API) | Network Events (CDP) |
172172
|-------------------|------------------------|----------------------|
173-
| Response status | Available | Available |
174-
| Response body | Available | Not available |
175-
| Response headers | ⚠️ Partial (CORS restricted) | Complete |
176-
| Request headers | Not accessible | Complete |
177-
| Set-Cookie headers | Hidden by browser | Available |
178-
| Timing information | ⚠️ Limited | Comprehensive |
179-
| Redirect chain | Only final URL | Full chain |
173+
| Response status | Available | Available |
174+
| Response body | Available | Not available |
175+
| Response headers | Partial (CORS restricted) | Complete |
176+
| Request headers | Not accessible | Complete |
177+
| Set-Cookie headers | Hidden by browser | Available |
178+
| Timing information | Limited | Comprehensive |
179+
| Redirect chain | Only final URL | Full chain |
180180

181181
**The Solution:** Combine both sources for complete information.
182182

@@ -322,11 +322,11 @@ Network events add overhead to request execution:
322322
### Optimization Pattern
323323
324324
```python
325-
# Inefficient - events enabled/disabled repeatedly
325+
# Inefficient - events enabled/disabled repeatedly
326326
for url in urls:
327327
response = await tab.request.get(url)
328328

329-
# Efficient - events enabled once
329+
# Efficient - events enabled once
330330
await tab.enable_network_events()
331331
for url in urls:
332332
response = await tab.request.get(url)

docs/en/deep-dive/architecture/event-architecture.md

Lines changed: 50 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -151,16 +151,23 @@ When an event arrives via WebSocket:
151151
Callbacks can be either synchronous or asynchronous. The event system handles both:
152152

153153
```python
154-
async def callback_wrapper(event):
155-
asyncio.create_task(callback(event))
156-
157-
if asyncio.iscoroutinefunction(callback):
158-
function_to_register = callback_wrapper
159-
else:
160-
function_to_register = callback
154+
async def _trigger_callbacks(self, event_name: str, event_data: dict):
155+
for cb_id, cb_data in self._event_callbacks.items():
156+
if cb_data['event'] == event_name:
157+
if asyncio.iscoroutinefunction(cb_data['callback']):
158+
await cb_data['callback'](event_data)
159+
else:
160+
cb_data['callback'](event_data)
161161
```
162162

163-
This ensures callbacks run in background tasks to prevent blocking the event loop.
163+
Asynchronous callbacks are awaited sequentially. This means each callback completes before the next one executes, which is important for:
164+
165+
- **Predictable Execution Order**: Callbacks execute in registration order
166+
- **Error Handling**: Exceptions in one callback don't prevent others from executing
167+
- **State Consistency**: Callbacks can rely on sequential state changes
168+
169+
!!! info "Sequential vs Concurrent Execution"
170+
Callbacks execute sequentially within the same event. However, different events can be processed concurrently since the event loop handles multiple connections simultaneously.
164171

165172
## Event Flow and Lifecycle
166173

@@ -294,35 +301,45 @@ The `ConnectionHandler` is the central component managing WebSocket communicatio
294301
```python
295302
class ConnectionHandler:
296303
def __init__(self, ...):
297-
self._callbacks = {} # Event name -> list of callbacks
298-
self._next_callback_id = 0
304+
self._events_handler = EventsManager()
299305
self._websocket = None
300306
# ... other attributes
301307

302308
async def register_callback(self, event_name, callback, temporary):
303-
callback_id = self._next_callback_id
304-
self._next_callback_id += 1
305-
306-
if event_name not in self._callbacks:
307-
self._callbacks[event_name] = []
308-
309-
self._callbacks[event_name].append((callback_id, callback, temporary))
310-
return callback_id
309+
return self._events_handler.register_callback(event_name, callback, temporary)
310+
311+
class EventsManager:
312+
def __init__(self):
313+
self._event_callbacks = {} # Callback ID -> callback data
314+
self._callback_id = 0
311315

312-
async def _dispatch_event(self, event_name, event_data):
313-
if event_name not in self._callbacks:
314-
return
315-
316+
def register_callback(self, event_name, callback, temporary):
317+
self._callback_id += 1
318+
self._event_callbacks[self._callback_id] = {
319+
'event': event_name,
320+
'callback': callback,
321+
'temporary': temporary
322+
}
323+
return self._callback_id
324+
325+
async def _trigger_callbacks(self, event_name, event_data):
316326
callbacks_to_remove = []
317327

318-
for callback_id, callback, temporary in self._callbacks[event_name]:
319-
await callback(event_data)
320-
321-
if temporary:
322-
callbacks_to_remove.append(callback_id)
328+
for cb_id, cb_data in self._event_callbacks.items():
329+
if cb_data['event'] == event_name:
330+
# Execute callback (await if async, call directly if sync)
331+
if asyncio.iscoroutinefunction(cb_data['callback']):
332+
await cb_data['callback'](event_data)
333+
else:
334+
cb_data['callback'](event_data)
335+
336+
# Mark temporary callbacks for removal
337+
if cb_data['temporary']:
338+
callbacks_to_remove.append(cb_id)
323339

324-
for callback_id in callbacks_to_remove:
325-
await self.remove_callback(callback_id)
340+
# Remove temporary callbacks after all callbacks executed
341+
for cb_id in callbacks_to_remove:
342+
self.remove_callback(cb_id)
326343
```
327344

328345
This architecture ensures:
@@ -374,9 +391,12 @@ Each tab maintains its own:
374391

375392
- Event domain enablement state
376393
- Callback registry
377-
- WebSocket connection (or shared connection with target ID)
394+
- Event communication channel
378395
- Network logs (if network events enabled)
379396

397+
!!! info "Communication Architecture"
398+
Each tab has its own event communication channel to the browser. For technical details on how WebSocket connections and target IDs work at the protocol level, see [Browser Domain Architecture](./browser-domain.md).
399+
380400
### Shared Browser Context
381401

382402
Multiple tabs can share:

docs/en/deep-dive/architecture/find-elements-mixin.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ Pydoll faces a specific architectural challenge:
3535
- **`WebElement`** needs to find elements **relative to itself** (child elements)
3636
- Both need **identical selector logic** (CSS, XPath, attribute building)
3737

38-
**Option 1: Shared Base Class**
38+
**Option 1: Shared Base Class**
3939

4040
```python
4141
class ElementLocator:
@@ -53,7 +53,7 @@ class WebElement(ElementLocator):
5353
- Violates Single Responsibility: `Tab` shouldn't inherit from same class as `WebElement`
5454
- Hard to extend: Adding new capabilities requires modifying base class
5555

56-
**Option 2: Mixin Pattern**
56+
**Option 2: Mixin Pattern (Chosen Approach)**
5757

5858
```python
5959
class FindElementsMixin:

0 commit comments

Comments
 (0)