Skip to content

fix(xueqiu): fix 400 errors — use browser cookies, correct UA and Referer, fix hot posts endpoint#210

Open
MatrixA wants to merge 1 commit intoPanniantong:mainfrom
MatrixA:fix/xueqiu-cookie-auth
Open

fix(xueqiu): fix 400 errors — use browser cookies, correct UA and Referer, fix hot posts endpoint#210
MatrixA wants to merge 1 commit intoPanniantong:mainfrom
MatrixA:fix/xueqiu-cookie-auth

Conversation

@MatrixA
Copy link
Copy Markdown
Contributor

@MatrixA MatrixA commented Mar 27, 2026

Problem

Xueqiu returns HTTP 400 (`error_code: 400016`) for all API calls regardless of proxy setup. This is not a network issue — it's an auth issue in the code itself.

Root cause: the stock API requires `xq_a_token`, a session token generated by Xueqiu's frontend JavaScript. Visiting the homepage via `urllib.request` only ever yields `acw_tc` (an anti-DDoS cookie). `xq_a_token` is never obtainable this way, so `_ensure_cookies()` was fundamentally insufficient.

Additional issues found during investigation:

  • `User-Agent: "agent-reach/1.0"` is flagged by Xueqiu's anti-bot system
  • All API requests were missing `Referer: https://xueqiu.com/\`
  • `get_hot_posts()` was calling `/statuses/hot/listV3.json` which returns an empty body (deprecated endpoint)
  • `configure --from-browser` didn't extract Xueqiu cookies, even though browser cookies fix the 400

Changes

`agent_reach/channels/xueqiu.py`

  • `_ensure_cookies()`: new three-level priority — (1) config file (saved by `--from-browser`) → (2) live Chrome cookies via `browser_cookie3` if `xq_a_token` is present → (3) homepage fallback
  • `_get_json()`: use real Chrome User-Agent; add `Referer: https://xueqiu.com/\`
  • `get_hot_posts()`: switch to `/v4/statuses/public_timeline_by_category.json`; parse `item.data` as JSON string to extract author, text, likes
  • `check()`: error message now points to `configure --from-browser chrome` instead of suggesting a proxy
  • `tier` and `backends` updated to reflect cookie requirement

`agent_reach/cookie_extract.py`

  • Add Xueqiu to `PLATFORM_SPECS` (grab all cookies, like XHS)
  • `configure_from_browser()` now saves `xueqiu_cookie` when `xq_a_token` is present; warns the user to log in first if it's missing

`tests/test_channels.py`

  • Update `get_hot_posts` test data to v4 timeline format
  • Add `test_ensure_cookies_loads_from_config`: verifies config-file cookie injection
  • Add `test_get_json_sends_referer_and_browser_ua`: verifies Referer header and non-bot UA

Docs (style kept minimal, consistent with existing tone)

  • README / README_en: update Xueqiu row from "Zero config" → "Tell your Agent 'help me set up Xueqiu'"
  • `docs/install.md`: add Xueqiu to `--from-browser` tip; add brief setup section
  • `docs/troubleshooting.md`: add short Xueqiu 400 section
  • `agent_reach/skill/SKILL.md`: update section header and footer note
  • `CHANGELOG.md`: add 1.3.1 entry

Testing

75 passed in 25s

End-to-end verified locally with real browser cookies:

  • `check()` → ok
  • `get_stock_quote("SH600519")` → 贵州茅台 1413.9 元
  • `get_hot_stocks(limit=10)` → 泡泡玛特 rank 1, 赣锋锂业 rank 2 …
  • `get_hot_posts(limit=3)` → real authors, titles, like counts
  • `configure --from-browser chrome` → ✅ Xueqiu: 18 cookies (含 xq_a_token)

The Xueqiu stock API requires a login session token (xq_a_token) that
is generated by Xueqiu's frontend JavaScript and cannot be obtained by
simply visiting the homepage. This caused persistent HTTP 400 (error
code 400016) for all users.

Changes:
- _ensure_cookies(): add three-level priority — config file (saved by
  --from-browser) → live Chrome cookies via browser_cookie3 → homepage
  fallback. The homepage-only approach only ever got acw_tc (anti-DDoS
  token), never xq_a_token.
- _get_json(): switch User-Agent from "agent-reach/1.0" to a real Chrome
  UA, and add Referer: https://xueqiu.com/ to all API requests.
- get_hot_posts(): replace the defunct /statuses/hot/listV3.json endpoint
  (returns empty body) with the v4 public timeline endpoint; correctly
  parse item.data as a JSON string to extract author, text, and likes.
- cookie_extract.py: add Xueqiu to PLATFORM_SPECS and configure_from_browser
  so that `agent-reach configure --from-browser chrome` now also saves
  Xueqiu cookies (only when xq_a_token is present).
- check(): improve error message to direct users to --from-browser instead
  of suggesting a proxy.
- Fix urllib.parse.quote usage (was using urllib.request.quote).
- Update tier and backends description to reflect cookie requirement.
- Add 2 new tests: cookie loading from config, Referer/UA header verification.
- Update docs: README, install guide, troubleshooting, SKILL.md, CHANGELOG.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant