Skip to content

fix(realtime): explicit unsubscribeMarket on subscription cleanup (H0)#28

Closed
cyl19970726 wants to merge 41 commits into
mainfrom
fix/realtime-h0-explicit-unsubscribe
Closed

fix(realtime): explicit unsubscribeMarket on subscription cleanup (H0)#28
cyl19970726 wants to merge 41 commits into
mainfrom
fix/realtime-h0-explicit-unsubscribe

Conversation

@cyl19970726

Copy link
Copy Markdown
Owner

Summary

Polymarket WS market channel is stateful: subscribe adds, unsubscribe removes. RealtimeServiceV2.unsubscribe callback only removed local listeners + cleared accumulatedMarketTokenIds, but never sent `{operation:'unsubscribe'}` to the server.

Result: server-side subscription set kept growing across worker lifetime → capacity exhausted → silent drop. Reproduced as 38.5% crypto / 81% cs2 / 1.4% weather miss rate over 24h.

Fix

Send `unsubscribeMarket(tokenIds)` on the active client whenever a subscription is torn down. Try/catch wrapped to prevent recorder hot-path exceptions.

```typescript
if (this.client && this.connected && tokenIds.length > 0) {
try {
this.client.unsubscribeMarket(tokenIds);
} catch (err) {
this.log(`unsubscribeMarket failed (non-fatal): ${err}`);
}
}
```

Test plan

  • 6 new unit tests covering happy path, accumulated set cleanup, disconnected/null client guards, throw safety, isolation between subs
  • All 85 poly-sdk tests pass
  • Verified live on GCE strategy-cs2 (2026-05-03 09:41 UTC):
    • Diag-Raw unique_tokens collapsed from cumulative 1670 → live 8 (= 4 active markets × 2 tokens, no accumulation)
    • 1h crypto miss rate 59.3% → 5-15% (irreducible from market lifecycle)
    • MaxListeners warnings only at startup burst, zero new post-deploy
  • 23h+ zero regressions across crypto/cs2/weather domains

Paired PR

  • earning-engine: ml-5m-exploration → general-data-infra (submodule bump + Phase A scoring)

🤖 Generated with Claude Code

cyl19970726 and others added 30 commits February 26, 2026 16:56
在 AutoCopyTradingOptions 中添加 preOrderCheck 回调,
允许在下单前进行异步检查(如市场交易量、orderbook 深度),
返回 false 可跳过该笔交易。

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bug 1: handleCryptoConnect() had no re-subscription logic. When LIVE_DATA
WebSocket disconnects and reconnects, all crypto price subscriptions were
silently lost. This caused the 5m data worker to receive 0 crypto prices
(received=0) while 15m worker was lucky to not disconnect yet (received=2924).

Fix: Store active Binance/Chainlink symbols in Sets, replay subscriptions
in handleCryptoConnect() with 1s delay (same pattern as handleConnect and
handleUserConnect).

Bug 2: handleCryptoPriceMessage() used `Number(payload.value) || 0` which
converts undefined (subscription acks) to price=0, polluting JSONL data.

Fix: Validate payload.symbol exists and value is finite > 0 before emit.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bug 1: handleConnect() only replays subscriptionMessages on reconnect.
If the 100ms batch timer was cancelled by a disconnect before
sendMergedMarketSubscription() stored the message, the subscription
was permanently lost. Fix: check accumulatedMarketTokenIds and
re-schedule if subscription was never stored.

Bug 2: subscribe() (old API adapter) extracts token IDs from all 5
subscription types, producing 5x duplicate IDs (10 instead of 2).
This sends { type: "MARKET", assets_ids: [id1,id2,id1,id2,...] }
to Polymarket which may cause INVALID OPERATION.
Fix: deduplicate with Set before sending.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add EventService class with 4 methods:
  - listEvents(): Query events with filtering
  - getEventById(): Get event by ID
  - getEventBySlug(): Get event by slug
  - getEventTags(): Get available event tags
- Integrate with RateLimiter (ApiType.GAMMA_API)
- Add caching support (5min for events, 1hr for tags)
- Export PolymarketEvent, ListEventsParams, EventTag types
- Part of task-event-driven-market-data-infrastructure Phase 2

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Enable Safe-to-EOA or Safe-to-Safe USDC transfers via Relayer.
Useful for fund distribution from main wallet to strategy wallets.
- Support optional logger injection in RealtimeServiceV2 and RealTimeDataClient
- Use debug level for frequent events (price_change, raw messages)
- Use info level for important events (orderbook, connections)
- Compatible with @earning-engine/logger but works standalone
Make privateKey optional in CTFConfig to enable read-only mode.
Read-only mode supports: getMarketResolution(), all balance queries require wallet.
Transaction methods (split/merge/redeem) throw clear error in read-only mode.

Fixes: ResolutionCollector startup error (empty PRIVATE_KEY env var)
… issues

Replace JsonRpcProvider with StaticJsonRpcProvider and explicit network config
to prevent automatic network detection failures with some RPC providers.

Changes:
- Use StaticJsonRpcProvider with explicit chainId (default: 137 for Polygon)
- Specify network name ('polygon') to avoid eth_chainId calls
- More reliable for read-only operations in data workers
… clearPosition retry

Smart Money & Copy Trading:
- SELL direction: bypass all filters (minTradeSize, sideFilter, priceRange, tradeFilter, preOrderCheck)
- GTD order: gtdExpiration as remaining seconds; when 0, order expires
- SELL + insufficient position: retry with clearPosition (pm_tracker logic)
- Remove onSellInsufficientPosition callback

TradingService:
- Add clearPosition(params): market sell by tokenId, fetches size from Data API
- ClearPositionParams: tokenId, price, orderType (minimal API)
- Add optional dataApi to TradingServiceConfig for clearPosition

DataApiClient:
- Add getPositionByTokenId(address, tokenId): encapsulate position lookup by asset

Made-with: Cursor
- SmartMoneyServiceConfig 新增 debug 选项 (默认 false)
- 新增 debugLog() 方法,仅 debug=true 时输出
- Poll 轮询:拉取结果、去重、交易检测
- Mempool:WSS 连接、订阅、交易检测、去重
- Subscribe:地址/大小/SmartMoney 过滤
- 跟单流程:目标检查、各类过滤、余额检查、preOrderCheck、
  下单执行、成功/失败、重试、SELL 持仓不足重试、Split 自动缩减
- 便于排查跟单问题,生产环境默认静默

Made-with: Cursor
When enabled, SELL orders use our full token balance instead of
copying the target's size. Useful when copying makers who have
inventory from prior trades.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- RelayerService.split/merge: add isNegRisk param, routes to NEG_RISK_ADAPTER
- SmartMoneyTrade: add copySize, copyPrice, copyValue fields
- PolySDKOptions: add debug field

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Uses Safe MultiSend to redeem multiple markets in a single relayer call,
avoiding sequential rate limits and 2s delays between redeems.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
redeem() and redeemBatch() were hardcoded to CTF_CONTRACT.
negRisk markets need NEG_RISK_ADAPTER (same pattern as split/merge).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
NegRisk adapter uses different function signature:
  redeemPositions(bytes32 conditionId, uint256[] amounts)
vs CTF standard:
  redeemPositions(address collateral, bytes32 parent, bytes32 condition, uint256[] indexSets)

Using CTF ABI with NEG_RISK_ADAPTER caused all negRisk redeems to revert on-chain.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
MaxUint256 doesn't work for negRisk adapter redeemPositions.
Need actual token balance (6 decimal CTF tokens).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Layer 1 (trading-service): FOK orders now only trust `result.success === true`.
Previously, having an orderID was treated as success, but FOK may return orderID
with success=undefined when accepted but not filled.

Layer 2 (smart-money-service): Market order path now routes through OrderManager
when available. OrderManager awaits real terminal state (filled/cancelled) via
WebSocket/polling, preventing FOK ghost positions where DB records a position
but no actual fill occurred on-chain.

Root cause: 13 ghost positions found in copy-trading reconciliation (2026-03-14).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
3-layer fix for chronic Chainlink data loss on shared WS connection:

Layer 1 (RealTimeDataClient):
- maxReconnectAttempts default 0 (infinite) — 24/7 service must never give up
- Backoff capped at 60s (was uncapped, max 512s)
- Pending message queue (cap=100) — subscribe messages no longer silently dropped during reconnect

Layer 2 (RealtimeServiceV2):
- forceReconnectCrypto() — application layer can force crypto WS reconnect
- isCryptoConnected() — expose actual WS connection state

Root cause: TCP-level ping/pong stays alive while individual symbol
subscriptions silently stop receiving data. Connection-level health
checks are blind to subscription-level failures.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Polymarket RTDS (wss://ws-live-data.polymarket.com) requires PING messages
every 5 seconds to maintain connection. Using the default 30s interval caused
the server to consider the connection dead and silently stop pushing Chainlink
data, resulting in ~40 hard reconnects/hour.

Only the cryptoClient is affected — main and user clients connect to different
endpoints that don't have this requirement.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Extend scanCryptoShortTermMarkets() duration type to include '1h' and '4h'
- Add generate1hSlug() for 1h human-readable slug format (ET timezone)
- Add coinFullNames mapping (btc→bitcoin, eth→ethereum, etc.)
- 4h uses standard timestamp slug format like 5m/15m

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Activity interface and parsing now includes eventSlug for multi-outcome market grouping
- DipArbScanOptions duration accepts '1h' | '4h' in addition to '5m' | '15m'

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Split smart-money-service.ts (3271 lines) into focused modules:
- types.ts: all shared types (60+ interfaces)
- constants.ts: CATEGORY_KEYWORDS + categorizeMarket
- copy.ts: PositionTracker (EventEmitter, position polling) + CopyEngine (two-layer retry, GTC→createLimitOrder fix)
- monitor.ts: TradeMonitor (polling + mempool, unified TradeEvent)
- core.ts: SmartMoneyCore (leaderboard, wallet info)
- reports.ts: WalletReports (wallet/daily/lifecycle reports)

Clean break: no backward compat shim. Old SmartMoneyService retained
for V1 fixed-mode compatibility pending ST-4-6 migration.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…break

- Remove smart-money-service.ts (3271 lines, fully replaced by modular split)
- TradeEvent: add timestamp + marketSlug fields for latency tracking
- monitor.ts: timestamp filled from Activity.timestamp (polling) / detectedAt (mempool)
- index.ts: PolymarketSDK.smartMoney → smartMoneyCore (SmartMoneyCore)
- index.ts: remove all SmartMoneyService exports

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
cyl19970726 and others added 11 commits March 20, 2026 23:55
…nal 500ms

When a FAK order partially fills, Polymarket sends USER_ORDER CANCELLATION
before USER_TRADE fill event. The old code called resolveTerminal+cleanup
immediately on CANCELLATION, removing fill listeners before they could fire.

Fix: if no fills recorded yet when cancellation arrives, delay resolveTerminal
by 500ms to allow in-flight USER_TRADE events to populate _fills array.
If fills have already arrived, resolve immediately (no behavior change).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…dlers too

Previous fix only delayed resolveTerminal but invokeHandlers('cancelled') still
fired immediately, causing monitor.ts onCancelled to write status='failed' before
any in-flight USER_TRADE fill events could arrive.

Move the entire else branch (status assignment + invokeHandlers + resolveTerminal)
into the 500ms setTimeout, so business layer is notified only after the window
for in-flight fills has passed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…edup

Activity.transactionHash is now preserved through activityToTradeEvent()
conversion and exposed on TradeEvent. Used by copy-trading monitor to
write CopyTrade records with a unique dedup key (source_tx_hash UNIQUE index).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ar 58197 timestamps

DataApiClient.normalizeTimestamp() already converts Unix seconds to ms.
activityToTradeEvent() was multiplying by 1000 again, producing
timestamps ~56000 years in the future and -1.77×10^12ms latency values.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace all executable console.log/warn/error calls with structured
log.info/debug/warn/error via createLogger in 7 files:
- services/order-manager.ts (ws connect, user-trade fill details)
- services/ctf-manager.ts (event handler errors, debug log)
- services/dip-arb-service.ts (chainlink subscribe, market-end)
- services/arbitrage-service.ts (internal log method)
- services/realtime-service-v2.ts (fallback log path)
- clients/data-api.ts (offset limit warning)
- smart-money/monitor.ts (mempool ws warn + stats)

JSDoc/comment-only console examples were intentionally skipped.
Add @earning-engine/logger workspace:* to package.json dependencies.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…engine/logger dep

After first commit (97adc7e), GCE build failed because poly-sdk is a
standalone package that cannot depend on workspace:* siblings at runtime.

Switch all 7 migrated files to use createModuleLogger from ../core/logger.js
(the pre-existing internal logger bridge in poly-sdk). Consumers inject
their logger via setLogger() at app startup — poly-sdk stays dependency-free.

Also:
- Remove @earning-engine/logger from package.json dependencies
- Track src/core/logger.ts (was untracked)
- Fix type errors: wrap bare error/string args in Record for Logger interface

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- core/order-status.ts: console.warn → log.warn (unknown API status)
- core/unified-cache.ts: console.warn → log.warn (invalidate limitation)
- core/cache-adapter-bridge.ts: console.warn → log.warn (invalidate limitation)
- realtime/realtime-data-client.ts: console.log fallback → _sdkLog module logger
- services/trading-service.ts: console.warn → log.warn (missing side field)
- smart-money/copy.ts: add createModuleLogger + replace console.log in FOK fallback
- index.ts: export setLogger, getLogger, Logger from core/logger.js

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… gracefully

Replace this.emit('error', ...) with log.warn() in trackSettlement catch block.
Emitting 'error' on an EventEmitter with no registered listener causes an
unhandled rejection. Settlement tracking failure (transient RPC outage) is
non-fatal — the process should log and continue, not risk crashing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Required by monitor.ts auto-merge to verify CTF token balances
before executing merge transactions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ules

Export new smart-money V2 utilities:
- categorizeMarket() + CATEGORY_KEYWORDS from constants
- DailyWalletReport, WalletLifecycleReport, WalletChartData, TextReport types

These are part of the modularized copy-trading branch smart-money V2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Polymarket WS market channel is stateful: subscribe adds, unsubscribe removes.
RealtimeServiceV2.unsubscribe callback only removed local listeners + cleared
accumulatedMarketTokenIds, but never sent {operation:'unsubscribe'} to the server.

Result: server-side subscription set kept growing across worker lifetime →
capacity exhausted → silent drop. Reproduced as 38.5% crypto / 81% cs2 /
1.4% weather miss rate over 24h.

Fix sends unsubscribeMarket(tokenIds) on the active client whenever a
subscription is torn down (try/catch wrapped to prevent recorder hot-path
exceptions). Added unit tests covering happy path, accumulated set cleanup,
disconnected/null client guards, throw safety, and isolation between subs.

Verified post-deploy on GCE strategy-cs2 (2026-05-03 09:41 UTC):
  - Diag-Raw unique_tokens collapsed from cumulative 1670 → live 8
    (= 4 active markets × 2 tokens, no accumulation)
  - 1h crypto miss rate 59.3% → 5-15% (irreducible from market lifecycle)
  - MaxListeners warnings only at startup burst, zero new post-deploy

Refs:
- task-fix-ob-collector-short-duration-miss skill (B1 H0 root cause)
- guide-data-quality-validation skill (output-validation methodology)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cyl19970726

Copy link
Copy Markdown
Owner Author

Superseded — direct push to copy-trading (7c8c7b4) per fork owner's preference. Same H0 fix, no PR flow needed.

@cyl19970726 cyl19970726 closed this May 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants