fix(realtime): explicit unsubscribeMarket on subscription cleanup (H0)#28
Closed
cyl19970726 wants to merge 41 commits into
Closed
fix(realtime): explicit unsubscribeMarket on subscription cleanup (H0)#28cyl19970726 wants to merge 41 commits into
cyl19970726 wants to merge 41 commits into
Conversation
在 AutoCopyTradingOptions 中添加 preOrderCheck 回调, 允许在下单前进行异步检查(如市场交易量、orderbook 深度), 返回 false 可跳过该笔交易。 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bug 1: handleCryptoConnect() had no re-subscription logic. When LIVE_DATA WebSocket disconnects and reconnects, all crypto price subscriptions were silently lost. This caused the 5m data worker to receive 0 crypto prices (received=0) while 15m worker was lucky to not disconnect yet (received=2924). Fix: Store active Binance/Chainlink symbols in Sets, replay subscriptions in handleCryptoConnect() with 1s delay (same pattern as handleConnect and handleUserConnect). Bug 2: handleCryptoPriceMessage() used `Number(payload.value) || 0` which converts undefined (subscription acks) to price=0, polluting JSONL data. Fix: Validate payload.symbol exists and value is finite > 0 before emit. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bug 1: handleConnect() only replays subscriptionMessages on reconnect.
If the 100ms batch timer was cancelled by a disconnect before
sendMergedMarketSubscription() stored the message, the subscription
was permanently lost. Fix: check accumulatedMarketTokenIds and
re-schedule if subscription was never stored.
Bug 2: subscribe() (old API adapter) extracts token IDs from all 5
subscription types, producing 5x duplicate IDs (10 instead of 2).
This sends { type: "MARKET", assets_ids: [id1,id2,id1,id2,...] }
to Polymarket which may cause INVALID OPERATION.
Fix: deduplicate with Set before sending.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add EventService class with 4 methods: - listEvents(): Query events with filtering - getEventById(): Get event by ID - getEventBySlug(): Get event by slug - getEventTags(): Get available event tags - Integrate with RateLimiter (ApiType.GAMMA_API) - Add caching support (5min for events, 1hr for tags) - Export PolymarketEvent, ListEventsParams, EventTag types - Part of task-event-driven-market-data-infrastructure Phase 2 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Enable Safe-to-EOA or Safe-to-Safe USDC transfers via Relayer. Useful for fund distribution from main wallet to strategy wallets.
- Support optional logger injection in RealtimeServiceV2 and RealTimeDataClient - Use debug level for frequent events (price_change, raw messages) - Use info level for important events (orderbook, connections) - Compatible with @earning-engine/logger but works standalone
Make privateKey optional in CTFConfig to enable read-only mode. Read-only mode supports: getMarketResolution(), all balance queries require wallet. Transaction methods (split/merge/redeem) throw clear error in read-only mode. Fixes: ResolutionCollector startup error (empty PRIVATE_KEY env var)
… issues
Replace JsonRpcProvider with StaticJsonRpcProvider and explicit network config
to prevent automatic network detection failures with some RPC providers.
Changes:
- Use StaticJsonRpcProvider with explicit chainId (default: 137 for Polygon)
- Specify network name ('polygon') to avoid eth_chainId calls
- More reliable for read-only operations in data workers
… clearPosition retry Smart Money & Copy Trading: - SELL direction: bypass all filters (minTradeSize, sideFilter, priceRange, tradeFilter, preOrderCheck) - GTD order: gtdExpiration as remaining seconds; when 0, order expires - SELL + insufficient position: retry with clearPosition (pm_tracker logic) - Remove onSellInsufficientPosition callback TradingService: - Add clearPosition(params): market sell by tokenId, fetches size from Data API - ClearPositionParams: tokenId, price, orderType (minimal API) - Add optional dataApi to TradingServiceConfig for clearPosition DataApiClient: - Add getPositionByTokenId(address, tokenId): encapsulate position lookup by asset Made-with: Cursor
- SmartMoneyServiceConfig 新增 debug 选项 (默认 false) - 新增 debugLog() 方法,仅 debug=true 时输出 - Poll 轮询:拉取结果、去重、交易检测 - Mempool:WSS 连接、订阅、交易检测、去重 - Subscribe:地址/大小/SmartMoney 过滤 - 跟单流程:目标检查、各类过滤、余额检查、preOrderCheck、 下单执行、成功/失败、重试、SELL 持仓不足重试、Split 自动缩减 - 便于排查跟单问题,生产环境默认静默 Made-with: Cursor
When enabled, SELL orders use our full token balance instead of copying the target's size. Useful when copying makers who have inventory from prior trades. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- RelayerService.split/merge: add isNegRisk param, routes to NEG_RISK_ADAPTER - SmartMoneyTrade: add copySize, copyPrice, copyValue fields - PolySDKOptions: add debug field Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Uses Safe MultiSend to redeem multiple markets in a single relayer call, avoiding sequential rate limits and 2s delays between redeems. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
redeem() and redeemBatch() were hardcoded to CTF_CONTRACT. negRisk markets need NEG_RISK_ADAPTER (same pattern as split/merge). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
NegRisk adapter uses different function signature: redeemPositions(bytes32 conditionId, uint256[] amounts) vs CTF standard: redeemPositions(address collateral, bytes32 parent, bytes32 condition, uint256[] indexSets) Using CTF ABI with NEG_RISK_ADAPTER caused all negRisk redeems to revert on-chain. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
MaxUint256 doesn't work for negRisk adapter redeemPositions. Need actual token balance (6 decimal CTF tokens). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Layer 1 (trading-service): FOK orders now only trust `result.success === true`. Previously, having an orderID was treated as success, but FOK may return orderID with success=undefined when accepted but not filled. Layer 2 (smart-money-service): Market order path now routes through OrderManager when available. OrderManager awaits real terminal state (filled/cancelled) via WebSocket/polling, preventing FOK ghost positions where DB records a position but no actual fill occurred on-chain. Root cause: 13 ghost positions found in copy-trading reconciliation (2026-03-14). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
3-layer fix for chronic Chainlink data loss on shared WS connection: Layer 1 (RealTimeDataClient): - maxReconnectAttempts default 0 (infinite) — 24/7 service must never give up - Backoff capped at 60s (was uncapped, max 512s) - Pending message queue (cap=100) — subscribe messages no longer silently dropped during reconnect Layer 2 (RealtimeServiceV2): - forceReconnectCrypto() — application layer can force crypto WS reconnect - isCryptoConnected() — expose actual WS connection state Root cause: TCP-level ping/pong stays alive while individual symbol subscriptions silently stop receiving data. Connection-level health checks are blind to subscription-level failures. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Polymarket RTDS (wss://ws-live-data.polymarket.com) requires PING messages every 5 seconds to maintain connection. Using the default 30s interval caused the server to consider the connection dead and silently stop pushing Chainlink data, resulting in ~40 hard reconnects/hour. Only the cryptoClient is affected — main and user clients connect to different endpoints that don't have this requirement. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Extend scanCryptoShortTermMarkets() duration type to include '1h' and '4h' - Add generate1hSlug() for 1h human-readable slug format (ET timezone) - Add coinFullNames mapping (btc→bitcoin, eth→ethereum, etc.) - 4h uses standard timestamp slug format like 5m/15m Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Activity interface and parsing now includes eventSlug for multi-outcome market grouping - DipArbScanOptions duration accepts '1h' | '4h' in addition to '5m' | '15m' Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Split smart-money-service.ts (3271 lines) into focused modules: - types.ts: all shared types (60+ interfaces) - constants.ts: CATEGORY_KEYWORDS + categorizeMarket - copy.ts: PositionTracker (EventEmitter, position polling) + CopyEngine (two-layer retry, GTC→createLimitOrder fix) - monitor.ts: TradeMonitor (polling + mempool, unified TradeEvent) - core.ts: SmartMoneyCore (leaderboard, wallet info) - reports.ts: WalletReports (wallet/daily/lifecycle reports) Clean break: no backward compat shim. Old SmartMoneyService retained for V1 fixed-mode compatibility pending ST-4-6 migration. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…break - Remove smart-money-service.ts (3271 lines, fully replaced by modular split) - TradeEvent: add timestamp + marketSlug fields for latency tracking - monitor.ts: timestamp filled from Activity.timestamp (polling) / detectedAt (mempool) - index.ts: PolymarketSDK.smartMoney → smartMoneyCore (SmartMoneyCore) - index.ts: remove all SmartMoneyService exports Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…nal 500ms When a FAK order partially fills, Polymarket sends USER_ORDER CANCELLATION before USER_TRADE fill event. The old code called resolveTerminal+cleanup immediately on CANCELLATION, removing fill listeners before they could fire. Fix: if no fills recorded yet when cancellation arrives, delay resolveTerminal by 500ms to allow in-flight USER_TRADE events to populate _fills array. If fills have already arrived, resolve immediately (no behavior change). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…dlers too
Previous fix only delayed resolveTerminal but invokeHandlers('cancelled') still
fired immediately, causing monitor.ts onCancelled to write status='failed' before
any in-flight USER_TRADE fill events could arrive.
Move the entire else branch (status assignment + invokeHandlers + resolveTerminal)
into the 500ms setTimeout, so business layer is notified only after the window
for in-flight fills has passed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…edup Activity.transactionHash is now preserved through activityToTradeEvent() conversion and exposed on TradeEvent. Used by copy-trading monitor to write CopyTrade records with a unique dedup key (source_tx_hash UNIQUE index). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ar 58197 timestamps DataApiClient.normalizeTimestamp() already converts Unix seconds to ms. activityToTradeEvent() was multiplying by 1000 again, producing timestamps ~56000 years in the future and -1.77×10^12ms latency values. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace all executable console.log/warn/error calls with structured log.info/debug/warn/error via createLogger in 7 files: - services/order-manager.ts (ws connect, user-trade fill details) - services/ctf-manager.ts (event handler errors, debug log) - services/dip-arb-service.ts (chainlink subscribe, market-end) - services/arbitrage-service.ts (internal log method) - services/realtime-service-v2.ts (fallback log path) - clients/data-api.ts (offset limit warning) - smart-money/monitor.ts (mempool ws warn + stats) JSDoc/comment-only console examples were intentionally skipped. Add @earning-engine/logger workspace:* to package.json dependencies. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…engine/logger dep After first commit (97adc7e), GCE build failed because poly-sdk is a standalone package that cannot depend on workspace:* siblings at runtime. Switch all 7 migrated files to use createModuleLogger from ../core/logger.js (the pre-existing internal logger bridge in poly-sdk). Consumers inject their logger via setLogger() at app startup — poly-sdk stays dependency-free. Also: - Remove @earning-engine/logger from package.json dependencies - Track src/core/logger.ts (was untracked) - Fix type errors: wrap bare error/string args in Record for Logger interface Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- core/order-status.ts: console.warn → log.warn (unknown API status) - core/unified-cache.ts: console.warn → log.warn (invalidate limitation) - core/cache-adapter-bridge.ts: console.warn → log.warn (invalidate limitation) - realtime/realtime-data-client.ts: console.log fallback → _sdkLog module logger - services/trading-service.ts: console.warn → log.warn (missing side field) - smart-money/copy.ts: add createModuleLogger + replace console.log in FOK fallback - index.ts: export setLogger, getLogger, Logger from core/logger.js Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… gracefully
Replace this.emit('error', ...) with log.warn() in trackSettlement catch block.
Emitting 'error' on an EventEmitter with no registered listener causes an
unhandled rejection. Settlement tracking failure (transient RPC outage) is
non-fatal — the process should log and continue, not risk crashing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Required by monitor.ts auto-merge to verify CTF token balances before executing merge transactions. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ules Export new smart-money V2 utilities: - categorizeMarket() + CATEGORY_KEYWORDS from constants - DailyWalletReport, WalletLifecycleReport, WalletChartData, TextReport types These are part of the modularized copy-trading branch smart-money V2. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Polymarket WS market channel is stateful: subscribe adds, unsubscribe removes.
RealtimeServiceV2.unsubscribe callback only removed local listeners + cleared
accumulatedMarketTokenIds, but never sent {operation:'unsubscribe'} to the server.
Result: server-side subscription set kept growing across worker lifetime →
capacity exhausted → silent drop. Reproduced as 38.5% crypto / 81% cs2 /
1.4% weather miss rate over 24h.
Fix sends unsubscribeMarket(tokenIds) on the active client whenever a
subscription is torn down (try/catch wrapped to prevent recorder hot-path
exceptions). Added unit tests covering happy path, accumulated set cleanup,
disconnected/null client guards, throw safety, and isolation between subs.
Verified post-deploy on GCE strategy-cs2 (2026-05-03 09:41 UTC):
- Diag-Raw unique_tokens collapsed from cumulative 1670 → live 8
(= 4 active markets × 2 tokens, no accumulation)
- 1h crypto miss rate 59.3% → 5-15% (irreducible from market lifecycle)
- MaxListeners warnings only at startup burst, zero new post-deploy
Refs:
- task-fix-ob-collector-short-duration-miss skill (B1 H0 root cause)
- guide-data-quality-validation skill (output-validation methodology)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Owner
Author
|
Superseded — direct push to copy-trading (7c8c7b4) per fork owner's preference. Same H0 fix, no PR flow needed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Polymarket WS market channel is stateful: subscribe adds, unsubscribe removes. RealtimeServiceV2.unsubscribe callback only removed local listeners + cleared accumulatedMarketTokenIds, but never sent `{operation:'unsubscribe'}` to the server.
Result: server-side subscription set kept growing across worker lifetime → capacity exhausted → silent drop. Reproduced as 38.5% crypto / 81% cs2 / 1.4% weather miss rate over 24h.
Fix
Send `unsubscribeMarket(tokenIds)` on the active client whenever a subscription is torn down. Try/catch wrapped to prevent recorder hot-path exceptions.
```typescript
if (this.client && this.connected && tokenIds.length > 0) {
try {
this.client.unsubscribeMarket(tokenIds);
} catch (err) {
this.log(`unsubscribeMarket failed (non-fatal): ${err}`);
}
}
```
Test plan
Paired PR
🤖 Generated with Claude Code