docs: indexer services, packages, and top-level README#3260
docs: indexer services, packages, and top-level README#3260UnbornAztecKing wants to merge 1 commit intomainfrom
Conversation
20b3484 to
56455d9
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (10)
indexer/services/socks/README.md (2)
124-124: Minor grammar issue: "Interval for sending batched messages (250ms)."The static analysis tool suggests adding a space or correcting spelling. Consider clarifying this config description. The current wording is acceptable but could be more consistent with other descriptions in the section.
327-327: Minor style improvement: "message sending utilities" should be "message-sending utilities" (hyphenate compound adjective).The description reads: "WebSocket server wrapper and message sending utilities." Per the grammar hint, this should use a hyphen to clearly indicate it's a compound adjective modifying "utilities."
Consider the diff:
- - **`wss.ts`**: WebSocket server wrapper and message sending utilities. + - **`wss.ts`**: WebSocket server wrapper and message-sending utilities.indexer/packages/README.md (2)
43-43: Markdown formatting: remove spaces in emphasis markers (line 43).The static analysis tool (markdownlint) flagged improper spacing inside emphasis markers. Check this line for something like
* foo *or_ bar _which should be*foo*or_bar_without the spaces.Looking at the context, this is likely in a bullet list or table. Please ensure emphasis markers have no internal spaces for markdown compliance.
82-82: Markdown formatting: remove spaces in emphasis markers (line 82).Similar to the above, this line has spaces inside emphasis markers that should be corrected for markdown compliance:
- - byteToBase64/base64ToBytes, getOrderIdHash, isStatefulOrder, isLongTermOrder, requiresImmediateExecution, ORDER_FLAG_* constants. See ./v4-proto-parser/src/* + - bytesToBigInt/bigIntToBytes/bytesToBase64/base64ToBytes, getOrderIdHash, isStatefulOrder, isLongTermOrder, requiresImmediateExecution, ORDER_FLAG_* constants. See ./v4-proto-parser/src/*Actually, on closer inspection, the linting issue is about spaces in emphasis—check for patterns like
* text *and correct them to*text*.indexer/services/vulcan/README.md (1)
16-20: Minor style issue: word repetition at sentence beginnings.Static analysis flagged three successive sentences beginning with "Does not" in the "Out of scope" section:
- Does not persist orders...
- Does not manage WebSocket...
- Does not process on-chain...
- Does not perform order matching...
This is acceptable for an enumerated list of out-of-scope items, but if you want to follow strict style guidelines, consider rephrasing the last sentence. For example:
-- Does not perform order matching or settlement. +- Order matching and settlement are not performed (handled elsewhere).indexer/README.md (1)
119-119: Minor style issue: use hyphen for compound adjective.Line 119 reads: "@dydxprotocol-indexer/redis – typed Redis clients and structured caches (orderbooks, subaccounts, rate limiting primitives, etc.)."
The static analysis tool suggests: "If this is a compound adjective that modifies the following noun, use a hyphen."
Consider:
-- [@dydxprotocol-indexer/redis](packages/redis) – typed Redis clients and structured caches (orderbooks, subaccounts, rate limiting primitives, etc.). +- [@dydxprotocol-indexer/redis](packages/redis) – typed Redis clients and structured caches (orderbooks, subaccounts, rate-limiting primitives, etc.).indexer/services/roundtable/README.md (3)
21-26: Minor style note: word repetition in "Out of scope" section.Static analysis flagged three successive sentences beginning with "Does not":
- Does not process real-time blockchain events...
- Does not serve HTTP requests...
- Does not manage WebSocket connections...
- Does not write orders or execute trades.
This is acceptable for an enumerated list, but if you prefer to follow strict style guidelines, consider varying the sentence structure in the last item:
-- Does not write orders or execute trades. +- Order writing and trade execution are not performed (handled by trading stack).
54-54: Minor style note: word repetition at line 54.Static analysis flagged three sentences beginning with "**AWS":
- AWS RDS: Database snapshot management...
- AWS S3: Storage for database exports.
- AWS Athena: Query engine for research data analysis.
This is acceptable for a formatted list, but if you prefer variety:
-- **AWS S3**: Storage for database exports. +- **S3** (AWS): Object storage for database exports.This is a nitpick and the current format is clear.
421-421: Minor wording suggestion at line 421.Static analysis flagged: "'lag behind' might be wordy. Consider a shorter alternative."
Current: "PnL calculations are eventually consistent and may lag behind real-time data."
Consider:
-- PnL calculations are eventually consistent and may lag behind real-time data. +- PnL calculations are eventually consistent and may lag real-time data.Or:
-- PnL calculations are eventually consistent and may lag behind real-time data. +- PnL calculations are eventually consistent; real-time data may be ahead.This is optional; "lag behind" is acceptable in technical documentation.
indexer/services/auxo/README.md (1)
102-102: Minor wording improvement: simplify "outside of".Line 102 reads: "Auxo is designed to run as an AWS Lambda function and cannot be easily run locally outside of the AWS environment."
Static analysis flagged that "outside of" is redundant. Consider:
-Auxo is designed to run as an AWS Lambda function and cannot be easily run locally outside of the AWS environment. +Auxo is designed to run as an AWS Lambda function and cannot be easily run locally outside the AWS environment.Or:
-Auxo is designed to run as an AWS Lambda function and cannot be easily run locally outside of the AWS environment. +Auxo is designed to run as an AWS Lambda function and cannot be run locally except in AWS.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (9)
indexer/README.md(3 hunks)indexer/packages/README.md(1 hunks)indexer/services/auxo/README.md(1 hunks)indexer/services/bazooka/README.md(1 hunks)indexer/services/comlink/README.md(1 hunks)indexer/services/ender/README.md(1 hunks)indexer/services/roundtable/README.md(1 hunks)indexer/services/socks/README.md(1 hunks)indexer/services/vulcan/README.md(1 hunks)
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: hwray
Repo: dydxprotocol/v4-chain PR: 2597
File: indexer/services/ender/src/scripts/handlers/dydx_update_perpetual_v1_handler.sql:16-20
Timestamp: 2024-11-22T18:12:04.606Z
Learning: Avoid suggesting changes to deprecated functions such as `dydx_update_perpetual_v1_handler` in `indexer/services/ender/src/scripts/handlers/dydx_update_perpetual_v1_handler.sql` if they are unchanged in the PR.
📚 Learning: 2024-11-22T18:12:04.606Z
Learnt from: hwray
Repo: dydxprotocol/v4-chain PR: 2597
File: indexer/services/ender/src/scripts/handlers/dydx_update_perpetual_v1_handler.sql:16-20
Timestamp: 2024-11-22T18:12:04.606Z
Learning: Avoid suggesting changes to deprecated functions such as `dydx_update_perpetual_v1_handler` in `indexer/services/ender/src/scripts/handlers/dydx_update_perpetual_v1_handler.sql` if they are unchanged in the PR.
Applied to files:
indexer/services/ender/README.mdindexer/services/comlink/README.md
🪛 LanguageTool
indexer/services/auxo/README.md
[style] ~102-~102: This phrase is redundant. Consider using “outside”.
Context: ...nction and cannot be easily run locally outside of the AWS environment. For development an...
(OUTSIDE_OF)
indexer/services/roundtable/README.md
[style] ~25-~25: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...ocket connections (handled by Socks). - Does not write orders or execute trades. ##...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[style] ~54-~54: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...S3**: Storage for database exports. - AWS Athena: Query engine for research dat...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[style] ~421-~421: ‘lag behind’ might be wordy. Consider a shorter alternative.
Context: ...tions are eventually consistent and may lag behind real-time data. - Compliance data is c...
(EN_WORDINESS_PREMIUM_LAG_BEHIND)
indexer/services/socks/README.md
[grammar] ~124-~124: Ensure spelling is correct
Context: ... Interval for sending batched messages (250ms). ### Rate Limiting - **`RATE_LIMIT_E...
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
[grammar] ~327-~327: Use a hyphen to join words.
Context: ...**: WebSocket server wrapper and message sending utilities. - **message.ts...
(QB_NEW_EN_HYPHEN)
indexer/services/vulcan/README.md
[style] ~20-~20: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...s on-chain events or blockchain data. - Does not perform order matching or settlemen...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
indexer/services/comlink/README.md
[uncategorized] ~221-~221: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...82): Redis URL for rate limiting. ### Rate Limiting Configuration - **RATE_LIMIT_ENABLED`...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
indexer/README.md
[uncategorized] ~119-~119: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...ctured caches (orderbooks, subaccounts, rate limiting primitives, etc.). [@dydxprotocol-inde...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
🪛 markdownlint-cli2 (0.18.1)
indexer/packages/README.md
43-43: Spaces inside emphasis markers
(MD037, no-space-in-emphasis)
82-82: Spaces inside emphasis markers
(MD037, no-space-in-emphasis)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (13)
- GitHub Check: lint
- GitHub Check: call-build-ecs-service-vulcan / (vulcan) Check docker image build
- GitHub Check: call-build-ecs-service-socks / (socks) Check docker image build
- GitHub Check: check-build-bazooka
- GitHub Check: call-build-ecs-service-roundtable / (roundtable) Check docker image build
- GitHub Check: call-build-ecs-service-comlink / (comlink) Check docker image build
- GitHub Check: check-build-auxo
- GitHub Check: test / run_command
- GitHub Check: run_command
- GitHub Check: build-and-push-mainnet
- GitHub Check: build-and-push-testnet
- GitHub Check: Analyze (actions)
- GitHub Check: Summary
🔇 Additional comments (13)
indexer/services/socks/README.md (1)
3-3: Comprehensive and well-structured WebSocket service documentation.The overview clearly describes Socks' responsibilities as the WebSocket gateway. The subsequent sections on architecture, public interfaces, configuration, and local running provide excellent operational clarity. The detailed rate-limit and subscription-limit configurations are particularly helpful for operators.
indexer/services/bazooka/README.md (1)
3-3: Excellent documentation for Bazooka Lambda deployment/migration service.Clear description of Bazooka's purpose and scope. The section on responsibilities is well-delineated, with explicit "Out of scope" items that clarify boundaries. The Kafka topics managed and configuration sections provide operators with essential reference material.
indexer/packages/README.md (1)
1-100: Comprehensive and well-organized packages documentation.This is an excellent addition to the repository. It clearly documents the purpose and public API of each internal package, dependencies, constraints, initialization requirements, error handling, and concurrency considerations. The structure makes it easy for service developers to understand which package to use and how to integrate it properly. The notes on side effects (e.g., base logger transports on import) and lazy loading (Redis scripts) are particularly valuable for avoiding subtle bugs.
indexer/services/vulcan/README.md (1)
3-3: Comprehensive off-chain order processor documentation.Vulcan's responsibilities and architecture are clearly laid out. The processing flow diagram (step 1-7) makes the data flow easy to understand. Configuration section covers core settings, Kafka, Redis, batching, and importantly, feature flags that control behavior during fast sync and other operational modes.
indexer/README.md (1)
11-136: Excellent high-level overview and architecture documentation.This restructured README provides a clear introduction to the indexer system with well-organized sections on Overview, Infrastructure, Core Services, and Internal Packages. The descriptions of each service are concise yet informative, and the links to individual service READMEs and packages/README.md enable deeper exploration. The addition of links to packages/README.md at line 136 is particularly helpful for developers needing to understand the internal library structure.
indexer/services/ender/README.md (1)
3-5: Comprehensive on-chain data ingestion service documentation.Ender's role as the primary ingestion point is clearly articulated. The responsibilities section effectively distinguishes between in-scope (event processing, persistence, caching, publishing) and out-of-scope (HTTP endpoints, authentication, orderbook management) items. The processing flow from consuming blocks to publishing to multiple Kafka topics is well-documented and easy to follow.
indexer/services/roundtable/README.md (1)
1-523: Excellent comprehensive documentation for the Roundtable task scheduler.This is a thorough and well-organized documentation for a complex service with 18+ task loops. The responsibilities, architecture, and processing flow sections provide clear context. The configuration section is particularly comprehensive, with separate subsections for core settings, database, Redis, Kafka, loop enablement flags, intervals, lock multipliers, task-specific settings, compliance, AWS, and lag tracking. The observability section covering logging, metrics, failure modes, and limitations is valuable for operators. The directory layout helps developers navigate the codebase.
indexer/services/auxo/README.md (1)
3-220: Comprehensive Lambda deployment orchestrator documentation.Auxo's responsibilities and scope are clearly delineated. The processing flow diagram (7 steps) clearly explains the upgrade lifecycle. The event schema is well-documented, and the IAM permissions section is particularly valuable for security. The section on known limitations (no rollback, fixed timeouts, no health checks) is honest and helpful for operators to understand Auxo's constraints. The directory layout helps developers navigate the codebase.
indexer/services/comlink/README.md (5)
1-20: Excellent comprehensive documentation expansion. ✓The overview, scope, and out-of-scope sections clearly establish Comlink's role and boundaries. The architectural overview and dependency map are well-structured.
296-320: Verify Cache-Control directive defaults match production configuration.The Cache-Control directives (lines 299–320) specify defaults like
public, max-age=1for real-time endpoints andpublic, max-age=10for slower-changing data. Confirm these defaults align with actualconfig.tsvalues and production CDN caching expectations.
341-378: Approve local development and Docker setup instructions. ✓The prerequisites, step-by-step instructions, and Docker Compose guidance are clear and actionable. Ensure
.env.developmentand.env.testfiles are kept synchronized with the documented environment variables.
221-221: Static analysis flag: likely false positive.LanguageTool flagged "rate limiting" as potentially needing a hyphen, but in this context (
Redis URL for rate limiting), the phrase functions as a descriptive prepositional phrase, not a compound adjective modifying a noun. No change needed.
65-189: All handler file paths verified and exist in the codebase.The endpoint documentation is well-organized by domain, and all referenced controller files (addresses-controller, perpetual-positions-controller, orders-controller, fills-controller, transfers-controller, historical-pnl-controller, pnl-controller, perpetual-markets-controller, orderbook-controller, trades-controller, candles-controller, sparklines-controller, historical-funding-controller, funding-payments-controller, compliance-controller, compliance-v2-controller, affiliates-controller, vault-controller, turnkey-controller, skip-bridge-controller, time-controller, height-controller, social-trading-controller, historical-trading-reward-aggregations-controller, and historical-block-trading-rewards-controller) exist at the documented paths in
src/controllers/api/v4/.
| ## Configuration | ||
|
|
||
| Configuration is loaded from environment variables and parsed in [`src/config.ts`](src/config.ts). Environment-specific defaults are in `.env.development`, `.env.test`, and `.env.production`. | ||
|
|
||
| ### Core Settings | ||
|
|
||
| - **`SERVICE_NAME`** (string, default: `comlink`): Service identifier for logging and metrics. | ||
| - **`NODE_ENV`** (string, default: `development`): Environment mode (`development`, `test`, `production`). | ||
| - **`PORT`** (integer, default: `8080`): HTTP server port. | ||
| - **`CORS_ORIGIN`** (string, default: `*`): CORS allowed origins. | ||
| - **`LOG_GETS`** (boolean, default: `false`): Whether to log GET requests. | ||
| - **`KEEP_ALIVE_MS`** (integer, default: `61000`): Server keep-alive timeout. | ||
| - **`HEADERS_TIMEOUT_MS`** (integer, default: `65000`): Server headers timeout. | ||
|
|
||
| ### Database Configuration | ||
|
|
||
| - **`DB_HOSTNAME`** (string, required): PostgreSQL primary host. | ||
| - **`DB_READONLY_HOSTNAME`** (string, optional): PostgreSQL read replica host. | ||
| - **`DB_PORT`** (integer, default: `5432`): PostgreSQL port. | ||
| - **`DB_NAME`** (string, required): Database name. | ||
| - **`DB_USERNAME`** (string, required): Database username. | ||
| - **`DB_PASSWORD`** (string, required): Database password. | ||
|
|
||
| ### Redis Configuration | ||
|
|
||
| - **`REDIS_URL`** (string, default: `redis://localhost:6382`): Primary Redis URL for caching and orderbook data. | ||
| - **`REDIS_READONLY_URL`** (string, optional): Read-only Redis URL. | ||
| - **`REDIS_RECONNECT_TIMEOUT_MS`** (integer, default: `5000`): Redis reconnection timeout. | ||
| - **`RATE_LIMIT_REDIS_URL`** (string, default: `redis://localhost:6382`): Redis URL for rate limiting. | ||
|
|
||
| ### Rate Limiting Configuration | ||
|
|
||
| - **`RATE_LIMIT_ENABLED`** (boolean, default: `true`): Enable rate limiting. | ||
| - **`RATE_LIMIT_GET_POINTS`** (integer, default: `100`): Points per request for general GET endpoints. | ||
| - **`RATE_LIMIT_GET_DURATION_SECONDS`** (integer, default: `10`): Rate limit window duration. | ||
| - **`INDEXER_INTERNAL_IPS`** (string, default: `""`): Comma-separated list of internal IPs exempt from rate limiting. | ||
|
|
||
| Per-endpoint rate limits (all follow the pattern `RATE_LIMIT_<ENDPOINT>_POINTS` and `RATE_LIMIT_<ENDPOINT>_DURATION_SECONDS`): | ||
| - `RATE_LIMIT_ORDERS_*`: Orders endpoint limits. | ||
| - `RATE_LIMIT_FILLS_*`: Fills endpoint limits. | ||
| - `RATE_LIMIT_CANDLES_*`: Candles endpoint limits. | ||
| - `RATE_LIMIT_SPARKLINES_*`: Sparklines endpoint limits. | ||
| - `RATE_LIMIT_HISTORICAL_PNL_*`: Historical PnL endpoint limits. | ||
| - `RATE_LIMIT_PNL_*`: PnL endpoint limits. | ||
| - `RATE_LIMIT_FUNDING_*`: Funding payments endpoint limits. | ||
| - `RATE_LIMIT_SCREEN_QUERY_PROVIDER_*`: Compliance screening limits per IP. | ||
| - `RATE_LIMIT_SCREEN_QUERY_PROVIDER_GLOBAL_*`: Global compliance screening limits. | ||
|
|
||
| ### Compliance Configuration | ||
|
|
||
| - **`INDEXER_LEVEL_GEOBLOCKING_ENABLED`** (boolean, default: `false` in dev, `true` in production): Enable geo-blocking checks. | ||
| - **`ELLIPTIC_API_KEY`** (string, optional): Elliptic API key for compliance screening. | ||
| - **`MAX_AGE_SCREENED_ADDRESS_COMPLIANCE_DATA_SECONDS`** (integer, default: `86400`): Cache TTL for compliance data (1 day). | ||
| - **`EXPOSE_SET_COMPLIANCE_ENDPOINT`** (boolean, default: `false`): Expose internal compliance status endpoint (dev/staging only). | ||
|
|
||
| ### Turnkey & Bridging Configuration | ||
|
|
||
| - **`TURNKEY_API_BASE_URL`** (string, default: `https://api.turnkey.com`): Turnkey API base URL. | ||
| - **`TURNKEY_API_PRIVATE_KEY`** (string, required): Turnkey API private key for root user. | ||
| - **`TURNKEY_API_PUBLIC_KEY`** (string, required): Turnkey API public key for root user. | ||
| - **`TURNKEY_API_SENDER_PRIVATE_KEY`** (string, required): Turnkey API private key for bridge sender. | ||
| - **`TURNKEY_API_SENDER_PUBLIC_KEY`** (string, required): Turnkey API public key for bridge sender. | ||
| - **`TURNKEY_ORGANIZATION_ID`** (string, required): Turnkey parent organization ID. | ||
| - **`TURNKEY_MAGIC_LINK_TEMPLATE`** (string, optional): Custom magic link template for email authentication. | ||
| - **`TURNKEY_EMAIL_SENDER_ADDRESS`** (string, default: `notifications@mail.dydx.trade`): Email sender address. | ||
| - **`TURNKEY_EMAIL_SENDER_NAME`** (string, default: `dYdX Notifications`): Email sender name. | ||
| - **`SOLANA_SPONSOR_PUBLIC_KEY`** (string, required): Solana fee payer public key for sponsored transactions. | ||
| - **`ALCHEMY_API_KEY`** (string, required): Alchemy API key for webhook management and RPC calls. | ||
| - **`ALCHEMY_AUTH_TOKEN`** (string, required): Alchemy auth token for webhook updates. | ||
| - **`ALCHEMY_WEBHOOK_UPDATE_URL`** (string, default: `https://dashboard.alchemy.com/api/update-webhook-addresses`): Alchemy webhook update endpoint. | ||
| - **`ETHEREUM_WEBHOOK_ID`** (string, required): Alchemy webhook ID for Ethereum mainnet. | ||
| - **`ARBITRUM_WEBHOOK_ID`** (string, required): Alchemy webhook ID for Arbitrum. | ||
| - **`AVALANCHE_WEBHOOK_ID`** (string, required): Alchemy webhook ID for Avalanche. | ||
| - **`BASE_WEBHOOK_ID`** (string, required): Alchemy webhook ID for Base. | ||
| - **`OPTIMISM_WEBHOOK_ID`** (string, required): Alchemy webhook ID for Optimism. | ||
| - **`SOLANA_WEBHOOK_ID`** (string, required): Alchemy webhook ID for Solana. | ||
| - **`ZERODEV_API_KEY`** (string, required): ZeroDev API key for smart account sponsorship. | ||
| - **`ZERODEV_API_BASE_URL`** (string, default: `https://rpc.zerodev.app/api/v3`): ZeroDev RPC base URL. | ||
| - **`SKIP_SLIPPAGE_TOLERANCE_PERCENTAGE`** (string, default: `"0"`): Maximum slippage percentage for Skip Protocol swaps. | ||
| - **`SKIP_SLIPPAGE_TOLERANCE_USDC`** (integer, default: `100`): Maximum slippage in USDC for Skip Protocol swaps. | ||
| - **`BRIDGE_THRESHOLD_USDC`** (integer, default: `20`): Minimum USDC amount to trigger automatic bridging. | ||
| - **`MAXIMUM_BRIDGE_AMOUNT_USDC`** (integer, default: `999000`): Maximum USDC amount per bridge transaction. | ||
| - **`APPROVAL_SIGNER_PUBLIC_ADDRESS`** (string, required): Public address of the approval signer for policy enforcement. | ||
| - **`APPROVAL_ENABLED`** (boolean, default: `true`): Enable policy approvals for bridging. | ||
| - **`CALL_POLICY_VALUE_LIMIT`** (bigint, default: `100000000000000000000`): Maximum value limit for call policies. | ||
|
|
||
| ### Apple Sign-In Configuration | ||
|
|
||
| - **`APPLE_TEAM_ID`** (string, required): Apple Developer Team ID. | ||
| - **`APPLE_SERVICE_ID`** (string, required): Apple Service ID (client_id). | ||
| - **`APPLE_KEY_ID`** (string, required): Apple Key ID. | ||
| - **`APPLE_PRIVATE_KEY`** (string, required): Apple private key (base64-encoded). | ||
| - **`APPLE_APP_SCHEME`** (string, default: `dydxV4`): Deep link scheme for Apple OAuth redirect. | ||
|
|
||
| ### Firebase Configuration | ||
|
|
||
| - **`FIREBASE_PROJECT_ID`** (string, required): Firebase project ID. | ||
| - **`FIREBASE_PRIVATE_KEY`** (string, required): Firebase private key. | ||
| - **`FIREBASE_CLIENT_EMAIL`** (string, required): Firebase client email. | ||
|
|
||
| ### Amplitude Analytics Configuration | ||
|
|
||
| - **`AMPLITUDE_API_KEY`** (string, optional): Amplitude API key for event tracking. | ||
| - **`AMPLITUDE_SERVER_URL`** (string, default: `https://api.eu.amplitude.com/2/httpapi`): Amplitude server URL. | ||
|
|
||
| ### Cache-Control Directives | ||
|
|
||
| Each endpoint has a configurable `Cache-Control` header. All follow the pattern `CACHE_CONTROL_DIRECTIVE_<ENDPOINT>` (string, default varies by endpoint): | ||
| - `CACHE_CONTROL_DIRECTIVE_ADDRESSES` (default: `public, max-age=1`) | ||
| - `CACHE_CONTROL_DIRECTIVE_AFFILIATES` (default: `public, max-age=10`) | ||
| - `CACHE_CONTROL_DIRECTIVE_AFFILIATES_METADATA` (default: `no-cache`) | ||
| - `CACHE_CONTROL_DIRECTIVE_ASSET_POSITIONS` (default: `public, max-age=1`) | ||
| - `CACHE_CONTROL_DIRECTIVE_CANDLES` (default: `public, max-age=1`) | ||
| - `CACHE_CONTROL_DIRECTIVE_FILLS` (default: `public, max-age=1`) | ||
| - `CACHE_CONTROL_DIRECTIVE_FUNDING` (default: `public, max-age=10`) | ||
| - `CACHE_CONTROL_DIRECTIVE_HISTORICAL_BLOCK_TRADING_REWARDS` (default: `public, max-age=10`) | ||
| - `CACHE_CONTROL_DIRECTIVE_HISTORICAL_FUNDING` (default: `public, max-age=10`) | ||
| - `CACHE_CONTROL_DIRECTIVE_HISTORICAL_PNL` (default: `public, max-age=10`) | ||
| - `CACHE_CONTROL_DIRECTIVE_HISTORICAL_TRADING_REWARDS` (default: `public, max-age=10`) | ||
| - `CACHE_CONTROL_DIRECTIVE_ORDERBOOK` (default: `public, max-age=1`) | ||
| - `CACHE_CONTROL_DIRECTIVE_ORDERS` (default: `public, max-age=1`) | ||
| - `CACHE_CONTROL_DIRECTIVE_PERPETUAL_MARKETS` (default: `public, max-age=1`) | ||
| - `CACHE_CONTROL_DIRECTIVE_PERPETUAL_POSITIONS` (default: `public, max-age=1`) | ||
| - `CACHE_CONTROL_DIRECTIVE_SOCIAL_TRADING` (default: `public, max-age=10`) | ||
| - `CACHE_CONTROL_DIRECTIVE_SPARKLINES` (default: `public, max-age=10`) | ||
| - `CACHE_CONTROL_DIRECTIVE_TIME` (default: `no-cache, no-store, no-transform`) | ||
| - `CACHE_CONTROL_DIRECTIVE_TRADES` (default: `public, max-age=1`) | ||
| - `CACHE_CONTROL_DIRECTIVE_TRANSFERS` (default: `public, max-age=1`) | ||
| - `CACHE_CONTROL_DIRECTIVE_VAULTS` (default: `public, max-age=10`) | ||
| - `CACHE_CONTROL_DIRECTIVE_PNL` (default: `public, max-age=10`) | ||
|
|
||
| ### Vault Configuration | ||
|
|
||
| - **`VAULT_PNL_HISTORY_DAYS`** (integer, default: `90`): Number of days of daily PnL history to return. | ||
| - **`VAULT_PNL_HISTORY_HOURS`** (integer, default: `72`): Number of hours of hourly PnL history to return. | ||
| - **`VAULT_PNL_START_DATE`** (string, default: `2024-01-01T00:00:00Z`): Start date for vault PnL calculations. | ||
| - **`VAULT_LATEST_PNL_TICK_WINDOW_HOURS`** (integer, default: `1`): Window for fetching latest PnL tick. | ||
| - **`VAULT_FETCH_FUNDING_INDEX_BLOCK_WINDOWS`** (integer, default: `250000`): Block window size for fetching funding indices. | ||
| - **`VAULT_CACHE_TTL_MS`** (integer, default: `120000`): Cache TTL for vault PnL data (2 minutes). | ||
|
|
||
| ### Affiliate Configuration | ||
|
|
||
| - **`VOLUME_ELIGIBILITY_THRESHOLD`** (integer, default: `10000`): Minimum trading volume (in USDC) required for affiliate eligibility. | ||
|
|
||
| ### API Limits | ||
|
|
||
| - **`API_LIMIT_V4`** (integer, default: `1000`): Maximum number of results per API request. | ||
| - **`API_ORDERBOOK_LEVELS_PER_SIDE_LIMIT`** (integer, default: `100`): Maximum orderbook levels per side. |
There was a problem hiding this comment.
🧩 Analysis chain
Verify environment variable names, defaults, and Cache-Control/rate-limit settings against config.ts.
The configuration section lists 40+ environment variables with defaults and descriptions. These must match the actual configuration schema in indexer/services/comlink/src/config.ts to ensure consistency and prevent deployment issues.
Run this script to cross-reference environment variable names and defaults:
🏁 Script executed:
#!/bin/bash
# Description: Extract env vars from README and verify against config.ts.
echo "Extracting environment variable names from README..."
# Extract env var names from README (pattern: **\`VAR_NAME\`**)
readme_vars=$(grep -oP '\*\*`\K[A-Z_]+(?=`\*\*)' indexer/services/comlink/README.md | sort -u)
echo "Checking config.ts for variable definitions..."
config_file="indexer/services/comlink/src/config.ts"
if [ ! -f "$config_file" ]; then
echo "✗ config.ts not found at $config_file"
exit 1
fi
# For a sample of critical vars, check their presence and defaults in config.ts
critical_vars=("SERVICE_NAME" "NODE_ENV" "PORT" "DB_HOSTNAME" "DB_NAME" "REDIS_URL" "RATE_LIMIT_ENABLED" "CACHE_CONTROL_DIRECTIVE_ADDRESSES" "TURNKEY_API_BASE_URL" "APPLE_TEAM_ID" "FIREBASE_PROJECT_ID")
echo "Checking critical variables in config.ts..."
missing=0
for var in "${critical_vars[@]}"; do
if ! grep -q "$var" "$config_file"; then
echo "WARNING: $var not found in config.ts"
((missing++))
fi
done
if [ $missing -eq 0 ]; then
echo "✓ All critical environment variables found in config.ts."
else
echo "⚠ $missing variable(s) may be out of sync with config.ts."
fi
# List a few variable definitions for manual inspection
echo ""
echo "Sample variable definitions from config.ts:"
grep -A 2 "SERVICE_NAME\|NODE_ENV\|PORT" "$config_file" | head -20Length of output: 600
I need to inspect the actual config.ts file to understand the structure and verify which variables are actually defined.
🏁 Script executed:
#!/bin/bash
# Inspect the config.ts file structure to see variable definitions and naming patterns
echo "=== Viewing config.ts file structure ==="
config_file="indexer/services/comlink/src/config.ts"
# Show the first 100 lines to understand structure
echo "First 100 lines of config.ts:"
head -100 "$config_file"
echo ""
echo "=== Checking if variables use different naming conventions ==="
# Check for NODE_ENV or node_env or nodeEnv variations
echo "Searching for NODE_ENV variants:"
grep -i "node.env\|NODE_ENV" "$config_file" | head -5
echo ""
echo "Searching for SERVICE_NAME variants:"
grep -i "service.name\|SERVICE_NAME" "$config_file" | head -5
echo ""
echo "=== Total lines in config.ts ==="
wc -l "$config_file"Length of output: 5121
I need to see the rest of config.ts and check the imported schema files where the missing variables are likely defined.
🏁 Script executed:
#!/bin/bash
# View the rest of config.ts and locate the imported schema files
echo "=== Remaining lines of config.ts (lines 100-175) ==="
tail -n +100 "indexer/services/comlink/src/config.ts"
echo ""
echo "=== Locating imported schema packages ==="
# Look for the base schema package
fd -t f "baseConfig" indexer/services/comlink 2>/dev/null | head -5
echo ""
echo "=== Checking package.json for imported packages ==="
if [ -f "indexer/services/comlink/package.json" ]; then
echo "Dependencies with 'indexer' in name:"
grep -i "@dydxprotocol-indexer" "indexer/services/comlink/package.json"
fiLength of output: 5056
I need to locate and inspect the imported schema files from the workspace packages.
🏁 Script executed:
#!/bin/bash
# Find and inspect the imported schema files
echo "=== Searching for baseConfigSchema ==="
fd -t f "config.*schema\|base.*schema" indexer/packages --exec grep -l "baseConfigSchema\|SERVICE_NAME" {} \; 2>/dev/null | head -10
echo ""
echo "=== Looking for postgres schema ==="
fd -t f ".*config.*" indexer/packages/postgres 2>/dev/null | head -10
echo ""
echo "=== Checking base package structure ==="
find indexer/packages -maxdepth 3 -type d -name "base" 2>/dev/null
echo ""
echo "=== Searching for FIREBASE_PROJECT_ID in entire config.ts ==="
grep -n "FIREBASE" indexer/services/comlink/src/config.ts
echo ""
echo "=== Listing all exported variables from config.ts ==="
grep -oP '^\s+\K[A-Z_]+(?=:)' indexer/services/comlink/src/config.ts | sortLength of output: 3218
🏁 Script executed:
#!/bin/bash
# Inspect the imported schema files to verify missing variables
echo "=== Postgres schema variables ==="
grep -oP '^\s+\K[A-Z_]+(?=:)' indexer/packages/postgres/src/config.ts 2>/dev/null | head -20
echo ""
echo "=== Base schema variables ==="
grep -oP '^\s+\K[A-Z_]+(?=:)' indexer/packages/base/src/config.ts 2>/dev/null | head -30
echo ""
echo "=== Searching for FIREBASE in all config files ==="
rg -n "FIREBASE|Firebase" indexer/packages indexer/services/comlink/src 2>/dev/null | head -20
echo ""
echo "=== Checking if Firebase integration exists anywhere ==="
fd -t f ".*firebase" indexer/ 2>/dev/null | grep -v node_modules | head -10Length of output: 3777
🏁 Script executed:
#!/bin/bash
# Verify if comlink uses Firebase or notifications package
echo "=== Checking if comlink imports notifications ==="
grep -r "notifications" indexer/services/comlink/package.json 2>/dev/null
echo ""
echo "=== Searching for Firebase usage in comlink source ==="
rg -n "firebase\|Firebase\|FIREBASE" indexer/services/comlink/src 2>/dev/null | head -10
echo ""
echo "=== Checking comlink imports from config.ts ==="
head -20 indexer/services/comlink/src/config.ts | grep "import"
echo ""
echo "=== Verifying if notifications schema is used in comlink config ==="
grep "notificationsConfigSchema\|notifications" indexer/services/comlink/src/config.tsLength of output: 535
Remove Firebase configuration section from README; it is not part of comlink's config schema.
The Firebase variables (FIREBASE_PROJECT_ID, FIREBASE_PRIVATE_KEY, FIREBASE_CLIENT_EMAIL) documented in the README are not defined in indexer/services/comlink/src/config.ts. These variables belong to the notifications package (@dydxprotocol-indexer/notifications) and its config schema is not imported by comlink. Although comlink lists notifications as a dependency, it does not use Firebase configuration directly.
All other environment variables verified successfully: SERVICE_NAME, NODE_ENV, DB_* variables exist in imported base and postgres schemas; CACHE_CONTROL_DIRECTIVE_* and RATE_LIMIT_* variables are correctly documented with matching defaults in config.ts.
🧰 Tools
🪛 LanguageTool
[uncategorized] ~221-~221: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...82): Redis URL for rate limiting. ### Rate Limiting Configuration - **RATE_LIMIT_ENABLED`...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
🤖 Prompt for AI Agents
In indexer/services/comlink/README.md around lines 191 to 338, remove the
Firebase Configuration subsection (FIREBASE_PROJECT_ID, FIREBASE_PRIVATE_KEY,
FIREBASE_CLIENT_EMAIL) because these env vars are not defined in comlink's
src/config.ts and belong to the notifications package; update the README so it
only documents vars present in comlink's config (or move the Firebase entries to
the notifications package README) and re-run a quick scan to ensure no other
undocumented/mismatched env vars remain.
Summary
Details
Reviewer Notes
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.