Merged
Conversation
Remove dependency for external API. Only collector calls external API. Add exception handling for cache and DB access logic, wrapping with try~catch block
- Introduced single-flight mechanism to collapse concurrent cache misses - Added saveIfAbsent (SET NX) support to prevent redundant Redis writes - Separated cache write logic to skip when value already exists and is identical
- Modified getCurrentPrice to catch CompletionException from CompletableFuture.join() and rethrow underlying PriceNotFoundException directly - Updated unit tests to run synchronously with injected executor/backoff for determinism - Adjusted verifications to account for multiple cache lookups and saveIfAbsent usage
…ncurrent cache miss, and backoff behavior - Verified that cache hits return immediately without accessing DB or performing redundant writes - Ensured concurrent requests on cache miss collapse into a single DB read and Redis write - Confirmed that during backoff, if a peer fills the cache, the service reuses it and skips writing These tests improve confidence in cache coordination logic and concurrency handling.
…xecutor injection - Extracted cache retry count into PriceCacheProperties using @ConfigurationProperties - Registered PriceCacheProperties via @EnableConfigurationProperties - Introduced PriceCacheConfig to provide Executor and BackoffStrategy beans - Moved executor and backoff injection to constructor (supports @requiredargsconstructor) - Defaulted to fixed thread pool executor using availableProcessors() - Simplified loadOnce logic with Optional chaining for readability - Relocated configuration classes to shared-config module for modular clarity
- Implemented TimeSeriesPriceRepositoryImpl using JdbcTemplate - Removed mock-based test and integrated in-memory H2 test - Query now fetches latest price by ticker code from price_history table - Added debug logs for SQL execution and result mapping - Adapted test to use US stock symbols (e.g., AAPL) instead of mock - Ensured compatibility with Spring Boot and PostgreSQL-compatible H2 mode
…ticker module - Introduced TickerFormat class with static regex pattern for ticker validation - Defined TICKER_PATTERN supporting major international formats (e.g. AAPL, BRK.B) - Added isValid(String) helper method for reuse across controllers and services - Enforced separation of concerns by locating pattern in domain-ticker, not shared or app modules
… quotes - Implemented YahooFinancePriceClient using WebClient to fetch stock prices from Yahoo Finance API - Added YahooResponseParser to convert raw JSON responses into structured PriceSnapshot records - Created PriceSnapshot record to encapsulate ticker, price, volume, currency, and timestamp - Included a basic integration test (YahooFinancePriceClientTest) to validate API response - Configured Spring Boot application with scheduling and component scanning for batch collector
…ls and multi-store persistence - Added PriceCollectScheduler to periodically fetch stock prices using @scheduled - Introduced FetchIntervalService to apply per-ticker fetch intervals before collecting - Implemented PopularTickerService as a stub for retrieving popular ticker symbols - Developed PriceCollectorService to fetch prices via YahooFinancePriceClient and store them - Stored fetched prices into Redis (RedisPriceStore) with TTL and TimescaleDB (TimescalePriceStore) using upsert - Added robust error handling and logging across all services
- Removed legacy Yahoo Finance integration from price collection flow - Added AlphaVantageClient to fetch quotes using GLOBAL_QUOTE endpoint - Implemented AlphaVantageResponseParser to parse quote responses into PriceSnapshot objects - Updated PriceCollectorService to use Alpha Vantage client and parser - Added basic integration test for AlphaVantageClient - Formatted fetched quotes into a JSON array structure for parsing
PostgreSQL's TIMESTAMPTZ maps to java.sql.Timestamp, not Instant. Using Timestamp.from(Instant) ensures correct and explicit conversion.
- Added domain-level port interface `PriceDataProvider` to decouple app from infra - Moved `PriceSnapshot` into domain-price model to remove infra dependency - Implemented `AlphaVantageDataProvider` in infra/external as first provider * internally composes AlphaVantageClient + parser to return domain models - Updated `PriceCollectorService` to depend only on `PriceDataProvider` * no longer directly tied to AlphaVantage client/parser - Reworked unit tests (PriceCollectorServiceTest) to mock `PriceDataProvider` * provider implementations now tested in infra module independently This refactoring enables swapping between Alpha Vantage, Polygon.io, Yahoo Finance (or future providers) by configuration only, without modifying app logic.
- Added spring-kafka dependency to app-batch-collector for producing/consuming messages - Configured base Kafka settings in application.yml (producers, consumers, concurrency) - Introduced KafkaTopicsConfig in app-batch-collector to auto-create core topics: - quote.request - quote.normalized - quote.request.DLT (for dead-letter)
- Added FetchCommand DTO and Kafka topic configs (quote.request, quote.normalized) - Updated PriceCollectScheduler to enqueue tickers as FetchCommand messages to Kafka - Implemented FetchWorker consumer to process requests via PriceDataProvider and publish normalized PriceSnapshot events - Added RedisSinkConsumer and TimescaleSinkConsumer to consume normalized snapshots and persist them to Redis (real-time cache) and TimescaleDB (historical storage) - Initial configuration kept inside app-batch-collector for simplicity, with plan to refactor sinks and configs into infra modules later
…imescale - Implemented RedisSinkConsumer in app-batch-collector * listens to `quote.normalized` topic * stores latest PriceSnapshot into Redis with TTL - Implemented TimescaleSinkConsumer in app-batch-collector * listens to `quote.normalized` topic * upserts PriceSnapshot into TimescaleDB using ON CONFLICT for idempotency - Both consumers are placed in the app module temporarily * will be migrated to infra-redis and infra-timescaledb modules later
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
관련 이슈
Close #19, #36, #37, #38, #39, #40, #41, #42
작업 내용 요약
특이사항
Yahoo Finance API를 사용하려고 했지만, Unauthorized응답을 받게되어서 찾아보니, 최근에 비공식적인 API도 지원을 중단했다고 합니다. 대체할 무료 API를 찾다가 Alpha Vantage API를 사용하는 것으로 구현했습니다. 하지만 하루에 25건만 가능하고 사용에 제한이 있어서 추후에 좀 더 지원이 가능한 API로 대체하려고 합니다.
외부 시세 조회 API 변경에 대응하기 위해서 외부 API 사용 부분을 추상화해서 전략패턴을 적용하려고 합니다. 성능 테스트나 통합 테스트를 위해서 시세 데이터를 랜덤으로 생성하여 응답하는 별도의 서버를 간단히 구축하려고 합니다. 이때 간단히 외부 API 사용 부분을 대체할 수 있도록 구현하면 변경에도 유연한 대응이 가능하도록 합니다.