Skip to content

Optimize DirectionsRatesJob: batch INSERT instead of individual queries#66

Merged
dapi merged 3 commits intomasterfrom
chore/64-batch-insert-directions-rates
Jan 20, 2026
Merged

Optimize DirectionsRatesJob: batch INSERT instead of individual queries#66
dapi merged 3 commits intomasterfrom
chore/64-batch-insert-directions-rates

Conversation

@dapi
Copy link
Copy Markdown

@dapi dapi commented Dec 19, 2025

Summary

  • Replace ~6000 individual INSERT queries with single insert_all! call
  • Add currency rates caching for O(1) lookup by currency_pair
  • Reduce job execution time from 30-60 seconds to 1-2 seconds

Changes

  • Add Mathematic module for calculate_finite_rate method
  • Build all direction rate records in memory before inserting
  • Handle missing currency rates gracefully (skip instead of error)

Performance Impact

Metric Before After
SQL INSERTs ~6000 1
Execution time 30-60 sec 1-2 sec
DB load High Low

Test plan

  • All existing tests pass (424 examples, 0 failures)
  • New tests for batch insert behavior
  • Test for correct attribute values
  • Test for skipping exchange rates without matching currency rates

Fixes #64

🤖 Generated with Claude Code

@dapi
Copy link
Copy Markdown
Author

dapi commented Dec 19, 2025

Code review

Found 1 issue:

  1. CLAUDE.md Worker Architecture and File Organization sections not updated after Sidekiq-to-ActiveJob migration (CLAUDE.md should reflect actual codebase structure)

The PR migrated all workers to jobs (e.g., DirectionsRatesWorkerDirectionsRatesJob, app/workers/app/jobs/), but CLAUDE.md still references the old worker terminology:

gera/CLAUDE.md

Lines 23 to 28 in 8b9309d

### Worker Architecture
- **RatesWorker** concern for fetching external rates
- Individual workers for each rate source (ExmoRatesWorker, BitfinexRatesWorker, etc.)
- **CurrencyRatesWorker** - Builds currency rate matrix from external rates
- **DirectionsRatesWorker** - Calculates final direction rates with commissions
- **CreateHistory_intervalsWorker** - Aggregates historical data

Lines 23-28 reference RatesWorker, ExmoRatesWorker, CurrencyRatesWorker, DirectionsRatesWorker, etc. which should now be *Job.

gera/CLAUDE.md

Lines 103 to 104 in 8b9309d

- `app/workers/gera/` - Background job workers
- `lib/gera/` - Core engine logic and utilities

Line 103 references app/workers/gera/ which should be app/jobs/gera/.

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

dapi and others added 3 commits January 20, 2026 20:39
Replace individual create! calls with single insert_all! for significant
performance improvement when processing large numbers of exchange rates.

- Add Mathematic module for calculate_finite_rate
- Cache currency_rates by currency_pair for O(1) lookup
- Build all records in memory, then insert in one SQL statement
- Handle missing currency_rates gracefully (skip instead of error)

Fixes #64

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add explicit order(:interval_from) before .last to utilize the composite
index (cur_from_id, cur_to_id, interval_from) instead of falling back
to ORDER BY id DESC which requires a full table scan.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Rename "Worker Architecture" to "Job Architecture (ActiveJob/SolidQueue)"
- Update all class references from *Worker to *Job
- Update File Organization: app/workers/gera/ → app/jobs/gera/

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
@dapi dapi force-pushed the chore/64-batch-insert-directions-rates branch from 759d2cf to 5f73dc1 Compare January 20, 2026 17:43
@dapi dapi merged commit c42dd41 into master Jan 20, 2026
1 check passed
@dapi dapi deleted the chore/64-batch-insert-directions-rates branch January 20, 2026 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Оптимизация DirectionsRatesJob: batch INSERT вместо индивидуальных запросов

1 participant