Skip to content

performance optimized#486

Open
quangtuyen88 wants to merge 1 commit into
gensyn-ai:mainfrom
quangtuyen88:qt_optimized
Open

performance optimized#486
quangtuyen88 wants to merge 1 commit into
gensyn-ai:mainfrom
quangtuyen88:qt_optimized

Conversation

@quangtuyen88

Copy link
Copy Markdown
  1. Fixed Synchronous File I/O in modal-login/app/db.ts

Changes:

  • Converted all file operations from fs.readFileSync/fs.writeFileSync to async fs.promises
  • Made all exported functions async: upsertUser, getUser, getLatestApiKey, setApiKeyActivated
  • Added parallel reads/writes with Promise.all() in upsertUser (reads both files concurrently, writes both concurrently)
  • Proper error handling for ENOENT (file not found) cases

Impact: Eliminates event loop blocking, allows concurrent request handling, dramatically improves server throughput under load.

  1. Batched Model Inference in rgym_exp/src/trainer.py

Changes:

  • Rewrote _get_choice_logits to process all choices in a single batched forward pass
  • Tokenizes all choices upfront and pads them to the same length
  • Stacks sequences into a batch tensor
  • Single model forward pass instead of N separate passes
  • Manually computes per-sample log probabilities from batch output

Impact: Reduces from N model forward passes to 1 (where N = number of choices). For 4 choices, this is a ~75% reduction in inference time. For 10 choices, ~90% reduction.

  1. Optimized Nested Loops in web/api/dht_pub.py

Changes:

  • Pre-cache peer names for all peers at once (single batch lookup)
  • Reuse single timestamp for all messages instead of calling datetime.now() repeatedly
  • Extract peer name from cache using O(1) dictionary lookup instead of repeated function calls
  • Cleaner hash key construction (same MD5 operation but more explicit)

@quangtuyen88

Copy link
Copy Markdown
Author

@jcd496 can you help review this pr ? i have tested and all is working fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant