feat: optimize sync

[zy0n]

This pull request introduces several significant improvements to the processing and validation of Merkle trees and event handling, focusing on performance, reliability, and maintainability. The most important changes include the introduction of a larger batch size for commitment processing, improved concurrency control for write queue processing, enhanced error handling with retry logic, and optimizations for event mapping and batching. Additionally, configuration and type updates have been made to support these changes.

Merkletree Processing and Validation Improvements:

• Added a new CommitmentProcessingGroupSize.XXXXLarge (10,000) option to allow for larger batch processing, improving performance for large-scale updates (src/models/merkletree-types.ts, src/merkletree/txid-merkletree.ts, src/merkletree/utxo-merkletree.ts) [1] [2] [3].
• Updated the Merkle tree write queue processing logic to use a promise-based lock (processingWriteQueueLock) instead of a boolean flag, ensuring safe, sequential processing and preventing race conditions (src/merkletree/merkletree.ts) [1] [2] [3].
• Added support for skipping Merkle root validation in insertLeaves and related methods, enabling faster bulk imports or trusted syncs when desired (src/merkletree/merkletree.ts, src/railgun-engine.ts) [1] [2] [3] [4] [5] [6] [7] [8].

Event Handling and Batching Enhancements:

• Refactored event processing methods in V2Events to use synchronous mapping and batch event passing, reducing unnecessary async operations and improving efficiency (src/contracts/railgun-smart-wallet/V2/V2-events.ts) [1] [2] [3] [4].
• Optimized transaction batching in TXIDMerkletree to ensure that leaves for the same tree are grouped and queued together, reducing the number of write operations (src/merkletree/txid-merkletree.ts) [1] [2] [3].

Error Handling and Retry Logic:

• Enhanced error handling in updateTreesFromWriteQueue to implement retry logic with a delay, improving robustness against transient failures during batch processing (src/merkletree/merkletree.ts).

Configuration and Type Updates:

• Updated package.json to specify the Yarn version for package management, ensuring consistent dependency resolution (package.json).
• Minor type and import updates to support new functionality and maintain code clarity (src/merkletree/merkletree.ts).

These changes collectively improve the scalability, reliability, and maintainability of Merkle tree operations and event processing in the codebase.