Skip to content

Commit

Permalink
Update docs/design/2024-01-22-ticdc-large-txn-not-block-wm.md
Browse files Browse the repository at this point in the history
Co-authored-by: CharlesCheung <[email protected]>
  • Loading branch information
zhangjinpeng87 and CharlesCheung96 authored Jan 23, 2024
1 parent cd66b29 commit 8eb61cf
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/design/2024-01-22-ticdc-large-txn-not-block-wm.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ struct WatermarkAdvancer {
}
```

We separate normal transactions and large transactions in the TS Resolver module. Because each lock contains the transaction-size (or we expliciltely add large-transaction flag in the lock) information, when the Lock Observer receives a lock operation, we can dispatch the lock to related region's Resolver and according to the large transaction flag. We just store a little information like start-ts, primary-key, update-to-date min-commit-ts in the Resolver for a large transaction even though there are 1M rows changed in a region. This can significantly reduce the total memory usage of Resolver for a large transaction which can reduce the OOM issue of TiKV (we previously met some OOM issue because the Resolver occupies a lot of memory for large transactions).
We separate normal transactions and large transactions in the TS Resolver module. Because each lock contains the transaction-size (or we explicitly add large-transaction flag in the lock) information, when the Lock Observer receives a lock operation, we can dispatch the lock to related region's Resolver and according to the large transaction flag. We just store a little information like start-ts, primary-key, update-to-date min-commit-ts in the Resolver for a large transaction even though there are 1M rows changed in a region. This can significantly reduce the total memory usage of Resolver for a large transaction which can reduce the OOM issue of TiKV (we previously met some OOM issue because the Resolver occupies a lot of memory for large transactions).

![image](../media/large-txn-no-block-wm-4.png)

Expand Down

0 comments on commit 8eb61cf

Please sign in to comment.