KAFKA-16505: Fix source raw key and value in store caches #18739
Conversation
Co-authored-by: loicgreffier <[email protected]>
github-actions
bot
added
triage
There was a problem hiding this comment.
Thanks @loicgreffier for the PR!
You did not consider buffers in this PR as I described in my comment. Could you come up with a test that confirms that we also have the same issue with buffers and then provide a fix?
| @@ -259,6 +259,10 @@ public <K, V> void send(final String topic, | |||
|
|
|||
| final ProducerRecord<byte[], byte[]> serializedRecord = new ProducerRecord<>(topic, partition, timestamp, keyBytes, valBytes, headers); | |||
|
|
|||
| // As many records could be in-flight, | |||
| // freeing raw records in the context to reduce memory pressure | |||
| freeContext(context); | |||
There was a problem hiding this comment.
The name of this method is a bit misleading. It basically frees the raw record within the context, not the whole context. What about calling it freeRawInputRecordFromContext()?
|
|
||
| @Override | ||
| public boolean equals(final Object other) { | ||
| return super.equals(other); | ||
| } | ||
|
|
||
| @Override | ||
| public int hashCode() { | ||
| return super.hashCode(); | ||
| } |
| this.sourceRawKey = null; | ||
| this.sourceRawValue = null; |
There was a problem hiding this comment.
You also need to add these info to the serialize() and deserialize() so that the buffer values also get the source record. Here it gets a bit tricky, because you need to consider the case where a serialized record context does not contain the source record because it was written by a version of Streams that has not yet had the source record in the context.
More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.
Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.
Committer Checklist (excluded from commit message)