Skip to content

Conversation

@nikagra
Copy link

@nikagra nikagra commented Sep 5, 2025

Issue scylla-driver#81

repared statements invalidation used to be completely broken, you can read details on it here
Recently core merged PR that fixes problems for select statements.
After that PR, if driver supply SCYLLA_USE_METADATA_ID at the startup, when statements is prepared server hands out result queries metadata hash.
When driver executes this query it sends metadata id alongside with the request.
Server checks driver metadata id and what it has on its side, if there is any difference it will send new metadata and metadata id.
Driver suppose to pick it up and update metadata and metadata id on it's side, before deserializing response, solving issue with having outdated metadata on driver side.
Unfortunately it is solved only for SELECT statements, UPDATE/INSERT will be solved on separate occasion.

Core changes are in, we need to start working on driver side.

To Be Done:

  1. parse SCYLLA_USE_METADATA_ID protocol extension from SUPPORTED frame;
    • if present in SUPPORTED, send it in the STARTUP frame;
  2. if the extension is negotiated:
    • store the metadata ID, same way it is done for CQL v5
    • read METADATA_CHANGED (=3) flag in result metadata;
    • if the flag is set, read the new metadata ID (short bytes) and update it on prepare statement
  3. if received new result metadata that is different from the cached one:
    • use the new metadata for deserialization instead of the old one;
    • replace the old cached metadata with the new one.

Resolve skip metadata flag

Default value of skip metadata flag should be treated as safest option
Which means that if SCYLLA_USE_METADATA_ID was negotiated or CQL v5 is used then it is true, result metadata is skipped.
In other cases it is false.

Fixes

  • Enhancing forked native-protocol to support optional protocol options (PR#2) in FrameCodec, Codec's and SubCodec's.
  • Negotiating SCYLLA_USE_METADATA_ID extension during protocol initialization Options step.
  • Replacing FrameEncoder and FrameDecoder with updated ProtocolFeatures in pipeline after feature is negotiated.
  • These protocol features will be used during Frame encoding and decoding on native-protocol side

@nikagra nikagra force-pushed the scylla-4.x-use-metadataId branch from 3760023 to 22254de Compare September 5, 2025 12:03
@nikagra nikagra marked this pull request as ready for review September 16, 2025 13:24
@nikagra nikagra force-pushed the scylla-4.x-use-metadataId branch from 40d49d1 to 2feab3c Compare September 17, 2025 16:53
@dkropachev dkropachev self-requested a review September 18, 2025 15:04
@dkropachev dkropachev changed the title [#567] Initial implementation of SCYLLA_USE_METADATA_ID feature neg… [#567] Initial implementation of SCYLLA_USE_METADATA_ID feature Sep 18, 2025
Copy link

@dkropachev dkropachev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small changes, please cleanup commit history after you done.

@nikagra nikagra self-assigned this Sep 26, 2025
@nikagra nikagra changed the title [#567] Initial implementation of SCYLLA_USE_METADATA_ID feature Initial implementation of SCYLLA_USE_METADATA_ID feature Sep 26, 2025
@nikagra nikagra force-pushed the scylla-4.x-use-metadataId branch from 27d7a77 to 850cc81 Compare September 29, 2025 14:59
@dkropachev
Copy link

There is definitely a problem, I have restarted it three times, got same result should_not_struggle_to_fill_pools failed in all runs, it does not happen on other branches.

@nikagra nikagra force-pushed the scylla-4.x-use-metadataId branch from 61eac1b to 28b25fe Compare October 2, 2025 16:52
nikagra added 11 commits October 2, 2025 19:02
…ture negotiation during driver startup 🚀

# Conflicts:
#	core/src/main/java/com/datastax/oss/driver/internal/core/channel/ProtocolInitHandler.java
…Features` to `FrameEncoder` and `FrameDecoder` constructors. Updating integration tests.
# Conflicts:
#	core/src/main/java/com/datastax/oss/driver/internal/core/channel/DriverChannel.java
#	core/src/main/java/com/datastax/oss/driver/internal/core/channel/ProtocolInitHandler.java
…s` (used in `OsgiGeoTypesIT` & `OsgiGraphIT`).
@nikagra nikagra force-pushed the scylla-4.x-use-metadataId branch from 28b25fe to 3e967d7 Compare October 2, 2025 17:02
@dkropachev
Copy link

There is definitely a problem, I have restarted it three times, got same result should_not_struggle_to_fill_pools failed in all runs, it does not happen on other branches.

FYI, this problem exists on main branch, it is not specific to this PR.

@dkropachev dkropachev merged commit 4b53d10 into scylladb:scylla-4.x Oct 2, 2025
8 of 11 checks passed
@nikagra nikagra deleted the scylla-4.x-use-metadataId branch October 2, 2025 17:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants