KAFKA-18839: Drop EAGER rebalancing support in Kafka Streams

[ableegoldman]

In 3.1 we deprecated the eager rebalancing protocol and marked it for removal in a later release. We aim to officially drop support and remove the protocol from Streams in 4.0. Note that we are only removing the protocol itself from Streams for now, and will realize the actual benefit from dropping EAGER by cleaning up the task assignment & lifecycle management in a future PR.

The effect of this PR is that it will no longer be possible to perform a live upgrade Kafka Streams directly to 4.0 from version 2.3 or below. Users will have to go through a bridge release between 2.4 - 3.9 instead.

Note that several other incompatibilities that depend on the upgrade.from config still remain, which require it to be used when upgrading from [2.4, 3.4] to 3.5 or above. For this reason we recommend using a version in [3.5, 3.9] as the bridge release, so that only 3 rolling bounces are needed to upgrade. The recommended path is as follows:

1. apply the upgrade.from config and perform the 1st rolling bounce from 2.3 or below to a version in [3.5, 3.9]
2. perform a 2nd rolling bounce to remove the upgrade.from config but without changing the version
3. perform the 3rd a final rolling bounce to upgrade to 4.0

[mjsax]

Seems we would need to also check 2.5, ... 3.9, not just 2.4 ?

[ableegoldman]

we do, it loops over all the versions starting from the incompatible upgrades, we don't only check 2.4 it's just that 2.4 is where we switch from asserting that an exception is thrown to asserting that it can be instantiated without throwing

[mjsax]

Yes, we loop, but when upgradeFrom changes, should the condition not be:

if (upgradeFrom.toString().equals("2.4")
    || upgradeFrom.toString().equals("2.5")
   ...
    || upgradeFrom.toString().equals("3.9")
    // need to add a new version here for every new release    
    ) {

Otherwise we set beforeCooperative = false only for a single loop iteration when upgradeFrom = 2.4 ?

What actually makes me suggest to flip from logic and initialize beforeCooperative = false, and set it to true for versions 0.10.0 to 2.3 to avoid that we need to keep updating this test for very future release (what we would most likely forget...)

[ableegoldman]

nope because beforeCooperative is declared outside the loop so once it gets updated it stays on the new value..? am I going crazy here?

I mean the testing is passing so...

[mjsax]

\cc @dajac for visibility -- we want to get this into 4.0 release.

[ableegoldman]

a little confused about why 2.1 - 2.3 were not included in this matrix...?

[ijuma]

I think this change means we can also revert all the changes introduced via d1952e8 cc @FrankYang0529