Skip to content

Kroxylicious 1.0 #82

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Kroxylicious 1.0 #82

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
7590872
Propose a v1.0 and use of patch versions
tombentley Oct 20, 2025
f5f28bb
Tweaks
tombentley Oct 20, 2025
93410d2
Apply suggestion from @hrishabhg
tombentley Oct 28, 2025
95b26c3
Apply suggestion from @tombentley
tombentley Oct 28, 2025
848d1eb
Apply suggestion from @gracegrimwood
tombentley Oct 28, 2025
e11a8b0
Changes for review comments
tombentley Oct 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 94 additions & 0 deletions proposals/007-kroxylicious-1.0.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# Kroxylicious 1.0 and patch releases

This proposal is for a Kroxylicious 1.0 release, and an improved commitment to making patch releases.

## Current situation

Kroxylicious already uses [Semantic Versioning][semver], but so far all releases have been `0.x.y` versions.

The project currently promises API compatibilty covering at "least 3 releases, and at least 3 months".

So far, we have done patch releases (i.e. releases like `0.x.1`) only when a new minor release with a fix is not already released, and not imminent.

## Motivation

Under semantic versioning, releases with a major number of 0 are allowed to break compatibility with previous releases.
The software released by project is maturing, and existing users would benefit from having a stronger guarantee about when compatibility between releases might be broken.

At this time there are already a number of users using Kroxylicious in production, however, some potential users might be put off from adopting Kroxylicious by the `0.` version number.
A 1.0 release would be a signal that we believe Kroxylicious is production-ready.

## Proposal

### What will have a compatibility guarantee?

We propose that the release that was going to be called `0.18.0` instead be called `1.0.0`.
By implication of Semantic Versioning, that means we would be committing to not breaking compatibilty until some future `2.0` release.
The time of such a `2.0` release is to be decided, but anticipated to be very approximately 12 months after a `1.0` release.

For the avoidance of doubt, this compatibilty guarantee would cover:

1. Binary compatibility of the Java APIs of the following modules:
* `kroxylicious-api`
* `kroxylicious-kms`
2. The format of the proxy configuration file, including the configuration of certain plugins provided by the project. Compatibility here means that a released version `1.y` would accept any configuration file that was accepted as valid in any released `1.x` version, where `y > x`. "Valid" means the proxy starts up cleanly.
3. The plugins covered under item 2. are all those implementations of the APIs listed in 1. provided by the project, excluding those implementations in test packages and modules. This implies the contined inclusion of these plugins in the `1.x` series of releases. However, the existing multi-tenancy plugin will be marked as deprecated.
4. The Kubernetes custom resource APIs defined by the kroxylicious-operator.

If the compatibility guarantee is broken we will treat that as a bug.
Copy link

@mimaison mimaison Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only covers compatibility between Kroxylicious versions. I couldn't see anywhere compatibility with regards to Kafka. My understand is that Kroxy always tries to adopt the latest Kafka dependencies. Is that true? Should that be documented? If so it's takes the same Kafka compatibility guarantees as Kafka.
So for example if Kafka 5.0 drops support for Kafka < 3.0 (just a random example, this is not based on actual plans), Kroxylicious would also drop support for Kafka < 3.0.


### Minor and patch branches

The project needs to strike a balance between:

* what the community can sustainably deliver to end users,
* what users would like.

The use of the term "the community" in the above is important.
The promises being made here are somewhat contingent on support from the community:
If you want a bug fixed, but the maintainers do not think it's a priority, you will need to fix it and backport it yourself if you want to see a fix in an official release.

We expect the process to work similarly to the Kafka project itself:

Fixes should always be made to the tip-most branch that is affected by the bug, and subsequently back-ported to the branches for other affected versions. Often, this will mean PRs should be opened against `main`, and back ported to the `1.x` branch, and eventually released under a `1.x.y` tag.

Under semver, patch releases only contain bug fixes. They do not contain new features.
We interpret "bug fixes" to include:

* Bug fixes in Kroxylicious itself, where the bug has been fixed, and back ported to the relevant patch branch.
* exploitable CVEs in dependencies and transitive dependencies, where a fixed version of the dependency has been released.
* exploitable CVEs in Docker images, including in a base image, where a fixed version of the base image has been released.

Note that new features merged to `main` will not be eligible to be backported to a minor release branch.

# Patch releases

Although described in terms of `1.x`, the following would apply `2.0` and future major versions.

We propose to provide patch releases for current `1.x`, plus up to the previous two minor releases. I.e. If the current release is `1.x.y` then we are prepared to release a `1.x.(y+1)`, `1.(x-1).z` and `1.(x-2).z`.
Copy link
Member

@SamBarker SamBarker Oct 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if the example would be better expressed as a table.

Copy link
Member

@SamBarker SamBarker Oct 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given what we say about API(java/config) stability why would we offer anything other than roll forward to the latest patch release?

I am very wary about building up a branch of un-released commits/fixes. While I accept releases have a cost, ours are fairly cheap and getting cheaper (we keep automating stuff) I think we should aim to release every minor release branch we start. However i'd also advocate for lazy branching around this. Only start a 1.x-1 branch if and when requires

Copy link
Member

@robobario robobario Oct 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bleeding edge is more or less the first time our new code comes into contact with the real world. It's a likely place for performance etc to degrade. Conservative Users may be putting that version through it's paces in test environments before adopting in production, so supporting a couple of minor versions could be a good idea.

Copy link
Member Author

@tombentley tombentley Oct 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SamBarker the language around the number of maintained minor releases was based on what I thought we had discussed and agreed out of band, but maybe I misunderstood. I added the 6 month thing off my own bat purely as a way of making the commitment less open-ended. I'm happy to change it if that's not the consensus we reached, but we (the maintainers on the hook for actually making good on this) all need to agree what we're willing to do here.

Personally I feel that just "roll forward" is not really good enough. It means that users have to accept the additional risk of taking a bunch of unrelated features in order to get a fix. That's just not always acceptable in production, and I think it therefore stands to harm adoption in production environments.

Therefore I would advocate for:

  • Creating a 1.x branch at the time we release 1.x.
  • Cherry picking to this branch. What gets cherry-picked is partly up to the community. If someone from the community it willing to open a charry-pick PR targeted at 1.x that's great, and we should endevour to get that into a release. This is why a 1.x branch needs to be created when releasing 1.x.0, so that anyone can just go ahead and open such a PR.
  • However, I don't think we should be doing a release simply because a single commit got made to the 1.x branch. The costs of a release are not just bourne by the maintainers who are clicking the buttons on some automation (however fancy). There is a cost for the whole community in terms of assessing whether or not to pick up the new 1.x.y release, and in actually rolling it out. It's no good the project pushing out a new patch release every week if users cannot keep up. We also want the community to participate in testing. For example, if Billy Bob opened a cherry-pick PR that's going to be included in a release then Billy Bob might well want to confirm that the proposed 1.x.y release fixes their problem. Bear in mind what's in the release might be more than just Billy Bob's PR, so it reasonable to want to be sure that none of the other fixes as regressed Billy's fix. Also it's possible that the maintainers haven't been able to reproduce Billy's problem (e.g. maybe it's hardware specific), so their feedback is invaluable.
  • For all these reasons it's I think it's inevitable that some number of fixes get merged to the 1.x branch, building up, before a release of 1.x.y happens. This is OK! We can still decide to expedite a release if a particularly terrible bug/CVE is found. But otherwise batching a bunch of fixes actually makes the eventual release more consumable, and more likely to actually be used (because the risk/benefit looks more favourable if a release fixes several problems rather than a single one). And releases being used is, after all, why we do them.

We need to decide how many 'live' release branches we want on the go. Obviously the effort here is multiplicative, so we need to be realistic. Apache Kafka generally manages to release two patch releases for each minor (see https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan), and they're usually spaced about 3-4 months apart. Kafka also aims for 3 a.b.0 releases a year. This averages out to there being 3 'next releases' at any one time: A .0, a .1 and a .2. Personally my gut feeling is we could sustain something similar, but note the implication that we slow down the pace on the .0 releases.

So then the final loose end is when and how we stop making releases from a minor branch. Here the Apache Kafka project doesn't really have anything written down. But in practice patches do get merged to minor branches which are never publicly released. However, they are available to those motivated to do their own building and testing. That seems like a reasonable balance to me: The project is not obliged to release them, but the community has the ability (and some motivation) to contribute those patches to the project.

cc @k-wall @gracegrimwood @robobario @hrishabhg, because this discussion is kinda central to the whole thing.

Copy link
Member

@robobario robobario Oct 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only start a 1.x-1 branch if and when requires

On this point I would value having the release branches created as part of the standard workflow so that we don't have diverging release flows (release latest off main, vs other releases off release branch). Having them exist before we need them will enable community members to PR backports directly to them without having to request a branch creation. It makes things more democratic and approachable if all the processes are in place and Contributors just have to review/approve things and pull the release levers.

Copy link
Member

@robobario robobario Oct 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also wondering if it would be manageable to have things more time based. So have a scheme like:

  1. the latest minor version is actively supported
  2. when a new minor version is created, on date X ,the previous minor version is moved to a different support level
  3. the previous minor now has support dates set based on date X, like 3 months of bugfixing and 6 months of security fixing for example.

This would give Users an easy to interpret table of dates for when a minor goes EOL. And users would know that they are getting at least that many months of support for the latest minor version. Under this scheme there would always be a transition period when we put out a new minor version where we support two versions, where I think with the current proposal if you didn't release for 6 months the prior minor version would be unsupported on day 0 of the new release.

current actively supported version : 1.4.x

old versions:

version    | bugfix support until       | security support until (EOL)   |
1.3.x      | 01-03-2026                 | 01-06-2026                     |
1.2.x      | 01-02-2026                 | 01-05-2026                     |
...

But this could put us on the hook for more than the proposed 3 minor versions of maintenance. It feels like, as Tom said, we will probably slow down the minors due to these new processes and guarantees though. We may end up following Kafka's cadence, to keep up we have to bump kafka-clients and take minor or major changes to the Filter API.

Maybe those concerns are manageable by keeping the lengths of time tight enough

Copy link
Member

@gracegrimwood gracegrimwood Oct 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This got suuuuper long, sorry 😅

Here's a sort of TL;DR of my thoughts:

  • We should be actively creating and maintaining new patch releases for the last two minor versions, as well as the current minor version (i.e. 1.x, 1.x-1, and 1.x-2). To limit the overhead, there should be no more than 3 actively maintained minor versions at any time.
  • The EOL date for the oldest active minor version 1.x-2 should be upon the release of the third subsequent minor version 1.x+1 or six months after the initial release of the expiring version 1.x-2, whichever comes first.
  • We shouldn't try to schedule the EOL or maintenance windows of releases ahead of time, so we don't overcommit ourselves. We could revisit this after we've reached 1.0.0 and tried out a less rigid maintenance schedule for a bit.
  • We should stop merging PRs to a release branch once that version is no longer receiving new patch releases; the purpose of these branches is to create patch releases to serve users of the proxy, anything beyond that is unnecessary extra work. We could revisit this later if there is demand for it.
  • We should have release branches for minor releases in addition to the existing release tags, and these should be created during their initial (minor.0) release by our release automation. These branches can then accept PRs for backporting fixes, and new patch releases for their respective versions can be made from those branches. Patch releases only need release tags, not branches.
  • Backports should be community-driven via raising PRs for backports against active release branches. Maintainers would be responsible for approving these PRs, backporting high-severity security fixes, and for cutting patch releases.
  • The timing of patch releases should be driven by the community through a proposal process (separate from this repo), where a member of the community proposes a new patch release and others can vote on it. Urgent patch releases (such as for high-severity security issues) should bypass this. If we find we are creating patch releases too often, we could look at implementing rules to limit this.
  • Maintaining multiple minor releases might mean we're inclined to put out minor releases less often but it shouldn't mean the pace of development is meaningfully slower. Also, being more mindful about what we're releasing and how often might not be a bad thing.
  • Maintaining multiple minor releases will mean more things for us to review and more times we have to run the release workflow, but that would happen to some extent anyway as the project grows.
  • Not providing any active maintenance for prior releases and instead taking a stance of telling users to just roll forward to the latest version risks alienating a significant number of people who might otherwise use Kroxylicious, and isn't in the best interests of the project.
  • We really should not be talking about any of this as "support". It muddies the water as to what it is we're trying to accomplish as a project, so we should use the term "active maintenance" or just "maintenance" if we can.

the language around the number of maintained minor releases was based on what I thought we had discussed and agreed out of band

We discussed the idea of maintaining and issuing patches for 1.x, 1.x-1, and 1.x-2, but I don't know if we actually reached an agreement. Either way, it sounds like a good number to me; any less isn't really worth it, any more could become unsustainable.

I added the 6 month thing off my own bat purely as a way of making the commitment less open-ended.

I think 6 months is fine as a sort of hard-coded "expiry date" for minor releases. We currently release much more frequently than that, so I reckon it's unlikely any release would hit that deadline anyway. Even if we did let the oldest minor release "expire", having another minor release in there between oldest and latest means users still have a buffer zone to work in, and 6 months seems like plenty of time to arrange a proxy upgrade to a new minor version. We can revisit that number later on if we need to, but I think it's fine.

Personally I feel that just "roll forward" is not really good enough. It means that users have to accept the additional risk of taking a bunch of unrelated features in order to get a fix. That's just not always acceptable in production, and I think it therefore stands to harm adoption in production environments.

I fully agree. We can't know every user's environment and risk profile, and we risk alienating potential users if our response to requests for backported fixes is to just install the latest version. I don't think that stance is in the best interests of the project, even if it would mean less maintenance work for us. Besides, if it was feasible for everyone to just roll forward all the time, nobody would make any money selling LTS versions of things 😉

  • Creating a 1.x branch at the time we release 1.x.
  • Cherry picking to this branch. What gets cherry-picked is partly up to the community.

Agreed, starting out with a major.minor minor release branch at initial major.minor.0 release time to serve as the source of all future major.minor.x patch releases is the way to go. As @robobario suggested, let's have our release automation create it for us, perhaps when it creates the release tag? Then we let the community put up PRs against these minor release branches with backported fixes, and our job is mostly to approve or not, as we currently do with PRs to main.

Patch releases can then be tagged in git as usual but won't get their own release branches. We shouldn't put out patches of patch releases like major.minor.patch.x, as per semver, so any fixes should instead be backported to the respective minor branch and then released in the next patch release of that minor version. For example, if someone wanted to backport a fix to release major.minor.patch then instead they would raise a PR against the major.minor release branch, and that would get released as major.minor.patch+1.

I think there's still definitely a bigger role for us as maintainers in this process, like urgent backporting of high-severity exploitable CVEs, but otherwise I think this process really should be driven by the people using the proxy on the daily, as they're best placed to know which fixes they need and don't need. We can step in when something that isn't a fix (i.e. a feature) gets suggested, for anything urgent, or for anything that's too complicated to backport, but for the most part I see this as the kind of system that starts to run itself.

  • However, I don't think we should be doing a release simply because a single commit got made to the 1.x branch. The costs of a release are not just bourne by the maintainers who are clicking the buttons on some automation (however fancy). There is a cost for the whole community in terms of assessing whether or not to pick up the new 1.x.y release, and in actually rolling it out. It's no good the project pushing out a new patch release every week if users cannot keep up. [...]
  • For all these reasons it's I think it's inevitable that some number of fixes get merged to the 1.x branch, building up, before a release of 1.x.y happens. This is OK! We can still decide to expedite a release if a particularly terrible bug/CVE is found. But otherwise batching a bunch of fixes actually makes the eventual release more consumable, and more likely to actually be used

Yeah, batching fixes is the way to go. As for timing, I think the we should handle this through proposals, honestly. Maybe not anything as big or elaborate as what we do here in this repo, but having a forum or something (like Slack or GitHub discussions or just raising an issue with a special tag on it) where community members can suggest that they think it's time for a patch release of whatever active minor version and then others can vote on that idea. Obviously for anything high-risk we could just go ahead with creating a patch release anyway, but for everything else this lets us take a more hands-off approach.

If, by some chance, that proposal process gets out of hand and folks do start requesting new patch releases every week, then we could look at instituting some kind of limit on how often we will release new patch versions, but that's the kind of bridge we should cross if/when we get to it. I think trying to come up with a schedule for this is a bit of a fool's errand because there will be some times we get to a patch release date and there's nothing worth releasing, and some times when something will show up well ahead of schedule and we'll want to put patches out ASAP. We're better off letting the people who use the proxy determine (for the most part) when new patch releases are required.

We need to decide how many 'live' release branches we want on the go. Obviously the effort here is multiplicative, so we need to be realistic.

I think if we stick to only having branches for minor releases then the number of actively maintained "live" branches comes down to how many minor releases we intend to create new patch releases for. Like I said above, I think the current and the two previous minor releases (1.x, 1.x-1, and 1.x-2) is a good number. I don't think it means we will have to slow development of new features (.0's) because we'd be leaning on the community to do most of the heavy lifting as far as maintenance effort goes, but it might change how often we cut new releases from main and make us think more carefully about what goes into them, which isn't necessarily a bad thing. And of course, all of this will mean more things for us to review and more releases, but we're going to get more things to review and more stuff to release as our contributor base grows anyway.

We'll also have to keep in mind what happens when we hit a new major version, but I don't think that's an unnavigable problem, nor a particularly pressing one at this juncture.

So then the final loose end is when and how we stop making releases from a minor branch. Here the Apache Kafka project doesn't really have anything written down. But in practice patches do get merged to minor branches which are never publicly released. However, they are available to those motivated to do their own building and testing. That seems like a reasonable balance to me: The project is not obliged to release them, but the community has the ability (and some motivation) to contribute those patches to the project.

I've sort of said this already, but if we're only promising patch releases for the previous two minor versions, then the answer to "when do we stop releasing patches for 1.x-2?" is "once we've released 1.x+1", or at that 6 month expiration date, whichever comes first. Obviously, if we're churning out a new minor release every week then we're going to need to reassess both how many minor versions we're maintaining and how long we're maintaining them for. Though, if we were in a position to be churning out minor releases that frequently I think this whole discussion would be very different.

I think "once we've released 1.x+1" also answers the question of "when do we stop accepting PRs to the release branch for 1.x-2?", because if it's not going in a patch release then it doesn't need backporting IMO. The point of this is to make releases and upgrades of Kroxylicious safer, more stable, and more reliable for the people who use it, so merging backports that aren't going to end up in releases seems a bit like wasted effort. In saying that, there is some reasonable demand for this in the future, and we feel it wouldn't add any unreasonable burden on us to maintain, then we could do so. If we draw a hard line now we can always bend it later if we need to, but I don't think we need to do so pre-emptively.

I'm also wondering if it would be manageable to have things more time based. So have a scheme like:

  1. the latest minor version is actively supported
  2. when a new minor version is created, on date X ,the previous minor version is moved to a different support level
  3. the previous minor now has support dates set based on date X, like 3 months of bugfixing and 6 months of security fixing for example.

This would give Users an easy to interpret table of dates for when a minor goes EOL. And users would know that they are getting at least that many months of support for the latest minor version.

This is what NodeJS does and, to an extent, what Ubuntu does something something pro subscriptions and paid support tiers.

I don't dislike this model (quite the opposite actually, as a user it's great) but it feels very strict and rigid given where the project is at currently, and I just don't feel that it's the right fit for us right now. I think it also implies a level of "support" (I don't mean to nitpick on language but I really don't like using this word here, I think we should be calling this "maintenance" because that's what we're actually proposing to do) that I don't think we're trying to offer. It is not within our scope as an open source project to be offering —or appearing to offer— anything close to enterprise support, which is why I think we should be talking about "actively maintained" releases rather than "supported" releases.

If we are wanting some kind of tapered maintenance model then we could say something like "version 1.x-1 gets all the backports while version 1.x-2 only gets CVE fixes until we release 1.x+1", but trying to come up with a schedule for this at this stage isn't really feasible because we have no way of knowing how realistic these kinds of timelines are yet. We can guess, but we don't actually know. We aren't Kafka or NodeJS or Ubuntu; our development effort is not their development effort, our maintenance burden is not their maintenance burden, and we can only model ourselves off of them to a point. A system like this, with stricter timelines and more assurance for users, is the kind of thing we could revisit in the near future once we've actually reached 1.0.0 and tried out something a little more flexible, at which point we'll have a better idea of how much extra work we've actually created for ourselves and what kinds of timelines are actually practical.

Anyway, I'm off to bed. If you read to the end of this: here, have a cookie 🍪

Copy link

@hrishabhg hrishabhg Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it be more like this? (current version + 2 patches or current version, (current version - 1) patch and (current version - 2) patch.

Suggested change
We propose to provide patch releases for current `1.x`, plus up to the previous two minor releases. I.e. If the current release is `1.x.y` then we are prepared to release a `1.x.(y+1)`, `1.(x-1).z` and `1.(x-2).z`.
We propose to provide patch releases for current `1.x`, plus up to the previous two minor releases. I.e. If the current release is `1.x.y` then we are prepared to release a `1.(x-1).(y+1)` and `1.(x-2).(y+2)`.

OR

Suggested change
We propose to provide patch releases for current `1.x`, plus up to the previous two minor releases. I.e. If the current release is `1.x.y` then we are prepared to release a `1.x.(y+1)`, `1.(x-1).z` and `1.(x-2).z`.
We propose to provide patch releases for current `1.x`, plus up to the previous two minor releases. I.e. If the current release is `1.x.y` then we are prepared to release a `1.x.(y+1)` and `1.x.(y+2)`.

Copy link
Member Author

@tombentley tombentley Oct 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Holding off any change here while we work though the conversation above.

However, the "up to" is limited to 6 months after the initial minor release.
Copy link
Member

@robobario robobario Oct 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a distinction between the current latest minor and the 2 prior ones here? I'm wondering if somewhere we should say the latest minor will be bugfix supported beyond 6 months.

So if `1.(x-2).0` was released more than 6 months ago, there is no promise to release a `1.(x-2).z`; it will be at the maintainers' discretion.
This is not a commitment to either:
* fix any particular bug found in `1.x`
* nor to release back-ported fixes immediately, because releases have a cost.

When enough bugs have been fixed to justify the effort of doing a release, we will start a release process, creating a patch branch from the tip of the minor branch.

Just because some fixes have been merged to a given patch branch does not imply a patch release needs to be made. For example, if it's been more than 6 months since the initial release from that minor branch.

Users, or potential users, for whom the above commitments are insufficient are advised to find a commercial software vendor who can provide the necessary level of support for Kroxylicious.

## Affected/not affected projects

This proposal concerns the kroxylicious/kroxylicious github projects, and the releases that originate from it.

## Compatibility

This is the subject of this proposal and has already been described, above.

## Rejected alternatives

The only alternative is to stick with `0.x` numbering for some amount time.
At the end of the day, the move to 1.0 is more of a subject judgement call than something based on objective criteria.

[semver]: https://semver.org/