Bazel 8 + rules_java 8 updates, protoc toolchain

[mbland]

Description

Bumps dependencies to versions that are compatible with both Bazel 7.6.0 and 8.1.1, and adds protocol compiler toolchainization in //protoc for protobuf v29 and later.

Closes #1652. Part of #1482.

• ScalaPB jars: 0.11.17 => 1.0.0-alpha.1
• rules_python: 0.38.0 => 1.2.0
• rules_cc: 0.0.9 => 0.1.1
• rules_java: 7.12.4 => 8.11.0
• protobuf: 21.7 => 30.1
• rules_proto: 6.0.2 => 7.1.0

Bazel 6 is officially unsupported as of this change and the upcoming rules_scala 7.0.0 release. Updates .bazelci/presubmit.yml to bump the 7.x build to last_rc.

Registers a precompiled protocol compiler toolchain when --incompatible_enable_proto_toolchain_resolution is True. Otherwise, register_toolchains("@rules_scala_protoc_toolchains//:all") toolchains is a no-op, as it will be empty.

scripts/update_protoc_integrity.py automatically updates scala/private/protoc/protoc_integrity.bzl. The protobuf patch is the git diff output from protocolbuffers/protobuf#19679, which also inspired the updates to scala_proto/scala_proto_toolchain.bzl. The proto_lang_toolchain call in the BUILD file generated by protoc/private/protoc_toolchain.bzl was inspired by the README from:

• https://github.com/aspect-build/toolchains_protoc/

Loads java_proto_library from com_google_protobuf, replacing the officially deprecated version from rules_java.

Adds the scala parameter to scala_toolchains() to control whether it instantiates the builtin Scala toolchains. Removes the if len(toolchains) == 0 check from _scala_toolchains_repo_impl. The Scala version check will now happen only when both scala and validate_scala_version are True, which is essentially how the previous API worked.

Updates to README.md, and updates to WORKSPACE and third_party/repositories files precipitated by the dependency updates, comprise the remainder of this change.

Motivation

We're no longer planning to support Bazel 6 in the next major release per @simuons's decision in:

• Support Bzlmod and add rules_scala to bazel-central-registry #1482 (comment)

The plan is now to land the Bazel 7 and 8 compatibility updates first, then land the Bzlmod change. This enables us to make only one new major version release, instead of two (whereby the first release would've continued supporting Bazel 6).

It turns out the two major version plan wouldn't've been possible. Bazel 8 and rules_java 8 require protobuf >= v29, but this bump caused Windows builds to break when compiling protoc in #1710. src/google/protobuf/compiler/java/java_features.pb.h, the path specified in the error message, doesn't exist until protobuf v25.0.

@crt-31 and I found that this was related to the Windows/MSVC 260 character file path length limit. What's more, the protobuf team plans to drop MSVC support specifically because of this path length limit.

The protocol compiler toolchain prevents protoc recompilation, which fixes the Windows breakage while making all builds faster. Since Windows builds break since at least protobuf v25, but protoc toolchainization requires v29, the version bump and the protoc toolchain must land together.

Disabling the default Scala toolchain via scala_toolchains(scala = False) avoids instantiating any builtin compiler JAR repos or validating the Scala version. This enables users defining custom Scala toolchains using their own JARs to still use other builtin toolchains. This was prompted by: #1710 (comment)

Removing the if len(toolchains) == 0 covers the case in the upcoming Bzlmod implementation whereby the root module may explicitly disable all builtin toolchains. This avoids potential breakage of the register_toolchains("@rules_scala_toolchains//...:all") call from the upcoming MODULE.bazel file. Removing the scala_register_toolchains() calls from the dt_patches/test_dt_patches*/WORKSPACE files proves that those calls were harmless, but ultimately unnecessary.

---

I tried several things to get protocol compiler toolchainization to work with protobuf v28.2, described below. However, each path only led to the same suffering described in the new "Why this requires protobuf v29 or later" section of the README.

I discovered along the way that protobuf v30 isn't compatible with Bazel 6.5.0 at all. I added an explanation to the "Limited Bazel 6.5.0 compatibility" section of README.md.

---

I experimented with using protobuf v28.2, rules_proto 6.0.2, and rules_java 7.12.4 and 8.10.0. I updated the protobuf patch for v28.2 with the following statements:

load("//bazel/common:proto_common.bzl", "proto_common")
load("@rules_proto//proto:proto_common.bzl", "toolchains")

_PROTO_TOOLCHAIN = "@rules_proto//proto:toolchain_type"
_PROTO_TOOLCHAIN_ATTR = "INCOMPATIBLE_ENABLE_PROTO_TOOLCHAIN_RESOLUTION"
_PROTOC_TOOLCHAINS = toolchains.use_toolchain(_PROTO_TOOLCHAIN)

def _protoc_files_to_run(ctx):
    if getattr(proto_common, _PROTO_TOOLCHAIN_ATTR, False):

I updated protoc/private/protoc_toolchain.bzl to use proto_common from rules_proto.

I also created a rules_proto 6.0.2 patch for proto_toolchain() to fix a "no such package: //proto" breakage:

 6.0.2 patch for `proto_toolchain()`:

```diff
diff --git i/proto/private/rules/proto_toolchain.bzl w/proto/private/rules/proto_toolchain.bzl
index a091b80..def2699 100644
--- i/proto/private/rules/proto_toolchain.bzl
+++ w/proto/private/rules/proto_toolchain.bzl
@@ -33,7 +33,7 @@ def proto_toolchain(*, name, proto_compiler, exec_compatible_with = []):

     native.toolchain(
         name = name + "_toolchain",
-        toolchain_type = "//proto:toolchain_type",
+        toolchain_type = Label("//proto:toolchain_type"),
         exec_compatible_with = exec_compatible_with,
         target_compatible_with = [],
         toolchain = name,

I tried adding combinations of the following --incompatible_autoload_externally flag values to .bazelrc`:

common --incompatible_autoload_externally=+@protobuf,+@rules_java

Nothing worked.

---

After the protobuf v29 bump, and before the ScalaPB 1.0.0-alpha.1 bump, scala_proto targets would fail with the following error:

ERROR: .../external/com_google_protobuf/src/google/protobuf/BUILD.bazel:23:14:
  ProtoScalaPBRule
  external/com_google_protobuf/src/google/protobuf/any_proto_jvm_extra_protobuf_generator_scalapb.srcjar
  failed: (Exit 1): scalapb_worker failed:
  error executing ProtoScalaPBRule command
  (from target @@com_google_protobuf//src/google/protobuf:any_proto)
  bazel-out/.../bin/src/scala/scripts/scalapb_worker
    ... (remaining 2 arguments skipped)

--jvm_extra_protobuf_generator_out:
  java.lang.NoSuchMethodError:
  'java.lang.Object com.google.protobuf.DescriptorProtos$FieldOptions.getExtension(com.google.protobuf.GeneratedMessage$GeneratedExtension)'
    at scalapb.compiler.DescriptorImplicits$ExtendedFieldDescriptor.fieldOptions(DescriptorImplicits.scala:329)
  [ ...snip... ]

java.lang.RuntimeException: Exit with code 1
    at scala.sys.package$.error(package.scala:30)
    at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44)
    at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
    at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
    at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39)
    at scripts.ScalaPBWorker.main(ScalaPBWorker.scala)

ERROR: .../external/com_google_protobuf/src/google/protobuf/BUILD.bazel:23:14
  Building source jar external/com_google_protobuf/src/google/protobuf/any_proto_scalapb-src.jar
  failed: (Exit 1): scalapb_worker failed:
  error executing ProtoScalaPBRule command
  (from target @@com_google_protobuf//src/google/protobuf:any_proto)
  bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/scala/scripts/scalapb_worker
    ... (remaining 2 arguments skipped)

[mbland]

Hmm, an odd failure on the Windows build:

ERROR: C:/tools/msys64/home/b/_bazel_b/xknd5zlq/external/com_google_protobuf/src/google/protobuf/compiler/java/BUILD.bazel:87:11: Compiling src/google/protobuf/compiler/java/java_features.pb.cc [for tool] failed: (Exit 2): cl.exe failed: error executing CppCompile command (from target @@com_google_protobuf//src/google/protobuf/compiler/java:java_features_bootstrap) C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.39.33519\bin\HostX64\x64\cl.exe ... (remaining 1 argument skipped)
--
  | external/com_google_protobuf/src/google/protobuf/compiler/java/java_features.pb.cc(6): fatal error C1083: Cannot open include file: 'google/protobuf/compiler/java/java_features.pb.h': No such file or directory

src/google/protobuf/compiler/java/java_features.pb.h most definitely exists in protobuf v29.3.

All the other builds seem to pass. If they do, I'll kick this with an empty commit to see if we can get Windows to play along, in case it was some weird transient error.

[mbland]

Yep, just windows. And my empty commit bump just failed in the same way. Hmm... why can't it find the #include file right next to the source?

[crt-31]

I looked at the windows issue... the issue is that the full include path for that file is over the char limit (260 chars).
I think its a cc_library issue, but still investigating when it got broken... I think it has to do with the param file limitations, but I'm still looking into it.

Also, I don't remember having to compile those protobuf tools locally before... I feel like they came precompiled before?

[crt-31]

OK, I found the related issue here: protocolbuffers/protobuf#12947. (I'm going to look into it more cause I use MSVC all the time and haven't run into these issues.)

I wonder if we can go back to using the prebuilt protobuf libraries, then we don't need to require the c++ compiler in order to use rules_scala.

[mbland]

@crt-31 Heh, you beat me to it...I snuck a look on my phone while out this evening and found the same issue. Got online to post it, but was too late. 😛 What's more, my next Bzlmod blog post is going to talk about how I had to patch rules_rust to work around Windows path length breakages.

And I saw the paragraph that you just responded to:

Since we don't have any good way to mitigate this issue, and because bazel+msvc usage is low compared to other ways to build on Windows, we are planning to drop support for bazel+msvc in Protobuf 34 (the Q1 2026 release). Bazel 30 already disables support for this combination, with a flag to reenable support for it which gives time to gather feedback on this as well as give folks a longer horizon to migrate their builds to either make+msvc or bazel+clang-cl which don't suffer from this artificial limitation without being stuck on Protobuf 29 or lower:

https://protobuf.dev/news/v30/#poison-msvc--bazel

The linked announcement makes reference to protocolbuffers/protobuf#20085: "Breaking Change: Dropping support for Bazel+MSVC".

But to your point above:

I wonder if we can go back to using the prebuilt protobuf libraries, then we don't need to require the c++ compiler in order to use rules_scala.

I did experiment with using prebuilt Protobuf toolchains in my proto-toolchainization branch in mbland/rules_scala back in October, when I was first experimenting with Bzlmodification. I think I may dust it off and play with it again in light of this development; it may also provide a performance win, since protobuf wants to recompile constantly. (I'm also not aware of what "going back to using prebuilt protobuf libraries" would involve. We're already using the prebuilt protobuf-java artifact; I think protobuf keeps wanting to recompile protoc.)

[mbland]

UPDATE: Please skip this comment and read my new comment about the more recent fully working and polished protoc toolchainization.

The good news is, I went ahead and implemented protoc toolchainization in the latest commit, and it works really well. Please check out what I did in 7b7ab2a and let me know what you think of the README update.

The bad news is, the Windows build is still broken in the same way. I'm pretty sure it's because protobuf v29.3 still uses native.java_proto_library, which still depends on @com_google_protobuf//:protoc: Update: Nope, it's not. It's because the precompiled toolchain wasn't registered early enough.

• https://github.com/protocolbuffers/protobuf/blob/v29.3/bazel/java_proto_library.bzl#L11-L16

def java_proto_library(**kwattrs):
    # Only use Starlark rules when they are removed from Bazel
    if not hasattr(native, "java_proto_library"):
        _java_proto_library(**kwattrs)
    else:
        native.java_proto_library(**kwattrs)

I believe this is what @fmeum is trying to fix in protocolbuffers/protobuf#19679 (which I borrowed from in scala_proto/scala_proto_toolchain.bzl).

@simuons Until this gets resolved one way or another, what do you think of making Windows builds optional for now? Then when a fix comes, we can make it required again. Or maybe I can hack something to downgrade the Windows build down to protobuf v21.7 again, as it was before this pull request (i.e., currently remains on master).

@crt-31 I know you really don't want that to happen, but it's effectively blocking the next rules_scala release at this point.

@fmeum If you've any better ideas, or my understanding is incorrect in any way, I'm all ears.

At any rate, this gives me time to think about maybe adding more tests for this latest protoc toolchainization, if we need them.

[mbland]

Hold the presses... I just pushed an experiment that fixed the Windows build. I created a patch from protocolbuffers/protobuf#19679, then added a line to WORKSPACE to register the prebuilt toolchain early. Success!

Everything else is failing right now (all the tests for all the other WORKSPACE repos I haven't modified), but I'll fix that up later. The main question is whether to actually commit this patch.

[mbland]

OK, please ignore my update from yesterday. I've learned a lot, including that the protoc failure was due to the new toolchain not getting registered early enough, not that protobuf was selecting native rules. The latest commit polishes the protoc toolchainization implementation, and the Windows build is passing again (along with all the others).

The majority of the changes in the PR are still the boilerplate updates to WORKSPACE and .bazelversion files, and automated updates to third_party/repositories/scala_*.bzl files. The most substantial changes are in:

• Updating Bazel versions in .bazelci/presubmit.yml
• Adding common --incompatible_enable_proto_toolchain_resolution to .bazelrc and tools/bazel.rc.buildkite
• README.md
• The new examples/overridden_artifacts repo and new test in test/shell/test_examples.sh
• The new //protoc package with the protoc toolchainization logic
• The protobuf patch in scala/deps.bzl from bazel: Remove hardcoded dependency on //:protoc from language runtimes protocolbuffers/protobuf#19679
• The slight update to scala/private/macros/scala_repositories.bzl
• Updates to scala/toolchains.bzl and scala/toolchains_repo.bzl for toolchainization
• Minor tweaks to scala_config.bzl
• scala_proto/scala_proto_toolchain.bzl updates based on bazel: Remove hardcoded dependency on //:protoc from language runtimes protocolbuffers/protobuf#19679
• Version bumps in scripts/create_repository.py
• The new scripts/update_protoc_integrity.py
• Adding allow_empty = True to a glob() in test/semanticdb/BUILD
• Dropping Scala 2.11 tests from test_thirdparty_version.sh and test_version.sh

Still, this is a lot more than I'd originally planned to include in this pull request. Some of these updates can probably be teased out into separate PRs.

And/or, we could go back to the original 7.x and 8.x release plan. This would mean backing out the protobuf upgrade from v21.7 to v29.3, and other updates along with it, most notably protoc toolchainization.

In either case, Bzlmod could land in 7.0 or 7.1. I'll prepare my Bzlmod branch, but it would remain a relatively light update on top of whatever changes we decide to include, and when.

Anywho, as usual, options... @simuons Let me know what you'd prefer to do here.

[mbland]

P.S. I can confirm that this branch is still compatible with Bazel 6.5.0, both with the C++ compiler flags and with protocol compiler toolchainization enabled, as well as Bazel 8.1.1.

I have protoc toolchainized Bzlmod builds working on my unstable 'bzlmod-enable' branch and both of my stable bzlmod and bzlmod-bazel-8 branches. Can confirm they work with Bazels 7.5.0, 8.1.1, rolling, and last_green.

# rolling
Starting local Bazel server (9.0.0-pre.20250210.1) and connecting to it...

# last_green
2025/03/02 22:24:18 Using unreleased version at commit 15a3526621559170c2a28e9dae5e9d7c72a82f02                                                                  
2025/03/02 22:24:18 Downloading https://storage.googleapis.com/bazel-builds/artifacts/macos_arm64/15a3526621559170c2a28e9dae5e9d7c72a82f02/bazel...

The thing that's at once a bug and a feature of the Bzlmod branches is the fact that I had to manually copy the protobuf patch and add a single_version_override to each nested module to enable protoc toolchainization. This is because you can only apply patches in the main module, and they must be regular files in the same module. In a pathological case like this, with having to apply a patch to all these nested modules, it's a drag. In a typical case, when you're working with a single module, it's nice that Bzlmod forces things to be so explicit and correct, as opposed to WORKSPACE macros doing mysterious and important work.

And of course, in the best possible worlds, protocolbuffers/protobuf#19679 will get merged, and we can upgrade to at least v29.4 and forget about this patching and single_version_override business.

At any rate, for now, the patch is helping everything work, and I've documented the hell out of it in the README.

[mbland]

I've just gone ahead and added these bumps as well, and propagated them to bzlmod-enable, bzlmod, and bzlmod-bazel-8:

• Go: 1.24.0 => 1.24.1
• Scalafmt: 3.9.1 => 3.9.2
• org.scala-sbt:compiler-interface: 1.10.7 => 1.10.8
• org.scala-sbt:util-interface: 1.10.7 => 1.10.10
• rules_java: 8.9.0 => 8.10.0

[mbland]

Just bumped:

• grpc: 1.70.0 => 1.71.1
• protobuf-java: 4.29.3 => 4.30.0
• protobuf: v29.3 => v30.0

I'm also about to try breaking out separate pull requests from this one, in case that will prove more helpful. Then we'll have the option of landing the different pieces in smaller chunks, or at least making more sense of the pile of changes in this PR.

[mbland]

@simuons and @liucijus Before you review, approve, and merge this pull request, consider reviewing and merging these others first, which I just extracted from this one:

• Bazel 7 and MSVC compatible version updates #1711
• Add examples/overridden_artifacts and test #1713
• 7.0.0 README updates and minor code changes #1714

Each of those is orthogonal to one another, and is much easier to understand. Merging them will render this pull request much smaller and more focused itself.

Also, what do you think of bumping the default Scala version from the current 2.12.20 to 2.13.16?

Finally, #1712 (which is not for review at this point) shows that even protobuf v25.5 breaks MSVC builds by default. I may play with that branch/pull request some more, but I think it's proving that upgrading from protobuf v21.7 very rapidly leads to the toolchainization solution in this PR.

[mbland]

@simuons @crt-31 Sorry for being so spammy today...but I did convince myself to add a commit to extract a separate scala_protoc_toolchains() macro. It feels like the right thing to do after all.

[mbland]

To be clear, with the latest commit, the protocol compiler toolchain API no longer requires loading and calling scala_toolchains(), but now requires:

# Must come after loading `platforms` and `com_google_protobuf`.
load("@rules_scala//protoc:toolchains.bzl", "scala_protoc_toolchains")

scala_protoc_toolchains()

The register_toolchains("@rules_scala//protoc:all") call much earlier in WORKSPACE remains the same.

[mbland]

@crt-31 @simuons OK, last API update, I swear... 🙂

I've now tried updating the bzlmod-enable branch, and landed on making this a pure repo rule. So now the API is this (as documented in README.md and at the top of //protoc:toolchains.bzl):

# WORKSPACE

# Register this toolchain before any others.
register_toolchains("@rules_scala_protoc_toolchains//...:all")

load("@platforms//host:extension.bzl", "host_platform_repo")

# Instantiates the `@host_platform` repo to work around:
# - https://github.com/bazelbuild/bazel/issues/22558
host_platform_repo(name = "host_platform")

# ...load `com_google_protobuf`, `rules_proto`, etc...

load("@rules_scala//protoc:toolchains.bzl", "scala_protoc_toolchains")

# The name can be anything, but we recommend `rules_scala_protoc_toolchains`.
# Only include `platforms` if you need additional platforms other than the
# automatically detected host platform.
scala_protoc_toolchains(
    name = "rules_scala_protoc_toolchains",
    platforms = ["linux-x86_64"],
)

No need to use scala_toolchains() if you don't need it.

I've already pushed the corresponding module extension to bzlmod-enable, as //scala/extensions:protoc.bzl. This really should do it, I think.

[crt-31]

@mbland, I'll try it out on monday.

[crt-31]

@mbland , Tried latest version from today on my codebase and it worked. (Note I'm not using scalapb, not calling scala_toolchains()).

Also, thanks so much for setting up mechanism for prebuilt protoc. I can't beleive they don't have prebuilt support in the rules_proto or protobuf repo anymore, but awesome that you did that.

• So is the idea that we would now just only use prebuilt protoc in the rules_scala repo? That would make it simpler. My understanding was that rules_scala 6.x used prebuilt protoc, so this might be just fine. If someone wants to build protoc themselves, they could do it themeselves outside of rules_scala and bind it in.
• If rules_scala uses prebuilt protoc, then wouldn't this reduce a lot of the ScalaPB/abseil/protobuf versioning complexity described in Protobuf compatibility findings and suggestions #1647 ?
• Do we still need absil-cpp dependency if we are using prebuilts? Also maybe remove rules_cc as well. (built fine for me when I commented them out).
• Maybe for future: rules_proto is deprecated, so I think we might be able to remove dependency on that repo.
• In the scenario that I make my own custom scala_toolchain() like I do, but then still wanna use scala_proto. If I call scala_toolchains() with everything false except scalapb=true, I get an error that it can't find the scala_reflect jar. How would I do this? Shouldn't it use the already configured toolchain to get the compiler jars? Or should I be able to make a custom scala_proto_toolchain?

[mbland]

@mbland , Tried latest version from today on my codebase and it worked. (Note I'm not using scalapb, not calling scala_toolchains()).

Awesome! 😄

Also, thanks so much for setting up mechanism for prebuilt protoc. I can't beleive they don't have prebuilt support in the rules_proto or protobuf repo anymore, but awesome that you did that.

Heh, thanks. But I view it less as awesomeness on my part than necessity, since the Windows build was dead in the water without it. I'm just lucky I dabbled in exploring precompiled protoc toolchains back in October, when I was just starting to figure out how to update rules_scala for Bzlmod. Dusting off my old experimental branch greatly expedited the process.

That said, there are hints that precompiled protoc support in protobuf itself is forthcoming:

• Protobuf News: News Announcements for Version 29.x
• Bazel: provide pre-built binary toolchain for protoc protocolbuffers/protobuf#19558 (comment)
• proto_library: Bazel does not use toolchain since v29.0 protocolbuffers/protobuf#20182 (comment)

• So is the idea that we would now just only use prebuilt protoc in the rules_scala repo? That would make it simpler. My understanding was that rules_scala 6.x used prebuilt protoc, so this might be just fine. If someone wants to build protoc themselves, they could do it themeselves outside of rules_scala and bind it in.

I wouldn't go that far yet, because consumers might not be prepared for or comfortable with using the precompiled protoc toolchain. This here pull request does enable the precompiled protoc toolchain by default for rules_scala development and CI, and makes it available to consumers, but doesn't force consumers to adopt it.

That said, I would imagine many, if not most, consumers might take advantage of the option once they realize it exists.

Also, FWIW, I'm not aware that rules_scala 6.x used a prebuilt protoc toolchain at all. protobuf didn't really begin to support it until v27.0 (protocolbuffers/protobuf#16323), which isn't compatible with any version of rules_scala, per #1647. I couldn't even get it to work with protobuf < v29, as I mention in the new "Why this requires protobuf v29 or later" section in README.md (from this PR).

• If rules_scala uses prebuilt protoc, then wouldn't this reduce a lot of the ScalaPB/abseil/protobuf versioning complexity described in Protobuf compatibility findings and suggestions #1647 ?

Sadly, no, not entirely, not as I understand it. The abseil-cpp "incompatibiliy" is whether or not the Bazel version uses a new enough C++ flavor by default while compiling protoc. Once you've got a protoc binary, prebuilt or compiled as a build dependency (possibly with explicit C++ compiler flags), the more significant incompatibility lies within the scala_proto implementation.

The scala_proto rules use ScalaPB libraries and the related protoc-bridge library to implement an aspect that compiles proto files into Scala. That aspect creates a Unix pipe (which I suppose MSYS2 emulates on Windows?), launches protoc, and uses the pipe to communicate between the protoc process and the aspect. Both ends of that pipe have to agree on the version of the protobuf-java runtime and how to interpret its features.

The good news is that the protobuf-java Maven releases are backwards compatible with earlier versions of protoc. A lot of the build hangs described in #1647 were fixed by bumping protobuf-java alone. The bad news is that not all of the problems were fixed by an easy protobuf-java bump. Newer versions of ScalaPB and protoc-bridge only support protobuf v28 and later. I don't know if future versions of ScalaPB and protoc-bridge will be as backward compatible as protobuf-java is, or if there'll be another compatibility gap like there is between ScalaPB 0.11.17 and 1.0.0-alpha1 (i.e., no version supports protobuf v25.6 through v27).

I would hope these protobuf related dependencies would become less version sensitive over time, but I wouldn't assume that we're there yet.

• Do we still need absil-cpp dependency if we are using prebuilts? Also maybe remove rules_cc as well. (built fine for me when I commented them out).

Technically, no, we don't still need them when using the prebuilt protoc toolchain. But we still need to keep them so that consumers that don't opt for the prebuilt protoc can still build with rules_scala successfully (i.e., can still recompile protoc as part of the build).

• Maybe for future: rules_proto is deprecated, so I think we might be able to remove dependency on that repo.

It is, but we can't remove it yet for two reasons: one hard requirement, and one nice to have. And in writing this, I realize I need to add this information to README.md, which I'll do in a new commit shortly.

The hard requirement is that the precompiled protoc toolchain implementation depends on the toolchains helper struct from @rules_proto//proto:proto_common.bzl. The equivalent toolchains helpers from rules_proto are currently private in protobuf, and first appeared in v29 (protocolbuffers/protobuf#17397).

The nice to have is that it enables both older and newer protobuf versions to still work with rules_scala by smoothing API changes between them. Even though we officially don't support protobuf versions earlier than v28 with this change, by keeping rules_proto for now, it's still possible to use v25.5 and earlier with rules_proto 6.0.2.

Specifically, this pull request ended up using doing the following in protoc/private/protoc_toolchain.bzl:

First, it loads the following directly from protobuf, since the protobuf_version.bzl file has contained PROTOC_VERSION since v21.0 (added in protocolbuffers/protobuf#9900).

load("@com_google_protobuf//:protobuf_version.bzl", "PROTOC_VERSION")

Then it loads the toolchains helper struct from rules_proto, because it's the same API from rules_proto 6.0.2 to 7.1.0. But check out the implementations:

• 6.0.2 wraps the Bazel native 'proto_common' constants itself
• 7.1.0 wraps @com_google_protobuf//bazel/common:proto_common.bzl, which doesn't exist until protobuf v27.0 (Add redirects for proto rules protocolbuffers/protobuf#16323).

load("@rules_proto//proto:proto_common.bzl", "toolchains")

Finally, it pushes the following load() statements back into the generated rules_scala_protoc_toolchains repo, since these files also don't exist until protobuf v27 (same protocolbuffers/protobuf#16323). Since we require v29 to even generate this package, it avoids breaking builds using earlier versions.

load(
    "@com_google_protobuf//bazel/toolchains:proto_lang_toolchain.bzl",
    "proto_lang_toolchain",
)
load(
    "@com_google_protobuf//bazel/toolchains:proto_toolchain.bzl",
    "proto_toolchain",
)

• In the scenario that I make my own custom scala_toolchain() like I do, but then still wanna use scala_proto. If I call scala_toolchains() with everything false except scalapb=true, I get an error that it can't find the scala_reflect jar. How would I do this? Shouldn't it use the already configured toolchain to get the compiler jars? Or should I be able to make a custom scala_proto_toolchain?

scala_proto_toolchain() should contain an argument like (from docs/scala_toolchain.md):

scala_library_classpath = [
    "@maven//:org_scala_lang_scala_library",
    "@maven//:org_scala_lang_scala_reflect",
],

If it doesn't contain a scala_library_classpath argument at all, it will default to creating one from the builtin toolchain repos. In that case, you'll need to set scala_toolchains(validate_scala_version = False, scala_proto = True), with scala = True left as the default. (I just updated docs/scala_toolchain.md earlier today to mention this.)

[mbland]

@crt-31 I assume this change is good to go now from your point of view?

@simuons I just extracted #1720 to trim this PR down again. Up to you if you want to merge #1720 first. If you merge this pull request first, I'll repurpose #1720 to only include the Guava updates (which aren't in this PR).

[mbland]

@simuons Just rebased on master after #1719 and #1720 and everything's passing.

[mbland]

Rebased on master after #1721.

[mbland]

Rebased on master after #1718.

[simuons]

Thanks @mbland and sorry it took longer than necessary.

[mbland]

@simuons Thanks, and no worries. We definitely straightened out a few things in the process.

I've rebased and updated my local bzlmod-enable branch and am running ./test_all.sh on it now. Once it passes, I'll post what could possibly be the final pull request for #1482. 🙂

That said, it will include changes to update the release workflow to publish to the Bazel Central Registry, replete with provenance attestations. (There'll be comments and a .bcr/README.md with references to explain this.) If you wanted me to have that in a separate pull request, I could break it out, since it's technically orthogonal to the rest of the change.