Replace swc_core with direct dependencies for heavy Rust-side users

[kdy1]

Status	Last Updated	Target version
N/A	Sep.3.2023	N/A

Summary

• Use swc_core for lightweight Rust-side users, like plugin authors.
• Heavy Rust-side users should use direct dependencies.
• The team will provide a general CLI tool for upgrading SWC crates.

Motivation

swc_core is very good for plugin authors, but clearly, it's very bad for heavy users from the Rust side. It's convenient, but it's a known bottleneck for compile time. All used SWC crates must be compiled before swc_core, and all dependant crates, should be built after swc_core.

This imposes one more compile time issue for development. If you update swc_core because you need a bugfix of swc_css_parser, your crates depending on swc_ecma_ast should be built again because swc_core is built again.

Detailed design

Version solver

We can create a CLI tool for version solving.

cargo ddt solve-version -w [email protected]

• This will run cargo-metadata internally to get a list of crates to check compatibility. Note that the current versions of crates are ignored by default. This can be respected in e.g. minimal-upgrade mode, but not now.

• -w can be specified multiple times. If specified, the versions of crates in the printed answer will be compatible with all packages.

• If there's a conflict, the command fails.

This can be also used for partial upgrade of SWC, like cargo ddt solve-version -w [email protected].
This will print crates compatible with [email protected]. In many cases, multiple versions of swc_ecma_ast maps to one version of swc_common, so a user can upgrade swc_ecma_ast without upgrading swc_common.

--upgrade

-u flag can be used to update all resolved versions.

--json v1

This tool should allow the dumping of everything with --json v1, so it can be extended easily with jq and terminal pipe.

Drawbacks

More work, again.

Alternatives

Unresolved questions

[kdy1]

cc @kwonoj Thoughts?

[kwonoj]

1. One of the biggest reasons for having swc_core is to resolve version conflict for transitive dependencies, not direct dependencies.

2. We used to use a direct dependency model for several places, including swc itself + next-swc/turbopack, however after adapting swc_core build time didn't increase hugely if I recall correctly - instead it grows over time gradually as we add more codes / features.

I'm generally not great in favor of introducing another tool to manage version conflict, swc_core was supposed to do serve as resolver for the versions as well as single sdk to build swc.

If we want to go back to the direct dependency model, have we ever considered different version modeling entirely? Lot of version conflict coming from individual pkg have individual versions. Instead, a single change bumps up all of swc_* pkg at once to same version and each dependency relies on strict pinned version potentially removes conflict. I think some of large rust codebase actually uses similar approach.

[kdy1]

1. The problem of transitive deps can be solved by the version solver.

2. Actually, it increased a lot. I did various compile time optimizations at similar times and that's why the compile time of turbopack wasn't increased significantly. Even though I did lots of optimizations, the compile time of Parcel actually increased a lot.

A single version for all packages is even worse. It means recompilation of all swc crates and all dependant crates on every dependency update.

[kdy1]

Actual compile time difference of turbopack:

• Depend directly on swc crates vercel/turborepo#5879

1m 53s (old) vs 1m 43s (new)

Thoughts?
I also had to patch swc-project/plugins to depend on relevant crates directly.

[kdy1]

Timing difference for swc/plugins:

• Previous: 2m41s https://github.com/swc-project/plugins/actions/runs/6143566537/job/16667274344
• New: 2m4s https://github.com/swc-project/plugins/actions/runs/6154346757/job/16699586097