Digest dynamic/async imports in javascript assets

[tomtaylor]

esbuild supports ES6 dynamic module imports, but they're not currently compatible with the mix phx.digest process as the import paths aren't digested.

This PR adds support for digesting any async and promise based imports inside a JS file, so a developer can use these without giving up cache busting, etc. Dynamic imports are great for front end performance as expensive libraries can be loaded on-demand rather than up front.

Before this PR, the following code would be the same after mix phx.digest runs. A digested file will load a non-digested other-module.js, and so not benefit from cache busting, etc.

async function onClick() {
  const { usefulFunction } = await import("./other-module.js");
  usefulFunction();
}

After this PR mix phx.digest will switch the import path for the digested path, e.g.

async function onClick() {
  const { usefulFunction } = await import("./other-module-2ddf92b4c94e25bd296b2df90848a564.js");
  usefulFunction();
}

Note: esbuild in Phoenix doesn't use async imports out of the box, it need to be enabled by changing the args to include following flags: --chunk-names=js/chunks/[name]-[hash] --splitting --format=esm. (Technically, it does, but the promise just resolves instantly to content that's in the same file.) I think there's an argument for changing the default flags to do this, but I understand if that might add more complexity than desired to the default asset pipeline.

I think there is value in supporting dynamic imports in the digester even if we don't flip the defaults, because it allows users who want to implement their own esbuild config to benefit from splitting + dynamic imports, without having to throw out the Phoenix digesting process entirely. Throwing out the Phoenix digest process is fairly annoying, as it means finding a solution for digesting the non-JS assets, which esbuild doesn't really support.

This PR doesn't digest any static chunk imports, eg. import { foo } from "./chunks/chunk-abcdef.js". These already have a hash appended and so the benefit would mostly aesthetic, but it's probably still worth implementing to minimise any surprises.

Anyway, let me know what you think and whether this is something you'd like to add to Phoenix. If there's support for it, I suggest I add some documentation, support for chunk imports and handle the optional options argument that import() can support.

[SteffenDE]

I think I remember esbuild also automatically appending hashes to dynamic imports in one project I worked on using a config like you've shown. I cannot check at the moment though.

This approach feels a bit brittle with the regex. Currently, it wouldn't detect the import when it spans multiple lines, I think:

function promiseImport() {
  import("./app.js")
    .then((app) => {
       console.log(app);
    });
}

So I'm not sure it wouldn't be better to just leave that job to esbuild and maybe enable the splitting config by default for new projects?

[tomtaylor]

I think I remember esbuild also automatically appending hashes to dynamic imports in one project I worked on using a config like you've shown.

esbuild will add a checksum to e.g. chunk-ABCDEFGH.js files, where it's deciding the name of the chunk based on how it decides to collate the imports, however it won't do that for dynamic imports where a named file is being imported.

This approach feels a bit brittle with the regex. Currently, it wouldn't detect the import when it spans multiple lines, I think:

Agree it's not ideal (the 'proper' way would be to use an AST transformation). But then neither is what we're already doing with source maps. I think the Regex could catch all the common scenarios, including line breaks, and there's a natural place to extend the test cases we already have.

[SteffenDE]

Just tested:

diff --git a/assets/js/app.js b/assets/js/app.js
index d5e278a..c5965d7 100644
--- a/assets/js/app.js
+++ b/assets/js/app.js
@@ -22,10 +22,13 @@ import {Socket} from "phoenix"
 import {LiveSocket} from "phoenix_live_view"
 import topbar from "../vendor/topbar"
 
+import Hooks from "./hooks"
+
 let csrfToken = document.querySelector("meta[name='csrf-token']").getAttribute("content")
 let liveSocket = new LiveSocket("/live", Socket, {
   longPollFallbackMs: 2500,
-  params: {_csrf_token: csrfToken}
+  params: {_csrf_token: csrfToken},
+  hooks: Hooks
 })
 
 // Show progress bar on live navigation and form submits
diff --git a/assets/js/dynamic.js b/assets/js/dynamic.js
new file mode 100644
index 0000000..fdc6510
--- /dev/null
+++ b/assets/js/dynamic.js
@@ -0,0 +1,3 @@
+export default {
+  hello: "World"
+}
diff --git a/assets/js/hooks.js b/assets/js/hooks.js
new file mode 100644
index 0000000..70690c8
--- /dev/null
+++ b/assets/js/hooks.js
@@ -0,0 +1,9 @@
+export default {
+  MyHook: {
+    mounted() {
+      import("./dynamic.js").then(() => {
+        console.log("loaded dynamic import!")
+      }) 
+    }
+  }
+}
diff --git a/config/config.exs b/config/config.exs
index c8f1882..0b47363 100644
--- a/config/config.exs
+++ b/config/config.exs
@@ -36,7 +36,7 @@ config :esbuild,
   version: "0.17.11",
   esbuild_sample: [
     args:
-      ~w(js/app.js --bundle --target=es2017 --outdir=../priv/static/assets --external:/fonts/* --external:/images/*),
+      ~w(js/app.js --chunk-names=chunks/[name]-[hash] --bundle --splitting --format=esm --target=es2017 --outdir=../priv/static/assets --external:/fonts/* --external:/images/*),
     cd: Path.expand("../assets", __DIR__),
     env: %{"NODE_PATH" => Path.expand("../deps", __DIR__)}
   ]

$ mix esbuild esbuild_sample

  ../priv/static/assets/app.js                      245.4kb
  ../priv/static/assets/chunks/chunk-TLI7THVW.js      2.5kb
  ../priv/static/assets/chunks/dynamic-CNW7OOQS.js    133b 

⚡ Done in 19ms

// js/hooks.js
var hooks_default2 = {
  MyHook: {
    mounted() {
      import("./chunks/dynamic-CNW7OOQS.js").then(() => {
        console.log("loaded dynamic import!");
      });
    }
  }
};