Skip to content

[Janus] P1: repack the wasi-threads toolchain artifact with the box's clang builtin headers (v2 grafts musl headers; wint_t/wchar_t undeclared) #11

Description

@TJKoury

Graph task: sdk-isomorphic-toolchain-sysroot-artifact · state: open · priority: 1 · components: [module-sdk, sdn]
Agent: Janus

Defect

space-data-network kubo/sdn/flowcc/toolchain/build-threads-bits.sh:75-84
grafts iso646.h stdalign.h stdarg.h stdbool.h stddef.h stdnoreturn.h tgmath.h __stddef_max_align_t.h from the emscripten sysroot's top-level include into
sysroot-wasi-threads/include/. Those are musl libc headers. clang's builtin
include dir (the one that carries __need_wint_t / __stddef_wint_t.h /
__need_wchar_t) is never staged, and the box's driver emits
-internal-isystem lib/clang/16.0.0/include — a RELATIVE path with nothing
behind it in the box FS. Measured search list inside the box:

#include <...> search starts here:
 /sysroot/include/wasm32-wasip1-threads
 /sysroot/include
End of search list.

Result: wctype.h:27: error: unknown type name 'wint_t' and
stdlib.h:85: error: unknown type name 'wchar_t'. Every TU that reaches
<wctype.h> or <stdlib.h> — i.e. essentially every C++ TU and most C TUs —
cannot be compiled by the boxed toolchain. The comment at
build-threads-bits.sh:12-16 blames clang-16 and libc++; the real cause is
this graft. flow_runtime.threads.o ships as a prebuilt native object because
of it.

Required

  1. Stage the box's OWN clang builtin include dir at the resource-dir path the
    box driver reports (clang -print-resource-dir → lib/clang/<ver> — treat
    the relative path as a defect too and pass -resource-dir explicitly, as
    wasiThreadsToolchain.js:133-136 already does natively).
  2. Repack as toolchain v3: new SHA256SUMS with a
    CLANG_BUILTIN_INCLUDE_ROLLUP_SHA256 alongside the existing
    WASI_THREADS_SYSROOT_ROLLUP_SHA256; stage-toolchain.sh asserts it.
  3. Guardrail test: compile a C++17 TU that includes <vector> <string> <cmath>
    AND a C TU that includes <stdlib.h> <wctype.h> through the box; both must
    emit an object. Run it in the SDK toolchain suite, not only in kubo.
  4. Re-evaluate whether flow_runtime.threads.o still needs to ship prebuilt.
    If the box can compile it, the bake stops shipping a native-built object as
    staged data and the isomorphism argument gets simpler.

Evidence

lab/isomorphic-toolchain/cxxfix.mjs and bigcxx.mjs (SDK branch
janus/isomorphic-toolchain-spike, commit ccac68d) reproduce both the
failure and the fix. With builtin headers grafted:
foundation/orbits/src/orbits_module.cpp (177 KB C++17 + flatbuffers) compiles
in 2,629 ms cold / 1,531 ms warm, 57.6 MB guest heap, deterministic across
runs, object 125,806 B vs native 22.1.2's 124,082 B.

Cross-repo: the artifact lives in space-data-network, the consumer contract is
the SDK's. Coordinate with Hermes before touching kubo/sdn/flowcc/toolchain/.
Mirrored from the dev graph (graph/tasks/sdk-isomorphic-toolchain-sysroot-artifact.md in DigitalArsenal/spacedatanetwork-stack); closes automatically when the graph task completes.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions