Skip to content

feat: add .include directive for multi-file assembly#109

Open
marcelofeitoza wants to merge 3 commits intoblueshift-gg:masterfrom
marcelofeitoza:feat/include-directive
Open

feat: add .include directive for multi-file assembly#109
marcelofeitoza wants to merge 3 commits intoblueshift-gg:masterfrom
marcelofeitoza:feat/include-directive

Conversation

@marcelofeitoza
Copy link
Copy Markdown
Contributor

@marcelofeitoza marcelofeitoza commented Mar 5, 2026

Add .include directive for multi-file assembly

Summary

Adds support for .include "path" in sBPF assembly, allowing programs to be split across multiple files. The directive is expanded at build time by inlining the referenced file's contents before assembly.

Motivation

The sbpf CLI compiles a single .s file per program. Labels defined in other files (e.g. custom_log, shared utilities) are never resolved, causing "unsupported BPF instruction" or undefined symbol errors when trying to call across files. This change enables modular assembly projects without external tooling.

Changes

  • src/commands/build.rs: Added expand_includes() that:

    • Parses .include "path" (path in double quotes)
    • Resolves paths relative to the file containing the directive
    • Recursively expands nested includes
    • Inlines file contents before passing to the assembler
  • tests/utils.rs: Added write_include_file() helper for integration tests

  • tests/test_include.rs: New integration tests:

    • test_include_directive – main file includes log.s, builds successfully
    • test_include_nested – nested includes (main → log.s → log_impl.s)
    • test_include_missing_file_fails – missing include produces clear error

Syntax

.include "path/to/file.s"
  • Path is relative to the directory of the file containing the directive
  • Supports nested includes (included files may use .include)
  • Invalid or missing paths: Failed to read include 'path': <io error>

Example

main.s:

.globl entrypoint
.include "log.s"
.include "messages.s"
.text
entrypoint:
  lddw r1, greeting
  mov64 r2, 7
  call custom_log
  exit

log.s:

.global custom_log
custom_log:
  call sol_log_
  exit

Checklist

  • cargo fmt --all passes
  • cargo clippy --all-targets --all-features -- -D warnings passes
  • cargo test --workspace --all-features --all-targets passes

@alnoki
Copy link
Copy Markdown

alnoki commented Mar 5, 2026

@marcelofeitoza this is awesome!

@clairechingching plz accept, I'll use this

@deanmlittle
Copy link
Copy Markdown
Collaborator

this is a well-established pattern in gcc and i'm in favor. main thing is, it would be our first major divergence from solana toolchain behavior. might be worth creating an equivalent patch to the shitty toolchain nobody uses just to maintain compatibility. assembly macros would be another such change that would improve devex a lot.

@deanmlittle deanmlittle added the enhancement New feature or request label Mar 5, 2026
@deanmlittle
Copy link
Copy Markdown
Collaborator

deanmlittle commented Mar 5, 2026

@marcelofeitoza In discussing with @clairechingching, we probably need some kind of solution to not need the .globl in the include file. The main reason being, we can't have multiple .globl symbols or the concept of an entrypoint gets confused. We can define alternative symbol names in included files, but there isn't really any benefit to adding additional symbol names to the binary. We probably need to consider some kind of way around this. From looking at the rest of the code, it looks like it would all just work ™️.

@marcelofeitoza
Copy link
Copy Markdown
Contributor Author

@marcelofeitoza In discussing with @clairechingching, we probably need some kind of solution to not need the .globl in the include file. The main reason being, we can't have multiple .globl symbols or the concept of an entrypoint gets confused. We can define alternative symbol names in included files, but there isn't really any benefit to adding additional symbol names to the binary. We probably need to consider some kind of way around this. From looking at the rest of the code, it looks like it would all just work ™️.

@deanmlittle I tested and it already works without .globl in the included files. I've added enforcement so the build errors if any included file uses .globl or .global.

image
~/include-demo $ sbpf build
⚡️ Building "include-demo""include-demo" built successfully in 1.408ms!
~/include-demo $ sbpf test 
🧪 Running tests
    Finished `release` profile [optimized] target(s) in 0.15s
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.22s
     Running unittests src/lib.rs (target/debug/deps/include_demo-78841665d7ad6f91)

running 1 test
[2026-03-05T02:42:49.851350000Z DEBUG solana_runtime::message_processor::stable_log] Program 78zVfKLgPEotVzdGk2h5hyWBoe9fiwSCWbu2k4dJjLmh invoke [1]
[2026-03-05T02:42:49.852540000Z DEBUG solana_runtime::message_processor::stable_log] Program log: include works
[2026-03-05T02:42:49.852567000Z DEBUG solana_runtime::message_processor::stable_log] Program log: math works
[2026-03-05T02:42:49.852617000Z DEBUG solana_runtime::message_processor::stable_log] Program 78zVfKLgPEotVzdGk2h5hyWBoe9fiwSCWbu2k4dJjLmh consumed 213 of 1400000 compute units
[2026-03-05T02:42:49.852629000Z DEBUG solana_runtime::message_processor::stable_log] Program 78zVfKLgPEotVzdGk2h5hyWBoe9fiwSCWbu2k4dJjLmh success
test tests::test_hello_world ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s

   Doc-tests include_demo

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

✅ Tests completed successfully!

@datasalaryman
Copy link
Copy Markdown

need a little help here. hoping to come to an agreement for backwards compatibility

anza-xyz/llvm-project#192 (comment)

@marcelofeitoza marcelofeitoza force-pushed the feat/include-directive branch from 58fd90b to 889efa4 Compare March 6, 2026 00:51
When multiple includes define the same label in .rodata/.data (e.g. message),
the build would fail with "Duplicate label". Labels are now prefixed with
`{path}___` (e.g. log.s → log___message, instructions/transfer/format.s →
instructions_transfer_format___message), so they no longer collide.

Made-with: Cursor
@marcelofeitoza marcelofeitoza force-pushed the feat/include-directive branch from 9ae334a to 5583e2b Compare March 6, 2026 01:11
@marcelofeitoza
Copy link
Copy Markdown
Contributor Author

This commit adds automatic prefixing for data labels in included files. When multiple includes define the same label in .rodata/.data (e.g. message), the build would fail with "Duplicate label". Labels are now prefixed with {path}___ (e.g. log.slog___message, instructions/transfer/format.sinstructions_transfer_format___message), so they no longer collide.

@alnoki
Copy link
Copy Markdown

alnoki commented Mar 6, 2026

assembly macros would be another such change that would improve devex a lot.

@deanmlittle yes please! assert_eq! and similar for starters

@marcelofeitoza
Copy link
Copy Markdown
Contributor Author

marcelofeitoza commented Mar 6, 2026

assembly macros would be another such change that would improve devex a lot.

@deanmlittle yes please! assert_eq! and similar for starters

already have a PR prepared for in feat/macros, just will need this merged before I suppose. Or should I open a PR for this feat/include-directive branch?

@alnoki
Copy link
Copy Markdown

alnoki commented Mar 6, 2026

@marcelofeitoza will this PR allow for unit testing individual functions, once they are broken out into a separate file?

Perhaps it's worth a broader discussion, but the ability to test subroutines, especially with coverage would make ASM development significantly more auditable and verifiable

@marcelofeitoza
Copy link
Copy Markdown
Contributor Author

@alnoki With .include you can unit-test individual functions by building small test programs that only include and call them. For example, put log_msg in log.s, then add test_log.s that includes it and has an entrypoint that calls log_msg with chosen inputs. Run sbpf test on that program to exercise just that function.

Just tried it and it works. Layout:

unit-test-demo/
├── src/
│   ├── test_log/
│   │   ├── test_log.s          # entrypoint that only calls log_msg
│   │   └── modules/
│   │       └── log.s           # shared log_msg function
│   └── unit-test-demo/
│       └── unit-test-demo.s    # main program
└── deploy/
    ├── test_log.so             # built from test_log.s
    └── unit-test-demo.so

Running sbpf test:

test tests::test_log_msg ... ok
test tests::test_hello_world ... ok

And in the logs:

[2026-03-07T00:29:35.154955000Z DEBUG solana_runtime::message_processor::stable_log] Program 78ydmiDNpAhy8QoDABrmRZrBYdbC5LDfW6sh7wJPMGoA invoke [1]
[2026-03-07T00:29:35.154955000Z DEBUG solana_runtime::message_processor::stable_log] Program 78ydmiDNpAhy8QoDABrmRZrBYdbC5LDfW6sh7wJPMGoA invoke [1]
[2026-03-07T00:29:35.155289000Z DEBUG solana_runtime::message_processor::stable_log] Program log: include works
[2026-03-07T00:29:35.155289000Z DEBUG solana_runtime::message_processor::stable_log] Program log: Hello, Solana!
[2026-03-07T00:29:35.155347000Z DEBUG solana_runtime::message_processor::stable_log] Program 78ydmiDNpAhy8QoDABrmRZrBYdbC5LDfW6sh7wJPMGoA consumed 107 of 1400000 compute units
[2026-03-07T00:29:35.155378000Z DEBUG solana_runtime::message_processor::stable_log] Program 78ydmiDNpAhy8QoDABrmRZrBYdbC5LDfW6sh7wJPMGoA success
[2026-03-07T00:29:35.155394000Z DEBUG solana_runtime::message_processor::stable_log] Program 78ydmiDNpAhy8QoDABrmRZrBYdbC5LDfW6sh7wJPMGoA consumed 104 of 1400000 compute units
test tests::test_log_msg ... [2026-03-07T00:29:35.155440000Z DEBUG solana_runtime::message_processor::stable_log] Program 78ydmiDNpAhy8QoDABrmRZrBYdbC5LDfW6sh7wJPMGoA success

So the pattern works, even though it wasn’t the original goal of this PR with adding .include.

.include doesn’t add unit-test or coverage tooling by itself; it enables this pattern. Getting real coverage and subroutine-level testing would likely need more tooling.

@clairechingching
Copy link
Copy Markdown
Collaborator

clairechingching commented Mar 7, 2026

the goal of this PR is great and addresses a wanted feature! but the implementation is not okay.

we recently added debug feature to sbpf and doing this by just simply expanding the source program would loose source file/line tracking for the included assembly files

imo build.rs should just be responsible for taking inputs, setting up assembler config properly and passing the source files to the assembler. a more proper way to do this is to initiate a new parser to process the included assembly files but build them into the same AST (and handle the duplicate label there)

@deanmlittle what's your opinion on resolving duplicate labels across different files? personally i like the idea of adding prefix to the labels in callee programs

@deanmlittle
Copy link
Copy Markdown
Collaborator

@deanmlittle what's your opinion on resolving duplicate labels across different files? personally i like the idea of adding prefix to the labels in callee programs

When I said we should do something about the globl thing, I didn't actually mean we necessarily had to solve it here. I think there's two ways we can look at this.

  1. We do nothing to handle duplicates, not even the globl check, and simply allow duplicate symbols to cause an assembler error, and update the error to correctly tie both the line number and file together, then handle this correctly in our vscode plugin.
  2. We prefix imported symbols

Imo, includeimport. Include just means we're inlining a file. In any other language that manages to inline code by some means, such as macros or metaprogramming, the solution seems to always be 1.

@clairechingching
Copy link
Copy Markdown
Collaborator

Imo, includeimport. Include just means we're inlining a file. In any other language that manages to inline code by some means, such as macros or metaprogramming, the solution seems to always be 1.

^ this makes total sense to me!

@marcelofeitoza
Copy link
Copy Markdown
Contributor Author

I agree with both directions actually, think they will fit together:

Claire’s point: Move .include handling out of build.rs and into the assembler, so build.rs only handle inputs, config, and invoking the assembler. The assembler should parse .include, merge included files into a single AST, and handle duplicate labels there.

Dean’s point: Treat .include as pure inlining—no automatic prefixing and no special .globl handling. Let duplicate symbols produce normal assembler errors and improve error reporting (file/line) instead.

Proposed approach: Implement .include in the assembler (e.g. via sbpf.pest and the parser) so that included content is inlined into the main source before parsing. No label prefixing; if two included files define the same label, emit a standard duplicate-label error. Allow .globl/.global in included files. Whoever wants unique labels can name them explicitly (e.g. msg_log, msg_math).

@deanmlittle
Copy link
Copy Markdown
Collaborator

deanmlittle commented Mar 7, 2026

assembly macros would be another such change that would improve devex a lot.

@deanmlittle yes please! assert_eq! and similar for starters

this isn't quite how assembly macros look. if we were to copy NASM style, for example, it'd look something like this:

; Aligns a pointer to the next 8 bytes
; Parameters:
; 1. r_input - register pointing to value that needs alignment
; 2. r_temp - temporary register to store current input value for equality check
 
%macro align_8 2  
    mov64 %2, %1
    and64 %2, -8
    jeq %1, %2, 1
    add64 %1, 8
%endmacro

Which can then be called like this:

align_8 r3, r4

You can imagine how quickly multiple optimizations of common yet cumbersome functions like CPI would stack up to make writing ASM feel effortless.

@alnoki
Copy link
Copy Markdown

alnoki commented Mar 13, 2026

@clairechingching are you good to merge this too?

@deanmlittle
Copy link
Copy Markdown
Collaborator

@clairechingching are you good to merge this too?

Approve here is just concept ack. We still need to consider the implementation details and test it out with our greater tooling. See comment: #109 (comment)

@alnoki
Copy link
Copy Markdown

alnoki commented Mar 13, 2026

@clairechingching are you good to merge this too?

Approve here is just concept ack. We still need to consider the implementation details and test it out with our greater tooling. See comment: #109 (comment)

Got it, thanks!

I'm excited for this feature, already using it at DASMAC-com/dropset-beta#20

alnoki added a commit to DASMAC-com/dropset-beta that referenced this pull request Mar 13, 2026
# Background

The assembly build relies on recent `sbpf` features:
- [`--arch v3`](blueshift-gg/sbpf#98) — SBPFv3
architecture target
- [Multi-file assembly via
`.include`](blueshift-gg/sbpf#109)
- [`--deploy-dir` flag](blueshift-gg/sbpf#103)

# Changes

1. Add SBPF assembly source files for entrypoint, market registration,
and error handling
2. Add Cargo workspace configuration with `mollusk-svm` and `pinocchio`
dependencies
3. Extend `Algorithm` component with collapsible Shiki-highlighted
assembly blocks via new `asm` prop
4. Rename component `src` prop to `tex` and anchor IDs from `algo-` to
`algorithm-`
5. Update algorithm TeX to rename `data` → `insn`, `discriminator` →
`discriminant`, and related error/length constants
6. Add `asm` build target to Makefile and update `.gitignore` for
`deploy` and `target` directories
7. Improve `docs-dev` and `docs-prettier` targets to run `npm install`
and open browser on serve
8. Escape erroneous `chktex` errors
alnoki added a commit to DASMAC-com/dropset-beta that referenced this pull request Mar 19, 2026
…26)

# Changes

1. Add `rustfmt` and `clippy` as local pre-commit hooks in
`cfg/pre-commit/lint.yml`, using the system toolchain for fast execution
2. Add Rust toolchain (`dtolnay/rust-toolchain`) and shared dependency
cache (`Swatinem/rust-cache`) to the lint CI workflow
3. Add `test.yml` workflow that builds assembly via `make asm` and runs
`make test`, sharing the Rust dependency cache with lint
4. Add `.github/actions/install-sbpf` composite action that installs and
caches the `sbpf` CLI, pinned to the
[`feat/include-directive`](blueshift-gg/sbpf#109)
branch for multi-file assembly support
5. Update `make clean` to also remove docs artifacts (`node_modules`,
`.vitepress/cache`, `.vitepress/dist`) and the assembly deploy directory
6. Add `make lint` target and update `make all` to run `lint` and `test`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants