Skip to content

Add Boat backend for sandbox tools - #71725

Closed
zozo123 wants to merge 2 commits into
apache:mainfrom
zozo123:add-ascii-box-sandbox-backend
Closed

zozo123 wants to merge 2 commits into
apache:mainfrom
zozo123:add-ascii-box-sandbox-backend

Conversation

@zozo123

@zozo123 zozo123 commented Aug 17, 2026 •

Copy link
Copy Markdown
Contributor

Adds a hosted Boat backend for SandboxToolset, giving containerized Airflow workers a production-capable backend without a local daemon or KVM.

Boat is the service formerly called Ascii Box: the vendor renamed it on 2026-09-15 and published boat-sdk 1.0.0 the next day. The old ascii.dev/api/box/v1 base now answers with Deprecation and Sunset: 31 Oct 2026 headers (see Migrating from Box to Boat), so this PR ships the backend under the new name from the start: BoatSandboxBackend, the sandbox-boat extra on boat-sdk>=1.0.0, boat_conn_id / boat_default, and BOAT_API_KEY / BOAT_BASE_URL.

What changed

  • maps SandboxSpec.env at creation and refuses deny-all / per-domain egress policies Boat cannot enforce;
  • uses the native file write API, which has no command-line length ceiling to work around, and keeps the inherited shell read, which bounds a read inside the guest instead of after the bytes have reached the worker;
  • applies the API's command deadline (capped at 600s), destroys timed-out sandboxes and any sandbox that never becomes ready, and configures a server-side TTL as the orphan-cleanup backstop;
  • resolves the optional boat-sdk and a generic Airflow connection lazily, and exports the backend eagerly (the module imports the SDK on first use, so the package stays importable without it and a missing SDK surfaces as an actionable error);
  • documents the sandbox-boat extra and adds unit plus system coverage.

Validation

  • Unit tests cover connection resolution, fail-closed network policy, command bounding/timeout teardown, teardown of a sandbox that never reports ready, file ops, and the idempotent delete on the /sandboxes/{sandboxId} route;
  • the previously recorded system test Dag ran end to end against https://boat.dev/api/v1 with a real key on 2026-09-19 (create, rename, readiness, command, write, read, list, non-zero exit, delete): 1 passed in 13.47s;
  • the previous revision passed 23,548 non-database provider tests in CI;
  • local static checks pass with the container-dependent provider YAML check excluded, including ruff, dependency-lock validation, and provider metadata regeneration;
  • fixed the generated extras ordering and the short heading underline that failed static checks, documentation, and spellcheck in CI; and
  • local container-dependent unit tests and provider mypy could not be completed because a usable Airflow CI image was unavailable. Fresh CI validates the updated revision.

Related: #68847
See also: #71672, #71676


Was generative AI tooling used to co-author this PR?
  • Yes — Cursor (Grok), Claude Code (Opus 5), Claude Code (Fable 5.1), Codex (GPT-6)

Generated-by: Cursor (Grok), Claude Code (Opus 5), Claude Code (Fable 5.1), Codex (GPT-6) following the guidelines

@zozo123

zozo123 commented Sep 14, 2026

Copy link
Copy Markdown
Contributor Author

Airflow + Ascii Box CLI e2e proof

Committed to this PR as ascii_box_actual_cli_proof.gif. This is the actual CLI transcript from the successful real-box Airflow run: create, write, read, command output, listing, non-zero exit handling, destroy, and system test 1 passed.

@github-actions

github-actions Bot commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

uv.lock on main just moved via #73708 ("Mark the IBM MQ and IBM Db2 providers as ready for release"), commit 122e24c and this PR currently conflicts.

Quickest fix:

git fetch upstream main && git rebase upstream/main
rm uv.lock && uv lock
git add uv.lock && git rebase --continue
git push --force-with-lease

Automated nudge — ignore if you're not ready to rebase. This comment is updated in place on future uv.lock bumps.

@zozo123

zozo123 commented Sep 15, 2026

Copy link
Copy Markdown
Contributor Author

Here’s the Airflow UI view from a fresh run of this PR’s common_ai_sandbox_toolset_ascii_box system-test DAG.

I first ran the checked-in system test unchanged (1 passed). For the recording, I reran the same assertions with non-secret progress markers so every sandbox-boundary step is visible in the task log: write_file, successful run_command output validating file persistence + environment propagation + arithmetic, read_file, the expected exit-code-3/stderr failure path, list_directory, and teardown. The UI-triggered DAG and task both finished Success.

Airflow Ascii Box sandbox E2E

@zozo123
zozo123 force-pushed the add-ascii-box-sandbox-backend branch from 2b71bac to 41570aa Compare September 19, 2026 17:26
@zozo123 zozo123 changed the title Add Ascii Box backend for sandbox tools Add Boat backend for sandbox tools Sep 19, 2026
@zozo123
zozo123 force-pushed the add-ascii-box-sandbox-backend branch from cd6b138 to 195ad5e Compare September 22, 2026 07:57
SandboxToolset could only run off the worker through Modal, or on the worker
through sbx, which needs the sbx binary, a Docker login and host
virtualization that an unprivileged container cannot give it.
BoatSandboxBackend provisions each sandbox through Boat's hosted
cloud-computer API, so a containerized worker and Kubernetes get a second
off-worker option. Boat is the service formerly called Ascii Box, renamed on
2026-09-15; boat-sdk 1.0.0 is the first release under the new name.

Credentials come from a generic Airflow connection resolved lazily on first
use, so the API key can live in a secrets backend; boat_conn_id=None reads
BOAT_API_KEY from the worker environment instead.

Boat can enforce neither a deny-all network policy nor a per-domain egress
allowlist, so create refuses block_network=True and allow_egress_to rather
than provisioning something weaker than SandboxSpec asked for. Writes use the
native file API; reads keep the inherited bounded shell implementation so
max_bytes is enforced in the guest before output reaches worker memory.
@zozo123
zozo123 force-pushed the add-ascii-box-sandbox-backend branch from 195ad5e to 400d3d6 Compare September 22, 2026 08:48
@zozo123

zozo123 commented Sep 22, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main, which reorganised the provider docs twice today: docs/sandbox.rst, docs/toolsets.rst and docs/choosing_a_toolset.rst were replaced by docs/sandbox/ and docs/toolsets/, so the backend is now documented in docs/sandbox/backends.rst beside Modal and sbx rather than in toolsets.rst.

Also in this push: the temporary boat-sdk entry in [tool.uv.exclude-newer-package] is removed from both maps, since its own comment said to once the rolling window advanced past 2026-09-16 and it has; boat-sdk 1.0.0 now resolves without an override. main still claimed in sbx.py that no hosted backend ships with the provider, which Modal had already made untrue, so that docstring and the system-test README -- which never mentioned the Modal test -- are corrected here.

The system test passed end to end against boat.dev (1 passed in 14.58s): create, the marker passed through SandboxSpec.env and read back from inside the sandbox, successful and non-zero command exits, write/read/list, and teardown. It requests open egress, because Boat can enforce neither a deny-all policy nor a per-domain allowlist and create refuses both rather than provisioning something weaker than the spec asked for.

41 unit tests, ruff and the applicable prek hooks pass locally; mypy is clean on the changed files with the SDK installed.


Drafted-by: Claude Code (Opus 5); reviewed by @zozo123 before posting

@potiuk potiuk added the closed because of open PR limit Closed as a one-time step of introducing the open pull request limit label Sep 25, 2026
@potiuk

potiuk commented Sep 25, 2026

Copy link
Copy Markdown
Member

Hello @zozo123 - thank you for your contributions to Apache Airflow!

The Airflow community has introduced a limit of 5 open pull requests at a time for contributors without write access to the repository. You currently have 6 open pull requests, so - as a one-time step of introducing the limit - we closed the ones where maintainers have not engaged yet:

These pull requests stay open because maintainers are already engaged in them - they count towards your limit:

This is not a judgement of you or of your changes. We never told contributors before that opening many pull requests at once was a problem, so there is nothing to feel bad about - and nothing is lost: your branches, commits and the review history stay where they are.

What we ask you to do is to make your first prioritization decision: choose which of the pull requests above matter most to you, and reopen them (up to 5 open at a time, including the ones still open) with the "Reopen pull request" button or gh pr reopen <PR_NUMBER> --repo apache/airflow. Reopen the ones you are ready to follow through - keep them rebased, respond to review comments and fix failing checks.

While your pull requests are waiting for review, the most valuable thing you can do is help in other ways - reviewing other contributors' pull requests, helping with issues, and taking part in the discussions on the devlist and Slack.

Why we introduced the limit, what it means for you and how to reopen or restore a pull request is explained in https://github.com/apache/airflow/blob/main/contributing-docs/32_open_pull_request_limit.rst.


Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers closed because of open PR limit Closed as a one-time step of introducing the open pull request limit kind:documentation provider:common-ai

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants