Skip to content

Security: henrycashe26/VLA-Synth-Data

Security

SECURITY.md

Security policy

Supported versions

VLA-Synth-Data is currently in alpha (0.1.0a0). Security fixes are only applied to the main branch; there is no supported release line yet.

Reporting a vulnerability

Please do not file security issues as public GitHub issues. Instead, email the maintainer at the address in CITATION.cff with:

  • A description of the issue and its impact.
  • A minimal reproduction (ideally a script or a failing test case).
  • Your disclosure timeline expectations.

You can expect an initial acknowledgment within three business days and a status update within two weeks. Credit will be given in the release notes unless you prefer to remain anonymous.

Known security risks

These are issues we know about and have not yet fixed. Treat them as the security threat model for any deployment of this pipeline.

1. Pickle-based checkpoints — MEDIUM (mitigated from HIGH)

scripts/run_pipeline.py serializes per-stage outputs as pickle envelopes under data/checkpoints/. Loading an untrusted checkpoint used to be equivalent to running arbitrary Python code.

Mitigation shipped: pipeline.safe_checkpoint.SafeUnpickler now intercepts every class resolution during pickle.load via the find_class hook and rejects anything outside pipeline.safe_checkpoint.ALLOWED_CLASSES. The whitelist covers numpy core types, stdlib primitives, pathlib Paths, and a few known reconstruction helpers — nothing that can execute arbitrary code on its own. Hostile payloads that try os.system, subprocess.Popen, eval, or random third-party classes are rejected with a CheckpointFormatError. See tests/test_safe_checkpoint.py for the four hostile-payload unit tests that lock in the behavior.

Residual risk (why still MEDIUM, not LOW): restricted unpickling is not a perfect sandbox. An attacker who finds a __reduce__ path through a whitelisted class can still cause side effects. The long- term fix is to migrate the checkpoint format entirely away from pickle to JSON + NumPy .npz or safetensors, which removes code execution from the threat model altogether.

Operational guidance today: still avoid loading checkpoints from truly untrusted sources (e.g. random downloads). Within a trusted development setup — your own generations or a colleague's — the restricted unpickling closes the immediate HIGH-severity path.

2. Anthropic API key handling — MEDIUM

Stage 1 (task generation) and Stage 3c (language annotation) call the Anthropic API. The SDK reads ANTHROPIC_API_KEY from the process environment. This repo never logs or serializes the key, but:

  • Users should keep their key out of commit history and out of any trajectory dictionaries they share externally.
  • data/annotation_cache/ stores API responses on disk. If you share that directory, anyone with access can read the prompts and responses (not the key itself).

3. HuggingFace token handling — MEDIUM

pipeline/hooks.py::finalize_run reads HUGGINGFACE_TOKEN from the environment when pushing to the HF Hub. Same guidance as the Anthropic key: do not commit, do not share the cached upload state.

4. Arbitrary-code execution via task specs — LOW

Task specs are Pydantic-validated dicts. Current validation enforces type shapes but does not block injected code in string fields such as task_description — those strings are passed to the Anthropic API and to the dataset builder without escaping. An attacker who controls the task spec cannot execute code in this process, but a downstream consumer that evaluates the description as code (for example, by eval()ing a caption) could be compromised. Treat task descriptions as untrusted input.

5. External scene MJCF ingestion — LOW (not yet implemented)

A future change will wire RoboCasaAdapter output into MuJoCoEnvironment._build_scene. When that lands, external scene XMLs will be loaded via dm_control.mjcf which parses and compiles them at runtime. MuJoCo XML is a restricted format but can reference external mesh files and plugin binaries on some platforms. Only load scene XMLs from sources you trust. Until the loader is wired up, this is not an active risk.

6. Replay evaluation uses numpy.random.Generator — LOW

The replay evaluator's perturbation sampling is deterministic per seed and per trajectory ID. It does not use pickle, does not write to disk outside the explicit report path, and does not execute any user code from the replayed trajectories. The trajectory ctrl arrays are plain NumPy floats and are applied verbatim to MuJoCo — no injection path known.

Hardening recommendations for production users

If you are deploying this pipeline (not just running it locally), the maintainer strongly recommends:

  1. Run the whole pipeline in a restricted user account with no write access to your home directory or cloud credentials.
  2. Rate-limit the Anthropic API and HF Hub calls at the network layer in addition to the application-layer rate_limit_rpm config.
  3. Verify every data/raw/*.h5 file's size before ingesting; anomalously large files may indicate a renderer or recorder bug and can OOM the dataset-builder step.
  4. Treat data/annotation_cache/ as sensitive; it contains verbatim prompt and response data from the Anthropic API.
  5. Never re-use the same seed for generation and for any security-relevant randomness in the same process. This pipeline is deterministic by design, which is good for reproducibility and bad for anything needing cryptographic unpredictability.

Out of scope

This pipeline is not a security tool, an authentication system, or a sandbox. It generates synthetic training data. The threat model assumes you are the operator and the only consumer of the data you produce. Cross-tenant or multi-user deployments are not supported and are not in scope for this security policy.

There aren't any published security advisories