Launch FrontierPhysics with surface ion-trap shuttling task

[bingran-you]

Motivation

This task is derived from "Temporally multiplexed ion-photon quantum interface via fast ion-chain transport" (You, Wu, Miron, Ke, Monga, Saglamyurek, Haeffner, arXiv:2405.10501), my own experimental work. That paper demonstrates a temporally multiplexed ion-photon interface by rapidly transporting a chain of nine 40Ca+ ions across 74 μm in 86 μs, verifying the non-classical character of the multiplexed photons with a measured second-order correlation of g⁽²⁾(0) = 0.060(13). It also reports the honest failure mode: after transport, the ion crystal is coherently excited to as much as n̄ ≈ 110 in the centre-of-mass mode. Multiplexing worked; the transport heated the chain.

That residual excitation is exactly what this task is about. Raising the entangling rate of a distributed quantum network means shuttling ion chains faster, and shuttling them faster without heating means you cannot move the trap naively — you have to know the trap's secular frequencies from a real boundary-element solve of the electrode geometry, and you have to design the trap-center trajectory by Lewis-Riesenfeld invariant inverse engineering so the final excitation cancels. The broader architectural motivation for shuttling as a primitive is the QCCD proposal of Kielpinski, Monroe, and Wineland.

This task belongs in FrontierPhysics because it is the full computational chain behind that experiment rather than a textbook fragment: mesh a real trap STL, run a constant-panel BEM solve, build the ponderomotive pseudopotential, fit its local curvature along the correct in-plane direction, propagate to the axial frequency through the anisotropy parameter, solve the nine-ion Coulomb equilibrium, and emit two inverse-engineered waveforms — one single-ion move and one piecewise nine-ion sequence with dwells. This is what a trapped-ion experimentalist actually computes before touching the apparatus, and the chain is unforgiving: the radial frequency feeds everything downstream, so an electrode-width estimate instead of a real solve poisons every later number.

The trap geometry and derived panel mesh are processed from the HaeffnerLab integrated_photonics_bem release — the same group's trap modelling code — pinned at commit a58710a15d665b04fb8ffca98ea58c76feab9f43, with STL and mesh SHA-256 digests recorded in model_provenance.json alongside the drive parameters. The bundled boundary-element solver is FastLap, © 1992 Massachusetts Institute of Technology, redistributed under the MIT-issued permission notice retained verbatim in bem_fastlap/LICENSE.md; the mesher is Triangle, redistributed under its own notice in the same file. No expected answers are stored anywhere in the environment — the verifier recomputes the BEM reference independently at verify time.

Task history

The original work is the experiment published as arXiv:2405.10501: building and characterising the temporally multiplexed ion-photon interface, of which the fast nine-ion transport is one component. The figures below cover that work — the span and cost of doing the physics, not of packaging it into a task. The paper was submitted on 17 May 2024, about two months after the end date below.

Report	This task	Minimum
Project time scale — start and end date	2023-07-24 → 2024-03-11 (≈ 7.5 months)	2 weeks
Actual working hours spent exploring the task	100 hours	40 hours
Estimated hours for a first-year PhD to reproduce it	15 hours	10 hours

Task

Field	Value
Task ID	surface-ion-trap-shuttling
Category	natural-science / trapped-ions
Difficulty	Hard
Why it's hard	The agent must run a real electrostatics solve rather than estimate: mesh the supplied surface-trap STL, solve for the RF field near the null with a constant-panel BEM, construct the ponderomotive pseudopotential for a 40Ca+ ion at 80 V / 39.15 MHz, and fit its Hessian along the in-plane radial direction — which is parallel to the trap surface and perpendicular to the RF rails, not the surface normal. It must then propagate that frequency through the anisotropy parameter, solve the coupled nine-ion Coulomb force-balance for equilibrium spacings rather than dividing the chain into equal intervals, and inverse-engineer two transport waveforms, distinguishing the designed classical ion trajectory from the trap-center trajectory that carries the acceleration correction. Every downstream number depends on the first one.
Source	Derived from arXiv:2405.10501. Geometry processed from the HaeffnerLab integrated_photonics_bem repository at commit a58710a15d665b04fb8ffca98ea58c76feab9f43 (examples/sqip/inter_results/htrap/htrap.stl). Bundled solver is FastLap (© 1992 MIT) with Triangle; both license notices retained in bem_fastlap/LICENSE.md.
Deliverables	/root/result.md with five keyed values, plus /root/1.csv and /root/2.csv waveforms
Verifier	4 outcome tests; the BEM reference is recomputed independently at verify time, not stored
Skills Provided	See mentor skill summary below.

Mentor skill summary

FrontierPhysics permits task-specific mentor skills, so these are scoped narrower than a general skill library would be. Three carry reusable physics method; the fourth is deliberately a task recipe.

• surface-ion-trap-bem: Written for the fragile electrostatics stage — running the bundled FastLap helper against a provenance-pinned panel mesh, evaluating the RF field near the null, constructing the pseudopotential, and fitting its local radial curvature. It carries method, not answers: it explains why the electrode-width estimate is wrong and which in-plane direction is meant, but contains no frequency value and no verifier internals.

• ion-chain-equilibrium: Written for any one-dimensional Coulomb crystal in a harmonic axial trap given ion count, charge, mass, and axial frequency. It stays general by describing the dimensionless force-balance formulation and the equal-interval error that catches people out; it is not specialised to nine ions or to this trap.

• invariant-based-transport: Written for zero-final-excitation harmonic-trap transport by Lewis-Riesenfeld inverse engineering, including concatenated unequal-distance stages and dwell intervals. It stays general by covering the classical-trajectory / trap-center distinction and the acceleration correction, which apply to any shuttling waveform; it contains no distances, durations, or CSV values from this task.

• ion-shuttling-mentor: The one deliberately task-specific skill — an end-to-end ordering that tells the agent to invoke the three above in sequence and keep full precision until writing artifacts. It hardcodes no numeric answer; it supplies sequencing, which is exactly the thing being measured.

Checklist

• task.md prompt body is human-written and outcome-focused
• oracle/solve.sh and the oracle logic are human-written (LLM for syntax help OK, logic must be yours)
• Metadata follows taxonomy.yaml
• bench tasks check tasks/surface-ion-trap-shuttling passes
• Oracle passes 100%
• Tests verify outcomes, not implementation or skill usage
• task.md does NOT mention which skills to use
• Mentor skills contain no hardcoded final answers and no verifier internals
• No external API keys required for oracle or tests
• Source, code, and license provenance are documented
• No-skill and with-skill runs use the same task commit and model settings
• At least one agent passes the with-skill solvability control
• Trajectories and output artifacts were inspected
• PR contains only files under tasks/surface-ion-trap-shuttling/

Skill paths in Dockerfile

• COPY skills /root/.claude/skills
• COPY skills /root/.codex/skills
• COPY skills /root/.opencode/skill
• COPY skills /root/.agents/skills

Deliberately unchecked. This task bakes no skills into its image; a clean build contains zero SKILL.md files, and the four mentor skills are injected at runtime only when --skills-dir is passed. The Dockerfile creates the empty agent home directories and nothing else, which is what makes the no-skill condition meaningful.

Agent Performance

Agent	Model	With Skills	Without Skills
opencode	gpt-5.6-sol	1/1 (100%)	0/1 (0%)
claude-agent-acp	claude-opus-5	1/1 (100%)	1/1 (100%)

Every no-skill sandbox above was verified to contain zero reachable SKILL.md files before the agent started, on a benchflow build patched by benchflow-ai/benchflow#938. An earlier matrix was run before that verification existed and is not reported here.

I report one trial per cell rather than a multi-trial rate. The runs are long — up to 28 minutes each for the no-skill gpt-5.6-sol condition — and I would rather report a small honest N on a verified-clean harness than a larger N I cannot stand behind. Treat the pass rates as single observations.

I also could not apply the intended xhigh reasoning effort. Neither the OpenCode nor the Codex ACP harness declares an effort configuration option in benchflow 0.6.5, and passing --reasoning-effort aborts the run outright, so every row above is at the model default. Separately, codex-acp cannot reach gpt-5.6-sol at all — session/set_model returns -32603 for any model outside its built-in catalogue — which is why the GPT-family runs go through OpenCode.

Additional diagnostics

The verifier applies a 10% relative tolerance to the radial frequency and recomputes its own BEM reference at verify time. Here,

• "artifacts" means the run produced result.md with all five keyed values plus both waveform CSVs, parseable and schema-clean,
• "skills read" is counted from the run's own trajectory, so a no-skill row showing 0 is evidence of isolation rather than an assertion about it.

Agent	Model	Setting	Pass Rate	Artifacts	Tests	Wall clock	Tool calls	Skills read
oracle	—	reference	1/1 (100%)	complete	4/4	2m01s	—	—
opencode	gpt-5.6-sol	Without skills	0/1 (0%)	complete	3/4	28m21s	28	0
opencode	gpt-5.6-sol	With skills	1/1 (100%)	complete	4/4	3m30s	23	2
claude-agent-acp	claude-opus-5	Without skills	1/1 (100%)	complete	4/4	11m22s	24	0
claude-agent-acp	claude-opus-5	With skills	1/1 (100%)	complete	4/4	8m19s	15	not counted

The final cell is a measurement limitation, not a configuration one: that run was launched with --skills-dir and the skills were deployed, but Claude's ACP tool naming differs from OpenCode's, so my trajectory parser cannot evidence use the way it can for the OpenCode rows. I would rather mark it than report a zero that means something different from the two bolded zeros.

Failure analysis: gpt-5.6-sol without skills failed exactly one test, and it is the electrostatics one:

test_result_frequencies_are_derived_from_bem_and_anisotropy
  W_radial_freq = 0.555035 MHz
  BEM reference = 4.295858 MHz     (rel_tol = 0.10)

A factor of 7.7 low — far outside tolerance. The other three tests passed: the nine-ion equilibrium spacings, the single-ion inverse-engineered waveform, and the piecewise nine-ion waveform with dwells. So this was not a format or integration failure; the agent understood the deliverables, produced complete well-formed artifacts, handled the Coulomb statics, and designed both transport trajectories correctly. It failed specifically at recovering the trap frequency from the geometry, which is the one step where the mentor package carries real method — the pseudopotential expansion, the Hessian fit, and the fact that the requested in-plane radial direction is perpendicular to the RF rails rather than along the surface normal. With skills the same model lands on 4.295858148674524 MHz, matching the independently recomputed reference to ten significant figures. claude-opus-5 gets this step right unaided.

This is a capability gap, not a broken task: the oracle passes, both with-skill controls pass, and the failing run produced a clean result.md that simply contained wrong physics.

Development and Testing Reflection

The task localises failure to a single stage

A binary reward would have recorded the gpt-5.6-sol no-skill run as an undifferentiated zero. What actually happened is more useful: the run produced complete, well-formed artifacts and passed three of four tests, failing only at the boundary-element frequency. Because the chain is sequential and the verifier tests each link separately, the failure is localised rather than merely observed — the model could design invariant-based transport waveforms and solve the nine-ion Coulomb equilibrium, and could not extract the radial curvature from the real electrode geometry.

That is worth designing for deliberately. The temptation when authoring a physics task is to check only the final artifact, since that is what the experiment ultimately needs. Testing the intermediate quantities instead — radial frequency, axial frequency, spacings, then each waveform — costs little and turns a pass/fail into a diagnosis. It also makes the task robust to partial credit arguments: I can state precisely which physics the model did and did not have.

One skill package does not help both models equally

The two models give opposite verdicts. claude-opus-5 solves this unaided; gpt-5.6-sol does not, and needs the mentor package to reach the BEM frequency. Reporting a single model, or a single condition, would have hidden that entirely.

The skills also did more than flip a pass/fail bit. gpt-5.6-sol went from 28m21s to 3m30s, and claude-opus-5 — which needs no help to pass — still went from 24 to 15 tool calls. So the package is doing something even for the model that already knows the physics: not teaching the method, but removing the search for it. That mirrors the 100-hours-to-15-hours compression in the task history above, and it is visible in the trajectories even when the reward is identical at both ends.

What I would test next

I would narrow the mentor package — dropping ion-shuttling-mentor, which is pure sequencing — to see whether the three method skills alone close the gap for gpt-5.6-sol. My expectation is that surface-ion-trap-bem accounts for nearly all of the effect, since that is the only test the no-skill run failed, and that the sequencing skill mostly buys wall-clock rather than correctness. If that holds, the task would be a cleaner instrument with the mentor recipe removed entirely, and I would rather find that out than ship a package whose parts I have not separated.

Please let me know if any updates to the task, verifier, or skills would make this contribution better aligned with the FrontierPhysics framework; I would be happy to revise them.