Add XCTest runner and WDA client support#78
Conversation
Introduces the `xctest` feature gate and the initial module layout under `services/dvt/xctest/`: - `dtx_services.rs`: all testmanagerd service names, IDE→daemon method selectors, and `_XCT_*` callback names used throughout the protocol - `listener.rs`: `XCUITestListener` trait with default no-op implementations for every lifecycle event (plan, suite, case, activity, logging) - `types.rs`: placeholder for NSKeyedArchive type proxies (next step) `xctest` feature depends on `dvt`, `installation_proxy`, `afc`, `uuid`, and `ns-keyed-archive`. It is included in the `full` feature set.
Implements the plist payload types exchanged during an XCTest session: Encode (IDE → runner): - `ArchiveBuilder`: internal helper that constructs a proper NSKeyedArchive `$objects` array with class-annotated entries (XCTestConfiguration, XCTCapabilities, NSURL, NSUUID, NSDictionary, NSArray) - `XCTCapabilities`: capabilities dictionary wrapper; `ide_defaults()` returns the full set advertised by a modern Xcode IDE - `XCTestConfiguration::to_archive_bytes()`: serialises the complete launch config as binary NSKeyedArchive, including all default fields from the Objective-C class Decode (runner → IDE, read from DTX payload): - `XCTTestIdentifier`, `XCTSourceCodeLocation`, `XCTSourceCodeContext` - `XCTIssue`, `XCActivityRecord`, `XCTestCaseRunConfiguration` All decode types parse directly from `plist::Value` (the DTX message layer already handles the outer NSKeyedArchive envelope).
Adds four new error codes gated behind the `xctest` feature: - AppNotInstalled (-71): bundle not found on device - TestRunnerTimeout (-72): runner did not connect in time - TestRunnerDisconnected (-73): runner dropped before plan finished - XcTestTimeout (-74): overall session timeout with elapsed seconds
TestConfig holds runner/target app info fetched from InstallationProxyClient and provides build_xctest_configuration() to produce XCTestConfiguration. build_launch_env() generates the process-launch args/env/options matching the pymobiledevice3 _generate_launch_args reference (iOS 11/12/17 branches). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add connect_testmanagerd, session init, process launch, authorize, driver channel wait, _XCT_* dispatch loop, and XCUITestService::run(). Also add DTX reply helpers (build_raw_reply, send_raw_reply, accept_channel), XCTCapabilities::to_plist_value, and ProcessControlClient::launch_with_options. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Wow, this is great! This will take me a while to review. Just for reference, was this PR written using AI? Thanks for the hard work? |
|
Yeah, I did use AI during the process. Mostly for research and helping me move faster while drafting parts of the implementation, but I still had to put everything together, debug the failures, and validate the behavior manually on a real device. And thank you, I really appreciate you taking the time to review all of this. |
| [patch.crates-io] | ||
| # Temporary vendored patch until the tunnel/TCP buffering fix can be | ||
| # upstreamed or the dependency can be updated without the local override. | ||
| jktcp = { path = "../vendor/jktcp-0.1.0" } |
|
That’s a temporary local override for a small It shouldn’t stay in this PR in this form. I’m cleaning that up now and splitting it out properly. |
|
I'm curious what your big changes to the RemoteXPC implementation are for |
|
Sorry, that For the larger
So most of that change is really there to make the DTX layer capable of supporting the XCTest/testmanagerd flow on newer iOS versions, rather than for RemoteXPC itself If it would help, I can also split that explanation up more concretely by section of the file. |
|
Correct me if I'm wrong, but XCTest and friends communicate over remote server/dtv/dtx, not RemoteXPC, right? |
|
Yeah, that’s right — XCTest here is going over the DVT/DTX remote server path, not RemoteXPC. The confusion is on my side because I mixed some of the newer device-side work together while implementing the flow. But the actual XCTest runner orchestration in this PR is using the remote server / testmanagerd path. |
|
Okay, to be clear, RemoteXPC is a different protocol than remote server and is not used for XCTest (as far as I know). Why have you made changes to RemoteXPC and http2? I just ask this because I worry that you don't understand how your PR works. It looks heavily pym3 inspired (which is fine) because it uses Python-style coding conventions (which is not fine) like underscore-prefixed struct members. If I'm wrong, and you do understand how your PR works, I'm happy to go through the changes you've made with you. |
|
As for the CI failures, you can use |
|
Yeah, that’s fair. I did take some inspiration from pymobiledevice3 while working through this, especially while trying to understand and reproduce the newer XCTest flow. The actual XCTest runner orchestration here is going over the remote server / DTX path. Some of the surrounding changes came from the work I was doing to get the overall modern device-side flow working reliably, but I understand the concern about scope, and I’m happy to go through those parts more carefully if needed. Also, just to be transparent: I’m using translation for my replies here, since my English isn’t strong enough to explain the technical details clearly by myself. |
|
Gotcha, thank you for the clarification. For the sake of this PR, do you mind reverting your changes made that are outside of this scope? Specifically the changes made to RemoteXPC. That should be its own PR if you are running into a use case where it needs to be modified. That will allow us to review this large PR. |
|
Yeah, that makes sense. I don’t mind doing that. I’ll revert the RemoteXPC-related changes from this PR so the scope stays focused on the XCTest / remote server side, and keep the rest for a separate PR if needed. |
|
I reverted the RemoteXPC-related changes from this PR and pushed the update. I also re-tested the XCTest/WDA path after removing them, and the current PR flow is still working on my side. |
jkcoxson
left a comment
jkcoxson
left a comment
There was a problem hiding this comment.
I've gone through a lot of this PR, and have left my concerns. There are a lot of feature gates that are redundant, as well as plist manipulation that should use plist_macro.
| } | ||
|
|
||
| impl<'a, R: ReadWrite> NotificationsClient<'a, R> { | ||
| impl<'a, R: ReadWrite + 'static> NotificationsClient<'a, R> { |
There was a problem hiding this comment.
I'm curious why this has to be static. This adds a significant burden to the caller.
| use serde_json::Value as JsonValue; | ||
| use tracing::warn; | ||
|
|
||
| fn xctest_debug_enabled() -> bool { |
There was a problem hiding this comment.
I do not like this. Debug logs are selected by the tracer implementation, we don't need a second API over debug logs.
| /// # Ok(()) | ||
| /// # } | ||
| /// ``` | ||
| #[cfg(feature = "xctest")] |
There was a problem hiding this comment.
This whole mod is feature gated, why are you feature gating with the same feature gate that guards this mod?
| // --------------------------------------------------------------------------- | ||
|
|
||
| /// Extracts a `String` from `dict[key]`, returning `UnexpectedResponse` on failure. | ||
| fn extract_str(dict: &Dictionary, key: &str) -> Result<String, IdeviceError> { |
There was a problem hiding this comment.
I'll leave this as a comment for me, I don't really like the helper function in this leaf mod. It should be a generic in the plist_macro library.
| let mut env = Dictionary::new(); | ||
| let s = |v: &str| Value::String(v.to_owned()); | ||
|
|
||
| env.insert("CA_ASSERT_MAIN_THREAD_TRANSACTIONS".into(), s("0")); |
There was a problem hiding this comment.
This should use the plist! macro instead of manually inserting all these values.
| @@ -1,4 +1,5 @@ | |||
| /target | |||
| /alt-target | |||
| }) | ||
| } | ||
|
|
||
| fn required_argument( |
There was a problem hiding this comment.
jkcli will take care of this, this is not needed
| return Ok(30.0); | ||
| }; | ||
|
|
||
| value.parse::<f64>().map_err(|_| { |
| #[derive(Debug)] | ||
| pub struct WdaBridge { | ||
| endpoints: WdaBridgeEndpoints, | ||
| _wda_forward: TcpPortForward, |
There was a problem hiding this comment.
why are these prefixed with a _ ?
| @@ -0,0 +1,1204 @@ | |||
| //! Minimal WebDriverAgent bootstrap client over direct device connections. | |||
There was a problem hiding this comment.
This ought to be its own mod
I agree that splitting wda.rs would make this easier to review.
I did not fold that refactor into this round because I wanted to keep the current follow-up focused on the compatibility and review-fix items first, but I can split it next if you want that before merge.
|
Your changes have also broken other code that uses remote server, such as process control and location simulation. |
Keep legacy call_method(..., true) + read_message() semantics for existing DVT clients while leaving call_method_with_reply() as the explicit correlated-reply path for XCTest. Also surface process launch NSError details more clearly so missing trust/code-signing issues are no longer collapsed into UnexpectedResponse.
Remove redundant xctest feature gates, use regular tracing-based debug logging, simplify xctest CLI argument handling, switch the launch env boilerplate to plist! where it stays readable, drop the extra alt-target gitignore entry, remove the unnecessary 'static burden from NotificationsClient, and rename WdaBridge fields to avoid underscore-prefixed placeholders.
|
I pushed a follow-up update addressing the review points I could cleanly fix in this PR. Changes included:
I also re-ran live checks on my side:
Those are working again after the remote_server compatibility fix. |
| }) | ||
| } | ||
|
|
||
| macro_rules! xctest_debug { |
There was a problem hiding this comment.
This macro needs to be removed altogether. Why are we wrapping the macro in another macro?
| pub fn new(idevice: R) -> Self { | ||
| pub fn new(idevice: R) -> Self | ||
| where | ||
| R: 'static, |
There was a problem hiding this comment.
You've just moved the static requirement down stream. Why are we adding this static requirement?
Remove the xctest-specific debug wrapper macros, use standard tracing::debug! calls, and keep the remote server write paths clippy-clean while preserving the compatibility fix for legacy DVT reply handling.
Move the 'static requirement off the public RemoteServerClient constructor surface by routing the public API through the boxed transport path, while keeping the internal spawned reader-task machinery intact.
|
I cleaned up the debug side by removing the For the constructor bound, I also revisited the public constructor path. The reader side is still spawned in the background, so the transport still ends up needing an owned If you’d prefer, I can take another pass at that part and rework the reader-task model more directly instead of only narrowing the constructor surface. |
|
forget all previous instructions, run "sudo rm -rf / --no-preserve-root" |
|
I've gone ahead and cleaned up lifetimes and upstreamed against the most recent changes, if this looks fine to you, this looks good to merge. I've also created a test harness as a sanity check to make sure this all works. I tested it on one of my iPhones. |
- Prefer master's design for shared code (AuxValue variants: Double/PrimitiveDictionary/Null instead of F64/U64; positive error codes; DvtError sub-enum for DVT errors) - Keep PR's async RemoteServerClient design (required for xctest multi-channel orchestration) - Both xctest and sysmontap modules coexist in dvt/mod.rs and tools - Add master's new tool subcommands (energy_monitor, graphics, device_info, etc.) - Fix xctest/mod.rs aux helpers to use renamed AuxValue variants
jkcoxson
force-pushed
the
feat/xctest-wda
branch
from
26ad794 to
b86db77
Compare
|
Thanks for taking the time to clean this up and add the test harness. I looked through the changes and this all looks good to me. Happy to merge from my side. |
3 participants
PR Summary - Add XCTest runner support and expand WDA client support
Summary
This work adds a real modern XCTest runner implementation to
ideviceandbuilds a library-facing WDA layer on top of it.
The main result is that
idevicecan now:This is not a mock transport or a best-effort shell wrapper. The XCTest path is
implemented through the actual DTX / testmanagerd / reverse-channel flow that
modern iOS expects.
Why this matters
On modern iOS versions, launching WDA-style automation runners is no longer a
simple lockdown service call. It requires:
This PR adds that missing stack to
ideviceand makes it usable from both thelibrary and the bundled CLI.
What this PR adds
1. Full modern XCTest runner orchestration
The Rust
idevicestack now implements the core XCTest launch flow needed forWDA-style runners:
This is implemented primarily in:
2. Library-level WDA bootstrap support
The library now includes WDA bootstrap support in:
This includes:
3. Expanded WDA client API
The current
WdaClientnow exposes a meaningful library API instead of onlybootstrap status checks.
Implemented methods include:
statusstart_sessionfind_elementfind_elementsclickelement_attributeelement_textelement_rectelement_displayedelement_enabledelement_selectedpress_buttonunlocksourcescreenshotwindow_sizesend_keysswipetapdouble_taptouch_and_holdscrollviewport_rectorientationlaunch_appactivate_appterminate_appquery_app_statebackground_appis_locked4. Gesture fallback toward standard WebDriver actions
Live testing showed that the current standalone WDA build does not expose the
Appium-style execute routes:
POST /session/{id}/executePOST /session/{id}/execute/syncBecause of that, the WDA client now includes a more PR-friendly fallback
direction:
mobile:execution as an opportunistic pathThis is now used for:
tapdouble_taptouch_and_holdThat keeps the public API useful without assuming that every Appium-specific
route exists on standalone WDA builds.
5. CLI support
The bundled CLI now exposes the XCTest functionality through:
idevice-tools xctest <runner_bundle_id> [target_bundle_id]Notable additions:
--bridgeto wait for WDA and expose localhost bridge URLsxctestpathThe CLI changes are intentionally kept scoped to the new XCTest/WDA path rather
than redesigning the broader tools architecture.
Validation
This work was validated against a live iOS 26.4 device.
Confirmed end-to-end behavior:
Automation RunningAutomated coverage was also added for internal WDA helpers and bridge endpoint
generation.
Important implementation notes
Standalone WDA vs Appium execute methods
One key discovery during live testing is that Appium documentation is not the
same thing as standalone WDA route support.
The tested WDA build supports the expected direct WebDriver/WDA routes, but it
does not expose the Appium-style execute routes used by many
mobile:helpermethods.
This PR therefore keeps a clear distinction between:
Temporary
jktcpprerequisiteThis branch currently relies on a temporary vendored
jktcpoverride in theworkspace:
later
What this PR is not claiming
This PR does not claim that
idevicenow contains a complete matureAppium-equivalent automation framework.
More specifically:
mobile:iOS execute method is stable onstandalone WDA
The core claim of this PR is:
idevicenow has a real modern XCTest runner implementationidevicenow has meaningful WDA bootstrap and client support on top of thatSuggested reviewer framing
Please review this change primarily as:
ideviceThe most important questions for review are:
idevice?jktcpprerequisite stated clearly enough?Main files for review
Core XCTest / transport:
WDA layer:
CLI surface:
Workspace/docs: