CI - cargo test runs C# and TypeScript tests

[bfops]

Description of Changes

C# and TypeScript tests can now be run via cargo test.

This also involved updating some test code to take a server url override, so they could use the existing SpacetimeDBGuard (which starts the server on an arbitrary port rather than a hardcoded 3000).

• master
  • #4972 cargo ci dlls -> cargo regen csharp dlls
    • #4971 Translate some bash scripts to Rust
      • [This PR] CI - cargo test runs C# and TypeScript tests
        • #4948 CI - Run unity tests through cargo test

API and ABI breaking changes

None. CI only.

Expected complexity level and risk

2

Testing

Failure reporting was checked with synthetic failing tests after the standard harness checks.

TypeScript harness:

• Run the wrapped TypeScript suite with cargo test -p spacetimedb-typescript-tests
• This runs pnpm build, pnpm vitest run with a JUnit report, then pnpm vitest typecheck --run. Reports are written under target/language-tests/typescript/.

C# harness:

• Pre-build the local server binaries before running the full C# harness:
  • cargo build -p spacetimedb-cli -p spacetimedb-standalone --features spacetimedb-standalone/allow_loopback_http_for_tests
• Run the wrapped C# suite with cargo test -p spacetimedb-csharp-tests
• This runs the bindings tests, packs/restores the SDK solution against local NuGet packages, runs the SDK tests, and then runs the C# regression script against a temporary SpacetimeDB server started through spacetimedb-guard. TRX reports are written under target/language-tests/csharp/.

Synthetic failure check

• Added a temporary failing Vitest test and ran:
  • cargo test -p spacetimedb-typescript-tests --test typescript -- --filter SyntheticHarnessFailure
• Verified the command exited with status 1 and reported the failure like this:

FAIL  tests/synthetic_harness_failure.test.ts > SyntheticHarnessFailure reports TypeScript assertion failures
AssertionError: expected 'actual TypeScript failure' to be 'expected TypeScript success'

Expected: "expected TypeScript success"
Received: "actual TypeScript failure"

command failed in .../crates/bindings-typescript:
  pnpm test -- --reporter=default --reporter=junit --outputFile=... -t SyntheticHarnessFailure
status: exit status: 1

• Added a temporary failing xUnit test and ran:
  • cargo test -p spacetimedb-csharp-tests --test csharp -- --filter SyntheticHarnessFailure
• Verified the command exited with status 1 and reported the failure like this:

SyntheticHarnessFailureTests.SyntheticHarnessFailureReportsCsharpAssertionFailures [FAIL]
Error Message:
 Assert.Equal() Failure: Strings differ
Expected: "expected C# success"
Actual:   "actual C# failure"

command failed in .../sdks/csharp:
  dotnet test -warnaserror --results-directory ... --logger trx;LogFileName=sdk.trx --no-restore --filter SyntheticHarnessFailure
status: exit status: 1

[jdetter]

Very unlikely to be related to this PR but the typescript tests don't pass on Windows:

src/sdk/logger.ts:1:42 - error TS2307: Cannot find module 'safe-stable-stringify' or its corresponding type declarations.

1 import { stringify as ssStringify } from 'safe-stable-stringify';
                                           ~~~~~~~~~~~~~~~~~~~~~~~

src/sdk/logger.ts:76:23 - error TS7006: Parameter 'key' implicitly has an 'any' type.

76   ssStringify(value, (key, current) => {
                         ~~~

src/sdk/logger.ts:76:28 - error TS7006: Parameter 'current' implicitly has an 'any' type.

76   ssStringify(value, (key, current) => {
                              ~~~~~~~


Found 3 errors in the same file, starting at: src/sdk/logger.ts:1

 ELIFECYCLE  Command failed with exit code 1.
command failed in C:\Users\boppy\clockwork\SpacetimeDB\crates/bindings-typescript:
  pnpm build
status: exit code: 1
error: test failed, to rerun pass `-p spacetimedb-typescript-tests --test typescript`

Caused by:
  process didn't exit successfully: `C:\Users\boppy\clockwork\SpacetimeDB\target\debug\deps\typescript-f5bc0a56063fb3d6.exe` (exit code: 1)

[jdetter]

csharp tests don't work on windows due to executing a bash script:

Workload updates are available. Run `dotnet workload list` for more information.
: invalid option nameconfig.sh: line 1: set: pipefail
command failed in C:\Users\boppy\clockwork\SpacetimeDB\sdks/csharp:
  bash ./tools~/write-nuget-config.sh ../..
status: exit code: 2
error: test failed, to rerun pass `-p spacetimedb-csharp-tests --test csharp`

Caused by:
  process didn't exit successfully: `C:\Users\boppy\clockwork\SpacetimeDB\target\debug\deps\csharp-2603e4bb2c95094a.exe` (exit code: 1)

[jdetter]

I think ideally this would work on both Unix and Windows based systems. There seem to be a few problems preventing this from working on Windows and I think that should maybe be a requirement for this PR. What do you think?

[jdetter]

In order for this to work on Windows as well it might be best to translate this script into rust and then inline it here, what do you think? Either that or we need an equivalent powershell script so that this will also work on windows and we need to invoke the powershell script instead if we're running on windows.

[bfops]

Yeah, that's fair.

I've created #4970 to do this more generally for all of our bash scripts.

[bfops]

translating these ones in #4971

[jdetter]

same here - what do you think about translating this script into rust and inlining it here?

[bfops]

Very unlikely to be related to this PR but the typescript tests don't pass on Windows:

@jdetter Interesting. Created #4969 to track.

[jdetter]

Other than the script failures on windows I don't have any objections left to this PR, LGTM. I've tested in places where I've left PR feedback and whatnot. I would like to test one more time once the PR with the script changes merges into this.