Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 81 additions & 0 deletions .claude/skills/qabot-fixture/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
---
name: qabot-fixture
description: Create a new qabot E2E test fixture interactively. Guides the user through defining test steps, expected outcomes, and saves the YAML fixture file. Use when user says "create fixture", "new test", "qabot fixture", or "/qabot-fixture".
allowed-tools: Bash, Read, Write, Edit, Glob, Grep, AskUserQuestion
---

# qabot Fixture Builder

You are helping the user create a new E2E test fixture for qabot. Fixtures are YAML files that define test steps for agent-browser to execute.

## Process

### 1. Gather context

Ask the user what they want to test. If they give a vague answer like "test the earn page", ask clarifying questions:
- What specific functionality? (page loads, specific interaction, full flow)
- What route? (e.g., /earn, /trade, /dashboard) - note: the base URL is a run-level arg, not per-fixture
- Does it need a connected wallet? (most tests do - use `depends_on: wallet-health.yaml`)

### 2. Read existing fixtures for reference

```bash
ls e2e/fixtures/*.yaml
```

Read them to understand the established patterns before building a new one.

### 3. Build steps interactively

For each step, ask the user:
- What should the agent do? (this becomes the `instruction`)
- What should be true after? (this becomes the `expected`)

Write instructions in natural language - agent-browser interprets them. No CSS selectors needed.

### 4. Name, describe, and save

Ask for:
- A fixture name (kebab-case for the filename, human-readable for the `name` field)
- A short description of what this fixture tests (one line)

Save to `e2e/fixtures/<name>.yaml`.

## Fixture YAML Format

```yaml
name: Human Readable Name
description: One-line description of what this fixture tests
route: /<route>
depends_on:
- wallet-health.yaml # runs BEFORE main steps (wallet unlock, onboarding dismiss)
post_depends_on:
- evm-chains-regression.yaml # runs AFTER main steps (optional)
steps:
- name: Step name
instruction: >
Natural language instruction for agent-browser.
Be specific about what to click, fill, or verify.
Reference UI elements by their visible text, not selectors.
expected: What should be true after this step completes
screenshot: true
```

### Composability

Fixtures can declare dependencies that run before or after the main steps:

- **`depends_on`** - fixtures that run BEFORE the main steps. Most fixtures should include `wallet-health.yaml` which handles onboarding dismiss + wallet unlock.
- **`post_depends_on`** - fixtures that run AFTER all main steps (e.g. regression suites, cleanup).
- Dependencies are resolved recursively and deduplicated (each fixture runs at most once).
- All fixtures run in one browser session - page state carries over.
- Step indices are continuous across all fixtures.

## Guidelines

- Instructions should be specific enough for an AI agent to follow
- Reference UI elements by visible text (e.g., "click the ETH selector", not "click #asset-select")
- For inputs that use React controlled components, note that `press` (char by char) works but `fill` may not trigger onChange
- Every step should have `screenshot: true` so test runs capture visual evidence
- Keep steps atomic - one action per step where possible
- The `expected` field is what the agent checks in the accessibility snapshot to determine pass/fail
453 changes: 453 additions & 0 deletions .claude/skills/qabot/SKILL.md

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,7 @@
- Main branch is `develop` - use this for PRs
- Branch naming: Use descriptive names (e.g., `feat_gridplus`, `fix_wallet_connect`)
- When opening PRs (via `gh`, Aviator `av`, or any CLI tool), ALWAYS use the `.github/PULL_REQUEST_TEMPLATE.md` template as the base for the PR body
- **Editing PR descriptions**: `gh pr edit --body` fails on this repo due to a deprecated Projects Classic GraphQL error. Use the REST API instead: `gh api repos/shapeshift/web/pulls/<number> -X PATCH -F "body=@/path/to/body.md"` (write the body to a temp file first)

### UI/UX Standards
- Account for light/dark mode using `useColorModeValue` hook
Expand Down
140 changes: 140 additions & 0 deletions e2e/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
# qabot - E2E QA Testing

qabot is an AI-powered QA testing platform for ShapeShift Web. Operators (humans using Claude Code) run E2E test fixtures via `agent-browser` and post results to the qabot dashboard at https://qabot-kappa.vercel.app/.

The `/qabot` skill in Claude Code handles everything - executing fixtures, capturing screenshots, and uploading results.

## Setup

### 1. Environment Variables

You need three secrets and two config vars. We recommend keeping secrets in `~/.secrets` and sourcing that from your shell rc - keeps secrets out of dotfiles you might commit:

```bash
# ~/.secrets
export QABOT_API_KEY="<ask gomes for the shared key>"
export NATIVE_WALLET_PASSWORD="<your native wallet password>"
```

Then source it from `~/.zshrc` (or `~/.bashrc`):

```bash
# ~/.zshrc
[ -f ~/.secrets ] && source ~/.secrets
```

These two don't need to be in secrets - put them wherever you source env vars:

```bash
export QABOT_URL="https://qabot-kappa.vercel.app"
export QABOT_OPERATOR="<your-name>" # e.g. "john", "clawdbot"
```

Summary:

| Variable | Required | Secret? | Description |
|---|---|---|---|
| `QABOT_API_KEY` | Yes | Yes | Shared API key for dashboard write access |
| `NATIVE_WALLET_PASSWORD` | Yes | Yes | Password for the native test wallet in agent-browser |
| `QABOT_URL` | Yes | No | Dashboard URL (default: `https://qabot-kappa.vercel.app`) |
| `QABOT_OPERATOR` | Yes | No | Your operator name - labels who ran the test |

### 2. agent-browser

[agent-browser](https://github.com/anthropics/agent-browser) is the headless browser automation tool that executes test steps.

```bash
npm install -g agent-browser
```

Requires v0.14.0+.

### 3. Wallet Profile Setup (one-time)

agent-browser stores wallet state per profile per origin. You need to import the native test wallet once for each origin you'll test against:

```bash
# Open a headed browser session
agent-browser --session qabot --profile ~/.agent-browser/profiles/qabot --headed open http://localhost:3000

# In the browser:
# 1. Create or import a native wallet
# 2. Set password to $NATIVE_WALLET_PASSWORD
# 3. Close the browser
```

Repeat for any other origin you want to test:

```bash
agent-browser --session qabot --profile ~/.agent-browser/profiles/qabot --headed open https://gome.shapeshift.com
agent-browser --session qabot --profile ~/.agent-browser/profiles/qabot --headed open https://release.shapeshift.com
```

After setup, the profile persists at `~/.agent-browser/profiles/qabot/` and subsequent runs reuse it.

### 4. ShapeShift Web Running

For local testing, start the dev server:

```bash
yarn dev
```

Or point `BASE_URL` at a staging environment:

```bash
BASE_URL=https://gome.shapeshift.com
```

## Running Tests

### Via Claude Code Skill

```
/qabot smoke-test.yaml
/qabot eth-to-fox-swap.yaml
/qabot evm-chains-regression.yaml
```

Or just `/qabot` for interactive mode - Claude will help you pick or craft a fixture.

### Available Fixtures

| Fixture | Description | Dependencies |
|---|---|---|
| `smoke-test.yaml` | Onboarding dismiss, wallet unlock, trade page verify | None |
| `eth-to-fox-swap.yaml` | Full ETH to FOX swap end-to-end | smoke-test |
| `thorchain-solana-swapper.yaml` | SOL/RUNE cross-chain swaps via THORChain | smoke-test |
| `evm-chains-regression.yaml` | $1 same-chain swaps across 7 EVM chains | smoke-test |

Fixtures live in `e2e/fixtures/`. Create new ones with `/qabot-fixture`.

## How It Works

1. The `/qabot` skill reads a YAML fixture and resolves dependencies
2. agent-browser opens a Chrome session with the test wallet profile
3. Each step is executed, verified via accessibility snapshots, and screenshotted
4. Screenshots are uploaded to the qabot dashboard (Vercel Blob storage)
5. Step results (pass/fail, agent observations, screenshots) are pushed per-step
6. The dashboard at https://qabot-kappa.vercel.app/ shows live progress

## Fixture Format

```yaml
name: My Test
description: What this tests
route: /trade
depends_on:
- smoke-test.yaml # runs BEFORE main steps
post_depends_on:
- cleanup.yaml # runs AFTER main steps
steps:
- name: Do something
instruction: >
Natural language instruction for agent-browser.
Click the swap button, type 1 in the amount field, etc.
expected: What should be visible/true after this step
screenshot: true
```

Dependencies are resolved recursively and deduplicated. All steps run in one browser session with continuous step indices.
Loading
Loading