🔧 chore(qa): add the beta-tester E2E testing skill

[mauretto78]

Summary

Adds beta-tester, a Claude Code skill that drives a real Chrome browser through a running
MateCat instance to test a given feature or flow, severity-scores the bugs it confirms, writes the
regression test in whichever harness will actually run it, and offers to file the confirmed ones
into the Matecat team backlog on Asana.

Two files, both under .claude/. No application code is touched.

Type

• feat — new user-facing feature
• fix — bug fix
• refactor — restructure without behavior change
• chore — build, deps, config, docs
• perf — performance improvement
• test — test coverage

Changes

File	Change
.claude/skills/beta-tester/SKILL.md	New skill. Usage guide for the team; a three-tier environment argument (local / staging / production) with a pre-flight gate that verifies the target rather than trusting it, and checks the daemons and the Vite bundle are not stale; execution workflow requiring an oracle before any assertion is scored; a bug-shape generator built from this repo's documented contracts; a severity model with floors; the reporting format; and a Filing to Asana section mapping confirmed findings onto the team backlog.
.claude/settings.json	Pre-allows five read-only Asana tools (get_me, get_project, get_projects, get_task, search_tasks) so a run does not stall on prompts while checking for duplicate tasks. Task creation and commenting are deliberately not pre-allowed — the permission prompt is the confirmation gate.

Testing

• vendor/bin/phpunit --exclude-group=ExternalServices --no-coverage passes
• ./vendor/bin/phpstan passes (0 errors, with baseline)
• Manual testing performed (describe below)
• New tests added for changed behavior
• Regression tests added for bug fixes

The two suites are unchecked because this PR adds Markdown and a permissions entry, and touches no
PHP or JS — there is nothing for them to exercise.

The skill was run end to end against the local stack (develop @ 43c500cc), driving Chrome through
the full project-creation flow: name → upload → convert → analyze → open the job in the editor. It
produced a correct pass, and independently found one real defect in
public/js/components/analyze/AnalyzeHeader.js (a mailto: href written as a literal string
instead of a JSX expression, at lines 191 and 205 — line 34 of the same file does it correctly).
That fix is not in this PR; it is reported separately so this change stays purely additive.

The safety gates were exercised: invoking with production refuses, and the pre-flight mismatch
check stops a run whose declared tier does not match the resolved host. For the Asana section, the
board, section and every custom-field option GID were read from the live API rather than assumed,
and the duplicate-search path was executed against the real backlog.

AI Disclosure

• No AI tools were used in this PR
• AI tools were used — name the agent/tool below

Claude Code (claude-opus-5)

Notes

The skill is deliberately scoped to what it can honestly do:

• It refuses production. Its third step is deliberately trying to break things, and
  www.matecat.com holds real customer data. Naming production in the skill is the point — so it
  recognises and refuses it rather than being silent. Post-release smoke-checking stays with
  deploy-matecat.
• It never emits a Playwright spec. This repo has no browser harness and no browser job in
  CI, so a spec nobody runs would look like coverage without being it. Regression tests are
  routed to Jest/RTL or PHPUnit instead, and a genuinely browser-only bug gets a precise
  reproduction and an explicit statement that no harness here can hold it.
• It will not invent expectations. Anything it cannot anchor to a PR, issue, spec or
  documented contract is reported as [OBSERVATION], left unscored, and never filed.
• It never files an Asana task without showing it first, and never sets P0 - Emergency.
  P0 means a production incident, and a skill barred from production cannot have observed one —
  escalating to P0 stays a human decision. Findings whose cause turns out to be environmental (a
  stale daemon, a disconnected VPN) are never filed either; during development those were the most
  convincing false positives by a wide margin.

.claude/ is covered by the .* rule on line 1 of .gitignore, so these files need
git add -f. That matches the deploy-matecat skill, which lives at the same path.

Follow-up, not blocking: with the VPN down, POST /api/app/convert-file hangs with no timeout
and no user-facing error — the upload sits at "Importing" indefinitely. Worth deciding whether
that endpoint should have a client-side timeout.

[mauretto78]

How to use the beta-tester skill

Quick guide for anyone who wants to try it. This is a copy of the Usage section at the top
of .claude/skills/beta-tester/SKILL.md —
the file is the version that stays up to date; this comment is here so you don't have to open the diff.

---

What it does

Ask Claude Code to beta-test something and it drives a real Chrome browser through it — clicking,
typing and navigating the way a person would — then tells you what it found.

/beta-tester <environment> <what you want tested>

The environment is optional and defaults to local:

argument	points at	what it is allowed to do
local (default)	your docker stack at dev.matecat.com	anything, including deliberately trying to break it
staging	freddy.matecat-staging.com	happy path and gentle edge cases; it asks first before anything that could disturb a colleague
production	www.matecat.com	nothing — it refuses, and says why

Some examples:

/beta-tester create a project with this XLIFF and check the job opens
/beta-tester test the job-scoped MT settings panel against PR #4784
/beta-tester staging check that login still works after the OAuth change

Before it can run

1. Install the Claude extension for Chrome — once, from the Chrome Web Store:
   https://chromewebstore.google.com/detail/claude/fcoeoabgfenejglbffodgkkbkcdhcgfn

   Then restart Chrome so the native-messaging host registers, and sign in to claude.ai. Use
   the same account you use for Claude Code: the extension keeps its own session, and if the
   two differ you get "signed in as a different account" and no browser actions will run. If you
   use more than one Chrome profile, check you are in the profile the extension is installed in.

2. Start Claude Code with the browser attached — claude --chrome. Without it the skill stops
   and tells you, rather than quietly writing a script and calling it a test run. /chrome reports
   the connection status if you need to check it.

3. Connect the VPN. File conversion goes through a service only reachable on it; without it,
   uploads hang at "Importing" with no error.

4. Have the stack running — docker compose up in docker_matecat/MateCat-Noble.

5. Credentials are optional. If your Chrome is already signed in to Matecat, it just works.
   Otherwise put them in ~/.claude/matecat-beta-tester.env (mode 0600, never committed).

If you have just switched branches, restart the container and rebuild the frontend first — the
skill checks for both and will tell you if either is stale.

What you get back

It opens by stating which environment it is on and what it verified, writes its test checklist
before touching the browser, then reports each bug the moment it finds it — severity label,
score, steps to reproduce, expected vs. actual — and closes with a summary table.

Two things it deliberately will not do. It will not report something as a bug when it cannot point
at a source of truth for the expected behaviour; those are labelled [OBSERVATION] so you can tell
them apart at a glance. And it will not write a Playwright spec — this repo has no browser test
harness, so regression tests go to Jest or PHPUnit where they will actually run.

---

Questions or something that behaves oddly: comment here and I'll fix the skill.

[github-actions]

🧪 Test-Guard Report

✅ PASS — All changed source files have adequate test coverage.

Coverage Analysis: ⏭️ SKIP

No source files to analyze — all changed files are tests or excluded.

Test File Matching: ✅ PASS

No source files to check

Per-File Evaluation: ✅ PASS

No source files to evaluate.

Result: ✅ PASS

[github-actions]

🧪 Test-Guard Report

✅ PASS — All changed source files have adequate test coverage.

Coverage Analysis: ⏭️ SKIP

No source files to analyze — all changed files are tests or excluded.

Test File Matching: ✅ PASS

No source files to check

Per-File Evaluation: ✅ PASS

No source files to evaluate.

Result: ✅ PASS