Skip to content

Conversation

@m1r0
Copy link
Member

@m1r0 m1r0 commented Oct 30, 2025

Resolves DOMAINS-1780

Proposed Changes

  • Add the six variations of the experiment.
  • Because this is probably the first time we're doing an experiment inside a Calypso package, some TypeScript fixes were needed related to importing the Explat client.

Why are these changes being made?

This is part of the domain search escape hatch experiment: pbxNRc-5td-p2

Testing Instructions

  • Go to /setup/onboarding/domains and search for a domain.
  • Make sure you're seeing the control version of the experiment:
image
  • Go to 22373-explat-experiment.
  • Assign yourself to the treatment_above_paid_domain_area by clicking the manual assignment Bookmarklet.
  • Refresh the page (possibly a few times), and you should see the treatment_above_paid_domain_area variation:
image
  • Assign yourself to treatment_paid_domain_area, refresh the page, and make sure you see that variation:
image
  • Assign yourself to treatment_paid_domain_area_free_emphasis, refresh the page, and make sure you see that variation:
image
  • Assign yourself to treatment_paid_domain_area_free_emphasis_extra_cta , refresh the page, and make sure you see that variation:
image
  • Assign yourself to treatment_paid_domain_area_skip_emphasis , refresh the page, and make sure you see that variation:
image

Pre-merge Checklist

  • Has the general commit checklist been followed? (PCYsg-hS-p2)
  • Have you written new tests for your changes?
  • Have you tested the feature in Simple (P9HQHe-k8-p2), Atomic (P9HQHe-jW-p2), and self-hosted Jetpack sites (PCYsg-g6b-p2)?
  • Have you checked for TypeScript, React or other console errors?
  • Have you tested accessibility for your changes? Ensure the feature remains usable with various user agents (e.g., browsers), interfaces (e.g., keyboard navigation), and assistive technologies (e.g., screen readers) (PCYsg-S3g-p2).
  • Have you used memoizing on expensive computations? More info in Memoizing with create-selector and Using memoizing selectors and Our Approach to Data
  • Have we added the "[Status] String Freeze" label as soon as any new strings were ready for translation (p4TIVU-5Jq-p2)?
    • For UI changes, have we tested the change in various languages (for example, ES, PT, FR, or DE)? The length of text and words vary significantly between languages.
  • For changes affecting Jetpack: Have we added the "[Status] Needs Privacy Updates" label if this pull request changes what data or activity we track or use (p4TIVU-aUh-p2)?

@m1r0 m1r0 self-assigned this Oct 30, 2025
@github-actions
Copy link

github-actions bot commented Oct 30, 2025

@matticbot
Copy link
Contributor

matticbot commented Oct 30, 2025

This PR modifies the release build for the following Calypso Apps:

For info about this notification, see here: PCYsg-OT6-p2

  • blaze-dashboard
  • command-palette-wp-admin
  • help-center
  • notifications
  • odyssey-stats
  • wpcom-block-editor

To test WordPress.com changes, run install-plugin.sh $pluginSlug add/domains-page-escape-hatch-experiment on your sandbox.

@m1r0 m1r0 force-pushed the add/domains-page-escape-hatch-experiment branch 2 times, most recently from ac90a23 to 3e86ab1 Compare October 30, 2025 16:21
@m1r0 m1r0 force-pushed the add/domains-page-escape-hatch-experiment branch from d9a719b to 1a7f959 Compare October 31, 2025 14:15
@m1r0 m1r0 force-pushed the add/domains-page-escape-hatch-experiment branch from 402f84f to 56124a2 Compare October 31, 2025 14:40
@m1r0 m1r0 marked this pull request as ready for review October 31, 2025 15:21
@m1r0 m1r0 requested review from a team as code owners October 31, 2025 15:21
@m1r0 m1r0 requested review from samacart and removed request for a team October 31, 2025 15:21
@matticbot matticbot added the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Oct 31, 2025
Copy link
Contributor

@leonardost leonardost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested all the variants and all of them show correctly according to the experiment. LGTM! Thanks for working on this. Left just a couple of non-blocking comments.

import { isOnboardingFlow } from '@automattic/onboarding';
import { useMemo } from 'react';
import { getFlowFromURL } from 'calypso/landing/stepper/utils/get-flow-from-url'; // eslint-disable-line no-restricted-imports
import { useExperiment } from 'calypso/lib/explat'; // eslint-disable-line no-restricted-imports
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you had to import the Explat code from Calypso here. I'm not sure if this might have implications for the build process, but everything seems to be working fine, at least in my tests 🙂

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I know, this rule is to avoid circular dependencies. Because Explat is so light, I think this won't be an issue.

@m1r0
Copy link
Member Author

m1r0 commented Oct 31, 2025

ExPlat Implementation Checklist

Please use this checklist as a guide for implementing ExPlat experiments.

All Platforms

ExPlat profile

  • Experiment name matches: Confirm experiment name matches the experiment profile in ExPlat.
  • Experiment platform matches: Confirm the experiment platform matches the experiment profile in ExPlat.

Assignment

  • Null assignments handled gracefully: Properly handle when the assignment function returns null (control group or no assignment).
  • All possible variations handled: Code handles all variations, plus null assignment.
  • Loading states implemented: Proper UX during assignment fetching.
  • No hardcoded assignments: Never force specific variations in production.
  • Variation names follow naming conventions: Use snake_case pattern for all variation name.
  • Manual assignment compatibility: Code works with manual assignment methods (bookmarklet, API).
  • All variations testable: All variations can be manually assigned and tested.

Eligibility and targeting

  • Client-side eligibility doesn't create bias: Eligibility checks are consistent and don't inadvertently exclude user groups.
  • Audience targeting respected: Implementation matches experiment's configured audience (new users, existing users, etc).
  • Cross-boundary handling: Appropriate handling for experiments that may cross login/logout. boundaries.
  • Anonymous user support: Proper handling of logged-out users when required.
  • E2E test exclusion: E2E tests (User-Agent containing wp-e2e-tests) properly excluded.

Tracking and Events

  • Exposure events fire after assignment: If your experiment has an exposure event, confirm it fires after the assignment is given to the user.
  • Event timing: Tracking events fire at the correct time in the user flow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

[Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants