Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
15 changes: 11 additions & 4 deletions README.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
---
title: Readme
description: <h1 align="center"> <a href="https://statsig.com/?ref=gh_docs"> <img width="300px" src="/images/Statsig_Favicon_Dark.png" /> </a> <div /> <a href="htt
---
## Welcome to the Statsig Docs

This page explains welcome to the statsig docs.


<h1 align="center">
<a href="https://statsig.com/?ref=gh_docs">
<img width="300px" src="/images/Statsig_Favicon_Dark.png" />
Expand All @@ -8,7 +15,7 @@
<a href="https://statsig.com/?ref=gh_docs">Statsig</a>
</h1>

Statsig empowers you to ship, measure, and learn from your releases using the same tools as the best tech companies in the world. With Statsig, you can run thousands of A/B tests, safely rollout features, and dive deep on core business metrics and user behavior—all on a single, unified platform. To get started with Statsig, visit our [website](https://statsig.com?ref=gh_docs) or [sign up](https://console.statsig.com/sign_up?ref=gh_docs).
Statsig empowers you to ship, measure, and learn from your releases using the same tools as the best tech companies in the world. With Statsig, you can run thousands of Experiments, safely rollout features, and dive deep on core business metrics and user behavior—all on a single, unified platform. To get started with Statsig, visit our [website](https://statsig.com?ref=gh_docs) or [sign up](https://console.statsig.com/sign_up?ref=gh_docs).

This repository is hosted at https://docs.statsig.com.

Expand All @@ -20,11 +27,11 @@ There are two ways to contribute:
1. Open a Pull Request on Github (via local dev, or by editing in the Github UI)
2. Use the Mintlify Visual Editor (Statsig Employees Only)

If you're used to docs-as-code or our old docusaurus setup, #1 is probably the right path - see below for local development instructions. If you'd prefer a snazzy visual interface, message Brock on slack to get setup with a Visual Editor account.
If you're used to docs-as-code or our old docusaurus setup, #1 is probably the right path - refer to the following example for local development instructions. If you'd prefer a snazzy visual interface, message Brock on slack to get setup with a Visual Editor account.

### Local development:

```
```bash
npm i -g mintlify
mintlify dev
```
Expand All @@ -47,4 +54,4 @@ You can find the allowlist of non-dictionary words in styles/config/vocabularies
#### Troubleshooting

- Mintlify dev isn't running - Run `mintlify install` it'll re-install dependencies.
- Page loads as a 404 - Make sure you are running in a folder with `docs.json`
- Page loads as a 404 - Make sure you are running in a folder with `docs.json`
1 change: 1 addition & 0 deletions access-management/discussions.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Discussions in Statsig
description: Feature rollout and experimentation are collaborative exercises where teams work together. Often this collaboration requires people taking screenshots
---

## Discussions
Expand Down
9 changes: 5 additions & 4 deletions access-management/guide.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Initial Setup Guide of your Workspace
description: <Info> Organizations and their related features are for Enterprise contracts only. Please reach out to our [support team](mailto:[email protected]),
---

<Info>
Expand All @@ -25,7 +26,7 @@ In Statsig, we have three constructs to help you organize your workspace and sca

**Organization** is an enterprise-level environment that allows companies to create project(s) and bring members to work inside the project.

**Project** is a workspace within the organization where the configs (e.g., feature gates, experiments, layers, etc.), metrics, and SDK keys you and your team created lives.
**Project** is a workspace within the organization where the configs (e.g., feature flags, experiments, layers, etc.), metrics, and SDK keys you and your team created lives.

**Team** is a group of members at the project level that can help your organization manage the permissions and ownership of resources.

Expand All @@ -35,7 +36,7 @@ In Statsig, we have three constructs to help you organize your workspace and sca
In Statsig, each project within the organization is isolated from one another.
This means that **none** of the resources or data is shared among different projects, even if they are part of the same organization.

The Statsig team believe that the **sensible default structure** for most customers is having an organization with a **single project** where multiple teams contribute and collaborate inside.
The Statsig team believe that the **sensible default structure** for most users is having an organization with a **single project** where multiple teams contribute and collaborate inside.

We believe that different teams and functions across your organization can stay well-organized within a single project by leveraging features such as **[teams](https://docs.statsig.com/access-management/teams), [roles](https://docs.statsig.com/access-management/projects#roles), [tags](https://docs.statsig.com/access-management/tags),** and **[templates](https://docs.statsig.com/experiments/templates/templates)**.
This also removes the risk of costly migrations if some projects ever need to be consolidated in the future for cross-functional collaborations.
Expand Down Expand Up @@ -73,7 +74,7 @@ New users who are provisioned via SSO will be assigned the *Member* Role unless
<img src="/images/access_management_project_view.png" alt="Organization project administration interface" />
</Frame>

A project in Statsig serves as a workspace that contains everything you and your team will create. This includes configs (e.g., feature gates, experiments, dynamic configs, layers), metrics, integrations, and more.
A project in Statsig serves as a workspace that contains everything you and your team will create. This includes configs (e.g., feature flags, experiments, dynamic configs, layers), metrics, integrations, and more.

When creating a new project, you can set its type to *Open* which would allow anyone in the organization to join freely, or to *Closed* which would allow people to join only by invitation or request.

Expand Down Expand Up @@ -140,7 +141,7 @@ Each **team** also has team-level **review settings** that can require reviews f
<img src="/images/access_management_sdk_env.png" alt="SDK environment interface" />
</Frame>

In Statsig, you have **Client API Keys** to initialize all Statsig [client SDKs](https://docs.statsig.com/client/introduction) and **Server Secret Keys** to initialize all Statsig [server SDKs](https://docs.statsig.com/server/introduction).
In Statsig, you have **Client API Keys** to initialize all Statsig [client SDKs](https://docs.statsig.com/client/introduction) and **Api Key Keys** to initialize all Statsig [server SDKs](https://docs.statsig.com/server/introduction).

We believe there is a **"Crawl, Walk, Run"** phase when it comes to configuring your API keys:

Expand Down
1 change: 1 addition & 0 deletions access-management/introduction.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Workspace Management Overview
sidebarTitle: Overview
description: Statsig provides a few different solutions for access management as you scale out adoption in your team/org/company. We have simple settings like auto
---

Statsig provides a few different solutions for access management as you scale out adoption in your team/org/company.
Expand Down
1 change: 1 addition & 0 deletions access-management/org-admin/experiment_policy.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Experiment Policy
description: <Info> Organization level Experiment Policies are an Enterprise only feature. </Info> * Set defaults for new experiments: While Statsig aims to provid
---

<Info>
Expand Down
1 change: 1 addition & 0 deletions access-management/org-admin/gates_policy.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Feature Gates Policy
description: <Info> Organization level Feature Gate Policies are an Enterprise only feature. </Info> Feature Gates Policy grants organization admins the ability to
---

<Info>
Expand Down
1 change: 1 addition & 0 deletions access-management/org-admin/organization_policies.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Organization Policies
description: <Info> Organization-level Experiment and Gate Policies are an Enterprise only feature. </Info> You can configure Organization-level Experiment and Gat
---

<Info>
Expand Down
3 changes: 2 additions & 1 deletion access-management/organizations.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Organization Settings & Administration
description: <Info> Organizations and their related features are for Enterprise contracts only. Please reach out to our [support team](mailto:[email protected]),
---

<Info>
Expand Down Expand Up @@ -27,7 +28,7 @@ You can view your Organization page by navigating to [Account Settings](https://
<img src="/images/access_management_account_info.png" alt="Account Settings organization management interface" />
</Frame>

#### Organization Information
### Organization Information
Your organization's **Info** sidebar includes the organization's name, SSO configuration, and other settings on access management and security settings.

As the organization's **Admin**, you can enable or disable SSO for all projects in the organization from this one place.
Expand Down
1 change: 1 addition & 0 deletions access-management/projects.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Project Access Management
description: <Info> This guide applies only to our on-demand customers. If you are an organization who has set up SSO, this guide will not apply to you. Please see
---
<Info>
This guide applies only to our on-demand customers. If you are an organization who has set up SSO, this guide will not apply to you. Please see our [SSO Guides](/access-management/sso/overview) for more information about how to manage access permissions through SSO.
Expand Down
1 change: 1 addition & 0 deletions access-management/scim/concepts.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: SCIM Concepts
description: <br /> Our SCIM implementation represents both Statsig users at the Organization level and Project level with their associated roles. There are two ma
---


Expand Down
1 change: 1 addition & 0 deletions access-management/scim/okta_scim_org_roles.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Okta SCIM Org Roles
sidebarTitle: Org Roles
description: For every user, Statsig surfaces a SCIM field named `statsigOrgRole`. Through this field, you can manage organization user roles. Currently, Okta can
---


Expand Down
1 change: 1 addition & 0 deletions access-management/scim/okta_scim_setup.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Okta SCIM Setup
sidebarTitle: Setup
description: This guide outlines the process for setting up SCIM (System for Cross-domain Identity Management) integration between Statsig and Okta. This integrati
---
This guide outlines the process for setting up SCIM (System for Cross-domain Identity
Management) integration between Statsig and Okta. This integration allows for automated
Expand Down
1 change: 1 addition & 0 deletions access-management/scim/okta_scim_team_management.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Okta SCIM Team Management
sidebarTitle: Team Management
description: <Note> Teams are specific Statsig groups that exist within projects. They are not shared across projects. They also have two possible roles: `Admin` a
---


Expand Down
1 change: 1 addition & 0 deletions access-management/scim/okta_scim_troubleshooting.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Okta SCIM Troubleshooting
sidebarTitle: Troubleshooting
description: | Error | Solution | |-------|----------| | User userID is not allowed to be created in this organization | Ensure that the users you are assigning to
---


Expand Down
1 change: 1 addition & 0 deletions access-management/scim/okta_scim_user_management.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Okta SCIM User and Project/Role Management
sidebarTitle: User and Project/Role Management
description: <Note> Users not assigned to the integration cannot be pushed into groups. </Note> - In Okta, go to the Statsig app's "Import" tab
---


Expand Down
1 change: 1 addition & 0 deletions access-management/scim/overview.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: SCIM User Provisioning
sidebarTitle: Overview
description: SCIM (System for Cross-domain Identity Management) is a standardized protocol that simplifies the automation of user provisioning and management acros
---


Expand Down
1 change: 1 addition & 0 deletions access-management/scim/scim-endpoints.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: SCIM API Overview
sidebarTitle: API Overview
description: The System for Cross-domain Identity Management (SCIM) specification is designed to make managing user identities in cloud-based applications and serv
---

The System for Cross-domain Identity Management (SCIM) specification is designed to make managing user identities in cloud-based applications and services easier. Statsig's SCIM API allows you to automate user provisioning and deprovisioning between your identity provider and Statsig.
Expand Down
1 change: 1 addition & 0 deletions access-management/sso/azuread.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Single Sign-On With Entra ID/Azure AD/Office 365
description: Microsoft Entra ID, formerly known as Azure AD, is a supported IdP for SSO into Statsig. - You will need to be the `Admin` of the Statsig Project you
---
Microsoft Entra ID, formerly known as Azure AD, is a supported IdP for SSO into Statsig.

Expand Down
1 change: 1 addition & 0 deletions access-management/sso/google.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: SSO with Google as your IdP
description: - You will need to be the `Admin` of the Statsig Project you intend to add SSO with Google Apps to.
---
## Requirements

Expand Down
1 change: 1 addition & 0 deletions access-management/sso/okta_sso.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Single Sign-On With Okta
description: - You will need to be the `Admin` of the Statsig Organization you intend to add SSO with Okta to. - You will need to be the Administrator of the Okta
---

## Requirements
Expand Down
1 change: 1 addition & 0 deletions access-management/sso/overview.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Single Sign-On With OIDC
description: <Info> SSO is an Enterprise feature. Please reach out to our [support team](mailto:[email protected]), your sales contact, or via our slack channel
---

<Info>
Expand Down
1 change: 1 addition & 0 deletions access-management/tags.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Tags in Statsig
description: Tags let you apply light-weight organization to your Statsig config (e.g. gates, experiments and metrics) to allow easy filtering by team (or organiza
---

## Tags for organization
Expand Down
1 change: 1 addition & 0 deletions access-management/teams.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Teams
description: <Info> Teams are an Enterprise-only feature. If you are on the Developer or Pro tiers, this guide will not apply to you. To upgrade to Enterprise, fee
---
<Info>
Teams are an Enterprise-only feature. If you are on the Developer or Pro tiers, this guide will not apply to you. To upgrade to Enterprise, feel free to reach out to our team [here](https://www.statsig.com/contact/demo).
Expand Down
1 change: 1 addition & 0 deletions ai-evals/offline-evals.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Offline Evals
description: Offline evals offer a quick, automated grading of model outputs on a fixed test set. They catch wins / regressions early—before any real users are exp
---

## What are Offline Evals
Expand Down
3 changes: 2 additions & 1 deletion ai-evals/online-evals.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: Online Evals
description: Online evals let you grade your model output in production on real world use cases. You can run the "live" version of a prompt, but can also shadow ru
---

## What are Online Evals
Expand Down Expand Up @@ -59,7 +60,7 @@ const liveOutput = client.completions.create(
);

// simulateneously run completions on the candidate prompts to get their output
```
```kotlin

**3. Score your output using graders**

Expand Down
1 change: 1 addition & 0 deletions ai-evals/overview.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: AI Evals Overview
sidebarTitle: Overview
description: <Info> AI Evals are currently in beta; Reach out in Slack to get access. </Info> Statsig AI Evals have a few core components to help iterate and serve
---

<Info>
Expand Down
1 change: 1 addition & 0 deletions ai-evals/prompts.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Prompts & Graders
sidebarTitle: Prompts & Graders
description: A Prompt is a way to represent an LLM prompt or a task in Statsig, with it's config. Prompts are similar to Dynamic Configs, and allow you to evaluate
---

## What is a Prompt in Statsig?
Expand Down
1 change: 1 addition & 0 deletions autotune/bandit-faq.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ keywords:
- owner:craig
last_update:
date: 2025-09-18
description: You will see diagnostic data appear in near-real time in the logstream on your bandit. Data on the models/results section will depend on your settings
---

### When should I see data show up in a bandit?
Expand Down
1 change: 1 addition & 0 deletions autotune/bandit-introduction.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ keywords:
- owner:vm
last_update:
date: 2025-09-18
description: Multi-Armed Bandits are solutions that automatically find the best variant among a group of candidates, balancing between "exploring" options and "exp
---

Multi-Armed Bandits are solutions that automatically find the best variant among a group of candidates, balancing between "exploring" options and "exploiting" the best option by dynamically allocating traffic. On Statsig, Bandits are used to pick the best user experience to drive a target metric or action.
Expand Down
1 change: 1 addition & 0 deletions autotune/contextual-bandit.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ keywords:
- owner:vm
last_update:
date: 2025-09-18
description: Autotune UI implements a variant of the LinUCB algorithm. This estimates a user's outcome for each variant, and incorporates the model's uncertainty t
---

## Model
Expand Down
9 changes: 5 additions & 4 deletions autotune/contextual/getting-started.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ keywords:
- owner:vm
last_update:
date: 2025-09-18
description: Getting started with Autotune AI can be done very quickly. <Warning> Statsig supports contextual autotune in all Client SDKs, but only in the followin
---

Getting started with Autotune AI can be done very quickly.
Expand Down Expand Up @@ -99,7 +100,7 @@ We assume you have your server secret key for the following code. Before running

First, import and initialize Statsig:

```
```javascript
from statsig_python_core import Statsig, StatsigUser

key = <your_key_here>
Expand All @@ -111,14 +112,14 @@ statsig.initialize().wait()

Then, create a user object and fetch your config:

```
```text
user = StatsigUser('user_id', custom={'key1': 'value1', 'key2': 'value2'})
cfg = statsig.get_experiment(user, autotune_name)
```

Now you have your cfg and can apply it!

```
```python
color = cfg.get_string("color", "default color")
print(f"Going to use {color} for my color now")
```
Expand All @@ -134,7 +135,7 @@ That's it! Your code is now serving personalized variants to your users.

Statsig requires a few hundred units to train a model, and will also not start training until those units' attribution window has elapsed. If you want to test the functionality, we highly recommend "faking a test" to confirm things work like you expect - use logic like

```
```text
fetch_autotune_value()
if(user country == 'us'):
log_click()
Expand Down
1 change: 1 addition & 0 deletions autotune/contextual/introduction.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ keywords:
- owner:vm
last_update:
date: 2025-09-18
description: Contextual Multi-Armed Bandits are a subset of Multi-Armed-Bandits which use context about a user to personalize their experience. This is generally a
---

Contextual Multi-Armed Bandits are a subset of Multi-Armed-Bandits which use context about a user to personalize their experience. This is generally achieved by predicting outcomes from among the variants, and picking the best outcome while factoring for uncertainty. Specifically, they will tend to prefer a slightly worse prediction that has a lot of uncertainty, thereby exploring that variant.
Expand Down
1 change: 1 addition & 0 deletions autotune/contextual/methodology.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ keywords:
- owner:craig
last_update:
date: 2025-09-18
description: This page covers the high level approach that Statsig takes to running contextual bandits across cloud and warehouse native. Specifics of implementati
---

## Methodology
Expand Down
3 changes: 2 additions & 1 deletion autotune/contextual/monitoring.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ keywords:
- owner:vm
last_update:
date: 2025-09-18
description: There are three primary ways we recommend you monitor autotune performance. The best way to evaluate if a bandit is working is seeing if it drives mor
---

There are three primary ways we recommend you monitor autotune performance.
Expand All @@ -17,7 +18,7 @@ This is the gold standard of measurement and is highly encouraged.

Standard practice is to wrap the autotune in a experiment with a binary parameter, either as 50/50 or a 90/10 holdback. You can link the experiment to the autotune to get the results on the autotune page. In code, this might look like:

```
```yaml
experiment_value = statsig.get_experiment('wrapping_experiment').get('flag')
default_param = '..."
if(experiment_value):
Expand Down
Loading