Skip to content

resource control: support more mode for BURSTABLE #21138

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 27 additions & 21 deletions information-schema/information-schema-resource-groups.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,17 @@ DESC resource_groups;
```

```sql
+------------+-------------+------+------+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------+-------------+------+------+---------+-------+
| NAME | varchar(32) | NO | | NULL | |
| RU_PER_SEC | bigint(21) | YES | | NULL | |
| PRIORITY | varchar(6) | YES | | NULL | |
| BURSTABLE | varchar(3) | YES | | NULL | |
+------------+-------------+------+------+---------+-------+
3 rows in set (0.00 sec)
+-------------+--------------+------+------+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+--------------+------+------+---------+-------+
| NAME | varchar(32) | NO | | NULL | |
| RU_PER_SEC | varchar(21) | YES | | NULL | |
| PRIORITY | varchar(6) | YES | | NULL | |
| BURSTABLE | varchar(3) | YES | | NULL | |
| QUERY_LIMIT | varchar(256) | YES | | NULL | |
| BACKGROUND | varchar(256) | YES | | NULL | |
+-------------+--------------+------+------+---------+-------+
6 rows in set (0.00 sec)
```

## Examples
Expand All @@ -35,11 +37,11 @@ SELECT * FROM information_schema.resource_groups; -- View all resource groups. T
```

```sql
+---------+------------+----------+-----------+
| NAME | RU_PER_SEC | PRIORITY | BURSTABLE |
+---------+------------+----------+-----------+
| default | UNLIMITED | MEDIUM | YES |
+---------+------------+----------+-----------+
+---------+------------+----------+----------------+-------------+------------+
| NAME | RU_PER_SEC | PRIORITY | BURSTABLE | QUERY_LIMIT | BACKGROUND |
+---------+------------+----------+----------------+-------------+------------+
| default | UNLIMITED | MEDIUM | YES(UNLIMITED) | NULL | NULL |
+---------+------------+----------+----------------+-------------+------------+
```

```sql
Expand Down Expand Up @@ -68,11 +70,11 @@ SELECT * FROM information_schema.resource_groups WHERE NAME = 'rg1'; -- View the
```

```sql
+------+------------+----------+-----------+-------------+
| NAME | RU_PER_SEC | PRIORITY | BURSTABLE | QUERY_LIMIT |
+------+------------+----------+-----------+-------------+
| rg1 | 1000 | MEDIUM | NO | NULL |
+------+------------+----------+-----------+-------------+
+------+------------+----------+-----------+-------------+------------+
| NAME | RU_PER_SEC | PRIORITY | BURSTABLE | QUERY_LIMIT | BACKGROUND |
+------+------------+----------+-----------+-------------+------------+
| rg1 | 1000 | MEDIUM | NO | NULL | NULL |
+------+------------+----------+-----------+-------------+------------+
1 row in set (0.00 sec)
```

Expand All @@ -81,8 +83,12 @@ The descriptions of the columns in the `RESOURCE_GROUPS` table are as follows:
* `NAME`: the name of the resource group.
* `RU_PER_SEC`: the backfilling speed of the resource group. The unit is RU/second, in which RU means [Request Unit](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru).
* `PRIORITY`: the absolute priority of tasks to be processed on TiKV. Different resources are scheduled according to the `PRIORITY` setting. Tasks with high `PRIORITY` are scheduled first. For resource groups with the same `PRIORITY`, tasks will be scheduled proportionally according to the `RU_PER_SEC` configuration. If `PRIORITY` is not specified, the default priority is `MEDIUM`.
* `BURSTABLE`: whether to allow the resource group to overuse the available system resources.
* `BURSTABLE`: whether to allow the resource group to overuse the available remaining system resources. The following three modes are supported. If no value is specified for `BURSTABLE`, the `MODERATED` mode is enabled by default.

- `OFF`: indicates that the resource group is not allowed to overuse any remaining system resources.
- `MODERATED`: indicates that the resource group is allowed to overuse remaining system resources to a limited extent, with priority given to allocated resources within its quota.
- `UNLIMITED`: indicates that the resource group can overuse remaining system resources without limitation, and the excess system resources are treated as equal to the resources within the quota during allocation.

> **Note:**
>
> TiDB automatically creates a `default` resource group during cluster initialization. For this resource group, the default value of `RU_PER_SEC` is `UNLIMITED` (equivalent to the maximum value of the `INT` type, that is, `2147483647`) and it is in `BURSTABLE` mode. All requests that are not bound to any resource group are automatically bound to this `default` resource group. When you create a new configuration for another resource group, it is recommended to modify the `default` resource group configuration as needed.
> TiDB automatically creates a `default` resource group during cluster initialization. For this resource group, the default value of `RU_PER_SEC` is `UNLIMITED` (equivalent to the maximum value of the `INT` type, that is, `2147483647`) and its `BURSTABLE` mode is `UNLIMITED`. All requests that are not bound to any resource group are automatically bound to this `default` resource group. You cannot delete the `default` resource group, but can modify its RU configuration.
16 changes: 9 additions & 7 deletions sql-statements/sql-statement-alter-resource-group.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,9 @@ DirectResourceGroupOption ::=
"RU_PER_SEC" EqOpt LengthNum
| "PRIORITY" EqOpt ResourceGroupPriorityOption
| "BURSTABLE"
| "BURSTABLE" EqOpt Boolean
| "BURSTABLE" EqOpt "MODERATED"
| "BURSTABLE" EqOpt "UNLIMITED"
| "BURSTABLE" EqOpt "OFF"
| "QUERY_LIMIT" EqOpt '(' ResourceGroupRunawayOptionList ')'
| "QUERY_LIMIT" EqOpt '(' ')'
| "QUERY_LIMIT" EqOpt "NULL"
Expand Down Expand Up @@ -84,7 +86,7 @@ TiDB supports the following `DirectResourceGroupOption`, where [Request Unit (RU
|---------------|-------------------------------------|------------------------|
| `RU_PER_SEC` | Rate of RU backfilling per second | `RU_PER_SEC = 500` indicates that this resource group is backfilled with 500 RUs per second |
| `PRIORITY` | The absolute priority of tasks to be processed on TiKV | `PRIORITY = HIGH` indicates that the priority is high. If not specified, the default value is `MEDIUM`. |
| `BURSTABLE` | If the `BURSTABLE` attribute is set, TiDB allows the corresponding resource group to use the available system resources when the quota is exceeded. |
| `BURSTABLE` | Whether to allow the resource group to overuse the available remaining system resources | Three modes are supported. `OFF` indicates that the resource group is not allowed to overuse any remaining system resources. `MODERATED` indicates that the resource group is allowed to overuse remaining system resources to a limited extent. `UNLIMITED` indicates that the resource group can overuse remaining system resources without limitation. If no value is specified for `BURSTABLE`, the `MODERATED` mode is enabled by default. |
| `QUERY_LIMIT` | When the query execution meets this condition, the query is identified as a runaway query and the corresponding action is executed. | `QUERY_LIMIT=(EXEC_ELAPSED='60s', ACTION=KILL, WATCH=EXACT DURATION='10m')` indicates that the query is identified as a runaway query when the execution time exceeds 60 seconds. The query is terminated. All SQL statements with the same SQL text will be terminated immediately in the coming 10 minutes. `QUERY_LIMIT=()` or `QUERY_LIMIT=NULL` means that runaway control is not enabled. See [Runaway Queries](/tidb-resource-control-runaway-queries.md). |
| `BACKGROUND` | Configure the background tasks. For more details, see [Manage background tasks](/tidb-resource-control-background-tasks.md). | `BACKGROUND=(TASK_TYPES="br,stats", UTILIZATION_LIMIT=30)` indicates that the backup and restore and statistics collection related tasks are scheduled as background tasks, and background tasks can consume 30% of the TiKV resources at most. |

Expand Down Expand Up @@ -168,11 +170,11 @@ SELECT * FROM information_schema.resource_groups WHERE NAME ='default';
```

```sql
+---------+------------+----------+-----------+-------------+-------------------------------------------+
| NAME | RU_PER_SEC | PRIORITY | BURSTABLE | QUERY_LIMIT | BACKGROUND |
+---------+------------+----------+-----------+-------------+-------------------------------------------+
| default | UNLIMITED | MEDIUM | YES | NULL | TASK_TYPES='br,ddl', UTILIZATION_LIMIT=30 |
+---------+------------+----------+-----------+-------------+-------------------------------------------+
+---------+------------+----------+----------------------+-------------+-------------------------------------------+
| NAME | RU_PER_SEC | PRIORITY | BURSTABLE | QUERY_LIMIT | BACKGROUND |
+---------+------------+----------+----------------------+-------------+-------------------------------------------+
| default | UNLIMITED | MEDIUM | YES(UNLIMITED) | NULL | TASK_TYPES='br,ddl', UTILIZATION_LIMIT=30 |
+---------+------------+----------+----------------------+-------------+-------------------------------------------+
1 rows in set (1.30 sec)
```

Expand Down
20 changes: 11 additions & 9 deletions sql-statements/sql-statement-create-resource-group.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,9 @@ DirectResourceGroupOption ::=
"RU_PER_SEC" EqOpt stringLit
| "PRIORITY" EqOpt ResourceGroupPriorityOption
| "BURSTABLE"
| "BURSTABLE" EqOpt Boolean
| "BURSTABLE" EqOpt "MODERATED"
| "BURSTABLE" EqOpt "UNLIMITED"
| "BURSTABLE" EqOpt "OFF"
| "QUERY_LIMIT" EqOpt '(' ResourceGroupRunawayOptionList ')'
| "QUERY_LIMIT" EqOpt '(' ')'
| "QUERY_LIMIT" EqOpt "NULL"
Expand Down Expand Up @@ -81,13 +83,13 @@ TiDB supports the following `DirectResourceGroupOption`, where [Request Unit (RU
|---------------|-------------------------------------|------------------------|
| `RU_PER_SEC` | Rate of RU backfilling per second | `RU_PER_SEC = 500` indicates that this resource group is backfilled with 500 RUs per second |
| `PRIORITY` | The absolute priority of tasks to be processed on TiKV | `PRIORITY = HIGH` indicates that the priority is high. If not specified, the default value is `MEDIUM`. |
| `BURSTABLE` | If the `BURSTABLE` attribute is set, TiDB allows the corresponding resource group to use the available system resources when the quota is exceeded. |
| `BURSTABLE` | Whether to allow the resource group to overuse the available remaining system resources | Three modes are supported. `OFF` indicates that the resource group is not allowed to overuse any remaining system resources. `MODERATED` indicates that the resource group is allowed to overuse remaining system resources to a limited extent. `UNLIMITED` indicates that the resource group can overuse remaining system resources without limitation. If no value is specified for `BURSTABLE`, the `MODERATED` mode is enabled by default. |
| `QUERY_LIMIT` | When the query execution meets this condition, the query is identified as a runaway query and the corresponding action is executed. | `QUERY_LIMIT=(EXEC_ELAPSED='60s', ACTION=KILL, WATCH=EXACT DURATION='10m')` indicates that the query is identified as a runaway query when the execution time exceeds 60 seconds. The query is terminated. All SQL statements with the same SQL text will be terminated immediately in the coming 10 minutes. `QUERY_LIMIT=()` or `QUERY_LIMIT=NULL` means that runaway control is not enabled. See [Runaway Queries](/tidb-resource-control-runaway-queries.md). |

> **Note:**
>
> - The `CREATE RESOURCE GROUP` statement can only be executed when the global variable [`tidb_enable_resource_control`](/system-variables.md#tidb_enable_resource_control-new-in-v660) is set to `ON`.
> TiDB automatically creates a `default` resource group during cluster initialization. For this resource group, the default value of `RU_PER_SEC` is `UNLIMITED` (equivalent to the maximum value of the `INT` type, that is, `2147483647`) and it is in `BURSTABLE` mode. All requests that are not bound to any resource group are automatically bound to this `default` resource group. When you create a new configuration for another resource group, it is recommended to modify the `default` resource group configuration as needed.
> TiDB automatically creates a `default` resource group during cluster initialization. For this resource group, the default value of `RU_PER_SEC` is `UNLIMITED` (equivalent to the maximum value of the `INT` type, that is, `2147483647`) and its `BURSTABLE` mode is `UNLIMTED`. All requests that are not bound to any resource group are automatically bound to this `default` resource group. You cannot delete the `default` resource group, but can modify its RU configuration.
> - Currently, only the `default` resource group supports modifying the `BACKGROUND` configuration.

## Examples
Expand Down Expand Up @@ -127,12 +129,12 @@ SELECT * FROM information_schema.resource_groups WHERE NAME ='rg1' or NAME = 'rg
```

```sql
+------+------------+----------+-----------+---------------------------------+
| NAME | RU_PER_SEC | PRIORITY | BURSTABLE | QUERY_LIMIT |
+------+------------+----------+-----------+---------------------------------+
| rg1 | 100 | HIGH | YES | NULL |
| rg2 | 200 | MEDIUM | NO | EXEC_ELAPSED=100ms, ACTION=KILL |
+------+------------+----------+-----------+---------------------------------+
+------+------------+----------+----------------------+---------------------------------+
| NAME | RU_PER_SEC | PRIORITY | BURSTABLE | QUERY_LIMIT |
+------+------------+----------+----------------------+---------------------------------+
| rg1 | 100 | HIGH | YES(MODERATED) | NULL |
| rg2 | 200 | MEDIUM | NO | EXEC_ELAPSED=100ms, ACTION=KILL |
+------+------------+----------+----------------------+---------------------------------+
2 rows in set (1.30 sec)
```

Expand Down
10 changes: 5 additions & 5 deletions sql-statements/sql-statement-drop-resource-group.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,11 +55,11 @@ SELECT * FROM information_schema.resource_groups WHERE NAME ='rg1';
```

```sql
+------+------------+----------+-----------+-------------+
| NAME | RU_PER_SEC | PRIORITY | BURSTABLE | QUERY_LIMIT |
+------+------------+----------+-----------+-------------+
| rg1 | 500 | MEDIUM | YES | NULL |
+------+------------+----------+-----------+-------------+
+------+------------+----------+----------------------+-------------+
| NAME | RU_PER_SEC | PRIORITY | BURSTABLE | QUERY_LIMIT |
+------+------------+----------+----------------------+-------------+
| rg1 | 500 | MEDIUM | YES(MODERATED) | NULL |
+------+------------+----------+----------------------+-------------+
1 row in set (0.01 sec)
```

Expand Down
10 changes: 5 additions & 5 deletions tidb-resource-control-background-tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,11 +77,11 @@ By default, the task types that are marked as background tasks are `""`, and the
The output is as follows:

```
+---------+------------+----------+-----------+-------------+-------------------------------------------+
| NAME | RU_PER_SEC | PRIORITY | BURSTABLE | QUERY_LIMIT | BACKGROUND |
+---------+------------+----------+-----------+-------------+-------------------------------------------+
| default | UNLIMITED | MEDIUM | YES | NULL | TASK_TYPES='br,ddl', UTILIZATION_LIMIT=30 |
+---------+------------+----------+-----------+-------------+-------------------------------------------+
+---------+------------+----------+----------------------+-------------+-------------------------------------------+
| NAME | RU_PER_SEC | PRIORITY | BURSTABLE | QUERY_LIMIT | BACKGROUND |
+---------+------------+----------+----------------------+-------------+-------------------------------------------+
| default | UNLIMITED | MEDIUM | YES(UNLIMITED) | NULL | TASK_TYPES='br,ddl', UTILIZATION_LIMIT=30 |
+---------+------------+----------+----------------------+-------------+-------------------------------------------+
```

5. To explicitly mark tasks in the current session as the background type, you can use `tidb_request_source_type` to explicitly specify the task type. The following is an example:
Expand Down
10 changes: 8 additions & 2 deletions tidb-resource-control-ru-groups.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ You can delete a resource group by using [`DROP RESOURCE GROUP`](/sql-statements

The following is an example of how to create a resource group.

1. Create a resource group `rg1`. The resource limit is 500 RUs per second and allows applications in this resource group to overrun resources.
1. Create a resource group `rg1`. The resource limit is 500 RUs per second and allows applications in this resource group to overrun resources. If no value is specified for `BURSTABLE`, the `MODERATED` mode is enabled by default.

```sql
CREATE RESOURCE GROUP IF NOT EXISTS rg1 RU_PER_SEC = 500 BURSTABLE;
Expand All @@ -203,6 +203,12 @@ The following is an example of how to create a resource group.
CREATE RESOURCE GROUP IF NOT EXISTS rg3 RU_PER_SEC = 100 PRIORITY = HIGH;
```

4. Create a `rg4` resource group with a quota of 500 RUs per second, and allow applications in this group to use additional resources beyond the quota by setting the `BURSTABLE` mode to `UNLIMITED`.

```sql
CREATE RESOURCE GROUP IF NOT EXISTS rg4 RU_PER_SEC = 500 BURSTABLE=UNLIMITED;
```

### Bind resource groups

TiDB supports three levels of resource group settings as follows.
Expand Down Expand Up @@ -232,7 +238,7 @@ If there are too many requests that result in insufficient resources for the res
> **Note:**
>
> - When you bind a user to a resource group by using `CREATE USER` or `ALTER USER`, it will not take effect for the user's existing sessions, but only for the user's new sessions.
> - TiDB automatically creates a `default` resource group during cluster initialization. For this resource group, the default value of `RU_PER_SEC` is `UNLIMITED` (equivalent to the maximum value of the `INT` type, that is, `2147483647`) and it is in `BURSTABLE` mode. Statements that are not bound to a resource group are automatically bound to this resource group. This resource group does not support deletion, but you can modify the configuration of its RU.
> - TiDB automatically creates a `default` resource group during cluster initialization. For this resource group, the default value of `RU_PER_SEC` is `UNLIMITED` (equivalent to the maximum value of the `INT` type, that is, `2147483647`) and its `BURSTABLE` is `UNLIMITED` mode. All requests that are not bound to any resource group are automatically bound to this `default` resource group. You cannot delete the `default` resource group, but can modify its RU configuration.

To unbind users from a resource group, you can simply bind them to the `default` group again as follows:

Expand Down