Skip to content

sql-statements: BR / PITR SQL enhancement #13663

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions TOC.md
Original file line number Diff line number Diff line change
Expand Up @@ -698,6 +698,7 @@
- [`BACKUP`](/sql-statements/sql-statement-backup.md)
- [`BATCH`](/sql-statements/sql-statement-batch.md)
- [`BEGIN`](/sql-statements/sql-statement-begin.md)
- [`BR JOB ADMIN](/sql-statements/sql-statement-br-job-admin.md)
- [`CANCEL LOAD DATA` 和 `DROP LOAD DATA`](/sql-statements/sql-statement-operate-load-data-job.md)
- [`CALIBRATE RESOURCE`](/sql-statements/sql-statement-calibrate-resource.md)
- [`CHANGE COLUMN`](/sql-statements/sql-statement-change-column.md)
Expand Down Expand Up @@ -750,6 +751,7 @@
- [`LOCK STATS`](/sql-statements/sql-statement-lock-stats.md)
- [`LOCK TABLES` 和 `UNLOCK TABLES`](/sql-statements/sql-statement-lock-tables-and-unlock-tables.md)
- [`MODIFY COLUMN`](/sql-statements/sql-statement-modify-column.md)
- [`PITR](/sql-statements/sql-statement-pitr.md)
- [`PREPARE`](/sql-statements/sql-statement-prepare.md)
- [`RECOVER TABLE`](/sql-statements/sql-statement-recover-table.md)
- [`RENAME INDEX`](/sql-statements/sql-statement-rename-index.md)
Expand All @@ -770,6 +772,7 @@
- [`SET TRANSACTION`](/sql-statements/sql-statement-set-transaction.md)
- [`SET [GLOBAL|SESSION] <variable>`](/sql-statements/sql-statement-set-variable.md)
- [`SHOW [BACKUPS|RESTORES]`](/sql-statements/sql-statement-show-backups.md)
- [`SHOW BACKUP METADATA](/sql-statements/sql-statement-show-backup-meta.md)
- [`SHOW ANALYZE STATUS`](/sql-statements/sql-statement-show-analyze-status.md)
- [`SHOW [GLOBAL|SESSION] BINDINGS`](/sql-statements/sql-statement-show-bindings.md)
- [`SHOW BUILTINS`](/sql-statements/sql-statement-show-builtins.md)
Expand Down
3 changes: 3 additions & 0 deletions sql-statements/sql-statement-backup.md
Original file line number Diff line number Diff line change
Expand Up @@ -187,3 +187,6 @@ BACKUP DATABASE `test` TO 'local:///mnt/backup/hist03'

* [RESTORE](/sql-statements/sql-statement-restore.md)
* [SHOW BACKUPS](/sql-statements/sql-statement-show-backups.md)
* [PITR](/sql-statements/sql-statement-pitr.md)
* [BR_JOB_ADMIN](/sql-statements/sql-statement-br-job-admin.md)
* [SHOW_BACKUP_META](/sql-statements/sql-statement-show-backup-meta.md)
85 changes: 85 additions & 0 deletions sql-statements/sql-statement-br-job-admin.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
---
title: BR JOB ADMIN
summary: TiDB 数据库中 BR 作业管理语句的使用概况。
aliases: ['/docs-cn/dev/sql-statements/sql-statement-br-job-admin']
---

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

缺少一级标题

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SHOW BR JOB [QUERY] 和 CANCEL BR JOB

本文介绍 TiDB BR (BACKUP & RESTORE) 作业管理语句,包括下面几种语句:
```sql
SHOW BR JOB --show information of a br task
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

admin show br job 还是 show br job ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

admin show br job

SHOW BR JOB QUERY --show the query statement of a br task
CANCEL BR JOB --cancel a br task
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

admin cancel br job

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DDL 的 CANCEL 时使用 CANCEL DDL JOBS 关键字,是否和 DDL 保持一致

ADMIN CANCEL DDL JOBS job_id [, job_id] ...;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
CANCEL BR JOB --cancel a br task
CANCEL BR JOBS --cancel br tasks

```
> **警告:**
>
> TiDB BR 作业管理语句在 v7.1.0 是实验特性,其语法或者行为表现在 GA 前可能会发生变化。
>
### 显示 br 作业信息

按照 task id 展示一个 br 作业的信息

#### 语法图

```ebnf+diagram
StreamShowJobStmt ::=
"SHOW" "BR" "JOB" Int64Num
```

#### 示例

{{< copyable "sql" >}}

```sql
SHOW BR JOB 13456;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
SHOW BR JOB 13456;
SHOW BR JOBS where job_id = 13456;

Copy link
Contributor

@benmaoer benmaoer Sep 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-- 100 是分页展示历史
SQL> SHOW BR JOBS 100;

-- 100 是 job_id
SQL> SHOW BR JOBS where job_id = 100;

-- 展示原始执行的 SQL 语句
SQL> SHOW BR JOB QUERIES 58;

```

### 显示 br 作业运行的 SQL 语句

按照 task id 展示一个 br 作业运行的 SQL 语句

#### 语法图

```ebnf+diagram
StreamShowQueryStmt ::=
"SHOW" "BR" "JOB" "QUERY" Int64Num
```

#### 示例

{{< copyable "sql" >}}

```sql
SHOW BR JOB QUERY 13456;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

建议添加上 SHOW BR JOB QUERIES where job_id = 13456 的返回结果

```

### 取消一个 br 作业

按照 task id 取消一个 br 作业的运行

#### 语法图

```ebnf+diagram
StreamCancelJobStmt ::=
"CANCEL" "BR" "JOB" Int64Num
```

#### 示例

{{< copyable "sql" >}}

```sql
CANCEL BR JOB 13456;
```

## MySQL 兼容性

该语句是 TiDB 对 MySQL 语法的扩展。

## 另请参阅

* [BACKUP](/sql-statements/sql-statement-backup.md)
* [SHOW BACKUPS](/sql-statements/sql-statement-show-backups.md)
* [SHOW RESTORES](/sql-statements/sql-statement-show-backups.md)
* [RESTORE](/sql-statements/sql-statement-restore.md)
* [PITR](/sql-statements/sql-statement-pitr.md)
* [SHOW_BACKUP_META](/sql-statements/show-backp-meta.md)
246 changes: 246 additions & 0 deletions sql-statements/sql-statement-pitr.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,246 @@
---
title: TiDB 日志备份与 PITR 语句
summary: 介绍 TiDB 日志备份与 PITR 的 SQL 语句。
aliases: ['/zh/tidb/dev/br-log-statement/']
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
aliases: ['/zh/tidb/dev/br-log-statement/']

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

新文档不需要 aliases,aliases 用于添加被删除的文档的别名

---

# TiDB 日志备份与 PITR

> **警告:**
>
> TiDB 日志备份与 PITR SQL 语句支持在 v7.1.0 是实验特性,其语法或者行为表现在 GA 前可能会发生变化。

本文介绍 TiDB 日志备份和 PITR (Point-in-time recovery) SQL 语句。

TiDB 日志备份与 PITR SQL 语句的大部分功能功能以及使用的参数的含义,和 BR 工具命令行基本是一致的,不同的是 SQL 语句备份恢复过程是由 TiDB 本身驱动,而非单独的 BR 工具。BR 工具的优势和警告也适用。如果你想了解如何进行日志备份与 PITR,可以参考以下教程:

- [ TiDB 日志备份与 PITR 命令行手册](/br/br-pitr-manual.md)
- [TiDB 日志备份与 PITR 使用指南](/br/br-pitr-guide.md)
- [TiDB 集群备份与恢复实践示例](/br/backup-and-restore-use-cases.md)
对于命令行已经存在功能,本文档重点是描述语法图和参数列表,对于 SQL 语句特有的功能会在本文档中做详细描述。

## 日志备份语句

执行日志备份语句需要 `BACKUP_ADMIN` 或 `SUPER` 权限。此外,执行备份的 TiDB 节点和集群中的所有 TiKV 节点都必须有对目标存储的读或写权限。

```sql
BACKUP LOGS TO --start a log backup task
SHOW BACKUP LOGS STATUS --get status for the log backup task\
PAUSE BACKUP LOGS --pause a log backup task
RESUME BACKUP LOGS --resume a log backup task
STOP BACKUP LOGS --stop a log backup task
SHOW BACKUP LOGS METADATA FROM --get the metadata of log dir
PURGE BACKUP JOBS FROM --purge the log data until sometime
```

### 启动日志备份

等价于执行 `br log start` 命令,你可以在备份集群启动一个日志备份任务。该任务在 TiDB 集群持续地运行,及时地将 KV 变更日志保存到备份存储中。

#### 语法图

```ebnf+diagram
StreamStartStmt ::=
"BACKUP" "LOGS" "TO" stringLit StreamStartOption*

StreamStartOption ::=
"START_TS" '='? StartTSO

StartTSO ::=
LengthNum | stringLit
```

#### 示例

{{< copyable "sql" >}}

Comment on lines +55 to +56
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{{< copyable "sql" >}}

```sql
BACKUP LOGS TO 'local:///mnt/backup/2020/07/26';
BACKUP LOGS TO 'local:///mnt/backup/2022/07/26' START_TS = '2022-07-26 21:20:00+0800';
```

### 查询日志备份任务

等价于执行 `br log status` 命令,查询日志备份任务状态。

#### 语法图

```ebnf+diagram
StreamStatusStmt ::=
"SHOW" "BACKUP" "LOGS" "STATUS"
```

#### 示例

{{< copyable "sql" >}}

Comment on lines +75 to +76
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{{< copyable "sql" >}}

```sql
SHOW BACKUP LOGS STATUS;
```

### 暂停日志备份任务

等价于执行 `br log pause` 命令,暂停正在运行的日志备份任务。

#### 语法图

```ebnf+diagram
StreamPauseStmt ::=
"PAUSE" "BACKUP" "LOGS" StreamPauseOption*

StreamPauseOption ::=
"GC_TTL" '='? GCTTLTSO

GCTTLTSO ::=
LengthNum | stringLit
```

#### 示例

{{< copyable "sql" >}}

Comment on lines +100 to +101
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{{< copyable "sql" >}}

```sql
PAUSE BACKUP LOGS;
PAUSE BACKUP LOGS GC_TTL = '2022-08-26 21:20:00+0800';
```

### 恢复日志备份任务

等价于执行 `br log resume` 命令,恢复被暂停的日志备份任务。

#### 语法图

```ebnf+diagram
StreamResumeStmt ::=
"Resume" "BACKUP" "LOGS"
```

#### 示例

{{< copyable "sql" >}}

```sql
RESUME BACKUP LOGS;
```

### 停止日志备份任务

等价于执行 `br log stop` 命令,停止日志备份任务。

#### 语法图

```ebnf+diagram
StreamStopStmt ::=
"Stop" "BACKUP" "LOGS"
```

#### 示例

{{< copyable "sql" >}}

Comment on lines +139 to +140
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{{< copyable "sql" >}}

```sql
STOP BACKUP LOGS;
```

### 查看备份数据元信息

等价于执行 `br log metadata` 命令,查看备份存储中保存的日志备份的元信息,例如最早和最近的可恢复时间点。

#### 语法图

```ebnf+diagram
StreamMetaDataStmt ::=
"SHOW" "BACKUP" "LOGS" "METADATA" "FROM" stringLit
```

#### 示例

{{< copyable "sql" >}}

Comment on lines +158 to +159
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{{< copyable "sql" >}}

```sql
SHOW BACKUP LOGS METADATA FROM 's3://backup-101/logbackup?access-key=${access-key}&secret-access-key=${secret-access-key}"';
```

### 清理日志备份数据

等价于执行 `br log truncate` 命令,从备份存储中删除过期或不再需要的备份日志数据。

#### 语法图

```ebnf+diagram
StreamMetaPurgeStmt ::=
"PURGE" "BACKUP" "LOGS" "METADATA" "FROM" stringLit StreamPurgeOption*

StreamPurgeOption ::=
"UNTIL_TS" '='? UNTILTSO

UNTILTSO ::=
LengthNum | stringLit
```

#### 示例

{{< copyable "sql" >}}

Comment on lines +183 to +184
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{{< copyable "sql" >}}

```sql
PURGE BACKUP LOGS METADATA FROM 's3://backup-101/logbackup?access-key=${access-key}&secret-access-key=${secret-access-key}"';
PURGE BACKUP LOGS METADATA FROM 's3://backup-101/logbackup?access-key=${access-key}&secret-access-key=${secret-access-key}"' UNTIL_TS = '2022-08-26 21:20:00+0800' ;
```

## 恢复到指定时间点 PITR

执行 PITR 语句需要 `RESTORE_ADMIN` 或 `SUPER` 权限。此外,执行恢复的 TiDB 节点和集群中的所有 TiKV 节点都必须有对目标存储的读或写权限。

```sql
RESTORE POINT FROM --restore cluster to a point
```

### 恢复到指定时间点

等价于执行 `br restore point` 命令,在新集群上进行 PITR,或者只恢复日志备份数据。

#### 语法图

```ebnf+diagram
StreamRestorePITStmt ::=
"RESTORE" "POINT" "FROM" stringLit StreamRestorePITOption*

StreamRestorePITOption ::=
"START_TS" '='? StartTSO
| "FULL_BACKUP_STORAGE" '='? stringLit
| "START_TS" '='? StartTSO
| "STORED_TS" '='? StoredTSO

StartTSO ::=
LengthNum | stringLit

StoredTSO ::=
LengthNum | stringLit
```

#### 示例

{{< copyable "sql" >}}

Comment on lines +223 to +224
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{{< copyable "sql" >}}

```sql
RESTORE POINT FROM 's3://backup-101/logbackup?access-key=${access-key}&secret-access-key=${secret-access-key}"' FULL_BACKUP_STORAGE = 's3://backup-101/snapshot-202205120000?access-key=${access-key}&secret-access-key=${secret-access-key}"';

```
> **注意:**
>
> - 不支持重复恢复某段时间区间的日志,如多次重复恢复 `[t1=10, t2=20)` 区间的日志数据,可能会造成恢复后的数据不正确。
> - 多次恢复不同时间区间的日志时,需保证恢复日志的连续性。如先后恢复 `[t1, t2)`、`[t2, t3)` 和 `[t3, t4)` 三个区间的日志可以保证正确性,而在恢复 `[t1, t2)` 后跳过 `[t2, t3)` 直接恢复 `[t3, t4)` 的区间可能导致恢复之后的数据不正确。


## MySQL 兼容性

该语句是 TiDB 对 MySQL 语法的扩展。

## 另请参阅

* [BACKUP](/sql-statements/sql-statement-backup.md)
* [SHOW BACKUPS](/sql-statements/sql-statement-show-backups.md)
* [SHOW RESTORES](/sql-statements/sql-statement-show-backups.md)
* [RESTORE](/sql-statements/sql-statement-restore.md)
* [BR_JOB_ADMIN](/sql-statements/sql-statement-br-job-admin.md)
* [SHOW_BACKUP_META](/sql-statements/show-backp-meta.md)
3 changes: 3 additions & 0 deletions sql-statements/sql-statement-restore.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,3 +161,6 @@ RESTORE DATABASE * FROM 's3://example-bucket/inc-backup-2';

* [BACKUP](/sql-statements/sql-statement-backup.md)
* [SHOW RESTORES](/sql-statements/sql-statement-show-backups.md)
* [PITR](/sql-statements/sql-statement-pitr.md)
* [BR_JOB_ADMIN](/sql-statements/sql-statement-br-job-admin.md)
* [SHOW_BACKUP_META](/sql-statements/sql-statement-show-backup-meta.md)
Loading