From 200fcce1463dcecdb1b116dc34db2f1a2496b51b Mon Sep 17 00:00:00 2001 From: Shaun Struwig <41984034+Blargian@users.noreply.github.com> Date: Mon, 16 Jun 2025 10:21:31 +0200 Subject: [PATCH 1/4] Update export-backups-to-own-cloud-account.md --- .../backups/export-backups-to-own-cloud-account.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/docs/cloud/manage/backups/export-backups-to-own-cloud-account.md b/docs/cloud/manage/backups/export-backups-to-own-cloud-account.md index e255b79ad0a..06e77117a53 100644 --- a/docs/cloud/manage/backups/export-backups-to-own-cloud-account.md +++ b/docs/cloud/manage/backups/export-backups-to-own-cloud-account.md @@ -12,12 +12,16 @@ import EnterprisePlanFeatureBadge from '@theme/badges/EnterprisePlanFeatureBadge ClickHouse Cloud supports taking backups to your own cloud service provider (CSP) account (AWS S3, Google Cloud Storage, or Azure Blob Storage). For details of how ClickHouse Cloud backups work, including "full" vs. "incremental" backups, see the [backups](overview.md) docs. -Here we show examples of how to take full and incremental backups to AWS, GCP, Azure object storage as well as how to restore from the backups. +Here we show examples of how to take full and incremental backups to AWS, GCP, Azure object storage as well as how to restore from the backups. The BACKUP commands listed below are run within the original service. The RESTORE commands are run from a new service where the backup should be restored. :::note Users should be aware that any usage where backups are being exported to a different region in the same cloud provider, or to another cloud provider (in the same or different region) will incur [data transfer](../network-data-transfer.mdx) charges. ::: +:::note +Backup / Restore into your own bucket for services utilizing [TDE](https://clickhouse.com/docs/cloud/security/cmek#transparent-data-encryption-tde) is currently not supported. +::: + ## Requirements {#requirements} You will need the following details to export/restore backups to your own CSP storage bucket. @@ -59,6 +63,11 @@ You will need the following details to export/restore backups to your own CSP st
# Backup / Restore +:::note +1. For restoring the backup from your own bucket into a new service, you will need to update the trust policy of your backups storage bucket to allow access from the new service. +2. The Backup / Restore commands need to be run from the database command line. For restore to a new service, you will first need to create the service and then run the command. +::: + ## Backup / Restore to AWS S3 Bucket {#backup--restore-to-aws-s3-bucket} ### Take a DB Backup {#take-a-db-backup} From d25aad86c8379343df66d1e708779597f9fddbbd Mon Sep 17 00:00:00 2001 From: Shaun Struwig <41984034+Blargian@users.noreply.github.com> Date: Mon, 16 Jun 2025 10:26:08 +0200 Subject: [PATCH 2/4] Update export-backups-to-own-cloud-account.md --- .../export-backups-to-own-cloud-account.md | 71 +++++++++++++++---- 1 file changed, 57 insertions(+), 14 deletions(-) diff --git a/docs/cloud/manage/backups/export-backups-to-own-cloud-account.md b/docs/cloud/manage/backups/export-backups-to-own-cloud-account.md index 06e77117a53..cf62be00628 100644 --- a/docs/cloud/manage/backups/export-backups-to-own-cloud-account.md +++ b/docs/cloud/manage/backups/export-backups-to-own-cloud-account.md @@ -61,18 +61,18 @@ You will need the following details to export/restore backups to your own CSP st 2. Access HMAC key and HMAC secret.
-# Backup / Restore +# Backup / restore :::note 1. For restoring the backup from your own bucket into a new service, you will need to update the trust policy of your backups storage bucket to allow access from the new service. 2. The Backup / Restore commands need to be run from the database command line. For restore to a new service, you will first need to create the service and then run the command. ::: -## Backup / Restore to AWS S3 Bucket {#backup--restore-to-aws-s3-bucket} +## Backup / restore to AWS S3 bucket {#backup--restore-to-aws-s3-bucket} -### Take a DB Backup {#take-a-db-backup} +### Take a DB backup {#take-a-db-backup} -**Full Backup** +**Full backup** ```sql BACKUP DATABASE test_backups @@ -86,11 +86,11 @@ You will need to use a different UUID for each new backup in this subdirectory, For example, if you are taking daily backups, you will need to use a new UUID each day. ::: -**Incremental Backup** +**Incremental backup** ```sql BACKUP DATABASE test_backups -TO S3('https://testchbackups.s3.amazonaws.com/backups/', '', '') +TO S3('https://testchbackups.s3.amazonaws.com/backups//my_incremental', '', '') SETTINGS base_backup = S3('https://testchbackups.s3.amazonaws.com/backups/', '', '') ``` @@ -104,11 +104,11 @@ FROM S3('https://testchbackups.s3.amazonaws.com/backups/', '', '', '', '/', ', ) ``` + Where `uuid` is a unique identifier, used to differentiate a set of backups. -**Incremental Backup** +**Incremental backup** ```sql BACKUP DATABASE test_backups @@ -155,6 +156,48 @@ TO S3('https://storage.googleapis.com/test_gcs_backups//my_incremental', ' SETTINGS base_backup = S3('https://storage.googleapis.com/test_gcs_backups/', 'key', 'secret') ``` +# Granular backups {#granular-backups} + +The `BACKUP` command also works with granular backups of specific tables. Example AWS commands for backing up a specific table are listed below. GCP and Azure commands are similar to the ones explained above, except that they need to be customized to backup specific tables. + +### Take a granular backup {#take-a-granular-backup} + +**Full backup** + +```sql +BACKUP TABLE data TO S3('https://testchbackups.s3.amazonaws.com/backups/', '', '') +``` + +**Incremental backup** + +```sql +BACKUP TABLE data TO S3('https://testchbackups.s3.amazonaws.com/backups/my_incremental/', '', '') SETTINGS base_backup = S3('https://testchbackups.s3.amazonaws.com/backups/', '', '') +``` + +### Restore from a granular backup {#restore-from-granular-backup} + +```sql +RESTORE TABLE data AS data3 FROM +S3('https://testchbackups.s3.amazonaws.com/backups/my_incremental', '', '') +``` + +### Backup and restore all service data {#backup-and-restore-all-service-data} + +**Backup** + +```sql +BACKUP TABLE system.settings_profiles, TABLE system.row_policies, TABLE system.quotas, TABLE system.functions, ALL EXCEPT DATABASES INFORMATION_SCHEMA,information_schema,system TO S3('https://testchbackups.s3.amazonaws.com/backups/', '', '') +``` + +**Restore** + +```sql +RESTORE ALL FROM S3('https://testchbackups.s3.amazonaws.com/backups/', '', '') +``` + ### Restore from a backup {#restore-from-a-backup-2} ```sql From 650519679a0890a6e17cb93c6b60d50234a012c4 Mon Sep 17 00:00:00 2001 From: Shaun Struwig <41984034+Blargian@users.noreply.github.com> Date: Mon, 16 Jun 2025 10:28:16 +0200 Subject: [PATCH 3/4] Update Headings.yml --- styles/ClickHouse/Headings.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/styles/ClickHouse/Headings.yml b/styles/ClickHouse/Headings.yml index 9bee1cb4225..074e4f6f2ea 100644 --- a/styles/ClickHouse/Headings.yml +++ b/styles/ClickHouse/Headings.yml @@ -63,6 +63,8 @@ exceptions: - Docker Compose - Kafka - Google Cloud Run + - Google Cloud Storage + - GCS - NPM - OTel - - SQL \ No newline at end of file + - SQL From b9f4dc5a659582eaa9b30b7fa0f481236504fdba Mon Sep 17 00:00:00 2001 From: Shaun Struwig <41984034+Blargian@users.noreply.github.com> Date: Mon, 16 Jun 2025 10:30:56 +0200 Subject: [PATCH 4/4] Update export-backups-to-own-cloud-account.md --- .../backups/export-backups-to-own-cloud-account.md | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/docs/cloud/manage/backups/export-backups-to-own-cloud-account.md b/docs/cloud/manage/backups/export-backups-to-own-cloud-account.md index cf62be00628..32f55d9f140 100644 --- a/docs/cloud/manage/backups/export-backups-to-own-cloud-account.md +++ b/docs/cloud/manage/backups/export-backups-to-own-cloud-account.md @@ -61,7 +61,8 @@ You will need the following details to export/restore backups to your own CSP st 2. Access HMAC key and HMAC secret.
-# Backup / restore + +# Backup / restore {#backup-restore} :::note 1. For restoring the backup from your own bucket into a new service, you will need to update the trust policy of your backups storage bucket to allow access from the new service. @@ -197,11 +198,3 @@ BACKUP TABLE system.settings_profiles, TABLE system.row_policies, TABLE system.q ```sql RESTORE ALL FROM S3('https://testchbackups.s3.amazonaws.com/backups/', '', '') ``` - -### Restore from a backup {#restore-from-a-backup-2} - -```sql -RESTORE DATABASE test_backups -AS test_backups_restored_gcs -FROM S3('https://storage.googleapis.com/test_gcs_backups/', 'key', 'secret') -```