diff --git a/src/UserGuide/Master/Reference/Common-Config-Manual.md b/src/UserGuide/Master/Reference/Common-Config-Manual.md index 97a9414fe..0f5a6ff10 100644 --- a/src/UserGuide/Master/Reference/Common-Config-Manual.md +++ b/src/UserGuide/Master/Reference/Common-Config-Manual.md @@ -676,14 +676,14 @@ Different configuration parameters take effect in the following three ways: | Default | ms | | Effective | Only allowed to be modified in first start up | -* default\_ttl\_in\_ms - -| Name | default\_ttl\_in\_ms | -| :---------: | :------------------------------------- | -| Description | Default ttl when each database created | -| Type | Long | -| Default | Infinity | -| Effective | After restarting system | +* tier\_ttl\_in\_ms + +|Name| tier\_ttl\_in\_ms | +|:---:|:--------------| +|Description| Define the maximum age of data for which each tier is responsible | +|Type| long | +|Default| -1 | +|Effective| After restarting system | * max\_waiting\_time\_when\_insert\_blocked diff --git a/src/UserGuide/Master/Reference/DataNode-Config-Manual.md b/src/UserGuide/Master/Reference/DataNode-Config-Manual.md index 947a16546..4911486ab 100644 --- a/src/UserGuide/Master/Reference/DataNode-Config-Manual.md +++ b/src/UserGuide/Master/Reference/DataNode-Config-Manual.md @@ -497,3 +497,88 @@ There will be at most 10 gc.log.* files and each one can reach to 10MB. |Type| int32 | |Default| 5000 | |Effective| After restarting system | + + + +#### Storage engine configuration + +* dn\_default\_space\_usage\_thresholds + +|Name| dn\_default\_space\_usage\_thresholds | +|:---:|:--------------| +|Description| Define the minimum remaining space ratio for each tier data catalogue; when the remaining space is less than this ratio, the data will be automatically migrated to the next tier; when the remaining storage space of the last tier falls below this threshold, the system will be set to READ_ONLY | +|Type| double | +|Default| 0.85 | +|Effective| hot-load | + +* remote\_tsfile\_cache\_dirs + +|Name| remote\_tsfile\_cache\_dirs | +|:---:|:--------------| +|Description| Cache directory stored locally in the cloud | +|Type| string | +|Default| data/datanode/data/cache | +|Effective| After restarting system | + +* remote\_tsfile\_cache\_page\_size\_in\_kb + +|Name| remote\_tsfile\_cache\_page\_size\_in\_kb | +|:---:|:--------------| +|Description| Block size of locally cached files stored in the cloud | +|Type| int | +|Default| 20480 | +|Effective| After restarting system | + +* remote\_tsfile\_cache\_max\_disk\_usage\_in\_mb + +|Name| remote\_tsfile\_cache\_max\_disk\_usage\_in\_mb | +|:---:|:--------------| +|Description| Maximum Disk Occupancy Size for Cloud Storage Local Cache | +|Type| long | +|Default| 51200 | +|Effective| After restarting system | + +* object\_storage\_type + +|Name| object\_storage\_type | +|:---:|:--------------| +|Description| Cloud Storage Type | +|Type| string | +|Default| AWS_S3 | +|Effective| After restarting system | + +* object\_storage\_bucket + +|Name| object\_storage\_bucket | +|:---:|:--------------| +|Description| Name of cloud storage bucket | +|Type| string | +|Default| iotdb_data | +|Effective| After restarting system | + +* object\_storage\_endpoiont + +|Name| object\_storage\_endpoiont | +|:---:|:--------------| +|Description| endpoint of cloud storage | +|Type| string | +|Default| None | +|Effective| After restarting system | + +* object\_storage\_access\_key + +|Name| object\_storage\_access\_key | +|:---:|:--------------| +|Description| Authentication information stored in the cloud: key | +|Type| string | +|Default| None | +|Effective| After restarting system | + +* object\_storage\_access\_secret + +|Name| object\_storage\_access\_secret | +|:---:|:--------------| +|Description| Authentication information stored in the cloud: secret | +|Type| string | +|Default| None | +|Effective| After restarting system | diff --git a/src/UserGuide/Master/User-Manual/Tiered-Storage_timecho.md b/src/UserGuide/Master/User-Manual/Tiered-Storage_timecho.md index 11d32bdf5..1cb50b1ee 100644 --- a/src/UserGuide/Master/User-Manual/Tiered-Storage_timecho.md +++ b/src/UserGuide/Master/User-Manual/Tiered-Storage_timecho.md @@ -36,11 +36,11 @@ The specific parameter definitions and their descriptions are as follows. | Configuration | Default | Description | Constraint | | ---------------------------------------- | ------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | -| dn_data_dirs | None | specify different storage directories and divide the storage directories into tiers | Each level of storage uses a semicolon to separate, and commas to separate within a single level; cloud (OBJECT_STORAGE) configuration can only be used as the last level of storage and the first level can't be used as cloud storage; a cloud object at most; the remote storage directory is denoted by OBJECT_STORAGE | -| default_ttl_in_ms | None | Define the maximum age of data for which each tier is responsible | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs；"-1" means "unlimited". | -| dn_default_space_move_thresholds | 0.15 | Define the minimum remaining space ratio for each tier data catalogue; when the remaining space is less than this ratio, the data will be automatically migrated to the next tier; when the remaining storage space of the last tier falls below this threshold, the system will be set to READ_ONLY | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs | +| dn_data_dirs | data/datanode/data | specify different storage directories and divide the storage directories into tiers | Each level of storage uses a semicolon to separate, and commas to separate within a single level; cloud (OBJECT_STORAGE) configuration can only be used as the last level of storage and the first level can't be used as cloud storage; a cloud object at most; the remote storage directory is denoted by OBJECT_STORAGE | +| tier_ttl_in_ms | -1 | Define the maximum age of data for which each tier is responsible | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs；"-1" means "unlimited". | +| dn_default_space_usage_thresholds | 0.85 | Define the minimum remaining space ratio for each tier data catalogue; when the remaining space is less than this ratio, the data will be automatically migrated to the next tier; when the remaining storage space of the last tier falls below this threshold, the system will be set to READ_ONLY | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs | | object_storage_type | AWS_S3 | Cloud Storage Type | IoTDB currently only supports AWS S3 as a remote storage type, and this parameter can't be modified | -| object_storage_bucket | None | Name of cloud storage bucket | Bucket definition in AWS S3; no need to configure if remote storage is not used | +| object_storage_bucket | iotdb_data | Name of cloud storage bucket | Bucket definition in AWS S3; no need to configure if remote storage is not used | | object_storage_endpoiont | | endpoint of cloud storage | endpoint of AWS S3；If remote storage is not used, no configuration required | | object_storage_access_key | | Authentication information stored in the cloud: key | AWS S3 credential key；If remote storage is not used, no configuration required | | object_storage_access_secret | | Authentication information stored in the cloud: secret | AWS S3 credential secret；If remote storage is not used, no configuration required | @@ -55,8 +55,8 @@ The following is an example of a local two-level storage configuration. ```JavaScript //Required configuration items dn_data_dirs=/data1/data;/data2/data,/data3/data; -default_ttl_in_ms=86400000;-1 -dn_default_space_move_thresholds=0.2;0.1 +tier_ttl_in_ms=86400000;-1 +dn_default_space_usage_thresholds=0.2;0.1 ``` In this example, two levels of storage are configured, specifically: @@ -73,8 +73,8 @@ The following takes three-level storage as an example: ```JavaScript //Required configuration items dn_data_dirs=/data1/data;/data2/data,/data3/data;OBJECT_STORAGE -default_ttl_in_ms=86400000;864000000;-1 -dn_default_space_move_thresholds=0.2;0.15;0.1 +tier_ttl_in_ms=86400000;864000000;-1 +dn_default_space_usage_thresholds=0.2;0.15;0.1 object_storage_name=AWS_S3 object_storage_bucket=iotdb object_storage_endpoiont= diff --git a/src/UserGuide/V1.2.x/Reference/Common-Config-Manual.md b/src/UserGuide/V1.2.x/Reference/Common-Config-Manual.md index d227fd87e..56b7478f3 100644 --- a/src/UserGuide/V1.2.x/Reference/Common-Config-Manual.md +++ b/src/UserGuide/V1.2.x/Reference/Common-Config-Manual.md @@ -649,11 +649,11 @@ Different configuration parameters take effect in the following three ways: * default\_ttl\_in\_ms | Name | default\_ttl\_in\_ms | -| :---------: | :------------------------------------- | -| Description | Default ttl when each database created | -| Type | Long | -| Default | Infinity | -| Effective | After restarting system | +|:---:|:--------------| +|Description| Define the maximum age of data for which each tier is responsible | +|Type| long | +|Default| -1 | +|Effective| After restarting system | * max\_waiting\_time\_when\_insert\_blocked diff --git a/src/UserGuide/V1.2.x/Reference/DataNode-Config-Manual.md b/src/UserGuide/V1.2.x/Reference/DataNode-Config-Manual.md index f8feccdd8..c476186d2 100644 --- a/src/UserGuide/V1.2.x/Reference/DataNode-Config-Manual.md +++ b/src/UserGuide/V1.2.x/Reference/DataNode-Config-Manual.md @@ -357,3 +357,89 @@ sbin\start-datanode.bat printgc GC log is stored at `IOTDB_HOME/logs/gc.log`. There will be at most 10 gc.log.* files and each one can reach to 10MB. + + +#### Storage engine configuration + +* dn\_default\_space\_move\_thresholds + +|Name| dn\_default\_space\_move\_thresholds | +|:---:|:--------------| +|Description|Define the minimum remaining space ratio for each tier data catalogue; when the remaining space is less than this ratio, the data will be automatically migrated to the next tier; when the remaining storage space of the last tier falls below this threshold, the system will be set to READ_ONLY | +|Type| double | +|Default| 0.15 | +|Effective| hot-load | + + + +* remote\_tsfile\_cache\_dirs + +|Name| remote\_tsfile\_cache\_dirs | +|:---:|:--------------| +|Description| Cache directory stored locally in the cloud | +|Type| string | +|Default| data/datanode/data/cache | +|Effective| After restarting system | + +* remote\_tsfile\_cache\_page\_size\_in\_kb + +|Name| remote\_tsfile\_cache\_page\_size\_in\_kb | +|:---:|:--------------| +|Description| Block size of locally cached files stored in the cloud | +|Type| int | +|Default| 20480 | +|Effective| After restarting system | + +* remote\_tsfile\_cache\_max\_disk\_usage\_in\_mb + +|Name| remote\_tsfile\_cache\_max\_disk\_usage\_in\_mb | +|:---:|:--------------| +|Description| Maximum Disk Occupancy Size for Cloud Storage Local Cache | +|Type| long | +|Default| 51200 | +|Effective| After restarting system | + +* object\_storage\_type + +|Name| object\_storage\_type | +|:---:|:--------------| +|Description| Cloud Storage Type | +|Type| string | +|Default| AWS_S3 | +|Effective| After restarting system | + +* object\_storage\_bucket + +|Name| object\_storage\_bucket | +|:---:|:--------------| +|Description| Name of cloud storage bucket | +|Type| string | +|Default| iotdb_data | +|Effective| After restarting system | + +* object\_storage\_endpoiont + +|Name| object\_storage\_endpoiont | +|:---:|:--------------| +|Description| endpoint of cloud storage | +|Type| string | +|Default| None | +|Effective| After restarting system | + +* object\_storage\_access\_key + +|Name| object\_storage\_access\_key | +|:---:|:--------------| +|Description| Authentication information stored in the cloud: key | +|Type| string | +|Default| None | +|Effective| After restarting system | + +* object\_storage\_access\_secret + +|Name| object\_storage\_access\_secret | +|:---:|:--------------| +|Description| Authentication information stored in the cloud: secret | +|Type| string | +|Default| None | +|Effective| After restarting system | diff --git a/src/UserGuide/V1.2.x/User-Manual/Tiered-Storage_timecho.md b/src/UserGuide/V1.2.x/User-Manual/Tiered-Storage_timecho.md index c3d71e7f4..5d92b3779 100644 --- a/src/UserGuide/V1.2.x/User-Manual/Tiered-Storage_timecho.md +++ b/src/UserGuide/V1.2.x/User-Manual/Tiered-Storage_timecho.md @@ -36,11 +36,11 @@ The specific parameter definitions and their descriptions are as follows. | Configuration | Default | Description | Constraint | | ---------------------------------------- | ------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | -| dn_data_dirs | None | specify different storage directories and divide the storage directories into tiers | Each level of storage uses a semicolon to separate, and commas to separate within a single level; cloud (OBJECT_STORAGE) configuration can only be used as the last level of storage and the first level can't be used as cloud storage; a cloud object at most; the remote storage directory is denoted by OBJECT_STORAGE | -| default_ttl_in_ms | None | Define the maximum age of data for which each tier is responsible | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs | +| dn_data_dirs | data/datanode/data | specify different storage directories and divide the storage directories into tiers | Each level of storage uses a semicolon to separate, and commas to separate within a single level; cloud (OBJECT_STORAGE) configuration can only be used as the last level of storage and the first level can't be used as cloud storage; a cloud object at most; the remote storage directory is denoted by OBJECT_STORAGE | +| default_ttl_in_ms | -1 | Define the maximum age of data for which each tier is responsible | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs | | dn_default_space_move_thresholds | 0.15 | Define the minimum remaining space ratio for each tier data catalogue; when the remaining space is less than this ratio, the data will be automatically migrated to the next tier; when the remaining storage space of the last tier falls below this threshold, the system will be set to READ_ONLY | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs | | object_storage_type | AWS_S3 | Cloud Storage Type | IoTDB currently only supports AWS S3 as a remote storage type, and this parameter can't be modified | -| object_storage_bucket | None | Name of cloud storage bucket | Bucket definition in AWS S3; no need to configure if remote storage is not used | +| object_storage_bucket | iotdb_data | Name of cloud storage bucket | Bucket definition in AWS S3; no need to configure if remote storage is not used | | object_storage_endpoiont | | endpoint of cloud storage | endpoint of AWS S3；If remote storage is not used, no configuration required | | object_storage_access_key | | Authentication information stored in the cloud: key | AWS S3 credential key；If remote storage is not used, no configuration required | | object_storage_access_secret | | Authentication information stored in the cloud: secret | AWS S3 credential secret；If remote storage is not used, no configuration required | diff --git a/src/UserGuide/V1.3.0-2/Reference/Common-Config-Manual.md b/src/UserGuide/V1.3.0-2/Reference/Common-Config-Manual.md index 38e1d5d2b..1b146371e 100644 --- a/src/UserGuide/V1.3.0-2/Reference/Common-Config-Manual.md +++ b/src/UserGuide/V1.3.0-2/Reference/Common-Config-Manual.md @@ -640,11 +640,11 @@ Different configuration parameters take effect in the following three ways: * default\_ttl\_in\_ms | Name | default\_ttl\_in\_ms | -| :---------: | :------------------------------------- | -| Description | Default ttl when each database created | -| Type | Long | -| Default | Infinity | -| Effective | After restarting system | +|:---:|:--------------| +|Description| Define the maximum age of data for which each tier is responsible | +|Type| long | +|Default| -1 | +|Effective| After restarting system | * max\_waiting\_time\_when\_insert\_blocked diff --git a/src/UserGuide/V1.3.0-2/Reference/DataNode-Config-Manual.md b/src/UserGuide/V1.3.0-2/Reference/DataNode-Config-Manual.md index be96619a1..9b2164402 100644 --- a/src/UserGuide/V1.3.0-2/Reference/DataNode-Config-Manual.md +++ b/src/UserGuide/V1.3.0-2/Reference/DataNode-Config-Manual.md @@ -497,3 +497,96 @@ There will be at most 10 gc.log.* files and each one can reach to 10MB. |Type| int32 | |Default| 5000 | |Effective| After restarting system | + +#### Storage engine configuration + +* dn\_default\_space\_move\_thresholds + +|Name| dn\_default\_space\_move\_thresholds | +|:---:|:--------------| +|Description| Version 1.3.0/1: Define the minimum remaining space ratio for each tier data catalogue; when the remaining space is less than this ratio, the data will be automatically migrated to the next tier; when the remaining storage space of the last tier falls below this threshold, the system will be set to READ_ONLY | +|Type| double | +|Default| 0.15 | +|Effective| hot-load | + + +* dn\_default\_space\_usage\_thresholds + +|Name| dn\_default\_space\_usage\_thresholds | +|:---:|:--------------| +|Description| Version 1.3.2: Define the minimum remaining space ratio for each tier data catalogue; when the remaining space is less than this ratio, the data will be automatically migrated to the next tier; when the remaining storage space of the last tier falls below this threshold, the system will be set to READ_ONLY | +|Type| double | +|Default| 0.85 | +|Effective| hot-load | + +* remote\_tsfile\_cache\_dirs + +|Name| remote\_tsfile\_cache\_dirs | +|:---:|:--------------| +|Description| Cache directory stored locally in the cloud | +|Type| string | +|Default| data/datanode/data/cache | +|Effective| After restarting system | + +* remote\_tsfile\_cache\_page\_size\_in\_kb + +|Name| remote\_tsfile\_cache\_page\_size\_in\_kb | +|:---:|:--------------| +|Description| Block size of locally cached files stored in the cloud | +|Type| int | +|Default| 20480 | +|Effective| After restarting system | + +* remote\_tsfile\_cache\_max\_disk\_usage\_in\_mb + +|Name| remote\_tsfile\_cache\_max\_disk\_usage\_in\_mb | +|:---:|:--------------| +|Description| Maximum Disk Occupancy Size for Cloud Storage Local Cache | +|Type| long | +|Default| 51200 | +|Effective| After restarting system | + +* object\_storage\_type + +|Name| object\_storage\_type | +|:---:|:--------------| +|Description| Cloud Storage Type | +|Type| string | +|Default| AWS_S3 | +|Effective| After restarting system | + +* object\_storage\_bucket + +|Name| object\_storage\_bucket | +|:---:|:--------------| +|Description| Name of cloud storage bucket | +|Type| string | +|Default| iotdb_data | +|Effective| After restarting system | + +* object\_storage\_endpoiont + +|Name| object\_storage\_endpoiont | +|:---:|:--------------| +|Description| endpoint of cloud storage | +|Type| string | +|Default| None | +|Effective| After restarting system | + +* object\_storage\_access\_key + +|Name| object\_storage\_access\_key | +|:---:|:--------------| +|Description| Authentication information stored in the cloud: key | +|Type| string | +|Default| None | +|Effective| After restarting system | + +* object\_storage\_access\_secret + +|Name| object\_storage\_access\_secret | +|:---:|:--------------| +|Description| Authentication information stored in the cloud: secret | +|Type| string | +|Default| None | +|Effective| After restarting system | diff --git a/src/UserGuide/V1.3.0-2/User-Manual/Tiered-Storage_timecho.md b/src/UserGuide/V1.3.0-2/User-Manual/Tiered-Storage_timecho.md index 11d32bdf5..20f7e7d9d 100644 --- a/src/UserGuide/V1.3.0-2/User-Manual/Tiered-Storage_timecho.md +++ b/src/UserGuide/V1.3.0-2/User-Manual/Tiered-Storage_timecho.md @@ -36,11 +36,12 @@ The specific parameter definitions and their descriptions are as follows. | Configuration | Default | Description | Constraint | | ---------------------------------------- | ------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | -| dn_data_dirs | None | specify different storage directories and divide the storage directories into tiers | Each level of storage uses a semicolon to separate, and commas to separate within a single level; cloud (OBJECT_STORAGE) configuration can only be used as the last level of storage and the first level can't be used as cloud storage; a cloud object at most; the remote storage directory is denoted by OBJECT_STORAGE | -| default_ttl_in_ms | None | Define the maximum age of data for which each tier is responsible | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs；"-1" means "unlimited". | -| dn_default_space_move_thresholds | 0.15 | Define the minimum remaining space ratio for each tier data catalogue; when the remaining space is less than this ratio, the data will be automatically migrated to the next tier; when the remaining storage space of the last tier falls below this threshold, the system will be set to READ_ONLY | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs | +| dn_data_dirs | data/datanode/data | specify different storage directories and divide the storage directories into tiers | Each level of storage uses a semicolon to separate, and commas to separate within a single level; cloud (OBJECT_STORAGE) configuration can only be used as the last level of storage and the first level can't be used as cloud storage; a cloud object at most; the remote storage directory is denoted by OBJECT_STORAGE | +| default_ttl_in_ms | -1 | Define the maximum age of data for which each tier is responsible | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs；"-1" means "unlimited". | +| dn_default_space_move_thresholds（V1.3.0/1） | 0.15 | Define the minimum remaining space ratio for each tier data catalogue; when the remaining space is less than this ratio, the data will be automatically migrated to the next tier; when the remaining storage space of the last tier falls below this threshold, the system will be set to READ_ONLY | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs | +| dn_default_space_usage_thresholds（V1.3.2） | 0.85 | Define the minimum remaining space ratio for each tier data catalogue; when the remaining space is less than this ratio, the data will be automatically migrated to the next tier; when the remaining storage space of the last tier falls below this threshold, the system will be set to READ_ONLY | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs | | object_storage_type | AWS_S3 | Cloud Storage Type | IoTDB currently only supports AWS S3 as a remote storage type, and this parameter can't be modified | -| object_storage_bucket | None | Name of cloud storage bucket | Bucket definition in AWS S3; no need to configure if remote storage is not used | +| object_storage_bucket | iotdb_data | Name of cloud storage bucket | Bucket definition in AWS S3; no need to configure if remote storage is not used | | object_storage_endpoiont | | endpoint of cloud storage | endpoint of AWS S3；If remote storage is not used, no configuration required | | object_storage_access_key | | Authentication information stored in the cloud: key | AWS S3 credential key；If remote storage is not used, no configuration required | | object_storage_access_secret | | Authentication information stored in the cloud: secret | AWS S3 credential secret；If remote storage is not used, no configuration required | diff --git a/src/UserGuide/latest/Reference/Common-Config-Manual.md b/src/UserGuide/latest/Reference/Common-Config-Manual.md index 97a9414fe..0f5a6ff10 100644 --- a/src/UserGuide/latest/Reference/Common-Config-Manual.md +++ b/src/UserGuide/latest/Reference/Common-Config-Manual.md @@ -676,14 +676,14 @@ Different configuration parameters take effect in the following three ways: | Default | ms | | Effective | Only allowed to be modified in first start up | -* default\_ttl\_in\_ms - -| Name | default\_ttl\_in\_ms | -| :---------: | :------------------------------------- | -| Description | Default ttl when each database created | -| Type | Long | -| Default | Infinity | -| Effective | After restarting system | +* tier\_ttl\_in\_ms + +|Name| tier\_ttl\_in\_ms | +|:---:|:--------------| +|Description| Define the maximum age of data for which each tier is responsible | +|Type| long | +|Default| -1 | +|Effective| After restarting system | * max\_waiting\_time\_when\_insert\_blocked diff --git a/src/UserGuide/latest/Reference/DataNode-Config-Manual.md b/src/UserGuide/latest/Reference/DataNode-Config-Manual.md index 947a16546..94ede5013 100644 --- a/src/UserGuide/latest/Reference/DataNode-Config-Manual.md +++ b/src/UserGuide/latest/Reference/DataNode-Config-Manual.md @@ -497,3 +497,88 @@ There will be at most 10 gc.log.* files and each one can reach to 10MB. |Type| int32 | |Default| 5000 | |Effective| After restarting system | + + +#### Storage engine configuration + + +* dn\_default\_space\_usage\_thresholds + +|Name| dn\_default\_space\_usage\_thresholds | +|:---:|:--------------| +|Description| Define the minimum remaining space ratio for each tier data catalogue; when the remaining space is less than this ratio, the data will be automatically migrated to the next tier; when the remaining storage space of the last tier falls below this threshold, the system will be set to READ_ONLY | +|Type| double | +|Default| 0.85 | +|Effective| hot-load | + +* remote\_tsfile\_cache\_dirs + +|Name| remote\_tsfile\_cache\_dirs | +|:---:|:--------------| +|Description| Cache directory stored locally in the cloud | +|Type| string | +|Default| data/datanode/data/cache | +|Effective| After restarting system | + +* remote\_tsfile\_cache\_page\_size\_in\_kb + +|Name| remote\_tsfile\_cache\_page\_size\_in\_kb | +|:---:|:--------------| +|Description| Block size of locally cached files stored in the cloud | +|Type| int | +|Default| 20480 | +|Effective| After restarting system | + +* remote\_tsfile\_cache\_max\_disk\_usage\_in\_mb + +|Name| remote\_tsfile\_cache\_max\_disk\_usage\_in\_mb | +|:---:|:--------------| +|Description| Maximum Disk Occupancy Size for Cloud Storage Local Cache | +|Type| long | +|Default| 51200 | +|Effective| After restarting system | + +* object\_storage\_type + +|Name| object\_storage\_type | +|:---:|:--------------| +|Description| Cloud Storage Type | +|Type| string | +|Default| AWS_S3 | +|Effective| After restarting system | + +* object\_storage\_bucket + +|Name| object\_storage\_bucket | +|:---:|:--------------| +|Description| Name of cloud storage bucket | +|Type| string | +|Default| iotdb_data | +|Effective| After restarting system | + +* object\_storage\_endpoiont + +|Name| object\_storage\_endpoiont | +|:---:|:--------------| +|Description| endpoint of cloud storage | +|Type| string | +|Default| None | +|Effective| After restarting system | + +* object\_storage\_access\_key + +|Name| object\_storage\_access\_key | +|:---:|:--------------| +|Description| Authentication information stored in the cloud: key | +|Type| string | +|Default| None | +|Effective| After restarting system | + +* object\_storage\_access\_secret + +|Name| object\_storage\_access\_secret | +|:---:|:--------------| +|Description| Authentication information stored in the cloud: secret | +|Type| string | +|Default| None | +|Effective| After restarting system | diff --git a/src/UserGuide/latest/User-Manual/Tiered-Storage_timecho.md b/src/UserGuide/latest/User-Manual/Tiered-Storage_timecho.md index 11d32bdf5..1cb50b1ee 100644 --- a/src/UserGuide/latest/User-Manual/Tiered-Storage_timecho.md +++ b/src/UserGuide/latest/User-Manual/Tiered-Storage_timecho.md @@ -36,11 +36,11 @@ The specific parameter definitions and their descriptions are as follows. | Configuration | Default | Description | Constraint | | ---------------------------------------- | ------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | -| dn_data_dirs | None | specify different storage directories and divide the storage directories into tiers | Each level of storage uses a semicolon to separate, and commas to separate within a single level; cloud (OBJECT_STORAGE) configuration can only be used as the last level of storage and the first level can't be used as cloud storage; a cloud object at most; the remote storage directory is denoted by OBJECT_STORAGE | -| default_ttl_in_ms | None | Define the maximum age of data for which each tier is responsible | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs；"-1" means "unlimited". | -| dn_default_space_move_thresholds | 0.15 | Define the minimum remaining space ratio for each tier data catalogue; when the remaining space is less than this ratio, the data will be automatically migrated to the next tier; when the remaining storage space of the last tier falls below this threshold, the system will be set to READ_ONLY | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs | +| dn_data_dirs | data/datanode/data | specify different storage directories and divide the storage directories into tiers | Each level of storage uses a semicolon to separate, and commas to separate within a single level; cloud (OBJECT_STORAGE) configuration can only be used as the last level of storage and the first level can't be used as cloud storage; a cloud object at most; the remote storage directory is denoted by OBJECT_STORAGE | +| tier_ttl_in_ms | -1 | Define the maximum age of data for which each tier is responsible | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs；"-1" means "unlimited". | +| dn_default_space_usage_thresholds | 0.85 | Define the minimum remaining space ratio for each tier data catalogue; when the remaining space is less than this ratio, the data will be automatically migrated to the next tier; when the remaining storage space of the last tier falls below this threshold, the system will be set to READ_ONLY | Each level of storage is separated by a semicolon; the number of levels should match the number of levels defined by dn_data_dirs | | object_storage_type | AWS_S3 | Cloud Storage Type | IoTDB currently only supports AWS S3 as a remote storage type, and this parameter can't be modified | -| object_storage_bucket | None | Name of cloud storage bucket | Bucket definition in AWS S3; no need to configure if remote storage is not used | +| object_storage_bucket | iotdb_data | Name of cloud storage bucket | Bucket definition in AWS S3; no need to configure if remote storage is not used | | object_storage_endpoiont | | endpoint of cloud storage | endpoint of AWS S3；If remote storage is not used, no configuration required | | object_storage_access_key | | Authentication information stored in the cloud: key | AWS S3 credential key；If remote storage is not used, no configuration required | | object_storage_access_secret | | Authentication information stored in the cloud: secret | AWS S3 credential secret；If remote storage is not used, no configuration required | @@ -55,8 +55,8 @@ The following is an example of a local two-level storage configuration. ```JavaScript //Required configuration items dn_data_dirs=/data1/data;/data2/data,/data3/data; -default_ttl_in_ms=86400000;-1 -dn_default_space_move_thresholds=0.2;0.1 +tier_ttl_in_ms=86400000;-1 +dn_default_space_usage_thresholds=0.2;0.1 ``` In this example, two levels of storage are configured, specifically: @@ -73,8 +73,8 @@ The following takes three-level storage as an example: ```JavaScript //Required configuration items dn_data_dirs=/data1/data;/data2/data,/data3/data;OBJECT_STORAGE -default_ttl_in_ms=86400000;864000000;-1 -dn_default_space_move_thresholds=0.2;0.15;0.1 +tier_ttl_in_ms=86400000;864000000;-1 +dn_default_space_usage_thresholds=0.2;0.15;0.1 object_storage_name=AWS_S3 object_storage_bucket=iotdb object_storage_endpoiont= diff --git a/src/zh/UserGuide/Master/Reference/Common-Config-Manual.md b/src/zh/UserGuide/Master/Reference/Common-Config-Manual.md index fd76cb4e3..70f113a59 100644 --- a/src/zh/UserGuide/Master/Reference/Common-Config-Manual.md +++ b/src/zh/UserGuide/Master/Reference/Common-Config-Manual.md @@ -705,6 +705,15 @@ IoTDB ConfigNode 和 DataNode 的公共配置参数位于 `conf` 目录下。 | 默认值 | ms | | 改后生效方式 | 仅允许在第一次启动服务前修改 | +* tier\_ttl\_in\_ms + +|名字| tier\_ttl\_in\_ms | +|:---:|:--------------| +|描述| 定义每个层级负责的数据范围，通过 TTL 表示 | +|类型| long | +|默认值| -1 | +|改后生效方式| 重启生效 | + * max\_waiting\_time\_when\_insert\_blocked | 名字 | max\_waiting\_time\_when\_insert\_blocked | diff --git a/src/zh/UserGuide/Master/Reference/DataNode-Config-Manual.md b/src/zh/UserGuide/Master/Reference/DataNode-Config-Manual.md index 434744aeb..fb5fdde55 100644 --- a/src/zh/UserGuide/Master/Reference/DataNode-Config-Manual.md +++ b/src/zh/UserGuide/Master/Reference/DataNode-Config-Manual.md @@ -489,3 +489,87 @@ GC 日志会被存储在`IOTDB_HOME/logs/gc.log`. 至多会存储 10 个 gc.log |类型| int32 | |默认值| 5000 | |改后生效方式| 重启生效 | + + +#### 存储引擎配置 + +* dn\_default\_space\_usage\_thresholds + +|名字| dn\_default\_space\_usage\_thresholds | +|:---:|:--------------| +|描述| 定义每个层级数据目录的最小剩余空间比例；当剩余空间少于该比例时，数据会被自动迁移至下一个层级；当最后一个层级的剩余存储空间到低于此阈值时，会将系统置为 READ_ONLY | +|类型| double | +|默认值| 0.85 | +|改后生效方式| 热加载 | + +* remote\_tsfile\_cache\_dirs + +|名字| remote\_tsfile\_cache\_dirs | +|:---:|:--------------| +|描述| 云端存储在本地的缓存目录 | +|类型| string | +|默认值| data/datanode/data/cache | +|改后生效方式| 重启生效 | + +* remote\_tsfile\_cache\_page\_size\_in\_kb + +|名字| remote\_tsfile\_cache\_page\_size\_in\_kb | +|:---:|:--------------| +|描述| 云端存储在本地缓存文件的块大小 | +|类型| int | +|默认值| 20480 | +|改后生效方式| 重启生效 | + +* remote\_tsfile\_cache\_max\_disk\_usage\_in\_mb + +|名字| remote\_tsfile\_cache\_max\_disk\_usage\_in\_mb | +|:---:|:--------------| +|描述| 云端存储本地缓存的最大磁盘占用大小 | +|类型| long | +|默认值| 51200 | +|改后生效方式| 重启生效 | + +* object\_storage\_type + +|名字| object\_storage\_type | +|:---:|:--------------| +|描述| 云端存储类型 | +|类型| string | +|默认值| AWS_S3 | +|改后生效方式| 重启生效 | + +* object\_storage\_bucket + +|名字| object\_storage\_bucket | +|:---:|:--------------| +|描述| 云端存储 bucket 的名称 | +|类型| string | +|默认值| iotdb_data | +|改后生效方式| 重启生效 | + +* object\_storage\_endpoiont + +|名字| object\_storage\_endpoiont | +|:---:|:--------------| +|描述| 云端存储的 endpoint | +|类型| string | +|默认值| 无 | +|改后生效方式| 重启生效 | + +* object\_storage\_access\_key + +|名字| object\_storage\_access\_key | +|:---:|:--------------| +|描述| 云端存储的验证信息 key | +|类型| string | +|默认值| 无 | +|改后生效方式| 重启生效 | + +* object\_storage\_access\_secret + +|名字| object\_storage\_access\_secret | +|:---:|:--------------| +|描述| 云端存储的验证信息 secret | +|类型| string | +|默认值| 无 | +|改后生效方式| 重启生效 | \ No newline at end of file diff --git a/src/zh/UserGuide/Master/User-Manual/Tiered-Storage_timecho.md b/src/zh/UserGuide/Master/User-Manual/Tiered-Storage_timecho.md index b368f1613..e46de729f 100644 --- a/src/zh/UserGuide/Master/User-Manual/Tiered-Storage_timecho.md +++ b/src/zh/UserGuide/Master/User-Manual/Tiered-Storage_timecho.md @@ -36,11 +36,11 @@ | 配置项 | 默认值 | 说明 | 约束 | | ---------------------------------------- | ------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | -| dn_data_dirs | 无 | 用来指定不同的存储目录，并将存储目录进行层级划分 | 每级存储使用分号分隔，单级内使用逗号分隔；云端配置只能作为最后一级存储且第一级不能作为云端存储；最多配置一个云端对象；远端存储目录使用 OBJECT_STORAGE 来表示 | -| default_ttl_in_ms | 无 | 定义每个层级负责的数据范围，通过 TTL 表示 | 每级存储使用分号分隔；层级数量需与 dn_data_dirs 定义的层级数一致；"-1" 表示"无限制" | -| dn_default_space_move_thresholds | 0.15 | 定义每个层级数据目录的最小剩余空间比例；当剩余空间少于该比例时，数据会被自动迁移至下一个层级；当最后一个层级的剩余存储空间到低于此阈值时，会将系统置为 READ_ONLY | 每级存储使用分号分隔；层级数量需与 dn_data_dirs 定义的层级数一致 | +| dn_data_dirs | data/datanode/data | 用来指定不同的存储目录，并将存储目录进行层级划分 | 每级存储使用分号分隔，单级内使用逗号分隔；云端配置只能作为最后一级存储且第一级不能作为云端存储；最多配置一个云端对象；远端存储目录使用 OBJECT_STORAGE 来表示 | +| tier_ttl_in_ms | -1 | 定义每个层级负责的数据范围，通过 TTL 表示 | 每级存储使用分号分隔；层级数量需与 dn_data_dirs 定义的层级数一致；"-1" 表示"无限制" | +| dn_default_space_usage_thresholds | 0.85 | 定义每个层级数据目录的最小剩余空间比例；当剩余空间少于该比例时，数据会被自动迁移至下一个层级；当最后一个层级的剩余存储空间到低于此阈值时，会将系统置为 READ_ONLY | 每级存储使用分号分隔；层级数量需与 dn_data_dirs 定义的层级数一致 | | object_storage_type | AWS_S3 | 云端存储类型 | IoTDB 当前只支持 AWS S3 作为远端存储类型，此参数不支持修改 | -| object_storage_bucket | 无 | 云端存储 bucket 的名称 | AWS S3 中的 bucket 定义；如果未使用远端存储，无需配置 | +| object_storage_bucket | iotdb_data | 云端存储 bucket 的名称 | AWS S3 中的 bucket 定义；如果未使用远端存储，无需配置 | | object_storage_endpoiont | | 云端存储的 endpoint | AWS S3 的 endpoint；如果未使用远端存储，无需配置 | | object_storage_access_key | | 云端存储的验证信息 key | AWS S3 的 credential key；如果未使用远端存储，无需配置 | | object_storage_access_secret | | 云端存储的验证信息 secret | AWS S3 的 credential secret；如果未使用远端存储，无需配置 | @@ -55,8 +55,8 @@ ```JavaScript // 必须配置项 dn_data_dirs=/data1/data;/data2/data,/data3/data; -default_ttl_in_ms=86400000;-1 -dn_default_space_move_thresholds=0.2;0.1 +tier_ttl_in_ms=86400000;-1 +dn_default_space_usage_thresholds=0.2;0.1 ``` 在该示例中，共配置了两个层级的存储，具体为： @@ -73,8 +73,8 @@ dn_default_space_move_thresholds=0.2;0.1 ```JavaScript // 必须配置项 dn_data_dirs=/data1/data;/data2/data,/data3/data;OBJECT_STORAGE -default_ttl_in_ms=86400000;864000000;-1 -dn_default_space_move_thresholds=0.2;0.15;0.1 +tier_ttl_in_ms=86400000;864000000;-1 +dn_default_space_usage_thresholds=0.2;0.15;0.1 object_storage_name=AWS_S3 object_storage_bucket=iotdb object_storage_endpoiont= diff --git a/src/zh/UserGuide/V1.2.x/Reference/Common-Config-Manual.md b/src/zh/UserGuide/V1.2.x/Reference/Common-Config-Manual.md index d3abf55a9..f50b42d9f 100644 --- a/src/zh/UserGuide/V1.2.x/Reference/Common-Config-Manual.md +++ b/src/zh/UserGuide/V1.2.x/Reference/Common-Config-Manual.md @@ -696,12 +696,12 @@ IoTDB ConfigNode 和 DataNode 的公共配置参数位于 `conf` 目录下。 * default\_ttl\_in\_ms -| 名字 | default\_ttl\_in\_ms | -| :----------: | :---------------------------------------------------------- | -| 描述 | 数据保留时间，会丢弃 now()-default\_ttl 之前的数据，单位 ms | -| 类型 | long | -| 默认值 | 36000000 | -| 改后生效方式 | 重启服务生效 | +|名字| default\_ttl\_in\_ms | +|:---:|:--------------| +|描述| 定义每个层级负责的数据范围，通过 TTL 表示 | +|类型| long | +|默认值| -1 | +|改后生效方式| 重启生效 | * max\_waiting\_time\_when\_insert\_blocked diff --git a/src/zh/UserGuide/V1.2.x/Reference/DataNode-Config-Manual.md b/src/zh/UserGuide/V1.2.x/Reference/DataNode-Config-Manual.md index 97e4fd413..300f71c7c 100644 --- a/src/zh/UserGuide/V1.2.x/Reference/DataNode-Config-Manual.md +++ b/src/zh/UserGuide/V1.2.x/Reference/DataNode-Config-Manual.md @@ -348,3 +348,86 @@ sbin\start-datanode.bat printgc ``` GC 日志会被存储在`IOTDB_HOME/logs/gc.log`. 至多会存储 10 个 gc.log 文件，每个文件最多 10MB。 + +#### 存储引擎配置 + +* dn\_default\_space\_move\_thresholds + +|名字| dn\_default\_space\_move\_thresholds | +|:---:|:--------------| +|描述| 定义每个层级数据目录的最小剩余空间比例；当剩余空间少于该比例时，数据会被自动迁移至下一个层级；当最后一个层级的剩余存储空间到低于此阈值时，会将系统置为 READ_ONLY | +|类型| double | +|默认值| 0.15 | +|改后生效方式| 热加载 | + +* remote\_tsfile\_cache\_dirs + +|名字| remote\_tsfile\_cache\_dirs | +|:---:|:--------------| +|描述| 云端存储在本地的缓存目录 | +|类型| string | +|默认值| data/datanode/data/cache | +|改后生效方式| 重启生效 | + +* remote\_tsfile\_cache\_page\_size\_in\_kb + +|名字| remote\_tsfile\_cache\_page\_size\_in\_kb | +|:---:|:--------------| +|描述| 云端存储在本地缓存文件的块大小 | +|类型| int | +|默认值| 20480 | +|改后生效方式| 重启生效 | + +* remote\_tsfile\_cache\_max\_disk\_usage\_in\_mb + +|名字| remote\_tsfile\_cache\_max\_disk\_usage\_in\_mb | +|:---:|:--------------| +|描述| 云端存储本地缓存的最大磁盘占用大小 | +|类型| long | +|默认值| 51200 | +|改后生效方式| 重启生效 | + +* object\_storage\_type + +|名字| object\_storage\_type | +|:---:|:--------------| +|描述| 云端存储类型 | +|类型| string | +|默认值| AWS_S3 | +|改后生效方式| 重启生效 | + +* object\_storage\_bucket + +|名字| object\_storage\_bucket | +|:---:|:--------------| +|描述| 云端存储 bucket 的名称 | +|类型| string | +|默认值| iotdb_data | +|改后生效方式| 重启生效 | + +* object\_storage\_endpoiont + +|名字| object\_storage\_endpoiont | +|:---:|:--------------| +|描述| 云端存储的 endpoint | +|类型| string | +|默认值| 无 | +|改后生效方式| 重启生效 | + +* object\_storage\_access\_key + +|名字| object\_storage\_access\_key | +|:---:|:--------------| +|描述| 云端存储的验证信息 key | +|类型| string | +|默认值| 无 | +|改后生效方式| 重启生效 | + +* object\_storage\_access\_secret + +|名字| object\_storage\_access\_secret | +|:---:|:--------------| +|描述| 云端存储的验证信息 secret | +|类型| string | +|默认值| 无 | +|改后生效方式| 重启生效 | \ No newline at end of file diff --git a/src/zh/UserGuide/V1.2.x/User-Manual/Tiered-Storage_timecho.md b/src/zh/UserGuide/V1.2.x/User-Manual/Tiered-Storage_timecho.md index 0f03ac151..8764220c9 100644 --- a/src/zh/UserGuide/V1.2.x/User-Manual/Tiered-Storage_timecho.md +++ b/src/zh/UserGuide/V1.2.x/User-Manual/Tiered-Storage_timecho.md @@ -36,11 +36,11 @@ | 配置项 | 默认值 | 说明 | 约束 | | ---------------------------------------- | ------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | -| dn_data_dirs | 无 | 用来指定不同的存储目录，并将存储目录进行层级划分 | 每级存储使用分号分隔，单级内使用逗号分隔；云端配置只能作为最后一级存储且第一级不能作为云端存储；最多配置一个云端对象；远端存储目录使用 OBJECT_STORAGE 来表示 | -| default_ttl_in_ms | 无 | 定义每个层级负责的数据范围，通过 TTL 表示 | 每级存储使用分号分隔；层级数量需与 dn_data_dirs 定义的层级数一致 | +| dn_data_dirs | data/datanode/data | 用来指定不同的存储目录，并将存储目录进行层级划分 | 每级存储使用分号分隔，单级内使用逗号分隔；云端配置只能作为最后一级存储且第一级不能作为云端存储；最多配置一个云端对象；远端存储目录使用 OBJECT_STORAGE 来表示 | +| default_ttl_in_ms | -1 | 定义每个层级负责的数据范围，通过 TTL 表示 | 每级存储使用分号分隔；层级数量需与 dn_data_dirs 定义的层级数一致 | | dn_default_space_move_thresholds | 0.15 | 定义每个层级数据目录的最小剩余空间比例；当剩余空间少于该比例时，数据会被自动迁移至下一个层级；当最后一个层级的剩余存储空间到低于此阈值时，会将系统置为 READ_ONLY | 每级存储使用分号分隔；层级数量需与 dn_data_dirs 定义的层级数一致 | | object_storage_type | AWS_S3 | 云端存储类型 | IoTDB 当前只支持 AWS S3 作为远端存储类型，此参数不支持修改 | -| object_storage_bucket | 无 | 云端存储 bucket 的名称 | AWS S3 中的 bucket 定义；如果未使用远端存储，无需配置 | +| object_storage_bucket | iotdb_data | 云端存储 bucket 的名称 | AWS S3 中的 bucket 定义；如果未使用远端存储，无需配置 | | object_storage_endpoiont | | 云端存储的 endpoint | AWS S3 的 endpoint；如果未使用远端存储，无需配置 | | object_storage_access_key | | 云端存储的验证信息 key | AWS S3 的 credential key；如果未使用远端存储，无需配置 | | object_storage_access_secret | | 云端存储的验证信息 secret | AWS S3 的 credential secret；如果未使用远端存储，无需配置 | diff --git a/src/zh/UserGuide/V1.3.0-2/Reference/Common-Config-Manual.md b/src/zh/UserGuide/V1.3.0-2/Reference/Common-Config-Manual.md index 01d48e57a..d8c0d25bc 100644 --- a/src/zh/UserGuide/V1.3.0-2/Reference/Common-Config-Manual.md +++ b/src/zh/UserGuide/V1.3.0-2/Reference/Common-Config-Manual.md @@ -685,12 +685,13 @@ IoTDB ConfigNode 和 DataNode 的公共配置参数位于 `conf` 目录下。 * default\_ttl\_in\_ms -| 名字 | default\_ttl\_in\_ms | -| :----------: | :---------------------------------------------------------- | -| 描述 | 数据保留时间，会丢弃 now()-default\_ttl 之前的数据，单位 ms | -| 类型 | long | -| 默认值 | 36000000 | -| 改后生效方式 | 重启生效 | +|名字| default\_ttl\_in\_ms | +|:---:|:--------------| +|描述| 定义每个层级负责的数据范围，通过 TTL 表示 | +|类型| long | +|默认值| -1 | +|改后生效方式| 重启生效 | + * max\_waiting\_time\_when\_insert\_blocked diff --git a/src/zh/UserGuide/V1.3.0-2/Reference/DataNode-Config-Manual.md b/src/zh/UserGuide/V1.3.0-2/Reference/DataNode-Config-Manual.md index 899805cb3..5e582d189 100644 --- a/src/zh/UserGuide/V1.3.0-2/Reference/DataNode-Config-Manual.md +++ b/src/zh/UserGuide/V1.3.0-2/Reference/DataNode-Config-Manual.md @@ -488,3 +488,95 @@ GC 日志会被存储在`IOTDB_HOME/logs/gc.log`. 至多会存储 10 个 gc.log |类型| int32 | |默认值| 5000 | |改后生效方式| 重启生效 | + +#### 存储引擎配置 + +* dn\_default\_space\_move\_thresholds + +|名字| dn\_default\_space\_move\_thresholds | +|:---:|:--------------| +|描述| 1.3.0/1版本：定义每个层级数据目录的最小剩余空间比例；当剩余空间少于该比例时，数据会被自动迁移至下一个层级；当最后一个层级的剩余存储空间到低于此阈值时，会将系统置为 READ_ONLY | +|类型| double | +|默认值| 0.15 | +|改后生效方式| 热加载 | + +* dn\_default\_space\_usage\_thresholds + +|名字| dn\_default\_space\_usage\_thresholds | +|:---:|:--------------| +|描述| 1.3.2版本：定义每个层级数据目录的最小剩余空间比例；当剩余空间少于该比例时，数据会被自动迁移至下一个层级；当最后一个层级的剩余存储空间到低于此阈值时，会将系统置为 READ_ONLY | +|类型| double | +|默认值| 0.85 | +|改后生效方式| 热加载 | + +* remote\_tsfile\_cache\_dirs + +|名字| remote\_tsfile\_cache\_dirs | +|:---:|:--------------| +|描述| 云端存储在本地的缓存目录 | +|类型| string | +|默认值| data/datanode/data/cache | +|改后生效方式| 重启生效 | + +* remote\_tsfile\_cache\_page\_size\_in\_kb + +|名字| remote\_tsfile\_cache\_page\_size\_in\_kb | +|:---:|:--------------| +|描述| 云端存储在本地缓存文件的块大小 | +|类型| int | +|默认值| 20480 | +|改后生效方式| 重启生效 | + +* remote\_tsfile\_cache\_max\_disk\_usage\_in\_mb + +|名字| remote\_tsfile\_cache\_max\_disk\_usage\_in\_mb | +|:---:|:--------------| +|描述| 云端存储本地缓存的最大磁盘占用大小 | +|类型| long | +|默认值| 51200 | +|改后生效方式| 重启生效 | + +* object\_storage\_type + +|名字| object\_storage\_type | +|:---:|:--------------| +|描述| 云端存储类型 | +|类型| string | +|默认值| AWS_S3 | +|改后生效方式| 重启生效 | + +* object\_storage\_bucket + +|名字| object\_storage\_bucket | +|:---:|:--------------| +|描述| 云端存储 bucket 的名称 | +|类型| string | +|默认值| iotdb_data | +|改后生效方式| 重启生效 | + +* object\_storage\_endpoiont + +|名字| object\_storage\_endpoiont | +|:---:|:--------------| +|描述| 云端存储的 endpoint | +|类型| string | +|默认值| 无 | +|改后生效方式| 重启生效 | + +* object\_storage\_access\_key + +|名字| object\_storage\_access\_key | +|:---:|:--------------| +|描述| 云端存储的验证信息 key | +|类型| string | +|默认值| 无 | +|改后生效方式| 重启生效 | + +* object\_storage\_access\_secret + +|名字| object\_storage\_access\_secret | +|:---:|:--------------| +|描述| 云端存储的验证信息 secret | +|类型| string | +|默认值| 无 | +|改后生效方式| 重启生效 | \ No newline at end of file diff --git a/src/zh/UserGuide/V1.3.0-2/User-Manual/Tiered-Storage_timecho.md b/src/zh/UserGuide/V1.3.0-2/User-Manual/Tiered-Storage_timecho.md index b368f1613..eafaee494 100644 --- a/src/zh/UserGuide/V1.3.0-2/User-Manual/Tiered-Storage_timecho.md +++ b/src/zh/UserGuide/V1.3.0-2/User-Manual/Tiered-Storage_timecho.md @@ -36,11 +36,12 @@ | 配置项 | 默认值 | 说明 | 约束 | | ---------------------------------------- | ------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | -| dn_data_dirs | 无 | 用来指定不同的存储目录，并将存储目录进行层级划分 | 每级存储使用分号分隔，单级内使用逗号分隔；云端配置只能作为最后一级存储且第一级不能作为云端存储；最多配置一个云端对象；远端存储目录使用 OBJECT_STORAGE 来表示 | -| default_ttl_in_ms | 无 | 定义每个层级负责的数据范围，通过 TTL 表示 | 每级存储使用分号分隔；层级数量需与 dn_data_dirs 定义的层级数一致；"-1" 表示"无限制" | -| dn_default_space_move_thresholds | 0.15 | 定义每个层级数据目录的最小剩余空间比例；当剩余空间少于该比例时，数据会被自动迁移至下一个层级；当最后一个层级的剩余存储空间到低于此阈值时，会将系统置为 READ_ONLY | 每级存储使用分号分隔；层级数量需与 dn_data_dirs 定义的层级数一致 | +| dn_data_dirs | data/datanode/data | 用来指定不同的存储目录，并将存储目录进行层级划分 | 每级存储使用分号分隔，单级内使用逗号分隔；云端配置只能作为最后一级存储且第一级不能作为云端存储；最多配置一个云端对象；远端存储目录使用 OBJECT_STORAGE 来表示 | +| default_ttl_in_ms | -1 | 定义每个层级负责的数据范围，通过 TTL 表示 | 每级存储使用分号分隔；层级数量需与 dn_data_dirs 定义的层级数一致；"-1" 表示"无限制" | +| dn_default_space_move_thresholds（V1.3.0/1） | 0.15 | 定义每个层级数据目录的最小剩余空间比例；当剩余空间少于该比例时，数据会被自动迁移至下一个层级；当最后一个层级的剩余存储空间到低于此阈值时，会将系统置为 READ_ONLY | 每级存储使用分号分隔；层级数量需与 dn_data_dirs 定义的层级数一致 | +| dn_default_space_usage_thresholds（V1.3.2） | 0.85 | 定义每个层级数据目录的最小剩余空间比例；当剩余空间少于该比例时，数据会被自动迁移至下一个层级；当最后一个层级的剩余存储空间到低于此阈值时，会将系统置为 READ_ONLY | 每级存储使用分号分隔；层级数量需与 dn_data_dirs 定义的层级数一致 | | object_storage_type | AWS_S3 | 云端存储类型 | IoTDB 当前只支持 AWS S3 作为远端存储类型，此参数不支持修改 | -| object_storage_bucket | 无 | 云端存储 bucket 的名称 | AWS S3 中的 bucket 定义；如果未使用远端存储，无需配置 | +| object_storage_bucket | iotdb_data | 云端存储 bucket 的名称 | AWS S3 中的 bucket 定义；如果未使用远端存储，无需配置 | | object_storage_endpoiont | | 云端存储的 endpoint | AWS S3 的 endpoint；如果未使用远端存储，无需配置 | | object_storage_access_key | | 云端存储的验证信息 key | AWS S3 的 credential key；如果未使用远端存储，无需配置 | | object_storage_access_secret | | 云端存储的验证信息 secret | AWS S3 的 credential secret；如果未使用远端存储，无需配置 | diff --git a/src/zh/UserGuide/latest/Reference/Common-Config-Manual.md b/src/zh/UserGuide/latest/Reference/Common-Config-Manual.md index fd76cb4e3..a5a82292f 100644 --- a/src/zh/UserGuide/latest/Reference/Common-Config-Manual.md +++ b/src/zh/UserGuide/latest/Reference/Common-Config-Manual.md @@ -705,6 +705,16 @@ IoTDB ConfigNode 和 DataNode 的公共配置参数位于 `conf` 目录下。 | 默认值 | ms | | 改后生效方式 | 仅允许在第一次启动服务前修改 | +* tier\_ttl\_in\_ms + +|名字| tier\_ttl\_in\_ms | +|:---:|:--------------| +|描述| 定义每个层级负责的数据范围，通过 TTL 表示 | +|类型| long | +|默认值| -1 | +|改后生效方式| 重启生效 | + + * max\_waiting\_time\_when\_insert\_blocked | 名字 | max\_waiting\_time\_when\_insert\_blocked | diff --git a/src/zh/UserGuide/latest/Reference/DataNode-Config-Manual.md b/src/zh/UserGuide/latest/Reference/DataNode-Config-Manual.md index 434744aeb..c88848432 100644 --- a/src/zh/UserGuide/latest/Reference/DataNode-Config-Manual.md +++ b/src/zh/UserGuide/latest/Reference/DataNode-Config-Manual.md @@ -489,3 +489,88 @@ GC 日志会被存储在`IOTDB_HOME/logs/gc.log`. 至多会存储 10 个 gc.log |类型| int32 | |默认值| 5000 | |改后生效方式| 重启生效 | + + + +#### 多级存储配置 + +* dn\_default\_space\_usage\_thresholds + +|名字| dn\_default\_space\_usage\_thresholds | +|:---:|:--------------| +|描述| 定义每个层级数据目录的最小剩余空间比例；当剩余空间少于该比例时，数据会被自动迁移至下一个层级；当最后一个层级的剩余存储空间到低于此阈值时，会将系统置为 READ_ONLY | +|类型| double | +|默认值| 0.85 | +|改后生效方式| 热加载 | + +* remote\_tsfile\_cache\_dirs + +|名字| remote\_tsfile\_cache\_dirs | +|:---:|:--------------| +|描述| 云端存储在本地的缓存目录 | +|类型| string | +|默认值| data/datanode/data/cache | +|改后生效方式| 重启生效 | + +* remote\_tsfile\_cache\_page\_size\_in\_kb + +|名字| remote\_tsfile\_cache\_page\_size\_in\_kb | +|:---:|:--------------| +|描述| 云端存储在本地缓存文件的块大小 | +|类型| int | +|默认值| 20480 | +|改后生效方式| 重启生效 | + +* remote\_tsfile\_cache\_max\_disk\_usage\_in\_mb + +|名字| remote\_tsfile\_cache\_max\_disk\_usage\_in\_mb | +|:---:|:--------------| +|描述| 云端存储本地缓存的最大磁盘占用大小 | +|类型| long | +|默认值| 51200 | +|改后生效方式| 重启生效 | + +* object\_storage\_type + +|名字| object\_storage\_type | +|:---:|:--------------| +|描述| 云端存储类型 | +|类型| string | +|默认值| AWS_S3 | +|改后生效方式| 重启生效 | + +* object\_storage\_bucket + +|名字| object\_storage\_bucket | +|:---:|:--------------| +|描述| 云端存储 bucket 的名称 | +|类型| string | +|默认值| iotdb_data | +|改后生效方式| 重启生效 | + +* object\_storage\_endpoiont + +|名字| object\_storage\_endpoiont | +|:---:|:--------------| +|描述| 云端存储的 endpoint | +|类型| string | +|默认值| 无 | +|改后生效方式| 重启生效 | + +* object\_storage\_access\_key + +|名字| object\_storage\_access\_key | +|:---:|:--------------| +|描述| 云端存储的验证信息 key | +|类型| string | +|默认值| 无 | +|改后生效方式| 重启生效 | + +* object\_storage\_access\_secret + +|名字| object\_storage\_access\_secret | +|:---:|:--------------| +|描述| 云端存储的验证信息 secret | +|类型| string | +|默认值| 无 | +|改后生效方式| 重启生效 | \ No newline at end of file diff --git a/src/zh/UserGuide/latest/User-Manual/Tiered-Storage_timecho.md b/src/zh/UserGuide/latest/User-Manual/Tiered-Storage_timecho.md index b368f1613..501a3966f 100644 --- a/src/zh/UserGuide/latest/User-Manual/Tiered-Storage_timecho.md +++ b/src/zh/UserGuide/latest/User-Manual/Tiered-Storage_timecho.md @@ -36,11 +36,11 @@ | 配置项 | 默认值 | 说明 | 约束 | | ---------------------------------------- | ------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | -| dn_data_dirs | 无 | 用来指定不同的存储目录，并将存储目录进行层级划分 | 每级存储使用分号分隔，单级内使用逗号分隔；云端配置只能作为最后一级存储且第一级不能作为云端存储；最多配置一个云端对象；远端存储目录使用 OBJECT_STORAGE 来表示 | -| default_ttl_in_ms | 无 | 定义每个层级负责的数据范围，通过 TTL 表示 | 每级存储使用分号分隔；层级数量需与 dn_data_dirs 定义的层级数一致；"-1" 表示"无限制" | -| dn_default_space_move_thresholds | 0.15 | 定义每个层级数据目录的最小剩余空间比例；当剩余空间少于该比例时，数据会被自动迁移至下一个层级；当最后一个层级的剩余存储空间到低于此阈值时，会将系统置为 READ_ONLY | 每级存储使用分号分隔；层级数量需与 dn_data_dirs 定义的层级数一致 | +| dn_data_dirs | data/datanode/data | 用来指定不同的存储目录，并将存储目录进行层级划分 | 每级存储使用分号分隔，单级内使用逗号分隔；云端配置只能作为最后一级存储且第一级不能作为云端存储；最多配置一个云端对象；远端存储目录使用 OBJECT_STORAGE 来表示 | +| tier_ttl_in_ms | -1 | 定义每个层级负责的数据范围，通过 TTL 表示 | 每级存储使用分号分隔；层级数量需与 dn_data_dirs 定义的层级数一致；"-1" 表示"无限制" | +| dn_default_space_usage_thresholds | 0.85 | 定义每个层级数据目录的最小剩余空间比例；当剩余空间少于该比例时，数据会被自动迁移至下一个层级；当最后一个层级的剩余存储空间到低于此阈值时，会将系统置为 READ_ONLY | 每级存储使用分号分隔；层级数量需与 dn_data_dirs 定义的层级数一致 | | object_storage_type | AWS_S3 | 云端存储类型 | IoTDB 当前只支持 AWS S3 作为远端存储类型，此参数不支持修改 | -| object_storage_bucket | 无 | 云端存储 bucket 的名称 | AWS S3 中的 bucket 定义；如果未使用远端存储，无需配置 | +| object_storage_bucket | iotdb_data | 云端存储 bucket 的名称 | AWS S3 中的 bucket 定义；如果未使用远端存储，无需配置 | | object_storage_endpoiont | | 云端存储的 endpoint | AWS S3 的 endpoint；如果未使用远端存储，无需配置 | | object_storage_access_key | | 云端存储的验证信息 key | AWS S3 的 credential key；如果未使用远端存储，无需配置 | | object_storage_access_secret | | 云端存储的验证信息 secret | AWS S3 的 credential secret；如果未使用远端存储，无需配置 | @@ -48,6 +48,7 @@ | remote_tsfile_cache_page_size_in_kb | 20480 | 云端存储在本地缓存文件的块大小 | 如果未使用远端存储，无需配置 | | remote_tsfile_cache_max_disk_usage_in_mb | 51200 | 云端存储本地缓存的最大磁盘占用大小 | 如果未使用远端存储，无需配置 | + ## 本地多级存储配置示例 以下以本地两级存储的配置示例。 @@ -55,8 +56,8 @@ ```JavaScript // 必须配置项 dn_data_dirs=/data1/data;/data2/data,/data3/data; -default_ttl_in_ms=86400000;-1 -dn_default_space_move_thresholds=0.2;0.1 +tier_ttl_in_ms=86400000;-1 +dn_default_space_usage_thresholds=0.2;0.1 ``` 在该示例中，共配置了两个层级的存储，具体为： @@ -73,8 +74,8 @@ dn_default_space_move_thresholds=0.2;0.1 ```JavaScript // 必须配置项 dn_data_dirs=/data1/data;/data2/data,/data3/data;OBJECT_STORAGE -default_ttl_in_ms=86400000;864000000;-1 -dn_default_space_move_thresholds=0.2;0.15;0.1 +tier_ttl_in_ms=86400000;864000000;-1 +dn_default_space_usage_thresholds=0.2;0.15;0.1 object_storage_name=AWS_S3 object_storage_bucket=iotdb object_storage_endpoiont=