Skip to content

Commit 1e25892

Browse files
leto-bleto-b
committed
add load sql to data import in 13x
1 parent abfab96 commit 1e25892

File tree

4 files changed

+188
-8
lines changed

4 files changed

+188
-8
lines changed

‎src/UserGuide/V1.3.x/Tools-System/Data-Import-Tool-1-3-4.md‎

Lines changed: 47 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,10 @@
11
# Data Import
22

33
## 1. Overview
4-
IoTDB supports two methods for data import:
4+
IoTDB supports three methods for data import:
55
* Data Import Tool: Use the import-data.sh (Unix/OS X) or import-data.bat (Windows) script in the tools directory to manually import CSV, SQL, or TsFile (open-source time-series file format) data into IoTDB.
66
* TsFile Auto-Loading Feature
7+
* Load `TsFile` SQL
78

89
<table style="text-align: left;">
910
<tbody>
@@ -21,13 +22,17 @@ IoTDB supports two methods for data import:
2122
<td>Can be used for single or batch import of SQL files into IoTDB</td>
2223
</tr>
2324
<tr>
24-
<td rowspan="2">TsFile</td>
25+
<td rowspan="3">TsFile</td>
2526
<td>Can be used for single or batch import of TsFile files into IoTDB</td>
2627
</tr>
2728
<tr>
2829
<td>TsFile Auto-Loading Feature</td>
2930
<td>Can automatically monitor a specified directory for newly generated TsFiles and load them into IoTDB.</td>
3031
</tr>
32+
<tr>
33+
<td>Load SQL</td>
34+
<td>Can be used for single or batch import of TsFile files into IoTDB</td>
35+
</tr>
3136
</tbody>
3237
</table>
3338

@@ -242,3 +247,43 @@ Add the following parameters to `iotdb-system.properties` (template: `iotdb-syst
242247
2. ​​**Restricted Directories**​: Do NOT set Pipe receiver directories, data directories, or other system paths as monitored directories.
243248
3. ​​**Directory Conflicts**​: Ensure `load_active_listening_fail_dir` does not overlap with `load_active_listening_dirs` or its subdirectories.
244249
4. ​​**Permissions**​: The monitored directory must have write permissions. Files are deleted after successful loading; insufficient permissions may cause duplicate loading.
250+
251+
## 4. Load SQL
252+
253+
IoTDB supports importing one or multiple TsFile files containing time series into another running IoTDB instance directly via SQL execution through the CLI.
254+
255+
### 4.1 Command
256+
257+
```SQL
258+
load '<path/dir>' with (
259+
'attribute-key1'='attribute-value1',
260+
'attribute-key2'='attribute-value2',
261+
)
262+
```
263+
264+
* `<path/dir>` : The path to a TsFile or a folder containing multiple TsFiles.
265+
* `<attributes>`: Optional parameters, as described below.
266+
267+
| Key | Key Description | Value Type | Value Range | Value is Required | Default Value |
268+
|--------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------|--------------------------------|-------------------|----------------------------|
269+
| `database-level` | When the database corresponding to the TsFile does not exist, the database hierarchy level can be specified via the ` database-level` parameter. The default is the level set in `iotdb-common.properties`. For example, setting level=1 means the prefix path of level 1 in all time series in the TsFile will be used as the database. | Integer | `[1: Integer.MAX_VALUE]` | No | 1 |
270+
| `on-success` | Action for successfully loaded TsFiles: `delete` (delete the TsFile after successful import) or `none` (retain the TsFile in the source folder). | String | `delete / none` | No | delete |
271+
| `convert-on-type-mismatch` | Whether to perform type conversion during loading if data types in the TsFile mismatch the target schema. | Boolean | `true / false` | No | true |
272+
| `verify` | Whether to validate the schema before loading the TsFile. | Boolean | `true / false` | No | true |
273+
| `tablet-conversion-threshold` | Size threshold (in bytes) for converting TsFiles into tablet format during loading. Default: `-1` (no conversion for any TsFile).(Available since v1.3.6) | Integer | `[-1,0 :`​`Integer.MAX_VALUE]` | No | -1 |
274+
275+
### 4.2 Example
276+
277+
```SQL
278+
-- Import tsfile by excuting load sql
279+
IoTDB> load '/home/dump1.tsfile' with ( 'on-success'='none')
280+
Msg: The statement is executed successfully.
281+
282+
-- Verify whether the import was successful
283+
IoTDB> select * from root.testdb.**
284+
+-----------------------------+------------------------------------+---------------------------------+-------------------------------+
285+
| Time|root.testdb.device.model.temperature|root.testdb.device.model.humidity|root.testdb.device.model.status|
286+
+-----------------------------+------------------------------------+---------------------------------+-------------------------------+
287+
|2025-04-17T10:35:47.218+08:00| 22.3| 19.4| true|
288+
+-----------------------------+------------------------------------+---------------------------------+-------------------------------+
289+
```

‎src/UserGuide/dev-1.3/Tools-System/Data-Import-Tool-1-3-4.md‎

Lines changed: 47 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,10 @@
11
# Data Import
22

33
## 1. Overview
4-
IoTDB supports two methods for data import:
4+
IoTDB supports three methods for data import:
55
* Data Import Tool: Use the import-data.sh (Unix/OS X) or import-data.bat (Windows) script in the tools directory to manually import CSV, SQL, or TsFile (open-source time-series file format) data into IoTDB.
66
* TsFile Auto-Loading Feature
7+
* Load `TsFile` SQL
78

89
<table style="text-align: left;">
910
<tbody>
@@ -21,13 +22,17 @@ IoTDB supports two methods for data import:
2122
<td>Can be used for single or batch import of SQL files into IoTDB</td>
2223
</tr>
2324
<tr>
24-
<td rowspan="2">TsFile</td>
25+
<td rowspan="3">TsFile</td>
2526
<td>Can be used for single or batch import of TsFile files into IoTDB</td>
2627
</tr>
2728
<tr>
2829
<td>TsFile Auto-Loading Feature</td>
2930
<td>Can automatically monitor a specified directory for newly generated TsFiles and load them into IoTDB.</td>
3031
</tr>
32+
<tr>
33+
<td>Load SQL</td>
34+
<td>Can be used for single or batch import of TsFile files into IoTDB</td>
35+
</tr>
3136
</tbody>
3237
</table>
3338

@@ -242,3 +247,43 @@ Add the following parameters to `iotdb-system.properties` (template: `iotdb-syst
242247
2. ​​**Restricted Directories**​: Do NOT set Pipe receiver directories, data directories, or other system paths as monitored directories.
243248
3. ​​**Directory Conflicts**​: Ensure `load_active_listening_fail_dir` does not overlap with `load_active_listening_dirs` or its subdirectories.
244249
4. ​​**Permissions**​: The monitored directory must have write permissions. Files are deleted after successful loading; insufficient permissions may cause duplicate loading.
250+
251+
## 4. Load SQL
252+
253+
IoTDB supports importing one or multiple TsFile files containing time series into another running IoTDB instance directly via SQL execution through the CLI.
254+
255+
### 4.1 Command
256+
257+
```SQL
258+
load '<path/dir>' with (
259+
'attribute-key1'='attribute-value1',
260+
'attribute-key2'='attribute-value2',
261+
)
262+
```
263+
264+
* `<path/dir>` : The path to a TsFile or a folder containing multiple TsFiles.
265+
* `<attributes>`: Optional parameters, as described below.
266+
267+
| Key | Key Description | Value Type | Value Range | Value is Required | Default Value |
268+
|--------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------|--------------------------------|-------------------|----------------------------|
269+
| `database-level` | When the database corresponding to the TsFile does not exist, the database hierarchy level can be specified via the ` database-level` parameter. The default is the level set in `iotdb-common.properties`. For example, setting level=1 means the prefix path of level 1 in all time series in the TsFile will be used as the database. | Integer | `[1: Integer.MAX_VALUE]` | No | 1 |
270+
| `on-success` | Action for successfully loaded TsFiles: `delete` (delete the TsFile after successful import) or `none` (retain the TsFile in the source folder). | String | `delete / none` | No | delete |
271+
| `convert-on-type-mismatch` | Whether to perform type conversion during loading if data types in the TsFile mismatch the target schema. | Boolean | `true / false` | No | true |
272+
| `verify` | Whether to validate the schema before loading the TsFile. | Boolean | `true / false` | No | true |
273+
| `tablet-conversion-threshold` | Size threshold (in bytes) for converting TsFiles into tablet format during loading. Default: `-1` (no conversion for any TsFile).(Available since v1.3.6) | Integer | `[-1,0 :`​`Integer.MAX_VALUE]` | No | -1 |
274+
275+
### 4.2 Example
276+
277+
```SQL
278+
-- Import tsfile by excuting load sql
279+
IoTDB> load '/home/dump1.tsfile' with ( 'on-success'='none')
280+
Msg: The statement is executed successfully.
281+
282+
-- Verify whether the import was successful
283+
IoTDB> select * from root.testdb.**
284+
+-----------------------------+------------------------------------+---------------------------------+-------------------------------+
285+
| Time|root.testdb.device.model.temperature|root.testdb.device.model.humidity|root.testdb.device.model.status|
286+
+-----------------------------+------------------------------------+---------------------------------+-------------------------------+
287+
|2025-04-17T10:35:47.218+08:00| 22.3| 19.4| true|
288+
+-----------------------------+------------------------------------+---------------------------------+-------------------------------+
289+
```

‎src/zh/UserGuide/V1.3.x/Tools-System/Data-Import-Tool-1-3-4.md‎

Lines changed: 47 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,11 @@
22

33
## 1. 功能概述
44

5-
IoTDB 支持两种方式进行数据导入
5+
IoTDB 支持三种方式进行数据导入
66

77
* 数据导入工具：tools 目录下的手动数据导入工具 `import-data.sh/bat`，可以将 CSV、SQL、及TsFile（开源时序文件格式）的数据导入 IoTDB。
88
* TsFile 自动加载功能
9+
* `Load SQL` 导入 TsFile
910

1011
<table style="text-align: left;">
1112
<tbody>
@@ -23,13 +24,17 @@ IoTDB 支持两种方式进行数据导入
2324
<td>可用于单个或一个目录的 SQL 文件批量导入 IoTDB</td>
2425
</tr>
2526
<tr>
26-
<td rowspan="2">TsFile</td>
27+
<td rowspan="3">TsFile</td>
2728
<td>可用于单个或一个目录的 TsFile 文件批量导入 IoTDB</td>
2829
</tr>
2930
<tr>
3031
<td>TsFile 自动加载功能</td>
3132
<td> 可以监听指定路径下新产生的TsFile文件，并将其加载进IoTDB</td>
3233
</tr>
34+
<tr>
35+
<td>Load SQL</td>
36+
<td>可用于单个或一个目录的 TsFile 文件批量导入 IoTDB</td>
37+
</tr>
3338
</tbody>
3439
</table>
3540

@@ -249,3 +254,43 @@ error: Invalid thread number '0'. Please set a positive integer.
249254
2. 禁止设置 Pipe 的 receiver 目录、存放数据的 data 目录等作为监听目录
250255
3. 禁止 `load_active_listening_fail_dir` 与 `load_active_listening_dirs` 存在相同的目录，或者互相嵌套
251256
4. 保证 `load_active_listening_dirs` 目录有足够的权限，在加载成功之后，文件将会被删除，如果没有删除权限，则会重复加载
257+
258+
## 4. Load SQL
259+
260+
IoTDB 支持通过 CLI 执行 SQL 直接将存有时间序列的一个或多个 TsFile 文件导入到另外一个正在运行的 IoTDB 实例中。
261+
262+
### 4.1 运行命令
263+
264+
```SQL
265+
load '<path/dir>' with (
266+
'attribute-key1'='attribute-value1',
267+
'attribute-key2'='attribute-value2',
268+
)
269+
```
270+
271+
* `<path/dir>` ：文件本身，或是包含若干文件的文件夹路径
272+
* `<attributes>`：可选参数，具体如下表所示
273+
274+
| Key | Key 描述 | Value 类型 | Value 取值范围 | Value 是否必填 | Value 默认值 |
275+
| --------------------------------------- |------------------------------------------------------------------------------------------------------------------------------------------------------------------------| ------------ | ----------------------------------------- | ---------------- | -------------------------- |
276+
| `database-level` | 当 tsfile 对应的 database 不存在时，可以通过` database-level`参数的值来制定 database 的级别，默认为`iotdb-common.properties`中设置的级别。<br>例如当设置 level 参数为 1 时表明此 tsfile 中所有时间序列中层级为1的前缀路径是 database。 | Integer | `[1: Integer.MAX_VALUE]` | 否 | 1 |
277+
| `on-success` | 表示对于成功载入的 tsfile 的处置方式:默认为`delete`，即tsfile 成功加载后将被删除;`none `表明 tsfile 成功加载之后依然被保留在源文件夹, | String | `delete / none` | 否 | delete |
278+
| `convert-on-type-mismatch` | 加载 tsfile 时，如果数据类型不一致，是否进行转换 | Boolean | `true / false` | 否 | true |
279+
| `verify` | 加载 tsfile 前是否校验 schema | Boolean | `true / false` | 否 | true |
280+
| `tablet-conversion-threshold` | 转换为 tablet 形式的 tsfile 大小阈值，针对小文件 tsfile 加载，采用将其转换为 tablet 形式进行写入:默认值为 -1，即任意大小 tsfile 都不进行转换 （V1.3.6及以后的V1.x版本支持） | Integer | `[-1,0 :`​`Integer.MAX_VALUE]` | 否 | -1 |
281+
282+
### 4.2 运行示例
283+
284+
```SQL
285+
-- 通过load sql 导入 tsfile
286+
IoTDB> load '/home/dump1.tsfile' with ( 'on-success'='none')
287+
Msg: The statement is executed successfully.
288+
289+
-- 验证数据导入成功
290+
IoTDB> select * from root.testdb.**
291+
+-----------------------------+------------------------------------+---------------------------------+-------------------------------+
292+
| Time|root.testdb.device.model.temperature|root.testdb.device.model.humidity|root.testdb.device.model.status|
293+
+-----------------------------+------------------------------------+---------------------------------+-------------------------------+
294+
|2025-04-17T10:35:47.218+08:00| 22.3| 19.4| true|
295+
+-----------------------------+------------------------------------+---------------------------------+-------------------------------+
296+
```

0 commit comments

Comments
 (0)