Skip to content

Commit c7315bd

Browse files
W1y1rW1y1r
authored
Data export script document merging (#415)
* Data export script document merging * Update document name * Change the sidebar name to Data Export
1 parent 6a85564 commit c7315bd

File tree

16 files changed

+828
-8
lines changed

16 files changed

+828
-8
lines changed

‎src/.vuepress/sidebar/V1.3.x/en.ts‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -99,8 +99,7 @@ export const enSidebar = {
9999
{ text: 'Monitor Tool', link: 'Monitor-Tool_apache' },
100100
{ text: 'Benchmark Tool', link: 'Benchmark' },
101101
{ text: 'Maintenance Tool', link: 'Maintenance-Tool_apache' },
102-
{ text: 'Data Import Export Script', link: 'Data-Import-Export-Tool' },
103-
{ text: 'TsFile Import Export Script', link: 'TsFile-Import-Export-Tool' },
102+
{ text: 'Data Export', link: 'Data-Export-Tool' },
104103
],
105104
},
106105
{

‎src/.vuepress/sidebar/V1.3.x/zh.ts‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -99,8 +99,7 @@ export const zhSidebar = {
9999
{ text: '监控工具', link: 'Monitor-Tool_apache' },
100100
{ text: '测试工具', link: 'Benchmark' },
101101
{ text: '运维工具', link: 'Maintenance-Tool_apache' },
102-
{ text: '数据导入导出脚本', link: 'Data-Import-Export-Tool' },
103-
{ text: 'TsFile导入导出脚本', link: 'TsFile-Import-Export-Tool' },
102+
{ text: '数据导出', link: 'Data-Export-Tool' },
104103
],
105104
},
106105
{

‎src/.vuepress/sidebar_timecho/V1.3.x/en.ts‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -108,8 +108,7 @@ export const enSidebar = {
108108
{ text: 'Monitor Tool', link: 'Monitor-Tool_timecho' },
109109
{ text: 'Benchmark Tool', link: 'Benchmark' },
110110
{ text: 'Maintenance Tool', link: 'Maintenance-Tool_timecho' },
111-
{ text: 'Data Import Export Script', link: 'Data-Import-Export-Tool' },
112-
{ text: 'TsFile Import Export Script', link: 'TsFile-Import-Export-Tool' },
111+
{ text: 'Data Export', link: 'Data-Export-Tool' },
113112
],
114113
},
115114
{

‎src/.vuepress/sidebar_timecho/V1.3.x/zh.ts‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -108,8 +108,7 @@ export const zhSidebar = {
108108
{ text: '监控工具', link: 'Monitor-Tool_timecho' },
109109
{ text: '测试工具', link: 'Benchmark' },
110110
{ text: '运维工具', link: 'Maintenance-Tool_timecho' },
111-
{ text: '数据导入导出脚本', link: 'Data-Import-Export-Tool' },
112-
{ text: 'TsFile导入导出脚本', link: 'TsFile-Import-Export-Tool' },
111+
{ text: '数据导出', link: 'Data-Export-Tool' },
113112
],
114113
},
115114
{

‎src/UserGuide/Master/Tools-System/Data-Export-Tool.md‎

Lines changed: 213 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,213 @@
1+
# Data Export
2+
3+
## 1. Introduction to Export Tools
4+
5+
Export tools can export data queried from SQL into specified formats, including the `export-tsfile.sh/bat` script for exporting TsFile files and the `export-data.sh/bat` script that supports exporting in CSV and SQL formats.
6+
7+
## 2. Supported Data Types
8+
9+
- CSV: A plain text format for storing formatted data, which needs to be constructed according to the specified CSV format mentioned below.
10+
11+
- SQL: A file containing custom SQL statements.
12+
13+
- TsFile: The file format for time series used in IoTDB.
14+
15+
## 3. export-tsfile Script
16+
17+
Supports TsFile: The file format for time series used in IoTDB.
18+
19+
20+
#### 3.1 Command
21+
22+
```Bash
23+
# Unix/OS X
24+
tools/export-tsfile.sh -h <ip> -p <port> -u <username> -pw <password> -td <directory> [-f <export filename> -q <query command> -s <sql file>]
25+
26+
# Windows
27+
tools\export-tsfile.bat -h <ip> -p <port> -u <username> -pw <password> -td <directory> [-f <export filename> -q <query command> -s <sql file>]
28+
```
29+
30+
#### 3.2 Parameter Introduction
31+
32+
| **Parameter** | **Definition** | **Required** | **Default** |
33+
| -------- | ------------------------------------------------------------ | ------------ | --------- |
34+
| -h | Hostname | No | root |
35+
| -p | Port | No | root |
36+
| -u | Username | No | 127.0.0.1 |
37+
| -pw | Password | No | 6667 |
38+
| -t | Target file directory, used to specify the directory where the output file should be saved | Yes | - |
39+
| -tfn | Name of the export file | No | - |
40+
| -q | Number of query commands to be executed, possibly used for batch execution of queries | No | - |
41+
| -s | SQL file path, used to specify the location of the file containing the SQL statements to be executed | No | - |
42+
| -timeout | Session query timeout, used to specify the maximum allowed time before the query operation is automatically terminated | No | - |
43+
44+
In addition, if the `-s` and `-q` parameters are not used, after the export script is started, you need to enter the query statement according to the program prompt, and different query results will be saved to different TsFile files.
45+
46+
47+
48+
#### 3.3 Running Examples
49+
50+
```Bash
51+
# Unix/OS X
52+
tools/export-tsfile.sh -h 127.0.0.1 -p 6667 -u root -pw root -td ./
53+
# or
54+
tools/export-tsfile.sh -h 127.0.0.1 -p 6667 -u root -pw root -td ./ -q "select * from root.**"
55+
# Or
56+
tools/export-tsfile.sh -h 127.0.0.1 -p 6667 -u root -pw root -td ./ -s ./sql.txt
57+
# Or
58+
tools/export-tsfile.sh -h 127.0.0.1 -p 6667 -u root -pw root -td ./ -s ./sql.txt -f myTsFile
59+
# Or
60+
tools/export-tsfile.sh -h 127.0.0.1 -p 6667 -u root -pw root -td ./ -s ./sql.txt -f myTsFile -t 10000
61+
62+
# Windows
63+
tools/export-tsfile.bat -h 127.0.0.1 -p 6667 -u root -pw root -td ./
64+
# Or
65+
tools/export-tsfile.bat -h 127.0.0.1 -p 6667 -u root -pw root -td ./ -q "select * from root.**"
66+
# Or
67+
tools/export-tsfile.bat -h 127.0.0.1 -p 6667 -u root -pw root -td ./ -s ./sql.txt
68+
# Or
69+
tools/export-tsfile.bat -h 127.0.0.1 -p 6667 -u root -pw root -td ./ -s ./sql.txt -f myTsFile
70+
# Or
71+
tools/export-tsfile.sh -h 127.0.0.1 -p 6667 -u root -pw root -td ./ -s ./sql.txt -f myTsFile -t 10000
72+
```
73+
74+
## 4. export-data Script
75+
76+
Supports CSV: A plain text format for storing formatted data, which needs to be constructed according to the specified CSV format below.
77+
78+
Supports SQL: A file containing custom SQL statements.
79+
80+
#### 4.1 Command
81+
82+
```Bash
83+
# Unix/OS X
84+
>tools/export-data.sh -h <ip> -p <port> -u <username> -pw <password> -t <directory> [-tf <time-format> -datatype <true/false> -q <query command> -s <source sql file> -tfn <target file name> -lpf <int> -type <export type> -aligned <true/false>]
85+
86+
# Windows
87+
>tools\export-data.bat -h <ip> -p <port> -u <username> -pw <password> -t <directory> [-tf <time-format> -datatype <true/false> -q <query command> -s <source sql file> -tfn <target file name> -lpf <int> -type <export type> -aligned <true/false>]
88+
```
89+
90+
#### 4.2 Parameter Introduction
91+
92+
| **Parameter** | **Definition** | **Required** | **Default** |
93+
| --------- | ------------------------------------------------------------ | ------------ | ------------------------ |
94+
| -h | Hostname | No | 127.0.0.1 |
95+
| -p | Port | No | 6667 |
96+
| -u | Username | No | root |
97+
| -pw | Password | No | root |
98+
| -t | Exported CSV or SQL file output path (In V1.3.2, the parameter was `-td`) | Yes | |
99+
| -datatype | Whether to print the corresponding data type behind the time series in the CSV file header, options are true or false | No | true |
100+
| -q | Directly specify the query statement to be executed in the command (currently only supports some statements, detailed list see below) Note: -q and -s parameters are required to fill in one, if filled in at the same time, -q takes effect. For detailed supported SQL statement examples, please refer to "SQL Statement Support Details" below | 否 | |
101+
| -s | Specify the SQL file, which may contain one or more SQL statements. If it contains multiple SQL statements, they should be separated by newlines (carriage returns). Each SQL statement corresponds to one or more output CSV or SQL files. Note: -q and -s parameters are required to fill in one, if filled in at the same time, -q takes effect. For detailed supported SQL statement examples, please refer to "SQL Statement Support Rules" below | 否 | |
102+
| -type | Specify the type of exported file, options are csv or sql | No | csv |
103+
| -tf | Specify the time format. The time format must comply with the[ISO 8601](https://calendars.wikia.org/wiki/ISO_8601)standard, or `timestamp` Explanation: Only effective when the - type is CSV | 否 | yyyy-MM-dd HH:mm:ss.SSSz |
104+
| -lpf | Specify the maximum number of lines in the dump file to be exported (V1.3.2 version parameter is`-linesPerFile`) | No | 10000 |
105+
| -timeout | Specify the timeout period for session queries, in ms | No | -1 |
106+
107+
#### 4.3 SQL 语句支持规则
108+
109+
1. Only query statements are supported; non-query statements (such as metadata management, system management, etc.) are not supported. For unsupported SQL, the program will automatically skip and output an error message.
110+
111+
2. In query statements, the current version only supports the export of raw data. If there are group by, aggregate functions, UDFs, operational operators, etc., they are not supported for export as SQL. When exporting raw data, please note that if exporting data from multiple devices, please use the align by device statement. Detailed examples are as follows:
112+
113+
114+
| | **支持导出** | **示例** |
115+
| ----------------------------------------- | ------------ | --------------------------------------------- |
116+
| Raw data single device query | Supported | select * from root.s_0.d_0 |
117+
| | 原始数据多设备查询（aligin by device） | Supported | select * from root.** align by device |
118+
| 支持 | select * from root.** align by device |
119+
| Raw data multi-device query (without align by device) | Not Supported | select * from root.**select * from root.s_0.* |
120+
121+
#### 4.4 Running Examples
122+
123+
- Export all data within the scope of a SQL execution to a CSV file.
124+
125+
```Bash
126+
# Unix/OS X
127+
>tools/export-data.sh -h <ip> -p <port> -u <username> -pw <password> -t <directory> [-tf <time-format> -datatype <true/false> -q <query command> -s <source sql file> -tfn <target file name> -lpf <int> -type <export type> -aligned <true/false>]
128+
129+
# Windows
130+
>tools\export-data.bat -h <ip> -p <port> -u <username> -pw <password> -t <directory> [-tf <time-format> -datatype <true/false> -q <query command> -s <source sql file> -tfn <target file name> -lpf <int> -type <export type> -aligned <true/false>]
131+
```
132+
133+
- Export results
134+
135+
```Bash
136+
Time,root.stock.Legacy.0700HK.L1_BidPrice,root.stock.Legacy.0700HK.Type,root.stock.Legacy.0700HK.L1_BidSize,root.stock.Legacy.0700HK.Domain,root.stock.Legacy.0700HK.L1_BuyNo,root.stock.Legacy.0700HK.L1_AskPrice
137+
2024-07-29T18:37:18.700+08:00,0.9666617,3.0,0.021367407654674264,-6.0,false,0.8926191
138+
2024-07-29T18:37:19.701+08:00,0.3057328,3.0,0.9965377284981661,-5.0,false,0.15167356
139+
```
140+
141+
- Export all data within the scope of all SQL executions in a SQL file to a CSV file.
142+
143+
144+
```Bash
145+
# Unix/OS X
146+
>tools/export-data.sh -t ./data/ -s export.sql
147+
# Windows
148+
>tools/export-data.bat -t ./data/ -s export.sql
149+
```
150+
151+
- Content of export.sql file (file pointed to by -s parameter)
152+
153+
154+
```Bash
155+
select * from root.stock.** limit 100
156+
select * from root.db.** limit 100
157+
```
158+
159+
- Export result file 1
160+
161+
162+
```Bash
163+
Time,root.stock.Legacy.0700HK.L1_BidPrice,root.stock.Legacy.0700HK.Type,root.stock.Legacy.0700HK.L1_BidSize,root.stock.Legacy.0700HK.Domain,root.stock.Legacy.0700HK.L1_BuyNo,root.stock.Legacy.0700HK.L1_AskPrice
164+
2024-07-29T18:37:18.700+08:00,0.9666617,3.0,0.021367407654674264,-6.0,false,0.8926191
165+
2024-07-29T18:37:19.701+08:00,0.3057328,3.0,0.9965377284981661,-5.0,false,0.15167356
166+
```
167+
168+
- Export result file 2
169+
170+
171+
```Bash
172+
Time,root.db.Random.RandomBoolean
173+
2024-07-22T17:16:05.820+08:00,true
174+
2024-07-22T17:16:02.597+08:00,false
175+
```
176+
177+
- Export data defined in the SQL file within the IoTDB database in an aligned format as SQL statements.
178+
179+
180+
```Bash
181+
# Unix/OS X
182+
>tools/export-data.sh -h 127.0.0.1 -p 6667 -u root -p root -t ./data/ -s export.sql -type sql -aligned true
183+
# Windows
184+
>tools/export-data.bat -h 127.0.0.1 -p 6667 -u root -p root -t ./data/ -s export.sql -type sql -aligned true
185+
```
186+
187+
- Export results
188+
189+
190+
```Bash
191+
INSERT INTO root.stock.Legacy.0700HK(TIMESTAMP,L1_BidPrice,Type,L1_BidSize,Domain,L1_BuyNo,L1_AskPrice) ALIGNED VALUES (1722249629831,0.62308747,2.0,0.012206747854849653,-6.0,false,0.14164352);
192+
INSERT INTO root.stock.Legacy.0700HK(TIMESTAMP,L1_BidPrice,Type,L1_BidSize,Domain,L1_BuyNo,L1_AskPrice) ALIGNED VALUES (1722249630834,0.7520042,3.0,0.22760657101910464,-5.0,true,0.089064896);
193+
INSERT INTO root.stock.Legacy.0700HK(TIMESTAMP,L1_BidPrice,Type,L1_BidSize,Domain,L1_BuyNo,L1_AskPrice) ALIGNED VALUES (1722249631835,0.3981064,3.0,0.6254559288663467,-6.0,false,0.9767922);
194+
```
195+
196+
- Export all data within the scope of a SQL execution to a CSV file, specifying the export time format as `yyyy-MM-dd HH:mm:ss`, and print the corresponding data type behind the table header time series.
197+
198+
199+
```Bash
200+
# Unix/OS X
201+
>tools/export-data.sh -h 127.0.0.1 -p 6667 -u root -p root -t ./data/ -s export.sql -type sql -aligned true
202+
# Windows
203+
>tools/export-data.bat -h 127.0.0.1 -p 6667 -u root -p root -t ./data/ -s export.sql -type sql -aligned true
204+
```
205+
206+
- Export results
207+
208+
209+
```Bash
210+
Time,root.stock.Legacy.0700HK.L1_BidPrice(DOUBLE),root.stock.Legacy.0700HK.Type(DOUBLE),root.stock.Legacy.0700HK.L1_BidSize(DOUBLE),root.stock.Legacy.0700HK.Domain(DOUBLE),root.stock.Legacy.0700HK.L1_BuyNo(BOOLEAN),root.stock.Legacy.0700HK.L1_AskPrice(DOUBLE)
211+
2024-07-30 10:33:55,0.44574088,3.0,0.21476832811611501,-4.0,true,0.5951748
212+
2024-07-30 10:33:56,0.6880933,3.0,0.6289119476165305,-5.0,false,0.114634395
213+
```

src/UserGuide/Master/Tools-System/Data-Import-Export-Tool.md renamed to src/UserGuide/Master/stage/Data-Import-Export-Tool.md

File renamed without changes.

src/UserGuide/Master/Tools-System/TsFile-Import-Export-Tool.md renamed to src/UserGuide/Master/stage/TsFile-Import-Export-Tool.md

File renamed without changes.

0 commit comments

Comments
 (0)