Alter column datatype with ANSI SQL statement #16027
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Conflicts: # iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/consensus/request/ConfigPhysicalPlanType.java # pom.xml
# Conflicts: # integration-test/src/main/java/org/apache/iotdb/it/env/cluster/config/MppDataNodeConfig.java # integration-test/src/main/java/org/apache/iotdb/it/env/remote/config/RemoteDataNodeConfig.java # integration-test/src/main/java/org/apache/iotdb/itbase/env/DataNodeConfig.java # iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/manager/ProcedureManager.java # iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/schema/table/column/TsTableColumnSchema.java # pom.xml
# Conflicts: # iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/sql/ast/AstVisitor.java
# Conflicts: # iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/memtable/AlignedWritableMemChunk.java
# Conflicts: # iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/execution/config/TableConfigTaskVisitor.java
# Conflicts: # integration-test/src/test/java/org/apache/iotdb/relational/it/db/it/IoTDBSetSystemStatusTableIT.java
(cherry picked from commit ea9f726)
# Conflicts: # iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/modification/ModificationFile.java
# Conflicts: # iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/plan/node/write/RelationalInsertRowNode.java # iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/plan/node/write/RelationalInsertTabletNode.java # iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/memtable/AbstractMemTable.java # iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/memtable/AlignedWritableMemChunk.java # iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/memtable/AlignedWritableMemChunkGroup.java # iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/read/reader/chunk/MemAlignedPageReader.java # iotdb-core/datanode/src/main/java/org/apache/iotdb/db/utils/datastructure/AlignedTVList.java
# Conflicts: # iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/manager/schema/ClusterSchemaManager.java # iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/persistence/schema/ClusterSchemaInfo.java # iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/plan/node/write/RelationalInsertTabletNode.java # iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/memtable/AlignedWritableMemChunkGroup.java # iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/memtable/WritableMemChunkGroup.java
…t timeseries and chunk and pagereader.
… statistics is not valid when any type transfer to STRING/TEXT type. For example, when try to get max() value in the sql, will occur '512' > '1024', max() value is 512;
… "String", but not "Binary".
…e value two sides of assert validation ago.
…ismatched: class StringStatistics vs BinaryStatistics" when tsfile is unclosed and current type and previous type is not compatible.
# Conflicts: # integration-test/src/test/java/org/apache/iotdb/relational/it/session/IoTDBSessionRelationalIT.java # iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/manager/pipe/receiver/protocol/IoTDBConfigNodeReceiver.java # iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/execution/config/executor/ClusterConfigTaskExecutor.java # iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/compaction/execute/performer/impl/FastCompactionPerformer.java # iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/compaction/execute/performer/impl/ReadChunkCompactionPerformer.java # iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/compaction/execute/utils/executor/fast/FastNonAlignedSeriesCompactionExecutor.java # iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/memtable/IWritableMemChunkGroup.java # iotdb-core/datanode/src/main/java/org/apache/iotdb/db/utils/CommonUtils.java # pom.xml
Fix the problem that valueChunkMetadata get wrong datatype so that can't generate valid statistics object in the switch code block cause by newChunkMetadata set the property value of TsDataType
# Conflicts: # iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/consensus/request/ConfigPhysicalPlanVisitor.java
…ccumulator query among tree and table model.
jt2594838
requested changes
Dec 30, 2025
Comment on lines
+330
to
+332
| // typesToTest.remove(TSDataType.STRING); | ||
| // typesToTest.remove(TSDataType.TEXT); | ||
| // typesToTest.remove(TSDataType.DATE); |
Comment on lines
+966
to
+982
| @Test | ||
| public void testAlterViewType() throws IoTDBConnectionException, StatementExecutionException { | ||
| String[] createTreeDataSqls = { | ||
| "CREATE ALIGNED TIMESERIES root.db.battery.b0(voltage INT32, current FLOAT)", | ||
| "INSERT INTO root.db.battery.b0(time, voltage, current) aligned values (" | ||
| + (System.currentTimeMillis() - 100000) | ||
| + ", 1, 1)", | ||
| "CREATE ALIGNED TIMESERIES root.db.battery.b1(voltage INT32, current FLOAT)", | ||
| "INSERT INTO root.db.battery.b1(time, voltage, current) aligned values (1, 1, 1)", | ||
| "INSERT INTO root.db.battery.b1(time, voltage, current) aligned values (2, 1, 1)", | ||
| "INSERT INTO root.db.battery.b1(time, voltage, current) aligned values (3, 1, 1)", | ||
| "INSERT INTO root.db.battery.b1(time, voltage, current) aligned values (4, 1, 1)", | ||
| "INSERT INTO root.db.battery.b1(time, voltage, current) aligned values (" | ||
| + System.currentTimeMillis() | ||
| + ", 1, 1)", | ||
| "CREATE TIMESERIES root.db.battery.b2.voltage INT32", | ||
| "CREATE TIMESERIES root.db.battery.b2.current FLOAT", |
Contributor
There was a problem hiding this comment.
Do not put tree model tests in this package.
Contributor
There was a problem hiding this comment.
Move this to a correct package (not relational)
| Arrays.asList(1d, 2d, 3)); | ||
| Assert.fail(); | ||
| } catch (StatementExecutionException e) { | ||
| System.out.println("hi: " + e.getMessage()); |
Comment on lines
-3078
to
+3081
| TSFileDescriptor.getInstance() | ||
| .getConfig() | ||
| .getValueEncoder(measurementSchema.getType()))) | ||
| String.valueOf( | ||
| TSFileDescriptor.getInstance() | ||
| .getConfig() | ||
| .getValueEncoder(measurementSchema.getType())))) |
Comment on lines
+364
to
+368
Comment on lines
+292
to
+300
| private static void addSymmetricPairs( | ||
| Set<Pair<TSDataType, TSDataType>> set, TSDataType... dataTypes) { | ||
| for (int i = 0; i < dataTypes.length; i++) { | ||
| for (int j = i + 1; j < dataTypes.length; j++) { | ||
| set.add(new Pair<>(dataTypes[i], dataTypes[j])); | ||
| set.add(new Pair<>(dataTypes[j], dataTypes[i])); | ||
| } | ||
| } | ||
| } |
Contributor
There was a problem hiding this comment.
This implementation is not very efficient (n^2 as you can imagine), may consider something like Disjoint Set.
Map<TsDataType, Class> dataTypeColumnClassMap;
dataTypeColumnClassMap.put(DATE, IntColumn.class);
dataTypeColumnClassMap.put(INT32, IntColumn.class);
dataTypeColumnClassMap.get(typeA) == dataTypeColumnClassMap.get(typeB)
Comment on lines
+318
to
+327
| timeseriesMetadata.setChunkMetadataList( | ||
| timeseriesMetadata.getChunkMetadataList().stream() | ||
| .map( | ||
| iChunkMetadata -> { | ||
| if (iChunkMetadata == null) return null; | ||
| iChunkMetadata.setModified(true); | ||
| return (ChunkMetadata) iChunkMetadata; | ||
| }) | ||
| .collect(Collectors.toList())); | ||
| } |
Contributor
There was a problem hiding this comment.
Is it necessary to collect as a new list? You may simply modify the original chunkMetadata
| return; | ||
| } | ||
| if (!SchemaUtils.isUsingSameColumn(sourceDataType, targetDataType) | ||
| && Arrays.asList(TSDataType.STRING, TSDataType.TEXT).contains(targetDataType)) { |
Contributor
There was a problem hiding this comment.
Use direct comparison, check other places.
Comment on lines
+1033
to
+1034
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
CHN doc:
https://timechor.feishu.cn/docx/AugedrYLqoybsbxzgEZc2eTinTg?from=from_copylink
IoTDB> alter timeseries root.db.battery.b0.current SET DATA TYPE DOUBLE;
Msg: The statement is executed successfully.
IoTDB> show timeseries root.db.battery.b0.current;
+--------------------------+-----+--------+--------+--------+-----------+----+----------+--------+------------------+--------+
| Timeseries|Alias|Database|DataType|Encoding|Compression|Tags|Attributes|Deadband|DeadbandParameters|ViewType|
+--------------------------+-----+--------+--------+--------+-----------+----+----------+--------+------------------+--------+
|root.db.battery.b0.current| null| root.db| DOUBLE| GORILLA| LZ4|null| null| null| null| BASE|
+--------------------------+-----+--------+--------+--------+-----------+----+----------+--------+------------------+--------+
Total line number = 1
It costs 0.043s
IoTDB> alter timeseries root.db.battery.b0.current SET DATA TYPE STRING;
Msg: The statement is executed successfully.
IoTDB> show timeseries root.db.battery.b0.current;
+--------------------------+-----+--------+--------+--------+-----------+----+----------+--------+------------------+--------+
| Timeseries|Alias|Database|DataType|Encoding|Compression|Tags|Attributes|Deadband|DeadbandParameters|ViewType|
+--------------------------+-----+--------+--------+--------+-----------+----+----------+--------+------------------+--------+
|root.db.battery.b0.current| null| root.db| STRING| GORILLA| LZ4|null| null| null| null| BASE|
+--------------------------+-----+--------+--------+--------+-----------+----+----------+--------+------------------+--------+
Total line number = 1
It costs 0.009s