Skip to content

Commit f3acc10

Browse files
Merge pull request #499 from cloudsufi/oracle-case-issue-1.10
[PLUGIN-1793] Oracle sink plugin issue with small case
2 parents 00677fe + 83a4b90 commit f3acc10

29 files changed

+2560
-51
lines changed

oracle-plugin/src/e2e-test/features/oracle/Datatype.feature

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616

1717
@Oracle
1818
Feature: Oracle - Verify Oracle source data transfer for multiple datatypes
19-
@ORACLE_SOURCE_DATATYPES_TEST @ORACLE_SINK_TEST @Oracle_Required
19+
@ORACLE_SOURCE_DATATYPES_TEST @ORACLE_TARGET_DATATYPES_TEST @Oracle_Required
2020
# Oracle Sanity test to transfer table data containing multiple datatypes
2121
Scenario: To verify data is getting transferred from Oracle to Oracle successfully
2222
Given Open Datafusion Project to configure pipeline
@@ -32,7 +32,7 @@ Feature: Oracle - Verify Oracle source data transfer for multiple datatypes
3232
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
3333
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
3434
Then Select radio button plugin property: "connectionType" with value: "service"
35-
Then Select radio button plugin property: "role" with value: "sysdba"
35+
Then Select radio button plugin property: "role" with value: "normal"
3636
Then Enter input plugin property: "referenceName" with value: "sourceRef"
3737
Then Replace input plugin property: "database" with value: "databaseName"
3838
Then Enter textarea plugin property: "importQuery" with value: "selectQuery"
@@ -51,7 +51,7 @@ Feature: Oracle - Verify Oracle source data transfer for multiple datatypes
5151
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
5252
Then Enter input plugin property: "referenceName" with value: "targetRef"
5353
Then Select radio button plugin property: "connectionType" with value: "service"
54-
Then Select radio button plugin property: "role" with value: "sysdba"
54+
Then Select radio button plugin property: "role" with value: "normal"
5555
Then Validate "Oracle2" plugin properties
5656
Then Close the Plugin Properties page
5757
Then Save the pipeline

oracle-plugin/src/e2e-test/features/oracle/DatatypeLong.feature

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616

1717
@Oracle
1818
Feature: Oracle - Verify Oracle source data transfer of type LONG
19-
@ORACLE_SOURCE_DATATYPES_TEST2 @ORACLE_SINK_TEST @Oracle_Required
19+
@ORACLE_SOURCE_DATATYPES_TEST2 @ORACLE_TARGET_DATATYPES_TEST2 @Oracle_Required
2020
# Oracle Sanity test to transfer table data containing LONG
2121
Scenario: To verify data is getting transferred from Oracle to Oracle successfully
2222
Given Open Datafusion Project to configure pipeline
@@ -32,7 +32,7 @@ Feature: Oracle - Verify Oracle source data transfer of type LONG
3232
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
3333
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
3434
Then Select radio button plugin property: "connectionType" with value: "service"
35-
Then Select radio button plugin property: "role" with value: "sysdba"
35+
Then Select radio button plugin property: "role" with value: "normal"
3636
Then Enter input plugin property: "referenceName" with value: "sourceRef"
3737
Then Replace input plugin property: "database" with value: "databaseName"
3838
Then Enter textarea plugin property: "importQuery" with value: "selectQuery"
@@ -51,7 +51,7 @@ Feature: Oracle - Verify Oracle source data transfer of type LONG
5151
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
5252
Then Enter input plugin property: "referenceName" with value: "targetRef"
5353
Then Select radio button plugin property: "connectionType" with value: "service"
54-
Then Select radio button plugin property: "role" with value: "sysdba"
54+
Then Select radio button plugin property: "role" with value: "normal"
5555
Then Validate "Oracle2" plugin properties
5656
Then Close the Plugin Properties page
5757
Then Save the pipeline

oracle-plugin/src/e2e-test/features/oracle/DatatypeLongRaw.feature

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616

1717
@Oracle
1818
Feature: Oracle - Verify Oracle source data transfer of type LONG RAW
19-
@ORACLE_SOURCE_LONGRAW_TEST @ORACLE_SINK_TEST @Oracle_Required
19+
@ORACLE_SOURCE_LONGRAW_TEST @ORACLE_TARGET_LONGRAW_TEST @Oracle_Required
2020
# Oracle Sanity test to transfer table data containing LONG RAW
2121
Scenario: To verify data is getting transferred from Oracle to Oracle successfully
2222
Given Open Datafusion Project to configure pipeline
@@ -32,7 +32,7 @@ Feature: Oracle - Verify Oracle source data transfer of type LONG RAW
3232
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
3333
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
3434
Then Select radio button plugin property: "connectionType" with value: "service"
35-
Then Select radio button plugin property: "role" with value: "sysdba"
35+
Then Select radio button plugin property: "role" with value: "normal"
3636
Then Enter input plugin property: "referenceName" with value: "sourceRef"
3737
Then Replace input plugin property: "database" with value: "databaseName"
3838
Then Enter textarea plugin property: "importQuery" with value: "selectQuery"
@@ -51,7 +51,7 @@ Feature: Oracle - Verify Oracle source data transfer of type LONG RAW
5151
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
5252
Then Enter input plugin property: "referenceName" with value: "targetRef"
5353
Then Select radio button plugin property: "connectionType" with value: "service"
54-
Then Select radio button plugin property: "role" with value: "sysdba"
54+
Then Select radio button plugin property: "role" with value: "normal"
5555
Then Validate "Oracle2" plugin properties
5656
Then Close the Plugin Properties page
5757
Then Save the pipeline

oracle-plugin/src/e2e-test/features/oracle/DatatypeLongVarchar.feature

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616

1717
@Oracle
1818
Feature: Oracle - Verify Oracle source data transfer of type LONG VARCHAR
19-
@ORACLE_SOURCE_DATATYPES_TEST4 @ORACLE_SINK_TEST @Oracle_Required
19+
@ORACLE_SOURCE_DATATYPES_TEST4 @ORACLE_TARGET_DATATYPES_TEST4 @Oracle_Required
2020
# Oracle Sanity test to transfer table data containing LONG VARCHAR
2121
Scenario: To verify data is getting transferred from Oracle to Oracle successfully
2222
Given Open Datafusion Project to configure pipeline
@@ -32,7 +32,7 @@ Feature: Oracle - Verify Oracle source data transfer of type LONG VARCHAR
3232
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
3333
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
3434
Then Select radio button plugin property: "connectionType" with value: "service"
35-
Then Select radio button plugin property: "role" with value: "sysdba"
35+
Then Select radio button plugin property: "role" with value: "normal"
3636
Then Enter input plugin property: "referenceName" with value: "sourceRef"
3737
Then Replace input plugin property: "database" with value: "databaseName"
3838
Then Enter textarea plugin property: "importQuery" with value: "selectQuery"
@@ -51,7 +51,7 @@ Feature: Oracle - Verify Oracle source data transfer of type LONG VARCHAR
5151
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
5252
Then Enter input plugin property: "referenceName" with value: "targetRef"
5353
Then Select radio button plugin property: "connectionType" with value: "service"
54-
Then Select radio button plugin property: "role" with value: "sysdba"
54+
Then Select radio button plugin property: "role" with value: "normal"
5555
Then Validate "Oracle2" plugin properties
5656
Then Close the Plugin Properties page
5757
Then Save the pipeline
Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
#
2+
# Copyright © 2023 Cask Data, Inc.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License"); you may not
5+
# use this file except in compliance with the License. You may obtain a copy of
6+
# the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
12+
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
13+
# License for the specific language governing permissions and limitations under
14+
# the License.
15+
#
16+
17+
@Oracle
18+
Feature: Oracle - Verify Oracle source data transfer for all Timestamp types
19+
@ORACLE_SOURCE_DATATYPE_TIMESTAMP @ORACLE_TARGET_DATATYPE_TIMESTAMP @Oracle_Required
20+
Scenario: To verify data is getting transferred from Oracle to Oracle successfully
21+
Given Open Datafusion Project to configure pipeline
22+
When Expand Plugin group in the LHS plugins list: "Source"
23+
When Select plugin: "Oracle" from the plugins list as: "Source"
24+
When Expand Plugin group in the LHS plugins list: "Sink"
25+
When Select plugin: "Oracle" from the plugins list as: "Sink"
26+
Then Connect plugins: "Oracle" and "Oracle2" to establish connection
27+
Then Navigate to the properties page of plugin: "Oracle"
28+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
29+
Then Replace input plugin property: "host" with value: "host" for Credentials and Authorization related fields
30+
Then Replace input plugin property: "port" with value: "port" for Credentials and Authorization related fields
31+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
32+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
33+
Then Select radio button plugin property: "connectionType" with value: "service"
34+
Then Select radio button plugin property: "role" with value: "normal"
35+
Then Select dropdown plugin property: "select-transactionIsolationLevel" with option value: "TRANSACTION_READ_COMMITTED"
36+
Then Enter input plugin property: "referenceName" with value: "sourceRef"
37+
Then Replace input plugin property: "database" with value: "databaseName"
38+
Then Enter textarea plugin property: "importQuery" with value: "selectQuery"
39+
Then Click on the Get Schema button
40+
Then Verify the Output Schema matches the Expected Schema: "outputTimestampDatatypesSchema"
41+
Then Validate "Oracle" plugin properties
42+
Then Close the Plugin Properties page
43+
Then Navigate to the properties page of plugin: "Oracle2"
44+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
45+
Then Replace input plugin property: "host" with value: "host" for Credentials and Authorization related fields
46+
Then Replace input plugin property: "port" with value: "port" for Credentials and Authorization related fields
47+
Then Replace input plugin property: "database" with value: "databaseName"
48+
Then Replace input plugin property: "tableName" with value: "targetTable"
49+
Then Replace input plugin property: "dbSchemaName" with value: "schema"
50+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
51+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
52+
Then Enter input plugin property: "referenceName" with value: "targetRef"
53+
Then Select radio button plugin property: "connectionType" with value: "service"
54+
Then Select radio button plugin property: "role" with value: "normal"
55+
Then Select dropdown plugin property: "select-transactionIsolationLevel" with option value: "TRANSACTION_READ_COMMITTED"
56+
Then Validate "Oracle2" plugin properties
57+
Then Close the Plugin Properties page
58+
Then Save the pipeline
59+
Then Preview and run the pipeline
60+
Then Verify the preview of pipeline is "success"
61+
Then Click on preview data for Oracle sink
62+
Then Verify preview output schema matches the outputSchema captured in properties
63+
Then Close the preview data
64+
Then Deploy the pipeline
65+
Then Run the Pipeline in Runtime
66+
Then Wait till pipeline is in running state
67+
Then Open and capture logs
68+
Then Verify the pipeline status is "Succeeded"
69+
Then Validate the values of records transferred to target table is equal to the values from source table

oracle-plugin/src/e2e-test/features/oracle/Oracle.feature

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616

1717
@Oracle
1818
Feature: Oracle - Verify Oracle source data transfer
19-
@ORACLE_SOURCE_TEST @ORACLE_SINK_TEST @Oracle_Required
19+
@ORACLE_SOURCE_TEST @ORACLE_TARGET_TEST @Oracle_Required
2020
Scenario: To verify data is getting transferred from Oracle to Oracle successfully
2121
Given Open Datafusion Project to configure pipeline
2222
When Expand Plugin group in the LHS plugins list: "Source"
@@ -31,7 +31,7 @@ Feature: Oracle - Verify Oracle source data transfer
3131
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
3232
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
3333
Then Select radio button plugin property: "connectionType" with value: "service"
34-
Then Select radio button plugin property: "role" with value: "sysdba"
34+
Then Select radio button plugin property: "role" with value: "normal"
3535
Then Enter input plugin property: "referenceName" with value: "sourceRef"
3636
Then Replace input plugin property: "database" with value: "databaseName"
3737
Then Enter textarea plugin property: "importQuery" with value: "selectQuery"
@@ -50,7 +50,7 @@ Feature: Oracle - Verify Oracle source data transfer
5050
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
5151
Then Enter input plugin property: "referenceName" with value: "targetRef"
5252
Then Select radio button plugin property: "connectionType" with value: "service"
53-
Then Select radio button plugin property: "role" with value: "sysdba"
53+
Then Select radio button plugin property: "role" with value: "normal"
5454
Then Validate "Oracle2" plugin properties
5555
Then Close the Plugin Properties page
5656
Then Save the pipeline
Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
#
2+
# Copyright © 2023 Cask Data, Inc.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License"); you may not
5+
# use this file except in compliance with the License. You may obtain a copy of
6+
# the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
12+
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
13+
# License for the specific language governing permissions and limitations under
14+
# the License.
15+
#
16+
17+
@Oracle @Oracle_Required
18+
Feature: Oracle sink - Verify Oracle sink plugin design time scenarios
19+
20+
Scenario: To verify Oracle sink plugin validation with connection and basic details for connectivity
21+
Given Open Datafusion Project to configure pipeline
22+
When Expand Plugin group in the LHS plugins list: "Sink"
23+
When Select plugin: "Oracle" from the plugins list as: "Sink"
24+
Then Navigate to the properties page of plugin: "Oracle"
25+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
26+
Then Replace input plugin property: "host" with value: "host" for Credentials and Authorization related fields
27+
Then Replace input plugin property: "port" with value: "port" for Credentials and Authorization related fields
28+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
29+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
30+
Then Select radio button plugin property: "role" with value: "normal"
31+
Then Select radio button plugin property: "connectionType" with value: "service"
32+
Then Enter input plugin property: "referenceName" with value: "targetRef"
33+
Then Replace input plugin property: "database" with value: "databaseName"
34+
Then Replace input plugin property: "tableName" with value: "targetTable"
35+
Then Replace input plugin property: "dbSchemaName" with value: "schema"
36+
Then Validate "Oracle" plugin properties
37+
Then Close the Plugin Properties page
38+
39+
Scenario: To verify Oracle sink plugin validation with connection arguments
40+
Given Open Datafusion Project to configure pipeline
41+
When Expand Plugin group in the LHS plugins list: "Sink"
42+
When Select plugin: "Oracle" from the plugins list as: "Sink"
43+
Then Navigate to the properties page of plugin: "Oracle"
44+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
45+
Then Replace input plugin property: "host" with value: "host" for Credentials and Authorization related fields
46+
Then Replace input plugin property: "port" with value: "port" for Credentials and Authorization related fields
47+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
48+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
49+
Then Select radio button plugin property: "role" with value: "normal"
50+
Then Select radio button plugin property: "connectionType" with value: "service"
51+
Then Enter input plugin property: "referenceName" with value: "targetRef"
52+
Then Replace input plugin property: "database" with value: "databaseName"
53+
Then Replace input plugin property: "tableName" with value: "targetTable"
54+
Then Replace input plugin property: "dbSchemaName" with value: "schema"
55+
Then Enter key value pairs for plugin property: "connectionArguments" with values from json: "connectionArgumentsList"
56+
Then Validate "Oracle" plugin properties
57+
Then Close the Plugin Properties page
58+
59+
Scenario: To verify Oracle sink plugin validation with advanced details with batch value
60+
Given Open Datafusion Project to configure pipeline
61+
When Expand Plugin group in the LHS plugins list: "Sink"
62+
When Select plugin: "Oracle" from the plugins list as: "Sink"
63+
Then Navigate to the properties page of plugin: "Oracle"
64+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
65+
Then Replace input plugin property: "host" with value: "host" for Credentials and Authorization related fields
66+
Then Replace input plugin property: "port" with value: "port" for Credentials and Authorization related fields
67+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
68+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
69+
Then Select radio button plugin property: "role" with value: "normal"
70+
Then Select radio button plugin property: "connectionType" with value: "service"
71+
Then Enter input plugin property: "referenceName" with value: "targetRef"
72+
Then Replace input plugin property: "database" with value: "databaseName"
73+
Then Replace input plugin property: "tableName" with value: "targetTable"
74+
Then Replace input plugin property: "defaultBatchValue" with value: "batchValue"
75+
Then Validate "Oracle" plugin properties
76+
Then Close the Plugin Properties page

0 commit comments

Comments
 (0)