[PLUGIN-1779] Add TRANSACTION_ISOLATION_LEVEL config in MySQL, PostgreSQL & SQL Server plugins #583

psainics · 2025-04-17T11:02:07Z

Add TRANSACTION_ISOLATION_LEVEL config in MySQL, PostgreSQL & SQL Server plugins

Added support for specifying the transaction isolation level in database plugins by introducing a new optional property "transactionIsolationLevel" in the AbstractDBSpecificConnectorConfig class — the parent class for MySQL, PostgreSQL, SQL Server and Oracle connector configurations.

damjad · 2025-04-18T17:06:04Z

Thank you so much for this. 🥇

We've been facing issues with this, and I was looking for a solution. I am happy that it has already been implemented.

Looking forward to it! 🤞

database-commons/src/main/java/io/cdap/plugin/db/connector/AbstractDBSpecificConnector.java

database-commons/src/main/java/io/cdap/plugin/db/connector/AbstractDBConnectorConfig.java

ritwiksahani · 2025-04-21T08:24:49Z

database-commons/src/main/java/io/cdap/plugin/db/config/AbstractDBSpecificSourceConfig.java

  public static final String DATABASE = "database";
  public static final String FETCH_SIZE = "fetchSize";
  public static final String DEFAULT_FETCH_SIZE = "1000";
+  public static final Logger LOG = LoggerFactory.getLogger(AbstractDBSpecificSourceConfig.class);


This is not being used anywhere why do we need to add this?

...ase-commons/src/main/java/io/cdap/plugin/db/connector/AbstractDBSpecificConnectorConfig.java

mysql-plugin/src/main/java/io/cdap/plugin/mysql/MysqlSource.java

postgresql-plugin/src/main/java/io/cdap/plugin/postgres/PostgresSource.java

postgresql-plugin/src/main/java/io/cdap/plugin/postgres/PostgresConnectorConfig.java

postgresql-plugin/docs/Postgres-batchsink.md

ritwiksahani · 2025-04-21T08:42:59Z

Add TRANSACTION_ISOLATION_LEVEL config in MySQL & PostgreSQL plugin

PLUGIN-1779

Added support for specifying the transaction isolation level in database plugins by introducing a new optional property "transactionIsolationLevel" in the AbstractDBSpecificConnectorConfig class — the parent class for MySQL, PostgreSQL, and Oracle connector configurations.

Please add testing evidence for mySql and postgres.

itsankit-google · 2025-04-21T08:46:50Z

...ase-commons/src/main/java/io/cdap/plugin/db/connector/AbstractDBSpecificConnectorConfig.java

+  @Description("The transaction isolation level for the database session.")
+  @Macro
+  @Nullable
+  protected String transactionIsolationLevel;


isn't this class also used by SQLServer?

Yes, it is. Should we hide the property from SQLServer Plugins for now? OR
add this property to the respective connector configuration classes of PostgreSQL and MySQL plugins and remove from parent connector config?

why not implement it in SQLServer as well?

ritwiksahani

Approval on the current code but please add SQLServer support as well as discussed in the other thread and I can then approve the PR.

postgresql-plugin/docs/Postgres-batchsink.md

psainics · 2025-04-22T09:38:18Z

Please add the same in postgreSQL-connector documentation as well.

psainics · 2025-04-22T09:39:11Z

mssql-plugin/widgets/SQL Server-connector.json

mssql-plugin/widgets/SqlServer-batchsink.json

mssql-plugin/widgets/SqlServer-batchsource.json

mssql-plugin/widgets/SQL Server-connector.json

mssql-plugin/widgets/SqlServer-batchsink.json

ritwiksahani · 2025-04-23T10:06:25Z

mssql-plugin/widgets/SqlServer-batchsource.json

+              "TRANSACTION_REPEATABLE_READ",
+              "TRANSACTION_SERIALIZABLE"
+            ],
+            "default": "TRANSACTION_SERIALIZABLE"


Change this to read commited as the default value as mentioned in the documentation.

Updated 38bd277

@ritwiksahani @itsankit-google Should we keep the default type as Serializable, which is currently used for all plugins, or should we change it to match the default isolation level supported by the database?

Is Serializable used for all plugins? I don't think so, correct me if I am wrong but we don't set transaction isolation level for other plugins currently. In that case it would be using the default isolation level of the DB.

We should maintain backward compatibility in terms of behaviour and have the defauilt isolation level match the DB one.

By default, we were setting TRANSACTION_SERIALIZABLE for all plugins except Oracle, ref:

database-plugins/database-commons/src/main/java/io/cdap/plugin/db/TransactionIsolationLevel.java

Line 66 in cde7167

return Connection.TRANSACTION_SERIALIZABLE;

Got it, agreed then in that case we should keep SERIALIZABLE as default and not change the behaviour.

Changed the default value to TRANSACTION_SERIALIZABLE 0881eb7

sgarg-CS · 2025-04-23T10:33:05Z

Approval on the current code but please add SQLServer support as well as discussed in the other thread and I can then approve the PR.

Added support for SQL Server as well.

sgarg-CS · 2025-04-23T10:36:58Z

Add TRANSACTION_ISOLATION_LEVEL config in MySQL & PostgreSQL plugin

PLUGIN-1779
Added support for specifying the transaction isolation level in database plugins by introducing a new optional property "transactionIsolationLevel" in the AbstractDBSpecificConnectorConfig class — the parent class for MySQL, PostgreSQL, and Oracle connector configurations.

Please add testing evidence for mySql and postgres.

Testing evidence

MySQL

PostgreSQL

Oracle

MS SQL Server

…lugins Add transaction isolation level to PostgreSQL, MYSQL and MSSQL Plugins.

psainics added the build label Apr 17, 2025

sgarg-CS self-assigned this Apr 17, 2025

sgarg-CS requested a review from ritwiksahani April 17, 2025 11:41

sgarg-CS changed the title ~~[PLUGIN-1779] Add TRANSACTION_ISOLATION_LEVEL config in PostgreSQL plugins~~ [PLUGIN-1779] Add TRANSACTION_ISOLATION_LEVEL config in MySQL & PostgreSQL plugin Apr 17, 2025

ritwiksahani requested changes Apr 21, 2025

View reviewed changes

ritwiksahani requested a review from itsankit-google April 21, 2025 08:43

itsankit-google reviewed Apr 21, 2025

View reviewed changes

psainics force-pushed the patch/PLUGIN-1779 branch from aa2f77c to d30d86c Compare April 21, 2025 10:18

ritwiksahani reviewed Apr 22, 2025

View reviewed changes

postgresql-plugin/docs/Postgres-batchsink.md Show resolved Hide resolved

ritwiksahani requested changes Apr 22, 2025

View reviewed changes

mssql-plugin/widgets/SQL Server-connector.json Show resolved Hide resolved

mssql-plugin/widgets/SqlServer-batchsink.json Show resolved Hide resolved

mssql-plugin/widgets/SqlServer-batchsource.json Show resolved Hide resolved

sgarg-CS changed the title ~~[PLUGIN-1779] Add TRANSACTION_ISOLATION_LEVEL config in MySQL & PostgreSQL plugin~~ [PLUGIN-1779] Add TRANSACTION_ISOLATION_LEVEL config in MySQL, PostgreSQL & SQL Server plugins Apr 23, 2025

ritwiksahani reviewed Apr 23, 2025

View reviewed changes

sgarg-CS requested a review from ritwiksahani April 24, 2025 07:28

ritwiksahani approved these changes Apr 24, 2025

View reviewed changes

Set TRANSACTION_ISOLATION_LEVEL config in MySQL, PostgreSQL & MSSQL p…

f4650e1

…lugins Add transaction isolation level to PostgreSQL, MYSQL and MSSQL Plugins.

sgarg-CS force-pushed the patch/PLUGIN-1779 branch from 0881eb7 to f4650e1 Compare April 24, 2025 07:49

sgarg-CS merged commit 7340037 into data-integrations:develop Apr 24, 2025
10 checks passed

psainics deleted the patch/PLUGIN-1779 branch April 24, 2025 15:16

This was referenced May 5, 2025

[🍒][Plugin-1779] Add TRANSACTION_ISOLATION_LEVEL config in MySQL, PostgreSQL & SQL Server plugins #589

Merged

[🍒][Plugin-1779] Add TRANSACTION_ISOLATION_LEVEL config in MySQL, PostgreSQL & SQL Server plugins #594

Merged

[PLUGIN-1779] Add TRANSACTION_ISOLATION_LEVEL config in MySQL, PostgreSQL & SQL Server plugins #583

[PLUGIN-1779] Add TRANSACTION_ISOLATION_LEVEL config in MySQL, PostgreSQL & SQL Server plugins #583

Uh oh!

Conversation

psainics commented Apr 17, 2025 • edited by sgarg-CS Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Add TRANSACTION_ISOLATION_LEVEL config in MySQL, PostgreSQL & SQL Server plugins

Uh oh!

damjad commented Apr 18, 2025

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ritwiksahani commented Apr 21, 2025 • edited by atlassian bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Add TRANSACTION_ISOLATION_LEVEL config in MySQL & PostgreSQL plugin

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ritwiksahani left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

psainics commented Apr 22, 2025

Uh oh!

psainics commented Apr 22, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ritwiksahani Apr 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sgarg-CS commented Apr 23, 2025

Uh oh!

sgarg-CS commented Apr 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Add TRANSACTION_ISOLATION_LEVEL config in MySQL & PostgreSQL plugin

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

psainics commented Apr 17, 2025 •

edited by sgarg-CS

Loading

ritwiksahani commented Apr 21, 2025 •

edited by atlassian bot

Loading

ritwiksahani Apr 24, 2025 •

edited

Loading

sgarg-CS commented Apr 23, 2025 •

edited

Loading