Skip to content

[PLUGIN-1779] Add TRANSACTION_ISOLATION_LEVEL config in MySQL, PostgreSQL & SQL Server plugins #583

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 24, 2025

Conversation

psainics
Copy link
Contributor

@psainics psainics commented Apr 17, 2025

Add TRANSACTION_ISOLATION_LEVEL config in MySQL, PostgreSQL & SQL Server plugins

PLUGIN-1779

Added support for specifying the transaction isolation level in database plugins by introducing a new optional property "transactionIsolationLevel" in the AbstractDBSpecificConnectorConfig class — the parent class for MySQL, PostgreSQL, SQL Server and Oracle connector configurations.

image

@psainics psainics added the build label Apr 17, 2025
@sgarg-CS sgarg-CS self-assigned this Apr 17, 2025
@sgarg-CS sgarg-CS requested a review from ritwiksahani April 17, 2025 11:41
@sgarg-CS sgarg-CS changed the title [PLUGIN-1779] Add TRANSACTION_ISOLATION_LEVEL config in PostgreSQL plugins [PLUGIN-1779] Add TRANSACTION_ISOLATION_LEVEL config in MySQL & PostgreSQL plugin Apr 17, 2025
@damjad
Copy link
Contributor

damjad commented Apr 18, 2025

Thank you so much for this. 🥇

We've been facing issues with this, and I was looking for a solution. I am happy that it has already been implemented.

Looking forward to it! 🤞

@@ -49,6 +51,7 @@ public abstract class AbstractDBSpecificSourceConfig extends PluginConfig implem
public static final String DATABASE = "database";
public static final String FETCH_SIZE = "fetchSize";
public static final String DEFAULT_FETCH_SIZE = "1000";
public static final Logger LOG = LoggerFactory.getLogger(AbstractDBSpecificSourceConfig.class);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not being used anywhere why do we need to add this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed !

@ritwiksahani
Copy link

ritwiksahani commented Apr 21, 2025

Add TRANSACTION_ISOLATION_LEVEL config in MySQL & PostgreSQL plugin

PLUGIN-1779

Added support for specifying the transaction isolation level in database plugins by introducing a new optional property "transactionIsolationLevel" in the AbstractDBSpecificConnectorConfig class — the parent class for MySQL, PostgreSQL, and Oracle connector configurations.

image

Please add testing evidence for mySql and postgres.

@Description("The transaction isolation level for the database session.")
@Macro
@Nullable
protected String transactionIsolationLevel;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't this class also used by SQLServer?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is. Should we hide the property from SQLServer Plugins for now? OR
add this property to the respective connector configuration classes of PostgreSQL and MySQL plugins and remove from parent connector config?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not implement it in SQLServer as well?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added here

@psainics psainics force-pushed the patch/PLUGIN-1779 branch from aa2f77c to d30d86c Compare April 21, 2025 10:18
Copy link

@ritwiksahani ritwiksahani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval on the current code but please add SQLServer support as well as discussed in the other thread and I can then approve the PR.

@psainics
Copy link
Contributor Author

Please add the same in postgreSQL-connector documentation as well.

image

@psainics
Copy link
Contributor Author

image
image
image

@sgarg-CS sgarg-CS changed the title [PLUGIN-1779] Add TRANSACTION_ISOLATION_LEVEL config in MySQL & PostgreSQL plugin [PLUGIN-1779] Add TRANSACTION_ISOLATION_LEVEL config in MySQL, PostgreSQL & SQL Server plugins Apr 23, 2025
"TRANSACTION_REPEATABLE_READ",
"TRANSACTION_SERIALIZABLE"
],
"default": "TRANSACTION_SERIALIZABLE"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change this to read commited as the default value as mentioned in the documentation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated 38bd277

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ritwiksahani @itsankit-google Should we keep the default type as Serializable, which is currently used for all plugins, or should we change it to match the default isolation level supported by the database?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is Serializable used for all plugins? I don't think so, correct me if I am wrong but we don't set transaction isolation level for other plugins currently. In that case it would be using the default isolation level of the DB.

We should maintain backward compatibility in terms of behaviour and have the defauilt isolation level match the DB one.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default, we were setting TRANSACTION_SERIALIZABLE for all plugins except Oracle, ref:

Copy link

@ritwiksahani ritwiksahani Apr 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, agreed then in that case we should keep SERIALIZABLE as default and not change the behaviour.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed the default value to TRANSACTION_SERIALIZABLE 0881eb7

@sgarg-CS
Copy link
Contributor

Approval on the current code but please add SQLServer support as well as discussed in the other thread and I can then approve the PR.

Added support for SQL Server as well.

@sgarg-CS
Copy link
Contributor

sgarg-CS commented Apr 23, 2025

Add TRANSACTION_ISOLATION_LEVEL config in MySQL & PostgreSQL plugin

PLUGIN-1779
Added support for specifying the transaction isolation level in database plugins by introducing a new optional property "transactionIsolationLevel" in the AbstractDBSpecificConnectorConfig class — the parent class for MySQL, PostgreSQL, and Oracle connector configurations.
image

Please add testing evidence for mySql and postgres.

Testing evidence

MySQL
image

PostgreSQL
image

Oracle
image

MS SQL Server
image

@sgarg-CS sgarg-CS requested a review from ritwiksahani April 24, 2025 07:28
…lugins

Add transaction isolation level to PostgreSQL, MYSQL and MSSQL Plugins.
@sgarg-CS sgarg-CS force-pushed the patch/PLUGIN-1779 branch from 0881eb7 to f4650e1 Compare April 24, 2025 07:49
@sgarg-CS sgarg-CS merged commit 7340037 into data-integrations:develop Apr 24, 2025
10 checks passed
@psainics psainics deleted the patch/PLUGIN-1779 branch April 24, 2025 15:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants