-
Notifications
You must be signed in to change notification settings - Fork 4.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SnowflakeIO: do not quote empty fields #33803
Conversation
1b0d71a
to
4269912
Compare
Assigning reviewers. If you would like to opt out of this review, comment R: @Abacn for label java. Available commands:
The PR bot will only process comments in the main thread (not review comments). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the fix. It sounds a valid edge case.
Ideally is there a way to unit test the change? e.g. to confirm
before: "1","","3","",
after "1",,"3",,
(not "1",,"3")
it does not introduce side effect like dropping a field, making the column mismatch
@@ -1192,7 +1192,13 @@ private String quoteField(String field) { | |||
} | |||
|
|||
private String quoteField(String field, String quotation) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
consider rename this private method (and its overload above) to sth like "quoteNonEmptyField" for clarity
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixup 016e641
Added in fixup 54ec742 |
When sending CSV to Snowflake for
COPY
, do not quote empty fields (''
).With some types, like integer, it leads to an error like this:
net.snowflake.client.jdbc.SnowflakeSQLException: Numeric value '' is not recognized
.GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.