-
Notifications
You must be signed in to change notification settings - Fork 1.6k
chore(deps): Update sqlparser to 0.56 #16456
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Ok(Expr::ScalarFunction(ScalarFunction::new_udf( | ||
fun, | ||
substring_args, | ||
))) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Without this change the test fails:
#[test]
fn test_order_by_to_sql_3() {
let statement = generate_round_trip_statement(
GenericDialect {},
r#"SELECT id, first_name, substr(first_name,0,5) FROM person ORDER BY id, substr(first_name,0,5)"#,
);
assert_snapshot!(
statement,
@r#"SELECT person.id, person.first_name, substr(person.first_name, 0, 5) FROM person ORDER BY person.id ASC NULLS LAST, substr(person.first_name, 0, 5) ASC NULLS LAST"#
);
}
In the previous version substr were hitting:
SQLExpr::Function(function) => {
self.sql_function_to_expr(function, schema, planner_context)
}
now it's hitting:
SQLExpr::Substring {
expr,
substring_from,
substring_for,
special: _,
shorthand: _,
} => self.sql_substring_to_expr(
expr,
substring_from,
substring_for,
schema,
planner_context,
),
Open for suggestions if there is a better way to handle it.
@@ -739,6 +739,12 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> { | |||
| SQLDataType::AnyType | |||
| SQLDataType::Table(_) | |||
| SQLDataType::VarBit(_) | |||
| SQLDataType::UTinyInt | |||
| SQLDataType::USmallInt |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure if the new data types should be handled here or Unsupported SQL type is fine.
(converted to draft while I'm investigating CI failures) |
(one of the test failures is related to #1898 and would be fixed if that patch is backported to 0.56) |
Cargo.toml
Outdated
@@ -167,7 +167,7 @@ recursive = "0.1.1" | |||
regex = "1.8" | |||
rstest = "0.25.0" | |||
serde_json = "1" | |||
sqlparser = { version = "0.56.0", default-features = false, features = ["std", "visitor"] } | |||
sqlparser = { git = "https://github.com/Dimchikkk/datafusion-sqlparser-rs.git", branch = "v0.56.1", default-features = false, features = ["std", "visitor"] } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FIXME
@@ -167,7 +167,10 @@ recursive = "0.1.1" | |||
regex = "1.8" | |||
rstest = "0.25.0" | |||
serde_json = "1" | |||
sqlparser = { version = "0.55.0", default-features = false, features = ["std", "visitor"] } | |||
sqlparser = { git = "https://github.com/Dimchikkk/datafusion-sqlparser-rs.git", branch = "v0.56.1", default-features = false, features = [ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As an apache project, we should probably not depend on a personal repository where an individual controls the branches without any additional checks -- not because I personally distrust you but because the OSS world has seen its fair share of supply chain attacks and we should use best practice. I see two options forward:
- use a branch within the apache org repo instead of your personal one
- actually publish a 0.56.1 backport release to crates.io
I would prefer the 2nd option. CC @alamb
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think publishing a 0.56.1 release of sqlparser makes sense to me. Perhaps we can file a ticket in the sqlparser repo to track.
I am happy to run the process
Another alternative is to release the next release (0.57.0) and just skip 0.56.0 entirely in Datafusion
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think not jumping multiple releases would be nice, so I've filed apache/datafusion-sqlparser-rs#1952
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@crepererum thanks for creating the issue for releasing 0.56.1!
I changed the link to my personal repo just to test that all CI checks pass, surely I change it back to official 0.56.1 once it is released.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will work on the release tomorrow -- hopefully we'll have it released by early next week
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it is impossible to backport the fix in a semver compatible way. See apache/datafusion-sqlparser-rs#1952
I will instead accelerate the timeline for releasing 0.58.0
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@alamb yeah, it's definitely not a semver compatible.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just for the record: I'm OK with version jumping (i.e. directly upgrading to 0.58). I was just hoping that the intermediate step would be easier for @Dimchikkk or whoever is gonna pick this up.
Which issue does this PR close?
Rationale for this change
What changes are included in this PR?
Are these changes tested?
Are there any user-facing changes?