Skip to content

chore(deps): Update sqlparser to 0.56 #16456

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Conversation

Dimchikkk
Copy link
Contributor

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions bot added the sql SQL Planner label Jun 19, 2025
Ok(Expr::ScalarFunction(ScalarFunction::new_udf(
fun,
substring_args,
)))
Copy link
Contributor Author

@Dimchikkk Dimchikkk Jun 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without this change the test fails:

#[test]
fn test_order_by_to_sql_3() {
    let statement = generate_round_trip_statement(
        GenericDialect {},
        r#"SELECT id, first_name, substr(first_name,0,5) FROM person ORDER BY id, substr(first_name,0,5)"#,
    );
    assert_snapshot!(
        statement,
        @r#"SELECT person.id, person.first_name, substr(person.first_name, 0, 5) FROM person ORDER BY person.id ASC NULLS LAST, substr(person.first_name, 0, 5) ASC NULLS LAST"#
    );
}

In the previous version substr were hitting:

            SQLExpr::Function(function) => {
                self.sql_function_to_expr(function, schema, planner_context)
            }

now it's hitting:

            SQLExpr::Substring {
                expr,
                substring_from,
                substring_for,
                special: _,
                shorthand: _,
            } => self.sql_substring_to_expr(
                expr,
                substring_from,
                substring_for,
                schema,
                planner_context,
            ),

Open for suggestions if there is a better way to handle it.

@@ -739,6 +739,12 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> {
| SQLDataType::AnyType
| SQLDataType::Table(_)
| SQLDataType::VarBit(_)
| SQLDataType::UTinyInt
| SQLDataType::USmallInt
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if the new data types should be handled here or Unsupported SQL type is fine.

@Dimchikkk Dimchikkk marked this pull request as draft June 19, 2025 12:58
@Dimchikkk
Copy link
Contributor Author

(converted to draft while I'm investigating CI failures)

@github-actions github-actions bot added the sqllogictest SQL Logic Tests (.slt) label Jun 20, 2025
@Dimchikkk
Copy link
Contributor Author

(one of the test failures is related to #1898 and would be fixed if that patch is backported to 0.56)

@Dimchikkk Dimchikkk marked this pull request as ready for review June 28, 2025 18:52
@Dimchikkk Dimchikkk changed the title chore(deps): Update sqlparser to 0.56.0 chore(deps): Update sqlparser to 0.56 Jun 28, 2025
Cargo.toml Outdated
@@ -167,7 +167,7 @@ recursive = "0.1.1"
regex = "1.8"
rstest = "0.25.0"
serde_json = "1"
sqlparser = { version = "0.56.0", default-features = false, features = ["std", "visitor"] }
sqlparser = { git = "https://github.com/Dimchikkk/datafusion-sqlparser-rs.git", branch = "v0.56.1", default-features = false, features = ["std", "visitor"] }
Copy link
Contributor Author

@Dimchikkk Dimchikkk Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FIXME

@@ -167,7 +167,10 @@ recursive = "0.1.1"
regex = "1.8"
rstest = "0.25.0"
serde_json = "1"
sqlparser = { version = "0.55.0", default-features = false, features = ["std", "visitor"] }
sqlparser = { git = "https://github.com/Dimchikkk/datafusion-sqlparser-rs.git", branch = "v0.56.1", default-features = false, features = [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As an apache project, we should probably not depend on a personal repository where an individual controls the branches without any additional checks -- not because I personally distrust you but because the OSS world has seen its fair share of supply chain attacks and we should use best practice. I see two options forward:

  • use a branch within the apache org repo instead of your personal one
  • actually publish a 0.56.1 backport release to crates.io

I would prefer the 2nd option. CC @alamb

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think publishing a 0.56.1 release of sqlparser makes sense to me. Perhaps we can file a ticket in the sqlparser repo to track.

I am happy to run the process

Another alternative is to release the next release (0.57.0) and just skip 0.56.0 entirely in Datafusion

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think not jumping multiple releases would be nice, so I've filed apache/datafusion-sqlparser-rs#1952

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@crepererum thanks for creating the issue for releasing 0.56.1!

I changed the link to my personal repo just to test that all CI checks pass, surely I change it back to official 0.56.1 once it is released.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will work on the release tomorrow -- hopefully we'll have it released by early next week

Copy link
Contributor

@alamb alamb Jul 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is impossible to backport the fix in a semver compatible way. See apache/datafusion-sqlparser-rs#1952

I will instead accelerate the timeline for releasing 0.58.0

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alamb yeah, it's definitely not a semver compatible.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for the record: I'm OK with version jumping (i.e. directly upgrading to 0.58). I was just hoping that the intermediate step would be easier for @Dimchikkk or whoever is gonna pick this up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sql SQL Planner sqllogictest SQL Logic Tests (.slt)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Upgrade to sqlparser 0.56.0
3 participants