Skip to content

Conversation

@mbwhite
Copy link
Contributor

@mbwhite mbwhite commented Dec 1, 2025

Starting from two examples of using Isthmus to go from and to SQL a few new methods make the process a lot simpler and symetrical.

We found that in use we needed helper wrapper clases, but these could be pushed into the main code.

Note to reviewers

I'll leave as draft until I'm sure the build is good and tests are fully working.

I did create a separate examples project; however it does feel a bit unwieldy to have this and the spark one. Maybe some rework to make a general examples project with sub directories per example 'category'... Happy to make that change if you wish.

@mbwhite mbwhite force-pushed the isthmus-api-updates branch 3 times, most recently from d5fc9df to eb50abf Compare December 1, 2025 10:58
@nielspardon
Copy link
Member

I did create a separate examples project; however it does feel a bit unwieldy to have this and the spark one. Maybe some rework to make a general examples project with sub directories per example 'category'... Happy to make that change if you wish.

I think it's fine having multiple example subprojects that each show how to use each of the 3 modules provided by substrait-java: Core, Isthmus, Spark. It would also show which dependency declarations / Gradle setup one needs to interact with each of the modules. If all of them were in a single examples project then it might be difficult to parse the Gradle setup and understand which dependencies / configuration is needed by which module.

@mbwhite mbwhite force-pushed the isthmus-api-updates branch from eb50abf to c0685ac Compare December 1, 2025 11:22
@mbwhite mbwhite marked this pull request as ready for review December 1, 2025 11:56
@mbwhite mbwhite force-pushed the isthmus-api-updates branch 3 times, most recently from 343277e to c7eb95d Compare December 1, 2025 12:20
@mbwhite mbwhite force-pushed the isthmus-api-updates branch 2 times, most recently from 79933c8 to 7baf2ca Compare December 2, 2025 14:24
Copy link
Member

@benbellick benbellick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There were some parts I didn't quite get around to reading, but just dumping some comments I have for now. Thank you for doing this! I think it will be a great help :)

Most of my comments are about code comments / the readme. Feel free to ignore whatever if you think my suggestions are less clear. Thanks!


## How does this work in theory?

In both cases, the Calcite library is used to do parsing and generation of the SQL String. Calcite has it's own relational object model, so there are classes within Isthmus to convert Substrait to and from Calcites object model.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Edited a bit for clarity

Suggested change
In both cases, the Calcite library is used to do parsing and generation of the SQL String. Calcite has it's own relational object model, so there are classes within Isthmus to convert Substrait to and from Calcites object model.
The [Calcite](https://calcite.apache.org/) library is used to do parsing and generation of the SQL String. Calcite has it's own relational object model, distinct from substrait's. There are classes within Isthmus to convert Substrait to and from Calcite's object model.

Starting from two examples of using Isthmus to go from and to
SQL a few new methods make the process a lot simpler and symetrical.

We found that in use we needed helper wrapper clases, but these could
be pushed into the main code.

Signed-off-by: MBWhite <[email protected]>
Signed-off-by: MBWhite <[email protected]>
@mbwhite mbwhite force-pushed the isthmus-api-updates branch from db1e1c9 to d768adb Compare December 5, 2025 10:26
@mbwhite
Copy link
Contributor Author

mbwhite commented Dec 5, 2025

Thanks @benbellick - changes made. I've used the batch accepted feature - so hopefully I've not missed any comment - as I agreed with the updates.

@mbwhite mbwhite requested a review from benbellick December 5, 2025 10:27
@mbwhite mbwhite force-pushed the isthmus-api-updates branch from d768adb to 7c1bb7f Compare December 5, 2025 10:28
@mbwhite mbwhite force-pushed the isthmus-api-updates branch 2 times, most recently from abadf67 to c8e010d Compare December 5, 2025 11:36
Signed-off-by: MBWhite <[email protected]>
@mbwhite mbwhite force-pushed the isthmus-api-updates branch from c8e010d to c9abb2b Compare December 5, 2025 11:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants