Spark 3.5 support

[nikolay-voskresenskiy-db]

Thank you for submitting a pull request!

Please make sure you have signed our Contributor License Agreement (CLA).
We are not asking you to assign copyright to us, but to give us the right to distribute your code without restriction. We ask this of all contributors in order to assure our users of the origin and continuing existence of the code.
You only need to sign the CLA once.

• I have signed the Contributor License Agreement (CLA)

[cla-checker-service]

❌ Author of the following commits did not sign a Contributor Agreement:
fc4f33b, f667aef

Please, read and sign the above mentioned agreement if you want to contribute to this project

[nikolay-voskresenskiy-db]

A few notes on the implementation:

1. The actual change that makes the runtime code compatible with spark 3.5.x is a single line in EsStreamQueryWriter.scala. There is no functional change, only a signature change inside internal spark API, The rest of the changes are related to building and testing:
2. We made the artifact for 3.5 into a separate build variant to prove out that the same code is compatible with both 3.4 and 3.5 via the test suite. This has a negative impact of prolonging build time, so the variant for 3.4 could be removed in future and 3.5 left as the primary variant
3. StreamingQueryLifecycleListener helper class in the tests now needs two versions because spark 3.5 splits out the stream idleness notification logic into a separate method
4. 3.5 has a slight change in dependencies since spark gets a separate spark-sql-api module
5. Had to make a few fixes to the build scaffolding to correctly handle 4 build variants instead of 2