Specify Gradle Input Registration Compatibility

[mgroth0]

I can not find any direct mention of the gradle configuration cache in this project in terms of whether or not it is guaranteed that file system reads from kotlinx-io will always be registered by Gradle.

Gradle instruments code to register file system reads as inputs. As of right now, kotlinx-io uses java.io.FileOutputStream and there is no issue. But theoretically, kotlinx-io's jvm module could one day use JNI. If it did, filesystem reads might not be registered by Gradle.

I raise this concern only because I am considering a general migration to kotlinx-io for file system operations in my gradle plugin, and want to make sure this is future proof.

Some ideas:

• Document somewhere this compatibility (to serve as a reminder and assurance)
• Create a test

Or maybe the idea that kotlinx-io would ever do filesystem operations outside of java.io or java.nio is silly? I don't really know to be honest. If so, then maybe this whole concern of mine is also silly in which case maybe this could just be closed.

Or if this isn't silly and a test would really be worthwhile, I imagine maybe something like this:

• Use TestKit
• Build logic should read from a text file
• Run build three times during the test, writing to the file in between the second and third
• Test asserts that the build was configured the 1st and 3rd times but loaded from cache the second time

I'm open to trying to contributing such a test if it seems worthwhile. I never tried TestKit before so I'd be curious if it would work.

[fzhinkin]

@mgroth0 thanks for raising the concern!

I am considering a general migration to kotlinx-io for file system operations in my gradle plugin

The current filesystem API exists only to fulfill a few practical scenarios, it won't be improved in the future, and instead it will be replaced with a new, most likely incompatible API.

Or maybe the idea that kotlinx-io would ever do filesystem operations outside of java.io or java.nio is silly?

It is definitely not silly, and there are some good reasons to implement the filesystem API layer using JNI or FFI, but the chances this road will be taken are pretty low at the moment.

This issue should be a good enough reminder of the potential problem if we'll try to implement FS-API using something other than java.io or java.nio on JVM in the future.

Speaking of a dedicated test, could you please elaborate what should be done on the kotlinx-io side once it fail? We don't have control over Gradle internals, so it looks like something that needs to be tracked and fixed on the Gradle side.

[mgroth0]

Thank you @fzhinkin !

The current filesystem API exists only to fulfill a few practical scenarios, it won't be improved in the future, and instead it will be replaced with a new, most likely incompatible API.

I'm curious when you say incompatible, if this is something that would really concern me or not. If a future update will just require a migration to new classes or functions, I'm not concerned. Could you possibily elaborate or point me to more information?

Speaking of a dedicated test, could you please elaborate what should be done on the kotlinx-io side once it fail? We don't have control over Gradle internals, so it looks like something that needs to be tracked and fixed on the Gradle side.

I think it might need to be a collaborative effort. If the day comes when kotlinx-io decides it wants to use native filesystem operations, I think that kotlinx-io devs should create an issue on Gradle's issue tracker requesting a solution. Then, I think that Gradle should collaborate with you. Maybe Gradle would request that you create a class such as kotlinx.io.GradleInputTrackingLayer, and call a specific method on this class like GradleInputTrackingLayer.registerRead(path: String). immediately before all filesystem operations. I think this method can probably be a no-op on the kotlinx-side with a totally empty body but with the strict specification that it is always called internally by kotlinx-io when a user reads from the filesystem (and some way to ensure it doesn't get optimized out by some compiler). Then, on Gradle's side, maybe they have a way they can easily expand the instrumentation they already have for java.io and java.nio methods to your GradleInputTrackingLayer class. Maybe Gradle could check if this class is on the classpath, and if so instrument it.

Technically, there are other paths to solve this issue as well. But this is just one idea off the top of my head, but I feel like it might check all of the boxes? No performance overhead, has zero effect when kotlinx-io is used outside of a gradle builld, and would be robust in that it would work the same way as it did for java.io. What do you think?

This issue should be a good enough reminder of the potential problem

Thanks, that's very reassuring. Most of all I just wanted to get this concern tracked because I know I'll probably forget and get affected by it years down the road. So knowing it is here is nice and we can shelf it for now :)

[fzhinkin]

I'm curious when you say incompatible, if this is something that would really concern me or not. If a future update will just require a migration to new classes or functions, I'm not concerned. Could you possibily elaborate or point me to more information?

Current design assumes that paths are "abstract", do not contain any filesystem-specific information and could be freely passed between different FileSystem implementations (although, there's only one such implementation). Such approach has some limitations (#292 (comment)) and we may choose something similar to java.nio.file.Path where a path is bound to a particular filesystem. So it might require migration to new classes/functions.

[mgroth0]

I really like that idea. I try to do this already, and have a set of OS-specific file path classes. Helps to handle things like case sensitivity better. Example: on a case insensitive filesystem like the default MacOS one, I like to make a path of Path("/myfile") == Path("/MyFile") , but in the context of a case sensitive file system (the default for linux) Path("/myfile") != ("/MyFile") . So I end up with MacPath and LinuxPath (though in reality it gets more complex since its technically possible for case sensitivity to go either way on either OS). Besides case sensitivity there's of course validations with file path seperators and also special file systems like zip which I think you mentioned. If you'd like me to add my input to the conversation you linked let me know. but I trust you probably have a good plan already and I mostly would be repeating what I said here.