Support .ELN-style crates in all zip readers and writers

[Pfeil]

Closes #251

See https://github.com/TheELNConsortium/TheELNFileFormat

From the view of RO-Crate Java these crates are zip crates with the special (incompatible) property that the crates are in a subfolder within the file.

This should be simple to add, but due to some weird Zip files, refactorings and deprecations this is a slightly larger task which adds also surprisigly larger amounts of code for testing and because I can't delete the deprecated code immediately.

Support reading

• Add a unit test if the readers already work with them, but as they assume a subfolder, this might be not the case.
• Zip: Consider simply making "get the first subfolder in the zip file" a fallback which then supports ELN files implicitly. Make sure it works with the test from above.
• ZipStream: Do the same for the reader.ZipStreamStrategy
• Ensure proper error propagation and fix issues we may detect doing so

Reading is implemented in the strategies directly, as the only difference is that they need to search in subfolders. The FolderStrategy does not implement ELN support as it does not read zip files.

Support writing

• Make also a test for writing
• Implement ELN support for zipStrategy
• Implement ELN support for zipStreamStrategy

Writers need to implement an additional interface which has a method (two, actually, for semantics) to turn a writer into ELN mode. The save method is not directly part of the strategy but of the CrateWriter. As not all writers will have this functionality, it stays a configuration in the strategy.

Other

• As I make less use of some deprected functions in the CratePreview implementations, the covarage sunk. I should make some sanity test for the time being and mark this test itself as deprecated or so.
• writer.ZipStrategy and writer.ZipStreamStrategy have some very similar code. Consider making a utility class or base class. Postponed. Can be done later without API change.
• Rename test classes consistently (writer vs strategy)
• Consider renaming the new strategy classes to contain "write"/"read", as java has no way of renaming imported classes.
• Make CratePreview generic? May be interesting in future, but for now I do not see a big advantage.

Documentation

• Document everything well in the implementations and in the tests. See/consider also Missing points in documentation #8
• Synchronize documentation (ensure consistency, especially regarding Zip and ZipStream reader / writers)

Notes to put into the documentation:

• Reading checks the first 50 non-recursive folders. I chose not to recurse and not to check all directories in order to avoid zip bombs and similar issues. Our goal is to support ELN file format and possibly a few other similar derivations.

Deprecations

• I'd like to move away from all the responsibilities the CratePreviews have regarding different formats. Instead, I propose an API which takes a crate and expects the result to be written somewhere. This is currently a file, but could be generic... will think about it. Anyway, the old APIs are currently marked as deprecated.

Breaking Changes

• The Readers and writers now expose IOExceptions which must be handled.
  I want this to have into the 2.x branch and therefore not want to break anything, but it turned out that the current implementation hid and logged errors to console when writing, similar to the rest of ro-crate-java. But as not every zip file is streaming-processable without a lot of memory, we need to provide a way to say "sorry, this is not going to work out". Also, reading and writing are important, high level tasks which should be handled by a developer using it.

  Therefore, I consider this not a breaking change, but a fix. In practice, it likely breaks for users, but it will be easy to handle.

• The methods to write an entity are now private methods in writer strategies. This is where they logically belong to. I do not consider it a breaking change because them being public was not intended anyway and just a matter of them being in the wrong place. So I consider it a fix regarding our approach to avoid invalid usage.

Amount of main code

Before

> tokei src/main
===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 FreeMarker              1          201          174            0           27
 Java                   65         6436         4070         1533          833
 JSON                    4         2736         2735            0            1
===============================================================================
 Total                  70         9373         6979         1533          861
===============================================================================

After / Current

> tokei src/main
===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 FreeMarker              1          201          174            0           27
 Java                   67         6689         4153         1682          854
 JSON                    4         2736         2735            0            1
===============================================================================
 Total                  72         9626         7062         1682          882
===============================================================================

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>, please review it.
  • Explain this complex logic.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
  • @coderabbitai explain this code block.
  • @coderabbitai modularize this function.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
  • @coderabbitai read src/utils.ts and explain its main purpose.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
  • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai full review to do a full review from scratch and review all the files again.
• @coderabbitai summary to regenerate the summary of the PR.
• @coderabbitai generate docstrings to generate docstrings for this PR.
• @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
• @coderabbitai help to get help.

Other keywords and placeholders

• Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
• Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
• Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

• You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
• Please see the configuration documentation for more information.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[coveralls]

Pull Request Test Coverage Report for Build #437

Details

• 235 of 253 (92.89%) changed or added relevant lines in 25 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.6%) to 90.791%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
src/main/java/edu/kit/datamanager/ro_crate/entities/AbstractEntity.java	4	5	80.0%
src/main/java/edu/kit/datamanager/ro_crate/preview/CratePreview.java	14	15	93.33%
src/main/java/edu/kit/datamanager/ro_crate/reader/FolderReader.java	0	1	0.0%
src/main/java/edu/kit/datamanager/ro_crate/util/ZipStreamUtil.java	0	1	0.0%
src/main/java/edu/kit/datamanager/ro_crate/writer/FolderWriter.java	0	1	0.0%
src/main/java/edu/kit/datamanager/ro_crate/writer/ZipWriter.java	0	1	0.0%
src/main/java/edu/kit/datamanager/ro_crate/reader/ReadZipStreamStrategy.java	49	51	96.08%
src/main/java/edu/kit/datamanager/ro_crate/reader/Readers.java	3	5	60.0%
src/main/java/edu/kit/datamanager/ro_crate/writer/WriteZipStreamStrategy.java	61	69	88.41%

Totals
Change from base Build #419:	0.6%
Covered Lines:	1962
Relevant Lines:	2161

---

💛 - Coveralls

[Pfeil]

Notes

• The current zip library has an error with the kadi4mat zip files. The issue is unclear at this point, but may be the same of the next point.
• Using the Java Zip API as a replacement has an issue with the same crates. The issue seems to be a zip format that is not possible to use in a streamed way.
  • https://www.baeldung.com/java-compress-and-uncompress#unzip
  • Bug: https://bugs.java.com/bugdatabase/view_bug?bug_id=8327690

Conclusions

• Revert to using the library we used before, for now.
• Make readers failible and document this. streamReaders may file due to zip files using a unstreamable format, and we should just accept this without weird and unexpected fallbacks. The user then may choose to download the file first before processing it.
• Tests will need some kind of blacklist hook where an implementation of the test can explicitly state that a certail file should not work. This way we will also detect if this changes (e.g. the file gets replaced with a better one).