Support for better Package URLs, better metadata capturing, and normalized a lot of functionality #53
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…software heritage id which is just a gitoid:sha1 so it was duplicate Signed-off-by: David Pollak <[email protected]>
Signed-off-by: David Pollak <[email protected]>
Signed-off-by: David Pollak <[email protected]>
…rt for package URLs, pom files, and general metadata Signed-off-by: David Pollak <[email protected]>
Signed-off-by: David Pollak <[email protected]>
Signed-off-by: David Pollak <[email protected]>
Signed-off-by: David Pollak <[email protected]>
…y improvements Signed-off-by: David Pollak <[email protected]>
Signed-off-by: David Pollak <[email protected]>
Signed-off-by: David Pollak <[email protected]>
Signed-off-by: David Pollak <[email protected]>
aredridel
approved these changes
Feb 3, 2025
| myHash < thatHash | ||
| } | ||
|
|
||
| def fixReferences(store: Storage): Item = { |
There was a problem hiding this comment.
"fix"? What's this actually doing?
There was a problem hiding this comment.
It might be worth separating the logic for creating an updated alias with modified connections from the side effect of writing to the database.
| "text/x-java-properties", | ||
| "text/x-scala", | ||
| "text/x-java-source", | ||
| "application/x-sh", |
There was a problem hiding this comment.
I regret to inform you shar.zip
There was a problem hiding this comment.
Sorry... what is the issue with shar.zip ? Missing what I need to enhance
There was a problem hiding this comment.
Oh, sorry. It's not like, super critical, but shell files can be archives (using sharutils), and there's one in that zip file. I fully expect we'd not support it but just archive formats are wild
earldouglas
reviewed
Feb 3, 2025
| .toDouble | ||
| val itemsPerMinute = itemsPerSecond * 60.0d | ||
| val left = totalItems.toDouble - updatedCnt.toDouble | ||
| f" Items/minute ${itemsPerMinute.round}, est remaining ${(left / itemsPerMinute).round} minutes" |
earldouglas
reviewed
Feb 3, 2025
earldouglas
approved these changes
Feb 3, 2025
…arentContext' to support better plugin functionality Signed-off-by: David Pollak <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
💻 Description of Change(s) (w/ context)
A substantial re-write of Goat Rodeo
Normalized traversal into containers. This means that filesystems, ISOs, JARs, TARs, etc. are all
treated the same way.
The result is that if there's an ISO with a TAR file that contains JAR/POM files, things will "just work"
Error handling is at the traversal level so that if there's an error processing or opening an
archive, that error is isolated.
Package URL handling is normalized via the PackageURL package
Metadata extraction is handled by specialized handlers for Maven and Debian types.
The Maven and Debian strategies are managed by passing all the artifacts to a handler and allowing the handler to
choose which files to handle. Thus, mixing Debian, Maven, etc. in one TAR will "just work"
🧠 Rationale Behind Change(s)
Goat Rodeo was a mess. This set of changes normalizes a lot of the function (choosing a per-artifact-type
strategy with implementations for Maven and Debian) as well as recursive file traversal.
The changes will allow rapid additions of new container types (beyond ISO, TAR, JAR, etc.)
The changes will allow addition of support for other artifact types (RPM, Python, etc.)
📝 Test Plan
Added a few tests and all existing tests pass.
📜 Documentation
A lot of ScalaDocs
💣 Quality Control
(All items must be checked before a PR is merged)
Did you…
Before Merging…