-
Notifications
You must be signed in to change notification settings - Fork 93
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Swift Package Manager component detection support #1316
base: main
Are you sure you want to change the base?
Add Swift Package Manager component detection support #1316
Conversation
@microsoft-github-policy-service agree company="Microsoft" |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1316 +/- ##
=======================================
+ Coverage 89.3% 89.5% +0.1%
=======================================
Files 384 389 +5
Lines 30106 30609 +503
Branches 1840 1851 +11
=======================================
+ Hits 26901 27405 +504
+ Misses 2812 2811 -1
Partials 393 393 ☔ View full report in Codecov by Sentry. |
src/Microsoft.ComponentDetection.Detectors/swiftpm/SwiftPMResolvedComponentDetector.cs
Outdated
Show resolved
Hide resolved
src/Microsoft.ComponentDetection.Detectors/swiftpm/SwiftResolvedComponentDetector.cs
Outdated
Show resolved
Hide resolved
src/Microsoft.ComponentDetection.Detectors/swiftpm/SwiftResolvedComponentDetector.cs
Outdated
Show resolved
Hide resolved
// We also register a Git component for the same package so that the git URL is registered. | ||
// Swift Package Manager directly downloads the package from the git URL. | ||
var detectedGitComponent = new GitComponent( | ||
repositoryUrl: new Uri(package.Location), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this location guaranteed to be a valid Git Url? I ask because if this is missing or an unexpected location it will effectively break the parsing of all Swift components in that scan.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When developing the detectector I tried to look into the SwiftPM documentation and found not format guarantee that this is a git url. Swift expects a repository when the kind is "remoteSourceControl". Anyway, I'm going to take a look into the SwiftPM source code to see what it's actually doing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok I found the documentation that I needed:
and
![Screenshot 2025-01-27 at 18 19 45](https://private-user-images.githubusercontent.com/6137860/407025182-3e043517-7ebc-4ac0-bae6-d95a924270b6.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mzg4MjQ3NjksIm5iZiI6MTczODgyNDQ2OSwicGF0aCI6Ii82MTM3ODYwLzQwNzAyNTE4Mi0zZTA0MzUxNy03ZWJjLTRhYzAtYmFlNi1kOTVhOTI0MjcwYjYucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI1MDIwNiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNTAyMDZUMDY0NzQ5WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YjkyZDhiYzc2NGJiNWRjMGI4MjM4Nzc1NjllZmY1MjMzMzYxNjQ5MTI1Y2UwZmVlYjExMDg4MTI4ODkwZTM1ZSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.nZl7UPBk2KFyxfat0RJ1QAo8U07QnTZABKdo_pv5Bz8)
and in the source code they are verifying that the URL is a valid git repository because SwiftPM depends on git tags to be able to download the specified version.
This is the source code where they are doing it:
![Screenshot 2025-01-27 at 18 18 26](https://private-user-images.githubusercontent.com/6137860/407025625-24f3f007-60aa-4470-ab1f-5b2eae46b5c0.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mzg4MjQ3NjksIm5iZiI6MTczODgyNDQ2OSwicGF0aCI6Ii82MTM3ODYwLzQwNzAyNTYyNS0yNGYzZjAwNy02MGFhLTQ0NzAtYWIxZi01YjJlYWU0NmI1YzAucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI1MDIwNiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNTAyMDZUMDY0NzQ5WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9MDRmNWRhZDQyZjVjYmYyY2I3NjFiOTlkY2MxMTg3ZDhmMjU0ZTRkYWQ4ZWI5MmI2ZmJlM2ZjYjcxMDk3ZjEwZCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.dced6O0rSVyEJ8-NxG-NFyB3rJnDsVEp3bKpOC9q0gY)
![Screenshot 2025-01-27 at 18 18 43](https://private-user-images.githubusercontent.com/6137860/407025643-198ba2a2-5396-4e3d-bd27-44738ad03fdc.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mzg4MjQ3NjksIm5iZiI6MTczODgyNDQ2OSwicGF0aCI6Ii82MTM3ODYwLzQwNzAyNTY0My0xOThiYTJhMi01Mzk2LTRlM2QtYmQyNy00NDczOGFkMDNmZGMucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI1MDIwNiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNTAyMDZUMDY0NzQ5WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9NTUzZGI1NzI2ZTYyNDlhNjUwNWM0YzlmYzhiMDZhNzc0NzMwMDEyMWIxYTk4NjkxY2RmMmNiYzMxY2M3NzI1MCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.j5dMdfGP6DWu34PxU5FO9vLKF-BboiAo_yW84czID5Y)
So yes, I would say it's very very probable that this is almost always going to be a valid git url for most cases.
These are all the available "Kinds" of packages and I'm covering one of the most common, which is the git url. I think that SwiftPM registries are not that common.
![Screenshot 2025-01-27 at 17 53 26](https://private-user-images.githubusercontent.com/6137860/407025949-91ee5514-180b-4eb7-9f9d-b00ddd90d856.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mzg4MjQ3NjksIm5iZiI6MTczODgyNDQ2OSwicGF0aCI6Ii82MTM3ODYwLzQwNzAyNTk0OS05MWVlNTUxNC0xODBiLTRlYjctOWY5ZC1iMDBkZGQ5MGQ4NTYucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI1MDIwNiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNTAyMDZUMDY0NzQ5WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9NWZlOWQ0Y2VmYjQ4NTliYWVlZmY5ZmU1M2Y2MTQ3MjAyODFkY2Y2YWM4M2MwZTRmNzAyZDgzMGE0ZTU3OWJkZSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.XKB5JzVLhrRsjJc0GsRZrqgtuD4ZsHSmE_NMw83v1bE)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As you told me in teams, I will make sure to enclose each for loop iteration with a try so that if a single dependency fails the rest of dependencies will be registered 👍🏼
type: "swift", | ||
@namespace: new Uri(this.packageUrl).Host, | ||
name: this.Name, | ||
version: this.hash, // Hash has priority over version when creating a PackageURL |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
have you confirmed in PURL spec hash is what is used as opposed to version? These are not meant to be arbitrary. We should be aligned with the spec has defined for Swift, I don't recall seeing hashes for purls in the past. See sample Swift PURL and purl-spec
Component detection does not currently support detection of Swift Package Manager. This package manager is the new default for swift-related projects now that CocoaPods (supported by Component detection) is on maintenance mode.
This new detector adds basic support to SwiftPM by parsing the JSON file
Package.resolved
that is generated when dependencies are resolved. This file is usually present in repositories and after building a Swift Package and its contents are enough to detect both direct and transitive dependencies.This detector does not parse the
Package.swift
file (hard to parse and does not include transitive dependencies) and does not differentiate between build dependencies, transitive dependencies, and direct dependencies (it is not possible by just parsingPackage.resolved
).This detector will register two kind of components: a
SwiftPMComponent
and aGitComponent
. I have decided to also register a Git component because SwiftPM does not rely in a repository infrastructure such as NPM, it directly downloads other Swift Packages from the git repositories. I need feedback in this regard, since I do not know what is the best practice (maybe having two registered components for the same dependency is not a good idea?)I had to invest time in troubleshooting the verification tests because I was not passing them. The issue is that the reference snapshot has less packages because the environments differ for
PipReport
since PR #1259. I have added the same environment vars to the snapshot-publish pipeline although maybe @pauld-msft could take a look at this PR to verify that my modification is correct.