-
Notifications
You must be signed in to change notification settings - Fork 93
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Swift Package Manager component detection support #1316
Open
raporpe
wants to merge
11
commits into
microsoft:main
Choose a base branch
from
raporpe:users/raulpo/implement-swiftpm-component-detection
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+858
−0
Open
Changes from all commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
bbf7515
Create Detector and Component types
raporpe 505075f
Implementation + tests
raporpe 29cb794
Verification file
raporpe 7c6631a
Add docs
raporpe 0cd36ad
Fix pipeline
raporpe 4f3f508
Merge branch 'main' into users/raulpo/implement-swiftpm-component-det…
raporpe a16b797
Rename SwiftPM to Swift
raporpe b637bbb
Change detector version to 1
raporpe 7a9e269
Make TargetSwiftPackageKind a private field
raporpe b16f488
Add error logging
raporpe ffb3aa5
Merge branch 'main' into users/raulpo/implement-swiftpm-component-det…
raporpe File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
# Swift Package Manager Detection | ||
|
||
## Requirements | ||
|
||
Swift Package Manager detection requires the following file to be present in the scan directory: | ||
|
||
- `Package.resolved` file | ||
|
||
## Detection strategy | ||
|
||
The detector `SwiftResolvedComponentDetector` only parses the `Package.resolved` file to get the dependencies. | ||
This file contains a json representation of the resolved dependencies of the project with the transitive dependencies. | ||
The version, the url and commit hash of the dependencies are stored in this file. | ||
|
||
[This is the only reference in the Apple documentation to the `Package.resolved` file.][1] | ||
|
||
## Known limitations | ||
|
||
Right now the detector does not support parsing `Package.swift` file to get the dependencies. | ||
It only supports parsing `Package.resolved` file. | ||
Some projects only commit the `Package.swift`, which is why it is planned to support parsing `Package.swift` in the future. | ||
|
||
[1]: https://docs.swift.org/package-manager/PackageDescription/PackageDescription.html#package-dependency |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -56,4 +56,7 @@ public enum ComponentType : byte | |
|
||
[EnumMember] | ||
Conan = 17, | ||
|
||
[EnumMember] | ||
Swift = 18, | ||
} |
50 changes: 50 additions & 0 deletions
50
src/Microsoft.ComponentDetection.Contracts/TypedComponent/SwiftComponent.cs
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
namespace Microsoft.ComponentDetection.Contracts.TypedComponent; | ||
|
||
using System; | ||
using System.Collections.Generic; | ||
using PackageUrl; | ||
|
||
/// <summary> | ||
/// Represents a Swift package manager component. | ||
/// </summary> | ||
public class SwiftComponent : TypedComponent | ||
{ | ||
private readonly string packageUrl; | ||
|
||
private readonly string hash; | ||
|
||
/// <summary> | ||
/// Initializes a new instance of the <see cref="SwiftComponent"/> class. | ||
/// </summary> | ||
/// <param name="name">The name of the component.</param> | ||
/// <param name="version">The version of the component.</param> | ||
/// <param name="packageUrl">The package URL of the component.</param> | ||
/// <param name="hash">The hash of the component.</param> | ||
public SwiftComponent(string name, string version, string packageUrl, string hash) | ||
{ | ||
this.Name = this.ValidateRequiredInput(name, nameof(name), nameof(ComponentType.Swift)); | ||
this.Version = this.ValidateRequiredInput(version, nameof(version), nameof(ComponentType.Swift)); | ||
this.packageUrl = this.ValidateRequiredInput(packageUrl, nameof(packageUrl), nameof(ComponentType.Swift)); | ||
this.hash = this.ValidateRequiredInput(hash, nameof(hash), nameof(ComponentType.Swift)); | ||
} | ||
|
||
public string Name { get; } | ||
|
||
public string Version { get; } | ||
|
||
public override ComponentType Type => ComponentType.Swift; | ||
|
||
public override string Id => $"{this.Name} {this.Version} - {this.Type}"; | ||
|
||
// The type is swiftpm | ||
public PackageURL PackageURL => new PackageURL( | ||
type: "swift", | ||
@namespace: new Uri(this.packageUrl).Host, | ||
name: this.Name, | ||
version: this.hash, // Hash has priority over version when creating a PackageURL | ||
qualifiers: new SortedDictionary<string, string> | ||
{ | ||
{ "repository_url", this.packageUrl }, | ||
}, | ||
subpath: null); | ||
} |
51 changes: 51 additions & 0 deletions
51
src/Microsoft.ComponentDetection.Detectors/swiftpm/Contracts/SwiftResolvedFile.cs
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
namespace Microsoft.ComponentDetection.Contracts.TypedComponent; | ||
|
||
using System.Collections.Generic; | ||
using Newtonsoft.Json; | ||
|
||
/// <summary> | ||
/// Represents a Swift Package Manager component. | ||
/// </summary> | ||
public class SwiftResolvedFile | ||
{ | ||
[JsonProperty("pins")] | ||
public IList<SwiftDependency> Pins { get; set; } | ||
|
||
[JsonProperty("version")] | ||
public int Version { get; set; } | ||
|
||
public class SwiftDependency | ||
{ | ||
// The name of the package | ||
[JsonProperty("identity")] | ||
public string Identity { get; set; } | ||
|
||
// How the package is imported. Example: "remoteSourceControl" | ||
// This is not an enum because the Swift contract does not specify the possible values. | ||
[JsonProperty("kind")] | ||
public string Kind { get; set; } | ||
|
||
// The unique path to the repository where the package is located. Example: Git repo URL. | ||
[JsonProperty("location")] | ||
public string Location { get; set; } | ||
|
||
// Data about the package version and commit hash. | ||
[JsonProperty("state")] | ||
public SwiftState State { get; set; } | ||
|
||
public class SwiftState | ||
{ | ||
// The commit hash of the package. | ||
[JsonProperty("revision")] | ||
public string Revision { get; set; } | ||
|
||
// The version of the package. Might be missing. | ||
[JsonProperty("version")] | ||
public string Version { get; set; } | ||
|
||
// The branch of the package. Might be missing. | ||
[JsonProperty("branch")] | ||
public string Branch { get; set; } | ||
} | ||
} | ||
} |
113 changes: 113 additions & 0 deletions
113
src/Microsoft.ComponentDetection.Detectors/swiftpm/SwiftResolvedComponentDetector.cs
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,113 @@ | ||
namespace Microsoft.ComponentDetection.Detectors.Swift; | ||
|
||
using System; | ||
using System.Collections.Generic; | ||
using System.IO; | ||
using System.Threading; | ||
using System.Threading.Tasks; | ||
using Microsoft.ComponentDetection.Contracts; | ||
using Microsoft.ComponentDetection.Contracts.Internal; | ||
using Microsoft.ComponentDetection.Contracts.TypedComponent; | ||
using Microsoft.Extensions.Logging; | ||
using Newtonsoft.Json; | ||
|
||
/// <summary> | ||
/// Detects Swift Package Manager components. | ||
/// </summary> | ||
public class SwiftResolvedComponentDetector : FileComponentDetector, IDefaultOffComponentDetector | ||
{ | ||
// We are only interested in packages coming from remote sources such as git | ||
// The Package Kind is not an enum because the Swift Package Manager contract does not specify the possible values. | ||
private const string TargetSwiftPackageKind = "remoteSourceControl"; | ||
|
||
public SwiftResolvedComponentDetector( | ||
IComponentStreamEnumerableFactory componentStreamEnumerableFactory, | ||
IObservableDirectoryWalkerFactory walkerFactory, | ||
ILogger<SwiftResolvedComponentDetector> logger) | ||
{ | ||
this.ComponentStreamEnumerableFactory = componentStreamEnumerableFactory; | ||
this.Scanner = walkerFactory; | ||
this.Logger = logger; | ||
} | ||
|
||
public override string Id { get; } = "Swift"; | ||
|
||
public override IEnumerable<string> Categories => [Enum.GetName(DetectorClass.Swift)]; | ||
|
||
public override IList<string> SearchPatterns { get; } = ["Package.resolved"]; | ||
|
||
public override IEnumerable<ComponentType> SupportedComponentTypes => [ComponentType.Swift]; | ||
|
||
public override int Version => 1; | ||
|
||
protected override Task OnFileFoundAsync( | ||
ProcessRequest processRequest, | ||
IDictionary<string, string> detectorArgs, | ||
CancellationToken cancellationToken = default) | ||
{ | ||
try | ||
{ | ||
this.ProcessPackageResolvedFile(processRequest.SingleFileComponentRecorder, processRequest.ComponentStream); | ||
} | ||
catch (Exception exception) | ||
{ | ||
this.Logger.LogError(exception, "SwiftComponentDetector: Error processing Package.resolved file: {Location}", processRequest.ComponentStream.Location); | ||
} | ||
|
||
return Task.CompletedTask; | ||
} | ||
|
||
private void ProcessPackageResolvedFile(ISingleFileComponentRecorder singleFileComponentRecorder, IComponentStream componentStream) | ||
{ | ||
var parsedResolvedFile = this.ReadAndParseResolvedFile(componentStream.Stream); | ||
|
||
foreach (var package in parsedResolvedFile.Pins) | ||
{ | ||
try | ||
{ | ||
if (package.Kind == TargetSwiftPackageKind) | ||
{ | ||
// The version of the package is not always available. | ||
var version = package.State.Version ?? package.State.Branch ?? package.State.Revision; | ||
|
||
var detectedSwiftComponent = new SwiftComponent( | ||
name: package.Identity, | ||
version: version, | ||
packageUrl: package.Location, | ||
hash: package.State.Revision); | ||
var newDetectedSwiftComponent = new DetectedComponent(component: detectedSwiftComponent, detector: this); | ||
singleFileComponentRecorder.RegisterUsage(newDetectedSwiftComponent); | ||
|
||
// We also register a Git component for the same package so that the git URL is registered. | ||
// Swift Package Manager directly downloads the package from the git URL. | ||
var detectedGitComponent = new GitComponent( | ||
repositoryUrl: new Uri(package.Location), | ||
commitHash: package.State.Revision, | ||
tag: version); | ||
var newDetectedGitComponent = new DetectedComponent(component: detectedGitComponent, detector: this); | ||
singleFileComponentRecorder.RegisterUsage(newDetectedGitComponent); | ||
} | ||
} | ||
catch (Exception exception) | ||
{ | ||
this.Logger.LogError(exception, "SwiftComponentDetector: Error processing package: {Package}", package.Identity); | ||
} | ||
} | ||
} | ||
|
||
/// <summary> | ||
/// Reads the stream of the package resolved file and parses it. | ||
/// </summary> | ||
/// <param name="stream">The stream of the file to parse.</param> | ||
/// <returns>The parsed object.</returns> | ||
private SwiftResolvedFile ReadAndParseResolvedFile(Stream stream) | ||
{ | ||
string resolvedFile; | ||
using (var reader = new StreamReader(stream)) | ||
{ | ||
resolvedFile = reader.ReadToEnd(); | ||
} | ||
|
||
return JsonConvert.DeserializeObject<SwiftResolvedFile>(resolvedFile); | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
80 changes: 80 additions & 0 deletions
80
test/Microsoft.ComponentDetection.Detectors.Tests/SwiftComponentTests.cs
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,80 @@ | ||
namespace Microsoft.ComponentDetection.Detectors.Tests.Swift; | ||
|
||
using System; | ||
using System.Collections.Generic; | ||
using FluentAssertions; | ||
using Microsoft.ComponentDetection.Contracts.TypedComponent; | ||
using Microsoft.VisualStudio.TestTools.UnitTesting; | ||
using PackageUrl; | ||
|
||
[TestClass] | ||
public class SwiftComponentTests | ||
{ | ||
[TestMethod] | ||
public void Constructor_ShouldInitializeProperties() | ||
{ | ||
var name = "alamofire"; | ||
var version = "5.9.1"; | ||
var packageUrl = "https://github.com/Alamofire/Alamofire"; | ||
var hash = "f455c2975872ccd2d9c81594c658af65716e9b9a"; | ||
|
||
var component = new SwiftComponent(name, version, packageUrl, hash); | ||
|
||
component.Name.Should().Be(name); | ||
component.Version.Should().Be(version); | ||
component.Type.Should().Be(ComponentType.Swift); | ||
component.Id.Should().Be($"{name} {version} - {component.Type}"); | ||
} | ||
|
||
[TestMethod] | ||
public void Constructor_ShouldThrowException_WhenNameIsNull() | ||
{ | ||
Action action = () => new SwiftComponent(null, "5.9.1", "https://github.com/Alamofire/Alamofire", "f455c2975872ccd2d9c81594c658af65716e9b9a"); | ||
action.Should().Throw<ArgumentException>().WithMessage("*name*"); | ||
} | ||
|
||
[TestMethod] | ||
public void Constructor_ShouldThrowException_WhenVersionIsNull() | ||
{ | ||
Action action = () => new SwiftComponent("alamofire", null, "https://github.com/Alamofire/Alamofire", "f455c2975872ccd2d9c81594c658af65716e9b9a"); | ||
action.Should().Throw<ArgumentException>().WithMessage("*version*"); | ||
} | ||
|
||
[TestMethod] | ||
public void Constructor_ShouldThrowException_WhenPackageUrlIsNull() | ||
{ | ||
Action action = () => new SwiftComponent("alamofire", "5.9.1", null, "f455c2975872ccd2d9c81594c658af65716e9b9a"); | ||
action.Should().Throw<ArgumentException>().WithMessage("*packageUrl*"); | ||
} | ||
|
||
[TestMethod] | ||
public void Constructor_ShouldThrowException_WhenHashIsNull() | ||
{ | ||
Action action = () => new SwiftComponent("alamofire", "5.9.1", "https://github.com/Alamofire/Alamofire", null); | ||
action.Should().Throw<ArgumentException>().WithMessage("*hash*"); | ||
} | ||
|
||
[TestMethod] | ||
public void PackageURL_ShouldReturnCorrectPackageURL() | ||
{ | ||
var name = "alamofire"; | ||
var version = "5.9.1"; | ||
var packageUrl = "https://github.com/Alamofire/Alamofire"; | ||
var hash = "f455c2975872ccd2d9c81594c658af65716e9b9a"; | ||
|
||
var component = new SwiftComponent(name, version, packageUrl, hash); | ||
|
||
var expectedPackageURL = new PackageURL( | ||
type: "swift", | ||
@namespace: "github.com", | ||
name: name, | ||
version: hash, | ||
qualifiers: new SortedDictionary<string, string> | ||
{ | ||
{ "repository_url", packageUrl }, | ||
}, | ||
subpath: null); | ||
|
||
component.PackageURL.Should().BeEquivalentTo(expectedPackageURL); | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
have you confirmed in PURL spec hash is what is used as opposed to version? These are not meant to be arbitrary. We should be aligned with the spec has defined for Swift, I don't recall seeing hashes for purls in the past. See sample Swift PURL and purl-spec