Skip to content

Conversation

@mthalman
Copy link
Member

@mthalman mthalman commented Mar 3, 2023

The VerifyVersionFile test in source-build is failing in the Ubuntu leg with the following failure:

System.InvalidOperationException : Failed to execute tar xzf /vmr/artifacts/x64/Release/dotnet-sdk-8.0.100-preview.3.23152.1-ubuntu.20.04-x64.tar.gz -C /vmr/test/Microsoft.DotNet.SourceBuild.SmokeTests/bin/Release/net8.0/sourcebuilt-artifacts ./sdk/*/.version
Exit code: 2

tar: Pattern matching characters used in file names
tar: Use --wildcards to enable pattern matching, or --no-wildcards to suppress this warning
tar: ./sdk/*/.version: Not found in archive
tar: Exiting with failure status due to previous errors

It seems the version of tar in Ubuntu has different behavior compared to the other distros where this was working. Because the file path being targeted to extract has a wildcard in it, tar requires that the --wildcards option be explicitly set. Otherwise, it defaults to not treating it as a wildcard, which then leads to the failure to find a file path of ./sdk/*/.version.

I've fixed this by explicitly setting the --wildcards option. The use of this option is compatible with the other distros as well.

Fixes dotnet/source-build#3294

@mthalman mthalman requested a review from a team as a code owner March 3, 2023 14:51
@mthalman mthalman enabled auto-merge (squash) March 3, 2023 14:53
@mthalman mthalman merged commit 9bd65f4 into dotnet:main Mar 3, 2023
mthalman added a commit to mthalman/installer that referenced this pull request Mar 7, 2023
Comment on lines 64 to +66
private void ExtractFileFromTarball(string tarballPath, string filePath, string outputDir)
{
ExecuteHelper.ExecuteProcessValidateExitCode("tar", $"xzf {tarballPath} -C {outputDir} {filePath}", OutputHelper);
ExecuteHelper.ExecuteProcessValidateExitCode("tar", $"--wildcards -xzf {tarballPath} -C {outputDir} {filePath}", OutputHelper);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this net8.0 project can reference https://github.com/dotnet/sdk/blob/5638e0ffabae3eb5505d7aaf81e3449ce3641c36/src/RazorSdk/Tasks/Microsoft.NET.Sdk.Razor.Tasks.csproj#L43 and use:

using System.Formats.Tar;
using System.IO;
using System.IO.Compression;
using Microsoft.Extensions.FileSystemGlobbing;
...
private void ExtractFileFromTarball(string tarballPath, string filePath, string outputDir)
{
    Matcher matcher = new();
    matcher.AddInclude(filePath);

    using FileStream fileStream = File.OpenRead(tarballPath);
    using GZipStream decompressorStream = new(fileStream, CompressionMode.Decompress);
    using TarReader reader = new(decompressorStream);

    TarEntry entry;
    while ((entry = reader.GetNextEntry()) is not null)
    {
        if (matcher.Match(entry.Name).HasMatches)
        {
            string outputPath = Path.Join(outputDir, entry.Name);
            Directory.CreateDirectory(Path.GetDirectoryName(outputPath));

            using FileStream outputFileStream = File.Create(outputPath);
            entry.DataStream.CopyTo(outputFileStream);
            // break; ?
        }
    }
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kasperk81 - Fixing this in #15765

@mthalman mthalman deleted the sb3294 branch March 8, 2023 13:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

VerifyVersionFile test is failing in Ubuntu CI leg

3 participants