Skip to content

Tar: support GNU numeric format. - #101172

Merged
carlossanlop merged 8 commits into
dotnet:mainfrom
tmds:tar_gnu_range
Jun 25, 2024
Merged

Tar: support GNU numeric format.#101172
carlossanlop merged 8 commits into
dotnet:mainfrom
tmds:tar_gnu_range

Conversation

@tmds

@tmds tmds commented Apr 17, 2024

Copy link
Copy Markdown
Member

The tar specification stores numeric fields using an octal representation. This limits the range of values that can be stored.

To increase the supported range, a GNU extension defines that when the leading byte is 0xff/0x80 the remaining bytes are a negative/positive big endian formatted value.

When writing under the PAX format, we continue to only use the only octal representation in the header fields. The values are overridden using extended attributes.

Fixes #93763.

@dotnet/area-system-formats-tar ptal.

@dotnet-policy-service dotnet-policy-service Bot added the community-contribution Indicates that the PR has been added by a community member label Apr 17, 2024
The tar specification stores numeric fields using an octal representation.
This limits the range of values that can be stored.

To increase the supported range, a GNU extension defines that when
the leading byte is 0xff/0x80 the remaining bytes are a
negative/positive big endian formatted value.

When writing under the PAX format, we continue to only use the
only octal representation in the header fields. The values are
overridden using extended attributes.
@tmds

tmds commented Apr 17, 2024

Copy link
Copy Markdown
Member Author

Test failures are unrelated.

@tmds

tmds commented Apr 29, 2024

Copy link
Copy Markdown
Member Author

@dotnet/area-system-formats-tar ptal.

@tmds

tmds commented May 8, 2024

Copy link
Copy Markdown
Member Author

@carlossanlop ptal.

@tmds

tmds commented May 21, 2024

Copy link
Copy Markdown
Member Author

@dotnet/area-system-formats-tar ptal.

@am11

am11 commented May 21, 2024

Copy link
Copy Markdown
Member

It would be nice to get this in for .NET 9. The sooner tar implementation matures the better for the ecosystem. Thanks for this @tmds! 👍

@carlossanlop carlossanlop left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this change. I left some questions. Haven't gone through the whole PR yet.

We also need to run runtime-extra-platforms tests.

Comment thread src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarEntry.cs
Comment thread src/libraries/System.Formats.Tar/tests/Manual/ManualTests.cs
@carlossanlop

Copy link
Copy Markdown
Contributor

/azp run runtime-extra-platforms

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@tmds
tmds force-pushed the tar_gnu_range branch from 9ca577e to 58f651a Compare May 23, 2024 08:23
@tmds

tmds commented May 28, 2024

Copy link
Copy Markdown
Member Author

@carlossanlop I have addressed your feedback, can you take another look?

@carlossanlop carlossanlop left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for addressing the feedback. I gave it another pass and left some comments for you to consider.

I'll run runtime-extra-platforms now to see if we catch any unexpected errors.

Comment thread src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarHeader.Write.cs Outdated
Comment thread src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarHeader.Write.cs Outdated
Comment thread src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarEntry.cs
Comment thread src/libraries/System.Formats.Tar/tests/Manual/ManualTests.cs
Comment thread src/libraries/System.Formats.Tar/tests/TarWriter/TarWriter.WriteEntry.Base.cs Outdated
@carlossanlop

Copy link
Copy Markdown
Contributor

/azp run runtime-extra-platforms

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@tmds

tmds commented Jun 6, 2024

Copy link
Copy Markdown
Member Author

@carlossanlop I've addressed the additional feedback except for the writerbig test. See #101172 (comment).

@tmds
tmds force-pushed the tar_gnu_range branch from 0160f2a to 0d17598 Compare June 6, 2024 13:44
@tmds

tmds commented Jun 14, 2024

Copy link
Copy Markdown
Member Author

@carlossanlop can you let me know if you want this test included: #101172 (comment)?

@carlossanlop

Copy link
Copy Markdown
Contributor

/azp run runtime-extra-platforms

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@carlossanlop carlossanlop left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved pending two things:

  • The re-addition of the removed writer-big test case as suggested in the snippet.
  • I ran runtime-extra-platforms, which executes tar tests in many other platforms (mobile maybe). Please verify we don't see any new failures related to this change.

@tmds

tmds commented Jun 17, 2024

Copy link
Copy Markdown
Member Author

@carlossanlop can you start another run of runtime-extra-platforms?

@carlossanlop

Copy link
Copy Markdown
Contributor

/azp run runtime-extra-platforms

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@tmds

tmds commented Jun 25, 2024

Copy link
Copy Markdown
Member Author

@carlossanlop this is good to merge.

@carlossanlop
carlossanlop merged commit c5e8f83 into dotnet:main Jun 25, 2024
@carlossanlop

Copy link
Copy Markdown
Contributor

Thanks @tmds!

@carlossanlop

Copy link
Copy Markdown
Contributor

Considering this is a problem that can be seen in runfo (one of our infrastructure tools) I'm inclined to backport this to .NET 8. Would you have any objections, @ericstj?

@ericstj

ericstj commented Jul 23, 2024

Copy link
Copy Markdown
Member

Runfo isn't a critical tool, its just for debugging. I'd instead suggest you seek out input from customers reporting this one. See if anyone is blocked on .NET 8.0 - that's more important. This definitely would meet the bar for servicing if we had a customer blocked by it.

@github-actions github-actions Bot locked and limited conversation to collaborators Aug 23, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-System.Formats.Tar community-contribution Indicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TarReader throws on archive that other tools accept

4 participants