[Breaking change]: Base64.DecodeFromUtf8{InPlace} ignore whitespace

[stephentoub]

Description

The System.Convert.FromBase64String, System.Convert.FromBase64CharArray, and corresponding Try methods on System.Convert have long ignored some ASCII whitespace characters (' ', '\t', '\r', '\n'), allowing any amount of such whitespace to be in the input. However, when the System.Buffers.Text.Base64.DecodeFromUtf8 and System.Buffers.Text.Base64.DecodeFromUtf8InPlace methods were added, they did not ignore these whitespace characters and would instead fail to decode any input that included whitespace. This makes the behavior when using the UTF16-based APIs different from that using the UTF8-based APIs, and means that the Base64.DecodeFromUtf8 and Base64.DecodeFromUtf8InPlace methods couldn't roundtrip UTF-encoded base-64 encoded data produced by Convert.ToBase64String with the Base64FormattingOptions.InsertLineBreaks option. It also means that the new Base64.IsValid(ReadOnlySpan<char>) and Base64.IsValid(ReadOnlySpan<byte>) methods would either need to have behavior inconsistent with each other or with their corresponding methods for UTF16 and UTF8 data on Convert and Base64.

Version

.NET 8 Preview 5

Previous behavior

System.Buffers.Text.Base64.DecodeFromUtf8 and System.Buffers.Text.Base64.DecodeFromUtf8InPlace would fail to process input containing whitespace, returning OperationStatus.InvalidData if any whitespace was encountered.

New behavior

System.Buffers.Text.Base64.DecodeFromUtf8 and System.Buffers.Text.Base64.DecodeFromUtf8InPlace now ignore whitespace (specifically ' ', '\t', '\r', and '\n') in the input, matching the behavior of Convert.FromBase64String.

Type of breaking change

• Binary incompatible: Existing binaries may encounter a breaking change in behavior, such as failure to load or execute, and if so, require recompilation.
• Source incompatible: When recompiled using the new SDK or component or to target the new runtime, existing source code may require source changes to compile successfully.
• Behavioral change: Existing binaries may behave differently at run time.

Reason for change

The change was made so that:

• the Base64 methods could decode a wider range of input data, including that produced by Convert.ToBase64String with the Base64FormattingOptions.InsertLineBreaks option as well as common formatting of data in configuration files and other real data sources.
• the Base64 methods would be consistent with the corresponding decoding APIs on System.Convert.
• the new Base64.IsValid APIs could be added in a manner where their behavior is consistent with each other and with the existing Convert and Base64 APIs.

Recommended action

If the new behavior is problematic for your code, you can use IndexOfAny(" \t\r\n"u8) to search the input for the whitespace that previously would have triggered an InvalidData result.

Feature area

Core .NET libraries

Affected APIs

• System.Buffers.Text.Base64.DecodeFromUtf8
• System.Buffers.Text.Base64.DecodeFromUtf8InPlace

---

Associated WorkItem - 97023