Regex Capture gives IndexOutOfRangeException on `(?:){93}`

[danmoseley]

Regex regex = new("@(a*)+?");
MatchCollection mc = regex.Matches("@");

foreach (Match m in mc)
{
    Console.WriteLine($"{m.Index} .. {m.Length + m.Index}");
    Console.WriteLine(m.ToString());
}

On other regex engines, like PCRE, Nim, Python, Javascript, and Golang you would get 0 .. 1

On .NET Framework this displays 1 .. 1

On .NET 6 it gives

1 .. 3
Unhandled exception. System.ArgumentOutOfRangeException: Index and length must refer to a location within the string. (Parameter 'length')
   at System.String.Substring(Int32 startIndex, Int32 length)
   at System.Text.RegularExpressions.Capture.get_Value()
   at System.Text.RegularExpressions.Capture.ToString()
   at Program.<Main>$(String[] args) in C:\Users\danmose\source\repos\ConsoleApp31\ConsoleApp31\Program.cs:line 191

@stephentoub first bug from running Nim tests.

Tagging subscribers to this area: @eerhardt, @dotnet/area-system-text-regularexpressions
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Regex regex = new("@(a*)+?");
MatchCollection mc = regex.Matches("@");

foreach (Match m in mc)
{
    Console.WriteLine($"{m.Index} .. {m.Length + m.Index - 1}");
    Console.WriteLine(m.ToString());
}

On other regex engines, like PCRE, Nim, Python, Javascript, and Golang you would get 0 .. 1

On .NET Framework this displays 1 .. 0

On .NET 6 it gives

1 .. 2
Unhandled exception. System.ArgumentOutOfRangeException: Index and length must refer to a location within the string. (Parameter 'length')
   at System.String.Substring(Int32 startIndex, Int32 length)
   at System.Text.RegularExpressions.Capture.get_Value()
   at System.Text.RegularExpressions.Capture.ToString()
   at Program.<Main>$(String[] args) in C:\Users\danmose\source\repos\ConsoleApp31\ConsoleApp31\Program.cs:line 191

@stephentoub first bug from running Nim tests.

Author:	danmoseley
Assignees:	-
Labels:	area-System.Text.RegularExpressions, untriaged
Milestone:	-

[danmoseley]

@(a?)+? with text of @will not crash, but gives 1 .. 1 whereas other engines will give 0 .. 1

So there seems to be distinct bugs

• incorrect starting index of the match in all .NET versions
• incorrect length in .NET Core, causing a crash.

[stephentoub]

incorrect length in .NET Core, causing a crash.

Looks like a regression in .NET 5, which also exhibits this. .NET Core 3.1 outputs 1..1 like .NET Framework does.

[stephentoub]

This is actually an issue in .NET Framework 4.8 and .NET Core 3.1 as well.

The change in .NET 5 is that it recognizes that the outer loop is actually atomic (because there's nothing after it to backtrack into it). So it effectively changes the expression to instead be:

"@(?>(a*)+?)"

Try that expression with .NET Framework, and you similarly get 1..3 and an out of range exception.

[stephentoub]

(Locally I also have the final changes that switches over all compiler / source generated expressions to using the new simpler code generation (except for anything that uses RightToLeft), and with that, this correctly outputs 0..1 and no exception, so it'll end up really only being an issue for the interpreter.)

[danmoseley]

When I fuzzed I believe I only used IsMatch. Next time we should enumerate the matches also.

[joperezr]

I just tested with Preview 7 bits and seems like all 4 engines (including Interpreted) are correctly printing:

0 .. 1
@

Not sure exactly which change fixed this as it was logged some time ago, but closing as completed.