Assertion failure in S_ReANY

[p5pRT]

Migrated from rt.perl.org#134003 (status was 'open')

Searchable as RT134003$

[p5pRT]

From @dur-randir

Created by @dur-randir

While fuzzing perl v5.29.9-63-g2496d8f3f7 built with afl and run
under libdislocator, I found the following program

for(0..6){0if split$p;($$v0l0e)&=($f,$v0l0e)=0;($e)=0}

to cause an assertion failure

perl​: ./inline.h​:182​: struct regexp *S_ReANY(const REGEXP *const)​:
Assertion `isREGEXP(re)' failed.

GDB stack trace is following

#0 __GI_raise (sig=sig@​entry=6) at ../sysdeps/unix/sysv/linux/raise.c​:50
#1 0x00007ffff7c25535 in __GI_abort () at abort.c​:79
#2 0x00007ffff7c2540f in __assert_fail_base (fmt=0x7ffff7d87ee0
"%s%s%s​:%u​: %s%sAssertion `%s' failed.\n%n", assertion=0x555555ac6c87
"isREGEXP(re)",
  file=0x555555ac6bd0 "inline.h", line=182, function=<optimized
out>) at assert.c​:92
#3 0x00007ffff7c330f2 in __GI___assert_fail (assertion=0x555555ac6c87
"isREGEXP(re)", file=0x555555ac6bd0 "inline.h", line=182,
  function=0x555555acda70 <__PRETTY_FUNCTION__.17109> "S_ReANY") at
assert.c​:101
#4 0x0000555555821313 in S_ReANY (re=0x555555b68e88) at inline.h​:182
#5 0x0000555555822da6 in Perl_pp_regcomp () at pp_ctl.c​:106
#6 0x000055555570b89b in Perl_runops_debug () at dump.c​:2537
#7 0x00005555555ed560 in S_run_body (oldscope=1) at perl.c​:2716
#8 0x00005555555ecade in perl_run (my_perl=0x555555b4c260) at perl.c​:2639
#9 0x00005555555a114e in main (argc=3, argv=0x7fffffffe1c8,
env=0x7fffffffe1e8) at perlmain.c​:127

While it looks like one from stack-not-refcounted pack, I don't see
how altering $$v0l0e and $f could affect $p. Bisected commit also
seems innocent (this is a blead-only failure)

commit 6ef7fe5
Author​: Karl Williamson <khw@​cpan.org>
Date​: Sun Mar 17 22​:11​:04 2019 -0600

  PATCH​: [perl #131551] Too deep regex compilation recursion

  This patch, started by Yves Orton, and refined in consultation with Tony
  Cook, imposes a maximum depth of unclosed left parentheses, at which
  point it croaks. This is to prevent the segfault in the ticket.

Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl 5.29.9:

Configured by dur-randir at Wed Feb 27 14:51:01 MSK 2019.

Summary of my perl5 (revision 5 version 29 subversion 9) configuration:
  Commit id: c1e47bad34ce1d9c84ed57c9b8978bcbd5a02e98
  Platform:
    osname=darwin
    osvers=13.4.0
    archname=darwin-thread-multi-2level
    uname='darwin isengard.local 13.4.0 darwin kernel version 13.4.0:
mon jan 11 18:17:34 pst 2016; root:xnu-2422.115.15~1release_x86_64
x86_64 '
    config_args='-de -Dusedevel -DDEBUGGING -Dusethreads'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-fno-common -DPERL_DARWIN -mmacosx-version-min=10.9
-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector
-I/usr/local/include -DPERL_USE_SAFE_PUTENV'
    optimize='-O3 -g'
    cppflags='-fno-common -DPERL_DARWIN -mmacosx-version-min=10.9
-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector
-I/usr/local/include'
    ccversion=''
    gccversion='4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags =' -mmacosx-version-min=10.9 -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.0/lib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib
/usr/lib
    libs=-lpthread -lgdbm -ldbm -ldl -lm -lutil -lc
    perllibs=-lpthread -ldl -lm -lutil -lc
    libc=
    so=dylib
    useshrplib=false
    libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=bundle
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags=' -mmacosx-version-min=10.9 -bundle -undefined
dynamic_lookup -L/usr/local/lib -fstack-protector'



@INC for perl 5.29.9:
    lib
    /usr/local/lib/perl5/site_perl/5.29.9/darwin-thread-multi-2level
    /usr/local/lib/perl5/site_perl/5.29.9
    /usr/local/lib/perl5/5.29.9/darwin-thread-multi-2level
    /usr/local/lib/perl5/5.29.9


Environment for perl 5.29.9:
    DYLD_LIBRARY_PATH (unset)
    HOME=/Users/dur-randir
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/Users/dur-randir/perlbrew/bin:/Users/dur-randir/perlbrew/perls/perl-5.22.1/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin
    PERLBREW_HOME=/Users/dur-randir/.perlbrew
    PERLBREW_MANPATH=/Users/dur-randir/perlbrew/perls/perl-5.22.1/man
    PERLBREW_PATH=/Users/dur-randir/perlbrew/bin:/Users/dur-randir/perlbrew/perls/perl-5.22.1/bin
    PERLBREW_PERL=perl-5.22.1
    PERLBREW_ROOT=/Users/dur-randir/perlbrew
    PERLBREW_SHELLRC_VERSION=0.84
    PERLBREW_VERSION=0.84
    PERL_BADLANG (unset)
    SHELL=/usr/local/bin/zsh

[p5pRT]

From @khwilliamson

On Sun, 07 Apr 2019 09​:51​:11 -0700, randir wrote​:

This is a bug report for perl from sergey.aleynikov@​gmail.com,
generated with the help of perlbug 1.41 running under perl 5.29.9.

-----------------------------------------------------------------
[Please describe your issue here]

While fuzzing perl v5.29.9-63-g2496d8f3f7 built with afl and run
under libdislocator, I found the following program

for(0..6){0if split$p;($$v0l0e)&=($f,$v0l0e)=0;($e)=0}

to cause an assertion failure

perl​: ./inline.h​:182​: struct regexp *S_ReANY(const REGEXP *const)​:
Assertion `isREGEXP(re)' failed.

GDB stack trace is following

#0 __GI_raise (sig=sig@​entry=6) at
../sysdeps/unix/sysv/linux/raise.c​:50
#1 0x00007ffff7c25535 in __GI_abort () at abort.c​:79
#2 0x00007ffff7c2540f in __assert_fail_base (fmt=0x7ffff7d87ee0
"%s%s%s​:%u​: %s%sAssertion `%s' failed.\n%n", assertion=0x555555ac6c87
"isREGEXP(re)",
file=0x555555ac6bd0 "inline.h", line=182, function=<optimized
out> ) at assert.c​:92
#3 0x00007ffff7c330f2 in __GI___assert_fail (assertion=0x555555ac6c87
"isREGEXP(re)", file=0x555555ac6bd0 "inline.h", line=182,
function=0x555555acda70 <__PRETTY_FUNCTION__.17109> "S_ReANY") at
assert.c​:101
#4 0x0000555555821313 in S_ReANY (re=0x555555b68e88) at inline.h​:182
#5 0x0000555555822da6 in Perl_pp_regcomp () at pp_ctl.c​:106
#6 0x000055555570b89b in Perl_runops_debug () at dump.c​:2537
#7 0x00005555555ed560 in S_run_body (oldscope=1) at perl.c​:2716
#8 0x00005555555ecade in perl_run (my_perl=0x555555b4c260) at
perl.c​:2639
#9 0x00005555555a114e in main (argc=3, argv=0x7fffffffe1c8,
env=0x7fffffffe1e8) at perlmain.c​:127

While it looks like one from stack-not-refcounted pack, I don't see
how altering $$v0l0e and $f could affect $p. Bisected commit also
seems innocent (this is a blead-only failure)

commit 6ef7fe5
Author​: Karl Williamson <khw@​cpan.org>
Date​: Sun Mar 17 22​:11​:04 2019 -0600

PATCH​: [perl #131551] Too deep regex compilation recursion

This patch, started by Yves Orton, and refined in consultation with
Tony
Cook, imposes a maximum depth of unclosed left parentheses, at which
point it croaks. This is to prevent the segfault in the ticket.

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags​:
category=core
severity=medium
---
Site configuration information for perl 5.29.9​:

Configured by dur-randir at Wed Feb 27 14​:51​:01 MSK 2019.

Summary of my perl5 (revision 5 version 29 subversion 9)
configuration​:
Commit id​: c1e47ba
Platform​:
osname=darwin
osvers=13.4.0
archname=darwin-thread-multi-2level
uname='darwin isengard.local 13.4.0 darwin kernel version 13.4.0​:
mon jan 11 18​:17​:34 pst 2016; root​:xnu-2422.115.15~1release_x86_64
x86_64 '
config_args='-de -Dusedevel -DDEBUGGING -Dusethreads'
hint=recommended
useposix=true
d_sigaction=define
useithreads=define
usemultiplicity=define
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
bincompat5005=undef
Compiler​:
cc='cc'
ccflags ='-fno-common -DPERL_DARWIN -mmacosx-version-min=10.9
-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector
-I/usr/local/include -DPERL_USE_SAFE_PUTENV'
optimize='-O3 -g'
cppflags='-fno-common -DPERL_DARWIN -mmacosx-version-min=10.9
-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector
-I/usr/local/include'
ccversion=''
gccversion='4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)'
gccosandvers=''
intsize=4
longsize=8
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=3
ivtype='long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries​:
ld='cc'
ldflags =' -mmacosx-version-min=10.9 -fstack-protector
-L/usr/local/lib'
libpth=/usr/local/lib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.0/lib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib
/usr/lib
libs=-lpthread -lgdbm -ldbm -ldl -lm -lutil -lc
perllibs=-lpthread -ldl -lm -lutil -lc
libc=
so=dylib
useshrplib=false
libperl=libperl.a
gnulibc_version=''
Dynamic Linking​:
dlsrc=dl_dlopen.xs
dlext=bundle
d_dlsymun=undef
ccdlflags=' '
cccdlflags=' '
lddlflags=' -mmacosx-version-min=10.9 -bundle -undefined
dynamic_lookup -L/usr/local/lib -fstack-protector'

---
@​INC for perl 5.29.9​:
lib
/usr/local/lib/perl5/site_perl/5.29.9/darwin-thread-multi-2level
/usr/local/lib/perl5/site_perl/5.29.9
/usr/local/lib/perl5/5.29.9/darwin-thread-multi-2level
/usr/local/lib/perl5/5.29.9

---
Environment for perl 5.29.9​:
DYLD_LIBRARY_PATH (unset)
HOME=/Users/dur-randir
LANG=en_US.UTF-8
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/Users/dur-randir/perlbrew/bin​:/Users/dur-
randir/perlbrew/perls/perl-
5.22.1/bin​:/usr/local/bin​:/usr/local/sbin​:/usr/bin​:/bin​:/usr/sbin​:/sbin​:/usr/texbin
PERLBREW_HOME=/Users/dur-randir/.perlbrew
PERLBREW_MANPATH=/Users/dur-randir/perlbrew/perls/perl-5.22.1/man
PERLBREW_PATH=/Users/dur-randir/perlbrew/bin​:/Users/dur-
randir/perlbrew/perls/perl-5.22.1/bin
PERLBREW_PERL=perl-5.22.1
PERLBREW_ROOT=/Users/dur-randir/perlbrew
PERLBREW_SHELLRC_VERSION=0.84
PERLBREW_VERSION=0.84
PERL_BADLANG (unset)
SHELL=/usr/local/bin/zsh

I am unable to reproduce this on Linux, -O0, g++ 7.3
--
Karl Williamson

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @dur-randir

On Wed, 10 Apr 2019 15​:54​:24 -0700, khw wrote​:

I am unable to reproduce this on Linux, -O0, g++ 7.3

Here's a full -V from a -O0 perl that triggers this for me, built with gcc 8.3.0​:

Summary of my perl5 (revision 5 version 29 subversion 10) configuration​:
  Derived from​: d304e73
  Platform​:
  osname=linux
  osvers=4.19.0-2-amd64
  archname=x86_64-linux
  uname='linux dorothy 4.19.0-2-amd64 #1 smp debian 4.19.16-1 (2019-01-17) x86_64 gnulinux '
  config_args='-des -Dusedevel -Dcc=gcc -DDEBUGGING -Doptimize=-O0 -g -ggdb3'
  hint=previous
  useposix=true
  d_sigaction=define
  useithreads=undef
  usemultiplicity=undef
  use64bitint=define
  use64bitall=define
  uselongdouble=undef
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='gcc'
  ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
  optimize='-O0 -g -ggdb3'
  cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='8.3.0'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='off_t'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='gcc'
  ldflags =' -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/8/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/8/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/8/include-fixed /usr/include/x86_64-linux-gnu /usr/lib
  libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.28.so
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version='2.28'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags='-Wl,-E'
  cccdlflags='-fPIC'
  lddlflags='-shared -O0 -g -ggdb3 -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl)​:
  Compile-time options​:
  DEBUGGING
  HAS_TIMES
  PERLIO_LAYERS
  PERL_COPY_ON_WRITE
  PERL_DONT_CREATE_GVSV
  PERL_MALLOC_WRAP
  PERL_OP_PARENT
  PERL_PRESERVE_IVUV
  PERL_USE_DEVEL
  USE_64_BIT_ALL
  USE_64_BIT_INT
  USE_LARGE_FILES
  USE_LOCALE
  USE_LOCALE_COLLATE
  USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC
  USE_LOCALE_TIME
  USE_PERLIO
  USE_PERL_ATOF
  Locally applied patches​:
  uncommitted-changes
  Built under linux
  Compiled at Apr 8 2019 14​:39​:05
  %ENV​:
  PERLBREW_BASHRC_VERSION="0.78"
  PERLBREW_HOME="/home/afl/.perlbrew"
  PERLBREW_MANPATH="/home/afl/perlbrew/perls/perl-5.26.0-dbg/man"
  PERLBREW_PATH="/home/afl/perlbrew/bin​:/home/afl/perlbrew/perls/perl-5.26.0-dbg/bin"
  PERLBREW_PERL="perl-5.26.0-dbg"
  PERLBREW_ROOT="/home/afl/perlbrew"
  PERLBREW_VERSION="0.78"
  @​INC​:
  lib/
  /usr/local/lib/perl5/site_perl/5.29.10/x86_64-linux
  /usr/local/lib/perl5/site_perl/5.29.10
  /usr/local/lib/perl5/5.29.10/x86_64-linux
  /usr/local/lib/perl5/5.29.10

[p5pRT]

From @dur-randir

It also crashes for me with gcc 7.4.0

[p5pRT]

From @hvds

On Wed, 10 Apr 2019 15​:54​:24 -0700, khw wrote​:

I am unable to reproduce this on Linux, -O0, g++ 7.3

I can reproduce it here with gcc 4.8.4, both at the specified commit and current blead (a719c52).

I can simplify it a bit to​:
  ./miniperl -e 'for (0..5) { split undef; $$x &= ($y, $x) = 0 }'
.. but the loop count seems quite sensitive.

For this variant, looping (0..2) instead gives just a bad free​:
% ./miniperl -e 'for (0..2) { split undef; $$x &= ($y, $x) = 0 }'
Attempt to free unreferenced scalar​: SV 0x20c9c10.
%

I suspect these are all symptoms of corruption via stack refcounting on $x.

Interestingly, building with clang and address-sanitizer I get a completely different result​:
% ./miniperl -e 'for(0..6){0if split$p;($$v0l0e)&=($f,$v0l0e)=0;($e)=0}'
ASAN​:SIGSEGV

==21782==ERROR​: AddressSanitizer​: SEGV on unknown address 0x000000000000 (pc 0x0000012ba34a sp 0x7ffe8601b400 bp 0x7ffe8601b910 T0)
  #0 0x12ba349 in Perl_sv_setiv sv.c​:1658
  #1 0x12bc8a1 in Perl_sv_setuv sv.c​:1705
  #2 0x12bd050 in Perl_sv_setuv_mg sv.c​:1726
  #3 0x15f7677 in Perl_pp_bit_and pp.c​:2323
  #4 0xefc9fc in Perl_runops_debug dump.c​:2537
  #5 0x6f7ca0 in S_run_body perl.c​:2716
  #6 0x6f364a in perl_run perl.c​:2639
  #7 0x1f134f8 in main miniperlmain.c​:133
  #8 0x7fecc34b9f44 (/lib/x86_64-linux-gnu/libc.so.6+0x21f44)
  #9 0x499cec in _start (miniperl+0x499cec)

AddressSanitizer can not provide additional info.
SUMMARY​: AddressSanitizer​: SEGV /src/package/lang/perl/gitperl/sv.c​:1658 Perl_sv_setiv
==21782==ABORTING
%

Hugo

[p5pRT]

From @dur-randir

On Thu, 11 Apr 2019 05​:34​:20 -0700, hv wrote​:

I suspect these are all symptoms of corruption via stack refcounting
on $x.

Good, so it can be shoved under some table)

[p5pRT]

From @hvds

On Thu, 11 Apr 2019 06​:24​:51 -0700, randir wrote​:

On Thu, 11 Apr 2019 05​:34​:20 -0700, hv wrote​:

I suspect these are all symptoms of corruption via stack refcounting
on $x.

Good, so it can be shoved under some table)

Bad, because attempting to diagnose it properly would eat up time and give little back. I don't know for sure that's the issue, but suspecting it is already makes me reluctant to spend more time on it.

Hugo