java.lang.VerifyError when using MethodHandle constructs

[maartenc]

Hi,

we have some issues with obfuscated code that is using MethodHandle constructs.
After obfuscating this code, we get a java.lang.VerifyError when that code is used.
It's certainly not with all MethodHandle related code, only in some specific situation.

We were able to track it down to the following simplified version of our code:

package com;

import static java.lang.invoke.MethodHandles.publicLookup;
import static java.lang.invoke.MethodType.methodType;

import java.io.RandomAccessFile;
import java.lang.invoke.MethodHandle;
import java.lang.invoke.MethodHandles;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.channels.FileChannel;

public class Test {

  public static ByteBuffer foo(String path, boolean useArena) throws Throwable {
    try (RandomAccessFile randomAccessFile = new RandomAccessFile(path, "r")) {
      ByteBuffer buffer;
      if (useArena) {
        Class<?> arenaClass = Class.forName("java.lang.foreign.Arena");
        Class<?> memorySegmentClass = Class.forName("java.lang.foreign.MemorySegment");

        MethodHandles.Lookup lookup = publicLookup();
        MethodHandle handle1 = lookup.findStatic(arenaClass, "ofAuto", methodType(arenaClass));
        MethodHandle handle2 = lookup.findVirtual(FileChannel.class, "map", methodType(memorySegmentClass, FileChannel.MapMode.class, long.class, long.class, arenaClass));
        MethodHandle handle3 = lookup.findVirtual(memorySegmentClass, "asByteBuffer", methodType(ByteBuffer.class));

        Object arena = handle1.invoke();
        Object memorySegment = handle2.invoke(randomAccessFile.getChannel(), FileChannel.MapMode.READ_ONLY, 0L, randomAccessFile.length(), arena);
        buffer = (ByteBuffer) handle3.invoke(memorySegment);
      } else {
        buffer = randomAccessFile.getChannel().map(FileChannel.MapMode.READ_ONLY, 0, randomAccessFile.length());
      }
      buffer.order(ByteOrder.BIG_ENDIAN);

      return buffer;
    }
  }
}

We compile and obfuscate this code with a JDK17 and ProGuard 7.7.
When we call this method, we always get a java.lang.VerifyError. It doesn't matter of we specify the useArena boolean to 'true' or 'false'. Or if we run it with a Java 17 JRE or Java 22+ JRE.

Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
Exception Details:
  Location:
    com/Test.foo(Ljava/lang/String;Z)Ljava/nio/ByteBuffer; @164: invokevirtual
  Reason:
    Type 'java/lang/Object' (current frame, stack[0]) is not assignable to 'java/nio/ByteBuffer'
  Current Frame:
    bci: @164
    flags: { }
    locals: { top, top, 'java/io/RandomAccessFile', 'java/lang/Object' }
    stack: { 'java/lang/Object', 'java/nio/ByteOrder' }
  Bytecode:
    0000000: bb00 0859 2a12 06b7 0019 4d1b 9900 8412
    0000010: 02b8 001d 3a04 1203 b800 1d3a 05b8 0023
    0000020: 3a06 1906 1904 1205 1904 b800 26b6 0024
    0000030: 3a07 1906 1214 1204 1905 1215 06bd 0009
    0000040: 5903 b200 1653 5904 b200 1653 5905 1904
    0000050: 53b8 0027 b600 253a 0819 0619 0512 0112
    0000060: 12b8 0026 b600 253a 0919 07b6 0020 3a0a
    0000070: 1908 2cb6 001b b200 1809 2cb6 001c 190a
    0000080: b600 223a 0b19 0919 0bb6 0021 4ea7 0013
    0000090: 2cb6 001b b200 1809 2cb6 001c b600 294e
    00000a0: 2db2 0017 b600 2857 2d3a 042c b600 1a19
    00000b0: 04b0 4e2c b600 1aa7 000b 3a04 2d19 04b6
    00000c0: 001f 2dbf                              
  Exception Handler Table:
    bci [11, 171] => handler: 178
    bci [179, 183] => handler: 186
  Stackmap Table:
    full_frame(@144,{Top,Top,Object[#8]},{})
    append_frame(@160,Object[#11])
    full_frame(@178,{Top,Top,Object[#8]},{Object[#13]})
    full_frame(@186,{Top,Top,Top,Object[#13]},{Object[#13]})
    same_frame(@194)

If I remove the MethodHandle usage, there is no issue.
Any idea what's going wrong, or suggestion to avoid this?

Maarten

[piazzesiNiccolo-GS]

Possibly related to this issue