Native android stacktraces not symbolicated in Flutter project using android gradle plugin

[jamesncl]

I have a flutter project (flutter 3.38.1) using sentry_flutter 9.9.0, sentry_dart_plugin 3.2.0 for uploading dart symbols, and Sentry Android Gradle Plugin 5.12.2 for uploading native android symbols.

Everything is working great except for deobfuscating crashes in native android code. Dart issues are correctly symbolicated, but native android is not. For example:

I'm trying to figure out why. I have added io.sentry.android.gradle to my top level settings.gradle.kts:

plugins {
    id("com.android.application") version "8.11.1" apply false
    ...
    id("io.sentry.android.gradle") version "5.12.2" apply false
}

and in my module-level build.gradle.kts with some configuration:

plugins {
    id("com.android.application")
    ...
    id("io.sentry.android.gradle")
}

...

sentry {
    debug.set(false)
    org.set("<REDACTED>")
    projectName.set("<REDACTED>")
    authToken.set(System.getenv("SENTRY_AUTH_TOKEN"))
    includeProguardMapping.set(true)
    autoUploadProguardMapping.set(true)
    dexguardEnabled.set(false)
    uploadNativeSymbols.set(true)
    autoUploadNativeSymbols.set(true)
    includeNativeSources.set(true)
    includeSourceContext.set(true)
    additionalSourceDirsForSourceContext.set(emptySet())

    tracingInstrumentation {
        enabled.set(false)
    }

    autoInstallation {
        enabled.set(false)
    }
    includeDependenciesReport.set(true)
    telemetry.set(false)
}

I have the following configuration in my Flutter app's pubspec.yaml:

sentry:
  upload_debug_symbols: true
  upload_source_maps: false
  upload_sources: true
  upload_source_context: true
  symbols_path: './build/app/outputs/symbols'
  dart_symbol_map_path: build/app/obfuscation.map.json
  project: '<REDACTED>'
  org: '<REDACTED>'

This is what I run to build my app:

flutter build appbundle --obfuscate --split-debug-info=build/app/outputs/symbols --extra-gen-snapshot-options=--save-obfuscation-map=build/app/obfuscation.map.json

This produces the following output:

console output

flutter build appbundle --obfuscate --split-debug-info=build/app/outputs/symbols --extra-gen-snapshot-options=--save-obfuscation-map=build/app/obfuscation.map.json
Resolving dependencies...
Downloading packages...
_fe_analyzer_shared 85.0.0 (92.0.0 available)
analyzer 7.7.1 (9.0.0 available)
animations 2.1.0 (2.1.1 available)

. ... snipped for brevity ...

win32_registry 1.1.5 (2.1.0 available)
Got dependencies!
76 packages have newer versions incompatible with dependency constraints.
Try flutter pub outdated for more information.
Variant 'debug', will keep symbols in binaries for:
'libunity.so'
'libil2cpp.so'
'libmain.so'
Variant 'release', symbols will be stripped from binaries.
Font asset "CupertinoIcons.ttf" was tree-shaken, reducing it from 257628 to 1752 bytes (99.3% reduction). Tree-shaking can be disabled by providing the --no-tree-shake-icons flag when building your app.
Font asset "MaterialIcons-Regular.otf" was tree-shaken, reducing it from 1645184 to 18020 bytes (98.9% reduction). Tree-shaking can be disabled by providing the --no-tree-shake-icons flag when building your app.

Found 8 debug information files
Resolved source code for 0 debug information files
Prepared debug information files for upload
Nothing to upload, all files are on the server
compressing mappings
uploading mappings
Uploaded a total of 0 new mapping files
Found 34 debug information files
Resolved source code for 4 debug information files
Prepared debug information files for upload
Uploaded 5 missing debug information files
File upload complete:

UPLOADED 0b3b25ac-f7d5-44f2-f900-e475569097a8 (mergeReleaseNativeLibs/out/lib/armeabi-v7a/libil2cpp.so; arm library)
UPLOADED edda1b4f-6c88-20c8-128a-5ff32ce813a9 (mergeReleaseNativeLibs/out/lib/arm64-v8a/libapp.so; arm64 library)
UPLOADED a1355e48-b376-6952-c9d1-2139354f4a84 (mergeReleaseNativeLibs/out/lib/arm64-v8a/libil2cpp.so; arm64 library)
UPLOADED 3dede911-a7bd-77e3-4ea9-e7ffd61e677e (mergeReleaseNativeLibs/out/lib/armeabi-v7a/libapp.so; arm library)
UPLOADED 00ce8f83-9f91-6887-cc74-84daffcbbf72 (mergeReleaseNativeLibs/out/lib/x86_64/libapp.so; x86_64 library)
Running Gradle task 'bundleRelease'... 126.4s
✓ Built build/app/outputs/bundle/release/app-release.aab (173.5MB)

I then run flutter packages pub run sentry_dart_plugin which results in the following output:

console output

flutter packages pub run sentry_dart_plugin
Deprecated. Use dart run instead.
[❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚] 0% reading config values..
Searching for pubspec.yaml or sentry.properties config...

Found config from pubspec.yaml

Sentry CLI binary checksum verification passed successfully (hash: d2e49a13bc253044dd8bc4bc822ca255ad8b3353488378e6ded1d5483cfad835).
☑ reading config values
☑ validating config values
[❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚] 18% uploading debug symbols..

Found 25 debug information files

Resolved source code for 3 debug information files

Prepared debug information files for upload

Uploaded 6 missing debug information files

File upload complete:

UPLOADED 3dede911-a7bd-77e3-4ea9-e7ffd61e677e (symbols/app.android-arm.symbols; arm debug companion)
UPLOADED 00ce8f83-9f91-6887-cc74-84daffcbbf72 (symbols/app.android-x64.symbols; x86_64 debug companion)
UPLOADED 00ce8f83-9f91-6887-cc74-84daffcbbf72 (symbols/app.android-x64.symbols; x86_64 sources)
UPLOADED edda1b4f-6c88-20c8-128a-5ff32ce813a9 (symbols/app.android-arm64.symbols; arm64 sources)
UPLOADED 3dede911-a7bd-77e3-4ea9-e7ffd61e677e (symbols/app.android-arm.symbols; arm sources)
UPLOADED edda1b4f-6c88-20c8-128a-5ff32ce813a9 (symbols/app.android-arm64.symbols; arm64 debug companion)

Found 174 debug information files (1 with embedded sources)

Resolved source code for 8 debug information files

Prepared debug information files for upload

Uploaded 3 missing debug information files

File upload complete:

UPLOADED 00ce8f83-9f91-6887-cc74-84daffcbbf72 (stripped_native_libs/release/stripReleaseDebugSymbols/out/lib/x86_64/libapp.so; x86_64 library)
UPLOADED 3dede911-a7bd-77e3-4ea9-e7ffd61e677e (stripped_native_libs/release/stripReleaseDebugSymbols/out/lib/armeabi-v7a/libapp.so; arm library)
UPLOADED edda1b4f-6c88-20c8-128a-5ff32ce813a9 (stripped_native_libs/release/stripReleaseDebugSymbols/out/lib/arm64-v8a/libapp.so; arm64 library)

Found 85 debug information files

Resolved source code for 37 debug information files

Prepared debug information files for upload

Uploaded 2 missing debug information files

File upload complete:

UPLOADED 3adcefbf-7bd5-34be-bad5-cbb9dd2a049b (Runner.xcarchive/dSYMs/App.framework.dSYM/Contents/Resources/DWARF/App; arm64 sources)
UPLOADED ec34821f-f6ba-3447-a154-90c7ccf7b8ee (Runner.xcarchive/dSYMs/UnityFramework.framework.dSYM/Contents/Resources/DWARF/UnityFramework; arm64 sources)

Found 0 debug information files
No debug information files found

Found 1 debug information file

Resolved source code for 1 debug information file

Prepared debug information files for upload

Nothing to upload, all files are on the server

Found 0 debug information files
No debug information files found

Found 1 debug information file

Resolved source code for 1 debug information file

Prepared debug information files for upload

Nothing to upload, all files are on the server

Found 1 debug information file

Resolved source code for 1 debug information file

Prepared debug information files for upload

Nothing to upload, all files are on the server

Found 1 debug information file

Resolved source code for 1 debug information file

Prepared debug information files for upload

Nothing to upload, all files are on the server

Found 0 debug information files
No debug information files found

Found 1 debug information file

Resolved source code for 1 debug information file

Prepared debug information files for upload

Nothing to upload, all files are on the server

Found 1 debug information file

Resolved source code for 1 debug information file

Prepared debug information files for upload

Nothing to upload, all files are on the server

Found 1 debug information file

Resolved source code for 1 debug information file

Prepared debug information files for upload

Nothing to upload, all files are on the server
[❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚❚] 18% uploading Dart symbol map(s)..
Uploading Dart symbol map '/Users//build/app/obfuscation.map.json' paired with '/Users//build/app/outputs/symbols/app.android-x64.symbols'

Uploaded 1 missing debug information file

File processing complete:

   OK 00ce8f83-9f91-6887-cc74-84daffcbbf72 (obfuscation.map.json; any)                                                                                         

Uploading Dart symbol map '/Users//build/app/obfuscation.map.json' paired with '/Users//build/app/outputs/symbols/app.android-arm64.symbols'

Uploaded 1 missing debug information file

File processing complete:

   OK edda1b4f-6c88-20c8-128a-5ff32ce813a9 (obfuscation.map.json; any)                                                                                         

Uploading Dart symbol map '/Users//build/app/obfuscation.map.json' paired with '/Users//build/app/outputs/symbols/app.android-arm.symbols'

Uploaded 1 missing debug information file

File processing complete:

   OK 3dede911-a7bd-77e3-4ea9-e7ffd61e677e (obfuscation.map.json; any)                                                                                         

Uploading Dart symbol map '/Users//build/app/obfuscation.map.json' paired with '/Users//build/ios/archive/Runner.xcarchive/dSYMs/App.framework.dSYM/Contents/Resources/DWARF/App'

Uploaded 1 missing debug information file

File processing complete:

   OK 3adcefbf-7bd5-34be-bad5-cbb9dd2a049b (obfuscation.map.json; any)                                                                                         

Dart symbol map upload summary: attempted=4, succeeded=4, failed=0
☑ uploading Dart symbol map(s)
☑ uploading debug symbols

Created release @5.18.3+2905230

uploadSourceMaps is disabled.

+---------------------------------+--------------+
| Repository | Revision |
+---------------------------------+--------------+
| | ac4dee93fcdb |
+---------------------------------+--------------+

Finalized release @5.18.3+2905230

I see a bunch of files in the 'debug information files' section of my project settings page on Sentry:

• obfuscation.map.json dartsymbolmap (4 of these)
• App SourceBundle (arm64)
• UnityFramework SourceBundle (arm64)
• WatchCompanion WatchKit Extension Mach-O executable (armv7k)
• WK Mach-O executable (arm64) (3 of these)
• libapp.so ELF dynamic library (arm64)
• libapp.so ELF dynamic library (arm)
• libapp.so ELF dynamic library (x86_64)
• libflutter.so ELF dynamic library (arm64)
• libflutter.so ELF dynamic library (arm)
• libflutter.so.sym ELF dynamic library (x86_64)
• libVkLayer_khronos_validation.so ELF dynamic library (arm64)
• librive_native.so ELF dynamic library (arm64)
• librive_native.so ELF dynamic library (x86)
• librive_native.so ELF dynamic library (arm)
• librive_native.so ELF dynamic library (x86_64)
• libdartjni.so ELF dynamic library (x86_64)
• libdartjni.so ELF dynamic library (arm)
• libdartjni.so ELF dynamic library (arm64)
• libdartjni.so ELF dynamic library (x86)
• lib_burst_generated.so ELF dynamic library (arm64)
• lib_burst_generated.so ELF dynamic library (arm)
• app.android-arm64.symbols SourceBundle (arm64)
• app.android-arm.symbols SourceBundle (arm)
• app.android-arm64.symbols ELF debug companion (arm64)
• app.android-x64.symbols SourceBundle (x86_64)
• app.android-x64.symbols ELF debug companion (x86_64)
• app.android-arm.symbols ELF debug companion (arm)
• libil2cpp.so ELF dynamic library (arm64)
• libil2cpp.so ELF dynamic library (arm)

What I'm trying to figure out is which ones are missing and how to fix it, but I don't have enough knowledge to understand what files are needed to deobfuscate native android issues.

One thing I did notice is that there is no file called mapping.txt in the debug information files list, which I believe is what ProGuard generates to symbolicate stack traces, is that right? I can see it in my build folder at build/app/outputs/mapping/release/mapping.txt but not in my list of debug information files on Sentry. Is this the problem? If yes what do I need to do to get the Android gradle plugin to upload it? If no what else can I do to try and figure this out?

Thanks so much

[linear]

DARTPL-34

[buenaflor]

@jamesncl hey can you share a link to the error at giancarlo.buenaflor@sentry.io

since it's unparsed it maybe means this error is thrown in a platform channel and propagated up to Flutter where we don't really have the capability to parse java/kotlin stacktraces (just a hunch from the bits I see)

[jamesncl]

Hi, yes you're absolutely right, I forgot that in some places I'm passing a stacktrace to flutter via a method channel and then sending it to Sentry - so it makes total sense why it's not being deobfuscated. Sorry for wasting your time!