Skip to content

Conversation

@andygrove
Copy link
Member

Which issue does this PR close?

Closes #.

Rationale for this change

Benchmark Size Before After Improvement
no_overflow_possible 1K 906 ns 60 ns 93.5% faster
no_overflow_possible 10K 7.26 µs 55 ns 99.2% faster
validation_no_overflow 1K 1.44 µs 1.26 µs 12.3% faster
validation_no_overflow 10K 11.79 µs 10.73 µs 9.3% faster
with_overflow_to_null 1K 925 ns 869 ns 6.3% faster
with_overflow_to_null 10K 7.39 µs 7.08 µs 4.5% faster

What changes are included in this PR?

  • no_overflow_possible (93-99% faster): When input precision ≤ target precision, we skip all validation and just update metadata - O(1) instead of O(n).
  • validation_no_overflow (9-12% faster): Uses the optimized is_valid_decimal_precision() that avoids error string allocation.
  • with_overflow_to_null (4-6% faster): Uses Arrow's optimized single-pass null_if_overflow_precision() method.

How are these changes tested?

@codecov-commenter
Copy link

codecov-commenter commented Dec 27, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 59.50%. Comparing base (f09f8af) to head (8b671d8).
⚠️ Report is 832 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #3001      +/-   ##
============================================
+ Coverage     56.12%   59.50%   +3.37%     
- Complexity      976     1381     +405     
============================================
  Files           119      167      +48     
  Lines         11743    15522    +3779     
  Branches       2251     2575     +324     
============================================
+ Hits           6591     9236    +2645     
- Misses         4012     4989     +977     
- Partials       1140     1297     +157     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@coderfender
Copy link
Contributor

Test failing :
test integral divide overflow for decimal

  == Results ==
  !== Correct Answer - 6 ==   == Spark Answer - 6 ==
   struct<(a div b):bigint>   struct<(a div b):bigint>
  ![null]                     [-1]
  ![null]                     [-1]
  ![null]                     [-1]
  ![null]                     [-1]
  ![null]                     [-4252079254050434654]
  ![null]                     [8770993997598117786] (QueryTest.scala:244)
 

@andygrove andygrove marked this pull request as draft December 28, 2025 16:14
@andygrove andygrove closed this Jan 8, 2026
@andygrove andygrove deleted the faster-checkoverflow branch January 8, 2026 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants