Fix next_up and next_down behavior for zero float values

[liamzwbao]

Which issue does this PR close?

• Closes Filtering and counting afterwards causes overflow panic in interval arithmetics #16736.

Rationale for this change

The root cause mentioned in the linked issue is caused by an invalid interval produced by satisfy_greater. For instance, calling satisfy_greater([0.0, 0.0], [-0.0, any], true) yields [0.0, 0.0], [-0.0, -ε] instead of the expected [0.0, 0.0], [-0.0, -0.0].

What changes are included in this PR?

As @berkaysynnada pointed out, the correct fix is to update next_up and next_down in rounding.rs, ensuring that next_up(-0.0) returns 0.0 and next_down(0.0) returns -0.0.

Are these changes tested?

Yes

Are there any user-facing changes?

No

[ozankabak]

Thanks for taking a look at this.

A cursory look suggests when a strict inequality is being propagated, if the next value of other side's lower bound is greater than the upper bound, the propagation result should be "infeasible".

@berkaysynnada will help with this issue

[berkaysynnada]

Currently, when computing the next representable float from +0.0 or -0.0, the behavior incorrectly skips directly to the smallest subnormal (±ε) instead of transitioning between -0.0 and +0.0. For example, next_down(+0.0) returns -ε, but we expect it to return -0.0. Similarly, next_up(-0.0) returns +ε, but we expect it to return +0.0.

This causes intervals like [-0.0, -ε] instead of the expected [-0.0, -0.0]. In ScalarValue comparisons we already treat -0.0 and +0.0 as NOT equal, but the rounding logic was skipping over them and jumping directly to subnormals.

To fix this, I locally updated next_up and next_down to handle ±0.0 explicitly. In next_up, if the input is -0.0, it now returns +0.0 instead of +ε. In next_down, if the input is +0.0, it now returns -0.0 instead of -ε. All other cases remain as they were. This keeps the fix localized to the specific ±0.0 boundary without unnecessarily affecting the general behavior of interval arithmetic logic.

pub fn next_up<F: FloatBits + Copy>(float: F) -> F {
    let bits = float.to_bits();
    if float.float_is_nan() || bits == F::infinity().to_bits() {
        return float;
    }

    // Special case: -0.0 → +0.0
    if bits == F::NEG_ZERO {
        return F::from_bits(F::ZERO);
    }
    ...

pub fn next_down<F: FloatBits + Copy>(float: F) -> F {
    let bits = float.to_bits();
    if float.float_is_nan() || bits == F::neg_infinity().to_bits() {
        return float;
    }
    
    // Special case: +0.0 → -0.0
    if bits == F::ZERO {
        return F::from_bits(F::NEG_ZERO);
    }
    ...

With these changes, the interval calculations now respect the special ±0.0 representations before moving into the subnormal range. This aligns the rounding behavior with how ScalarValue comparisons already work and avoids producing unexpected intervals.

[berkaysynnada]

BTW, maybe we should modify how floating point ScalarValue's are compared (

datafusion/datafusion/common/src/scalar/mod.rs

Line 484 in ce3f62a

(Some(f1), Some(f2)) => Some(f1.total_cmp(f2)),

). IDK why are we following total order convention, maybe @comphead have an idea, who is the original author of that change

[liamzwbao]

Thank you for pointing that out, @berkaysynnada and @ozankabak! The fix in next_up and next_down makes more sense.

As for the total_cmp changes, that was originally raised by @comphead.

[berkaysynnada]

Thank you for pointing that out, @berkaysynnada and @ozankabak! The fix in next_up and next_down makes more sense.

As for the total_cmp changes, that was originally raised by @comphead.

If this issue is not urgent for you, let's wait for a few days. This is not a trivial change, and we need to consider all consequences of the given decision. I need to do some readings and investigate how other engine/platforms behave.