Skip to content

Reject negative ln inputs#22276

Open
akmhatey-ai wants to merge 4 commits into
apache:mainfrom
akmhatey-ai:codex/ln-negative-input-error
Open

Reject negative ln inputs#22276
akmhatey-ai wants to merge 4 commits into
apache:mainfrom
akmhatey-ai:codex/ln-negative-input-error

Conversation

@akmhatey-ai

Copy link
Copy Markdown

Which issue does this PR close?

Rationale for this change

DataFusion currently returns NaN for negative ln inputs. PostgreSQL raises an error for the same case, and issue #22271 tracks aligning ln((-1.0)::float8) with that behavior.

This keeps the existing ln(0) behavior unchanged.

What changes are included in this PR?

  • Move ln from the shared unary math macro to a dedicated LnFunc implementation.
  • Return a compute error when ln receives a negative Float32 or Float64 value.
  • Update scalar.slt to cover positive column inputs plus negative scalar and column error cases.

Are these changes tested?

Yes. I ran:

  • cargo fmt --check -- datafusion/functions/src/math/mod.rs datafusion/functions/src/math/ln.rs
  • cargo check -p datafusion-functions
  • cargo test -p datafusion-functions math::ln --lib
  • cargo test -p datafusion-sqllogictest --test sqllogictests -- scalar:586
  • cargo test -p datafusion-sqllogictest --test sqllogictests -- scalar:593
  • cargo test -p datafusion-sqllogictest --test sqllogictests -- scalar:597
  • git diff --check

I also ran cargo test -p datafusion-sqllogictest --test sqllogictests -- scalar. The changed ln records passed, but the full file failed later at scalar.slt:1546+ because local aggregate_test_100 data was empty in this checkout; those failures were unrelated to this change.

Are there any user-facing changes?

Yes. ln now errors on negative inputs instead of returning NaN.

@github-actions github-actions Bot added sqllogictest SQL Logic Tests (.slt) functions Changes to functions implementation labels May 16, 2026

@kumarUjjawal kumarUjjawal left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we also need to look into other such log functions which also return NaN.

DataFusion CLI v53.1.0
> >   SELECT
    ln(2.0)    AS ln_pos,
    log10(100.0) AS log10_pos,
    log(-1.0)    AS log_neg,
    log2(-1.0)   AS log2_neg,
    log10(-1.0)  AS log10_neg;
+--------------------+-----------+---------+----------+-----------+
| ln_pos             | log10_pos | log_neg | log2_neg | log10_neg |
+--------------------+-----------+---------+----------+-----------+
| 0.6931471805599453 | 2.0       | NaN     | NaN      | NaN       |
+--------------------+-----------+---------+----------+-----------+
1 row(s) fetched.
Elapsed 0.004 seconds.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need a new file for the ln we can extend the already available macro for this.

@akmhatey-ai

Copy link
Copy Markdown
Author

Updated in 48152f0.

Covered the other negative-input cases from your example:

  • log now rejects negative values for float and decimal inputs.
  • log2 and log10 now use checked unary handling for negative Float32/Float64 inputs.
  • Invalid bases are unchanged and still follow the existing NaN behavior.

Validation run:

  • cargo test -p datafusion-functions test_log -- --nocapture
  • cargo test -p datafusion-sqllogictest --test sqllogictests -- scalar:640
  • cargo test -p datafusion-sqllogictest --test sqllogictests -- scalar:662
  • cargo test -p datafusion-sqllogictest --test sqllogictests -- scalar:711
  • cargo test -p datafusion-sqllogictest --test sqllogictests -- scalar:722
  • cargo test -p datafusion-sqllogictest --test sqllogictests -- scalar:750
  • cargo test -p datafusion-sqllogictest --test sqllogictests -- scalar:761
  • rustfmt --check datafusion/functions/src/math/log.rs datafusion/functions/src/macros.rs datafusion/functions/src/math/mod.rs
  • git diff --check
  • gitleaks detect --no-git on the changed files

One limitation: full cargo test -p datafusion-sqllogictest --test sqllogictests -- scalar still fails later on existing aggregate_test_100 expectations around lines 1551-1600, outside this log change.

@github-actions github-actions Bot removed the auto detected api change Auto detected API change label Jun 13, 2026

@kumarUjjawal kumarUjjawal left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also look into the power usage of log?

Float64Type,
Float64Type,
_,
>(&value, &base, checked_log)?,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This rewrite can skip the new error. log(-2.0, power(-2.0, 3.0)) should error because the log value is negative, but this can simplify to 3.0. Please only rewrite when the log domain is proven valid.

}

/// Returns true if the function is `PowerFunc`
fn is_pow(func: &ScalarUDF) -> bool {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This rewrite can skip the new error. log(-2.0, -2.0) should error, but this can simplify to 1. Please keep the original expression unless the value and base are proven valid

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PostgreSQL compatibility: ln(-1.0::float8) should error, not return NaN

2 participants