Skip to content

fix: BETWEEN optimization now triggers by comparing expr kind, not span#5738

Open
queelius wants to merge 1 commit intoPRQL:mainfrom
queelius:fix/between-optimization
Open

fix: BETWEEN optimization now triggers by comparing expr kind, not span#5738
queelius wants to merge 1 commit intoPRQL:mainfrom
queelius:fix/between-optimization

Conversation

@queelius
Copy link
Copy Markdown
Contributor

Summary

  • Fixes the try_into_between optimization which was dead code since it compared rq::Expr values including their span field
  • Changes comparison from a_l == b_l (full Expr including span) to a_l.kind == b_l.kind (semantic content only)
  • Adds regression test verifying BETWEEN is now generated

Fixes #5737

Before

from t | filter (a >= 5 && a <= 10)

Generated:

SELECT * FROM t WHERE a >= 5 AND a <= 10

After

SELECT * FROM t WHERE a BETWEEN 5 AND 10

Root Cause

rq::Expr derives PartialEq which includes the span field. The same column a referenced at two different source positions has the same CId but different Span values, so the equality check a_l == b_l always failed.

Test plan

  • Added test_between_optimization regression test
  • All 450 prqlc tests pass (including new test)
  • Change is a one-line fix in the comparison logic

Generated with Claude Code

Copilot AI review requested due to automatic review settings March 25, 2026 08:57
prql-bot
prql-bot previously approved these changes Mar 25, 2026
Copy link
Copy Markdown
Collaborator

@prql-bot prql-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both fixes are correct and well-tested.

BETWEEN optimization: The root cause is right — rq::Expr derives PartialEq over both kind and span, so the same column at two source positions always compared unequal. Comparing .kind only is the correct fix.

SQLite div_i: The old formula ROUND(ABS(l/r) - 0.5) fails when |l| < |r| because SQLite integer division yields 0, then ROUND(0 - 0.5) = -1 (SQLite rounds .5 away from zero). The new CAST(ABS(l * 1.0 / r) AS INTEGER) formula correctly truncates toward zero.

Note: PR #5736 (also by @queelius) contains only the SQLite div_i fix from this PR. One of the two should be closed to avoid merge conflicts.

@prql-bot prql-bot dismissed their stale review March 25, 2026 09:00

CI failure: PR title needs a conventional commit prefix (e.g. 'fix: ...'). Also measure-code-cov failed — investigating.

Copy link
Copy Markdown
Collaborator

@prql-bot prql-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI issues:

  1. Validate PR title: The title needs a conventional commit prefix. Should be: fix: BETWEEN optimization never triggering due to span comparison

  2. measure-code-cov: cargo llvm-cov failed — this may be a flake or may be related to the snapshot change in arithmetic.snap (the removed snapshot_kind: text line). Worth checking if the snapshot file format is valid.

  3. Overlap with #5736: That PR contains the same SQLite div_i fix. Consider closing #5736 if this PR is intended to supersede it, or removing the SQLite changes from this PR to keep the two fixes independent.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a missed SQL codegen optimization where a >= low && a <= high should compile to a BETWEEN low AND high, and also updates SQLite integer-division SQL generation (with associated snapshot updates).

Changes:

  • Adjust try_into_between to compare expression semantic content rather than full rq::Expr equality (which includes spans).
  • Add regression coverage ensuring the BETWEEN optimization triggers.
  • Update SQLite div_i SQL template and refresh affected integration snapshots.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
prqlc/prqlc/src/sql/gen_expr.rs Updates BETWEEN optimization matching logic.
prqlc/prqlc/tests/integration/sql.rs Adds regression test for BETWEEN output (and a SQLite // SQL regression).
prqlc/prqlc/src/sql/std.sql.prql Changes SQLite integer-division (div_i) SQL template.
prqlc/prqlc/tests/integration/snapshots/integration__queries__compileall__arithmetic.snap Updates expected SQL snapshots impacted by division template changes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

if a_l == b_l {
// Compare only `kind` (not `span`) since the same column
// referenced at two source positions has different spans.
if a_l.kind == b_l.kind {
Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if a_l.kind == b_l.kind moves the kind field out of a_l/b_l, so a_l can’t be passed to translate_operand(a_l, ...) afterward (partial move). Compare by reference instead (e.g., borrow the fields) or compare the underlying CId directly for ColumnRef to avoid moving the expressions.

Suggested change
if a_l.kind == b_l.kind {
if &a_l.kind == &b_l.kind {

Copilot uses AI. Check for mistakes.
Comment on lines 419 to 423
let div_f = l r -> s"({l} * 1.0 / {r:12})"

@{binding_strength=100}
let div_i = l r -> s"ROUND(ABS({l:11} / {r:12}) - 0.5) * SIGN({l:0}) * SIGN({r:0})"
let div_i = l r -> s"CAST(ABS({l:11} * 1.0 / {r:12}) AS INTEGER) * SIGN({l:0}) * SIGN({r:0})"

Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR introduces SQLite integer-division behavior changes (sqlite.div_i) and updates a large existing snapshot accordingly, but the PR title/description focuses on the BETWEEN optimization. Consider splitting these changes into a separate PR or updating the PR description/title to cover the SQLite // fix and snapshot churn so reviewers understand the full scope.

Copilot uses AI. Check for mistakes.
The try_into_between function compared rq::Expr values including their
span field. Since the same column referenced at two source positions
has different spans, the equality check always failed, making the
BETWEEN optimization dead code.

Fix by comparing only the kind field (a_l.kind == b_l.kind) instead of
the full Expr (a_l == b_l).

Fixes PRQL#5737

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@queelius queelius force-pushed the fix/between-optimization branch from 26010f5 to ef12bb0 Compare March 25, 2026 13:34
@queelius queelius changed the title Fix BETWEEN optimization never triggering due to span comparison fix: BETWEEN optimization now triggers by comparing expr kind, not span Mar 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BETWEEN optimization in SQL codegen never triggers due to span comparison

3 participants