Skip to content

fix sql_mode parser modes#25438

Open
LeftHandCold wants to merge 5 commits into
matrixorigin:mainfrom
LeftHandCold:codex/fix-sql-mode-parser-modes
Open

fix sql_mode parser modes#25438
LeftHandCold wants to merge 5 commits into
matrixorigin:mainfrom
LeftHandCold:codex/fix-sql-mode-parser-modes

Conversation

@LeftHandCold

@LeftHandCold LeftHandCold commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

What type of PR is this?

  • API-change
  • BUG
  • Improvement
  • Documentation
  • Feature
  • Test and CI
  • Code Refactoring

Which issue(s) this PR fixes:

Fixes #25407

What this PR does / why we need it:

This wires parser-level sql_mode settings into the MySQL parser and scanner:

  • ANSI_QUOTES makes double-quoted text parse as identifiers.
  • PIPES_AS_CONCAT keeps || as concat; otherwise || parses as logical OR.
  • NO_BACKSLASH_ESCAPES keeps backslashes as ordinary string characters.
  • REAL_AS_FLOAT maps REAL columns to FLOAT instead of DOUBLE.

Root Cause

sql_mode accepted these mode names, but frontend parsing only passed SQL text and lower_case_table_names into the parser. The scanner and yacc actions therefore used fixed behavior regardless of session sql_mode.

@matrix-meow matrix-meow added the size/M Denotes a PR that changes [100,499] lines label Jul 3, 2026
@LeftHandCold LeftHandCold force-pushed the codex/fix-sql-mode-parser-modes branch from dffadb5 to c62ab47 Compare July 3, 2026 08:44
@LeftHandCold LeftHandCold marked this pull request as ready for review July 3, 2026 08:46
@qodo-code-review

Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@mergify mergify Bot added the kind/bug Something isn't working label Jul 3, 2026
@LeftHandCold LeftHandCold force-pushed the codex/fix-sql-mode-parser-modes branch from 57ca47a to 8fe438e Compare July 3, 2026 13:57

@XuPeng-SH XuPeng-SH left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes: the top-level query parser now honors session sql_mode, but the prepared-string path is still not wired through, so this PR does not close the functional loop for SQL that is prepared from a string or user variable.

Blocking issue:

pkg/frontend/mysql_cmd_executor.go:1315 still parses PrepareString.Sql with mysql.Parse(execCtx.reqCtx, st.Sql, v.(int64)), and pkg/sql/plan/build_dcl.go:77 re-parses the same prepared string with mysql.Parse(ctx.GetContext(), pstmt.Sql, v.(int64)). Both calls force empty parser SQL mode. That means cases like these still parse the inner SQL with the old/default behavior instead of the current session mode:

set sql_mode = 'PIPES_AS_CONCAT';
prepare s from "select 'a'||'b'";
execute s;

set sql_mode = 'ANSI_QUOTES';
prepare s from 'select "id" from "t_ansi"';

set sql_mode = 'NO_BACKSLASH_ESCAPES';
prepare s from "select hex('a\\nb')";

set sql_mode = 'REAL_AS_FLOAT';
prepare s from 'create table t_real_float (r real)';

The fix should use the same mode derivation added for normal parsing, i.e. resolve sql_mode, pass it through mysql.SessionSQLModeForParser, and call ParseWithSQLMode / ParseOneWithSQLMode in both the frontend pre-parse and the plan builder re-parse. Please add regression coverage for prepared-string and PREPARE ... FROM @var paths, not only direct parser tests.

Also worth fixing while touching this block: in stmt_kind.go, preStmt.Free() is deferred before checking the parse error. If parsing the prepared string fails, preStmt can be nil and the error path can panic. Move the defer after the err != nil check.

Local checks run:

  • make err-check
  • go test -count=1 -timeout 300s ./pkg/sql/parsers/dialect/mysql with local CGo/rpath setup
  • go test -count=1 -timeout 300s ./pkg/sql/parsers -run 'TestParse|TestSplitSqlBySemicolon' with local CGo/rpath setup

CI was still mostly pending when reviewed.

@XuPeng-SH XuPeng-SH left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for one blocker:

  • pkg/frontend/mysql_cmd_executor.go:2468-2511 currently applies sql_mode only once to the full incoming SQL packet before any statement executes.
  • MatrixOne enables CLIENT_MULTI_STATEMENTS by default (pkg/frontend/mysql_protocol.go:54-65), so a request like SET sql_mode=ANSI_QUOTES; SELECT "c"; or SET sql_mode=REAL_AS_FLOAT; CREATE TABLE t(r REAL); still parses the second statement under the old mode. That means session-scoped parser behavior is still wrong for later statements in the same request.
  • Suggested fix: parse multi-statement packets incrementally (or split/reparse after SET sql_mode) so a leading SET sql_mode can affect subsequent statements in that same packet.

I would also add a regression test for the exact same-request pattern above once the execution path is fixed.

@aunjgr aunjgr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approach is correct. Wiring sql_mode into the scanner is the right level. Minor suggestions:

  1. reparseAfterStaticSQLModeSet silently returns on fragment count mismatch — a debug log would help catch unexpected mismatches (e.g., CREATE PROCEDURE bodies with semicolons).

  2. fragmentHasStatement could be simplified: strings.TrimSpace(fragment) != "" is sufficient since the parser already validates on reparse. The scanner.Scan() call is redundant.

  3. defer mysql.PutScanner(scanner) in fragmentHasStatement stacks defers in multi-statement batches. Consider manually putting the scanner back to avoid defer accumulation.

…er-modes

# Conflicts:
#	pkg/sql/parsers/dialect/mysql/mysql_sql.go
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/bug Something isn't working size/L Denotes a PR that changes [500,999] lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: SQL_MODE parser modes ANSI_QUOTES/PIPES_AS_CONCAT/NO_BACKSLASH_ESCAPES/REAL_AS_FLOAT are not MySQL-compatible

6 participants