[Bug] Nereids async MV transparent rewrite returns rows outside range predicate with partition union compensation

[Coogle2017]

Version:
Apache Doris 4.0.2-rc02

Summary:
When Nereids transparent rewrite chooses an async materialized view with grace_period > 0, queries with a range predicate on the MV partition column can return rows outside the predicate. In one case the rewritten plan uses UNION compensation, but the base-table compensation branch scans partitions that are outside the query range. In another case the chosen MV scan reads all MV partitions, and the range predicate is not applied to the MV scan.

Anonymized schema:

CREATE TABLE db.fact_t (
  data_date INT,
  account_id BIGINT,
  asset DECIMAL(20, 4)
)
PARTITION BY RANGE(data_date) (...);

CREATE TABLE db.dim_account (
  account_id BIGINT,
  branch_no INT,
  branch_zone VARCHAR(128)
);

CREATE MATERIALIZED VIEW db.mv_fact_account_daily
(data_date, branch_no, branch_zone, cnt, asset)
BUILD DEFERRED REFRESH AUTO ON MANUAL
DUPLICATE KEY(data_date, branch_no, branch_zone)
PARTITION BY (data_date)
DISTRIBUTED BY RANDOM BUCKETS 24
PROPERTIES (
  "grace_period" = "2000",
  "excluded_trigger_tables" = "dim_account"
)
AS
SELECT
  f.data_date,
  d.branch_no,
  d.branch_zone,
  COUNT(1) AS cnt,
  SUM(f.asset) AS asset
FROM db.fact_t f
LEFT JOIN db.dim_account d
  ON f.account_id = d.account_id
GROUP BY f.data_date, d.branch_no, d.branch_zone;

**Reproduce query:**
SELECT
  data_date,
  branch_no,
  branch_zone,
  COUNT(1)
FROM db.fact_t f
LEFT JOIN db.dim_account d
  ON f.account_id = d.account_id
WHERE data_date > 20260227
GROUP BY data_date, branch_no, branch_zone
LIMIT 50;

**Expected:**
No result row should have data_date <= 20260227.

**Actual:**
The result contains rows such as data_date = 20250101.

**Important explain evidence:**
MaterializedViewRewriteSuccessAndChose:
  CBO.internal.db.mv_fact_account_daily chose

10:VUNION

MV branch:
  output: partial_sum(cnt)
  TABLE: db.mv_fact_account_daily
  partitions include p_20260228 and later partitions

Base-table compensation branch:
  TABLE: db.fact_t
  partitions include p20250101, p20250131, ..., p20260227
  
  _This means the union compensation branch scans partitions outside the query predicate data_date > 20260227._

**Additional evidence:**
After increasing grace_period, the same query is rewritten to scan the MV directly:
TABLE: db.mv_fact_account_daily
partitions=163/163
final projections: data_date, branch_no, branch_zone, cnt
MaterializedViewRewriteSuccessAndChose:
  CBO.internal.db.mv_fact_account_daily chose
  
The range predicate on data_date is not applied to the MV scan, so rows from old partitions are returned.

Workaround observed:
Setting the MV property "grace_period" = "0" makes the MV become not chose when it is not strictly consistent, and the query falls back to the base table with correct partition pruning and correct results.

Suspected code areas:

AbstractMaterializedViewRule#predicatesCompensate
PartitionCompensator#calcInvalidPartitions
StructInfo#addFilterOnTableScan
AbstractMaterializedViewAggregateRule#rewriteQueryByView
PruneOlapScanPartition / PartitionPruner.prunePredicate
The suspected issue is that the original range predicate on the partition column is either lost after partition pruning before MV rewrite, or incorrectly converted during async MV partition union compensation. The rewritten MV plan must preserve the query partition predicate or reject the rewrite.  

**Additional diagnosis from ChatGPT-assisted source analysis:**

Based on a local source-code reading of Doris 4.0.2-rc02, this issue appears to be related to the interaction between Nereids async materialized-view rewrite, partition pruning, and partition union compensation.

The suspected chain is:

1. The original query has a range predicate on the partition column, for example `data_date > 20260227`.
2. Before or during MV rewrite, this predicate may be consumed by partition pruning and represented mainly as selected base-table partitions.
3. During async MV rewrite, the rewritten MV scan is generated from the materialization context. If the original range predicate is no longer available as a normal `LogicalFilter`, predicate compensation may treat it as already satisfied or fail to re-apply it to the MV scan.
4. When `grace_period > 0`, stale or partially valid MV partitions are allowed to participate in rewrite. This can trigger either:
   - direct MV scan with all MV partitions selected, or
   - `UNION` compensation where the base-table compensation side scans partitions outside the original query range.
5. As a result, rows with `data_date <= 20260227` can appear in the final result, violating the query predicate.

Relevant code areas worth checking:

- `org.apache.doris.nereids.rules.exploration.mv.AbstractMaterializedViewRule`
- `org.apache.doris.nereids.rules.exploration.mv.AbstractMaterializedViewAggregateRule`
- `org.apache.doris.nereids.rules.exploration.mv.PartitionCompensator`
- `org.apache.doris.nereids.rules.exploration.mv.StructInfo`
- `org.apache.doris.nereids.rules.rewrite.PruneOlapScanPartition`
- `org.apache.doris.nereids.rules.expression.rules.PartitionPruner`

In particular, I would suggest verifying whether a range predicate on an MV partition column is always preserved or correctly reconstructed after MV rewrite, especially when `PartitionCompensator` decides to build a union between MV partitions and base-table partitions.

This diagnosis was produced with ChatGPT-assisted analysis of the local Doris 4.0.2-rc02 source code and should be treated as a suspected root-cause direction, not a confirmed patch-level conclusion.

[debug-query-01-sanitized.txt](https://github.com/user-attachments/files/28048159/debug-query-01-sanitized.txt)