[SPARK-56202][SS] Refactor streaming join tests: split Base/Suite hierarchy and simplify mode dispatch

[cloud-fan]

What changes were proposed in this pull request?

Refactor streaming join test suites to separate test mode dispatch from the test hierarchy.

Before: AlsoTestWithVirtualColumnFamilyJoins runs every test in both VCF and non-VCF
modes within a single suite class. V4 suites extend these classes and use a string-matching
skip list to exclude incompatible tests.

After: Each join type has a two-layer hierarchy:

StreamingInnerJoinBase          (universal tests — all state format versions)
├── StreamingInnerJoinSuite     (adds V3-specific tests: window joins, metrics, schema)
│   ├── StreamingInnerJoinWithVCFSuite      (testMode = WithVCF)
│   └── StreamingInnerJoinWithoutVCFSuite   (testMode = WithoutVCF)
└── StreamingInnerJoinV4Suite   (testMode = WithVCF, stateFormatVersion = 4)

Mode dispatch uses a testMode: Mode enum in StreamingJoinSuite, replacing the
AlsoTestWithVirtualColumnFamilyJoins trait. Each concrete suite overrides testMode
with a single Mode value. The enum is more future-proof than a boolean — new modes
can be added without changing the dispatch type. V4 suites extend *Base directly —
no skip list needed.

Why are the changes needed?

1. Failure isolation: Suite name identifies the exact mode — no long disambiguating suffixes.
2. Selective retrigger: Rerun only the failing mode's suite without rerunning the passing mode.
3. No skip lists: V4 suites get only universal tests by inheriting from *Base, not *Suite. Adding V3-specific tests in the future won't accidentally break V4.
4. Extensible dispatch: A Mode enum replaces a trait with hard-coded dual-mode dispatch. Adding new modes (e.g., Avro encoding) is a new enum case + a one-liner suite.

Does this PR introduce any user-facing change?

No. Test infrastructure only. Same tests, same coverage.

How was this patch tested?

Existing tests — the *WithVCFSuite and *WithoutVCFSuite classes run the same tests as before, split by mode.

Was this patch authored or co-authored using generative AI tooling?

Yes

[Copilot]

Pull request overview

Refactors Structured Streaming join test suites to separate “mode dispatch” (VCF vs non-VCF) from the test hierarchy, enabling single-mode suites for clearer failures and more targeted reruns.

Changes:

• Replaces the dual-mode AlsoTestWithVirtualColumnFamilyJoins dispatch with a boolean useVirtualColumnFamilies in StreamingJoinSuite.
• Splits join suites into *Base (universal tests) vs *Suite (V1–V3-only tests), and updates V4 suites to extend *Base directly.
• Introduces concrete single-mode suites (*WithVCFSuite / *WithoutVCFSuite) to run modes independently.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingJoinSuite.scala	Introduces boolean-based mode dispatch, splits Base vs V1–V3 suite layers, and adds concrete single-mode suites.
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingJoinV4Suite.scala	Updates V4 suites to inherit only universal tests and enforces VCF mode via the new dispatch mechanism.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[HeartSaVioR]

+1
The change looks very reasonable - thanks for the change!