From 3e87302c89b2f4db1811f5ae376ca53d7628d827 Mon Sep 17 00:00:00 2001 From: Jieyou Xu Date: Fri, 5 Dec 2025 15:06:13 +0800 Subject: [PATCH] [DO NOT MERGE] Testing doc structure experiment --- src/SUMMARY.md | 112 +++++++++++++++++- src/__ci/how-to-use.md | 0 src/__ci/prologue.md | 0 .../explanations/build-pass-not-check-pass.md | 71 +++++++++++ src/__testing/explanations/intro.md | 0 src/__testing/how-to-guides/intro.md | 0 src/__testing/how-to-use.md | 0 src/__testing/prologue.md | 3 + src/__testing/reference/directives.md | 0 src/__testing/reference/intro.md | 0 src/__testing/tutorials/intro.md | 0 src/__tidy/how-to-use.md | 0 src/__tidy/prologue.md | 0 13 files changed, 183 insertions(+), 3 deletions(-) create mode 100644 src/__ci/how-to-use.md create mode 100644 src/__ci/prologue.md create mode 100644 src/__testing/explanations/build-pass-not-check-pass.md create mode 100644 src/__testing/explanations/intro.md create mode 100644 src/__testing/how-to-guides/intro.md create mode 100644 src/__testing/how-to-use.md create mode 100644 src/__testing/prologue.md create mode 100644 src/__testing/reference/directives.md create mode 100644 src/__testing/reference/intro.md create mode 100644 src/__testing/tutorials/intro.md create mode 100644 src/__tidy/how-to-use.md create mode 100644 src/__tidy/prologue.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index c136d3716..af02b427f 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -4,7 +4,7 @@ [About this guide](./about-this-guide.md) ---- + # Building and debugging `rustc` @@ -236,7 +236,113 @@ - [Sanitizers support](./sanitizers.md) - [Debugging support in the Rust compiler](./debugging-support-in-rustc.md) ---- +# Testing + +- [Prologue](./__testing/prologue.md) +- [How to use this section](./__testing/how-to-use.md) +- [Tutorials](./__testing/tutorials/intro.md) + - [Writing a `assembly-llvm` test]() + - [Writing a `codegen-llvm` test]() + - [Writing a `codegen-units` test]() + - [Writing a `coverage-run` test]() + - [Writing a `coverage-map` test]() + - [Writing a `crashes` test]() + - [Writing a `debuginfo` test]() + - [Writing a `mir-opt` test]() + - [Writing a `incremental` test]() + - [Writing a `pretty` test]() + - [Writing a `run-make` test]() + - [Writing a `run-make-cargo` test]() + - [Writing a `ui` test]() + - [Writing a `ui-fulldeps` test]() + - [Normalizing an unwanted string in a `ui` test]() + - [Only running a `ui` test under the nightly channel]() + - [Forcing a `rustc` to pretend to be stable in a `run-make` test]() +- [How-to guides](./__testing/how-to-guides/intro.md) +- [Reference](./__testing/reference/intro.md) + - [Local development]() + - [CI]() + - [compiletest]() + - [Interactions between bootstrap and compiletest]() + - [Interactions between rustc and compiletest]() + - [compiletest and staging]() + - [Test modes and test suites]() + - [The `assembly` test mode]() + - [The `assembly-llvm` test suite]() + - [The `codegen` test mode]() + - [The `codegen-llvm` test suite]() + - [The `codegen-units` test mode and test suite]() + - [The `crashes` test mode and test suite]() + - [The `coverage-map` and `coverage-run` test modes, and the `coverage` test suite]() + - [The `debuginfo` test mode and test suite]() + - [The `mir-opt` test mode and test suite]() + - [The `incremental` test mode and test suite]() + - [The `pretty` test mode and test suite]() + - [The `run-make` test mode]() + - [The `run-make` test suite]() + - [The `run-make-cargo` test suite]() + - [The `rustdoc` test mode]() + - [The `rustdoc-js` test suite]() + - [The `rustdoc-gui` test suite]() + - [The `rustdoc-json` test suite]() + - [The `ui` test mode]() + - [The `ui` test suite]() + - [The `ui-fulldeps` test suite]() + - [The `rustdoc-ui` test suite]() + - [Output snapshots]() + - [Error annotations]() + - [Normalizations]() + - [Controlling outcome expectations]() + - [Directives](./__testing/reference/directives.md) + - [run-make test infrastructure]() + - [run-make `rmake.rs` test recipes]() + - [`run-make-support` library]() + - [Remote testing and runners]() + - [Debuginfo test annotations]() + - [Pass modes]() + - [Compare modes]() + - [Clang-based tests]() + - [External dependencies]() + - [`nodejs`]() + - [`npm`]() + - [Debuggers]() + - [LLVM FileCheck]() + - [Auxiliary test support tools]() + - [`remote-test-client` and `remote-test-server`]() + - [jsondocck]() + - [jsondoclint]() + - [rustdoc-gui-tool]() + - [mir-opt-test-tool]() +- [Explanations](./__testing/explanations/intro.md) + - [Qualities of a good test]() + - [Writing good FileCheck assertions]() + - [Why does my `ui` test `//@ build-pass` but not `//@ check-pass`?](./__testing/explanations/build-pass-not-check-pass.md) +- [Frequently asked questions (FAQs)]() +- [Known problems and pain points]() + - [Debuginfo test infrastructure and test suite]() +- [Design notes]() +- [Additional notes and resources]() + - [Test case minimization]() + +# Tidy + +- [Prologue](./__tidy/prologue.md) +- [How to use this section](./__tidy/how-to-use.md) +- [Tutorials]() +- [How-to guides]() +- [Reference]() +- [Explanations]() + +# `rust-lang/rust` CI + +- [Prologue](./__ci/prologue.md) +- [How to use this section](./__ci/how-to-use.md) +- [Tutorials]() +- [How-to guides]() +- [Reference]() +- [Explanations]() + + [Appendix A: Background topics](./appendix/background.md) @@ -250,7 +356,7 @@ [Appendix Z: HumorRust](./appendix/humorust.md) ---- + [pau]: ./borrow_check/region_inference/placeholders_and_universes.md [opaque-infer]: ./borrow_check/opaque-types-region-inference-restrictions.md diff --git a/src/__ci/how-to-use.md b/src/__ci/how-to-use.md new file mode 100644 index 000000000..e69de29bb diff --git a/src/__ci/prologue.md b/src/__ci/prologue.md new file mode 100644 index 000000000..e69de29bb diff --git a/src/__testing/explanations/build-pass-not-check-pass.md b/src/__testing/explanations/build-pass-not-check-pass.md new file mode 100644 index 000000000..1bdb6bfb6 --- /dev/null +++ b/src/__testing/explanations/build-pass-not-check-pass.md @@ -0,0 +1,71 @@ +# Explanation: Why does my `ui` test `//@ build-pass` but not `//@ check-pass`? + +Alternative title: why is my `//@ build-pass` test failing in PR CI because no compiler errors are +emitted even though locally errors are in fact emitted? + +To understand this situation, we need to understand why this might happen. + +
+This particular explanation assumes that the root cause is due to PR CI forcing a check pass mode +(to be elaborated on in the following). Double-check the [*Differential +diagnosis*](#differential-diagnosis) section for some other possible root causes. +
+ + +## Background contexts + +### Compiler errors emitted past the `--emit=metadata` phase + +Compiler errors get emitted at different phases of a compilation session, so if certain phases +aren't reached, then errors that would only be emitted in those phases will not be emitted. + +The compiler `--emit=metadata` option (used on its own) skips encoding MIR for non-const functions +in the crate metadata among other things as check mode doesn't need those. So if a compiler error is +emitted only during the skipped encoding or later skipped codegen, only check-building a test source +with `--emit=metadata` will cause the compiler error to not be emitted. + +(A related option is `-Zno-codegen`, which only skips codegen.) + +### `//@ check-pass` vs `//@ build-pass` pass modes + +The key difference between the two pass modes are that `//@ check-pass` invokes only +`--emit=metadata`, whereas `//@ build-pass` will perform full crate metadata encoding and full +codegen. + +### Forcing a pass mode via `compiletest`'s `--pass=check` option, and certain PR CI jobs setting `--pass=check` + +The reason you only observe the failure in certain PR CI jobs, such as +[`x86_64-gnu-llvm-20`](https://github.com/rust-lang/rust/blob/864339abf952f07098dd82610256338520167d4a/src/ci/github-actions/jobs.yml#L125-L129) +is that selected PR CI jobs have special `./x test` invocations that force pass mode to `check` to +make sure that keeps working properly: + +```sh +../x.ps1 --stage 2 test tests/ui --pass=check --host='' --target=i686-unknown-linux-gnu + #^---------- + # forced `check` pass mode +``` + +See +[`x86_64-gnu-llvm.sh`](https://github.com/rust-lang/rust/blob/864339abf952f07098dd82610256338520167d4a/src/ci/docker/scripts/x86_64-gnu-llvm.sh#L20). + + +## Possible solutions + +### The `//@ ignore-pass` directive + +If it is *intentional* that the error is only emitted under `//@ build-pass`, then the test writer +can opt the test out of forcing pass modes by using the `//@ ignore-pass` directive. + +See also: the [*Directives*](../reference/directives.md) chapter. + + +## Differential diagnosis + +Note that there are other possible root causes, such as: + +- Semantic conflicts, e.g. the `HEAD` your branch was based on locally is outdated, and since then, + the remote `HEAD` contains commits that somehow causes your compiler error to be not emitted or + suppressed. In this case, rebase against latest `main`. +- Architecture or some other environmental conditions that might cause the error to not be emitted. + For instance, if the error is only emitted for 64-bit architectures, but the PR CI might be + explicitly testing against a 32-bit architecture. diff --git a/src/__testing/explanations/intro.md b/src/__testing/explanations/intro.md new file mode 100644 index 000000000..e69de29bb diff --git a/src/__testing/how-to-guides/intro.md b/src/__testing/how-to-guides/intro.md new file mode 100644 index 000000000..e69de29bb diff --git a/src/__testing/how-to-use.md b/src/__testing/how-to-use.md new file mode 100644 index 000000000..e69de29bb diff --git a/src/__testing/prologue.md b/src/__testing/prologue.md new file mode 100644 index 000000000..e2979f9b8 --- /dev/null +++ b/src/__testing/prologue.md @@ -0,0 +1,3 @@ +# Testing in `rust-lang/rust` + +TODO diff --git a/src/__testing/reference/directives.md b/src/__testing/reference/directives.md new file mode 100644 index 000000000..e69de29bb diff --git a/src/__testing/reference/intro.md b/src/__testing/reference/intro.md new file mode 100644 index 000000000..e69de29bb diff --git a/src/__testing/tutorials/intro.md b/src/__testing/tutorials/intro.md new file mode 100644 index 000000000..e69de29bb diff --git a/src/__tidy/how-to-use.md b/src/__tidy/how-to-use.md new file mode 100644 index 000000000..e69de29bb diff --git a/src/__tidy/prologue.md b/src/__tidy/prologue.md new file mode 100644 index 000000000..e69de29bb