Skip to content

Commit d7cdd18

Browse files
committed
[DO NOT MERGE] Testing doc structure experiment
1 parent 0afee7b commit d7cdd18

File tree

13 files changed

+180
-0
lines changed

13 files changed

+180
-0
lines changed

src/SUMMARY.md

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -236,6 +236,112 @@
236236
- [Sanitizers support](./sanitizers.md)
237237
- [Debugging support in the Rust compiler](./debugging-support-in-rustc.md)
238238

239+
# Testing
240+
241+
- [Prologue](./__testing/prologue.md)
242+
- [How to use this section](./__testing/how-to-use.md)
243+
- [Tutorials](./__testing/tutorials/intro.md)
244+
- [Writing a `assembly-llvm` test]()
245+
- [Writing a `codegen-llvm` test]()
246+
- [Writing a `codegen-units` test]()
247+
- [Writing a `coverage-run` test]()
248+
- [Writing a `coverage-map` test]()
249+
- [Writing a `crashes` test]()
250+
- [Writing a `debuginfo` test]()
251+
- [Writing a `mir-opt` test]()
252+
- [Writing a `incremental` test]()
253+
- [Writing a `pretty` test]()
254+
- [Writing a `run-make` test]()
255+
- [Writing a `run-make-cargo` test]()
256+
- [Writing a `ui` test]()
257+
- [Writing a `ui-fulldeps` test]()
258+
- [Normalizing an unwanted string in a `ui` test]()
259+
- [Only running a `ui` test under the nightly channel]()
260+
- [Forcing a `rustc` to pretend to be stable in a `run-make` test]()
261+
- [How-to guides](./__testing/how-to-guides/intro.md)
262+
- [Reference](./__testing/reference/intro.md)
263+
- [Local development]()
264+
- [CI]()
265+
- [compiletest]()
266+
- [Interactions between bootstrap and compiletest]()
267+
- [Interactions between rustc and compiletest]()
268+
- [compiletest and staging]()
269+
- [Test modes and test suites]()
270+
- [The `assembly` test mode]()
271+
- [The `assembly-llvm` test suite]()
272+
- [The `codegen` test mode]()
273+
- [The `codegen-llvm` test suite]()
274+
- [The `codegen-units` test mode and test suite]()
275+
- [The `crashes` test mode and test suite]()
276+
- [The `coverage-map` and `coverage-run` test modes, and the `coverage` test suite]()
277+
- [The `debuginfo` test mode and test suite]()
278+
- [The `mir-opt` test mode and test suite]()
279+
- [The `incremental` test mode and test suite]()
280+
- [The `pretty` test mode and test suite]()
281+
- [The `run-make` test mode]()
282+
- [The `run-make` test suite]()
283+
- [The `run-make-cargo` test suite]()
284+
- [The `rustdoc` test mode]()
285+
- [The `rustdoc-js` test suite]()
286+
- [The `rustdoc-gui` test suite]()
287+
- [The `rustdoc-json` test suite]()
288+
- [The `ui` test mode]()
289+
- [The `ui` test suite]()
290+
- [The `ui-fulldeps` test suite]()
291+
- [The `rustdoc-ui` test suite]()
292+
- [Output snapshots]()
293+
- [Error annotations]()
294+
- [Normalizations]()
295+
- [Controlling outcome expectations]()
296+
- [Directives]()
297+
- [run-make test infrastructure]()
298+
- [run-make `rmake.rs` test recipes]()
299+
- [`run-make-support` library]()
300+
- [Remote testing and runners]()
301+
- [Debuginfo test annotations]()
302+
- [Pass modes]()
303+
- [Compare modes]()
304+
- [Clang-based tests]()
305+
- [External dependencies]()
306+
- [`nodejs`]()
307+
- [`npm`]()
308+
- [Debuggers]()
309+
- [LLVM FileCheck]()
310+
- [Auxiliary test support tools]()
311+
- [`remote-test-client` and `remote-test-server`]()
312+
- [jsondocck]()
313+
- [jsondoclint]()
314+
- [rustdoc-gui-tool]()
315+
- [mir-opt-test-tool]()
316+
- [Explanations](./__testing/explanations/intro.md)
317+
- [Qualities of a good test]()
318+
- [Writing good FileCheck assertions]()
319+
- [Why does my `ui` test `//@ build-pass` but not `//@ check-pass`?](./__testing/explanations/build-pass-not-check-pass.md)
320+
- [Frequently asked questions (FAQs)]()
321+
- [Known problems and pain points]()
322+
- [Debuginfo test infrastructure and test suite]()
323+
- [Design notes]()
324+
- [Additional notes and resources]()
325+
- [Test case minimization]()
326+
327+
# Tidy
328+
329+
- [Prologue](./__tidy/prologue.md)
330+
- [How to use this section](./__tidy/how-to-use.md)
331+
- [Tutorials]()
332+
- [How-to guides]()
333+
- [Reference]()
334+
- [Explanations]()
335+
336+
# `rust-lang/rust` CI
337+
338+
- [Prologue](./__ci/prologue.md)
339+
- [How to use this section](./__ci/how-to-use.md)
340+
- [Tutorials]()
341+
- [How-to guides]()
342+
- [Reference]()
343+
- [Explanations]()
344+
239345
---
240346

241347
[Appendix A: Background topics](./appendix/background.md)

src/__ci/how-to-use.md

Whitespace-only changes.

src/__ci/prologue.md

Whitespace-only changes.
Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
# Explanation: Why does my `ui` test `//@ build-pass` but not `//@ check-pass`?
2+
3+
Alternative title: why is my `//@ build-pass` test failing in PR CI because no compiler errors are
4+
emitted even though locally errors are in fact emitted?
5+
6+
To understand this situation, we need to understand why this might happen.
7+
8+
<div class="warning">
9+
This particular explanation assumes that the root cause is due to PR CI forcing a check pass mode
10+
(to be elaborated on in the following). Double-check the [*Differential
11+
diagnosis*](#differential-diagnosis) section for some other possible root causes.
12+
</div>
13+
14+
15+
## Background contexts
16+
17+
### Compiler errors emitted past the `--emit=metadata` phase
18+
19+
Compiler errors get emitted at different phases of a compilation session, so if certain phases
20+
aren't reached, then errors that would only be emitted in those phases will not be emitted.
21+
22+
The compiler `--emit=metadata` option (used on its own) skips encoding MIR for non-const functions
23+
in the crate metadata among other things as check mode doesn't need those. So if a compiler error is
24+
emitted only during the skipped encoding or later skipped codegen, only check-building a test source
25+
with `--emit=metadata` will cause the compiler error to not be emitted.
26+
27+
(A related option is `-Zno-codegen`, which only skips codegen.)
28+
29+
### `//@ check-pass` vs `//@ build-pass` pass modes
30+
31+
The key difference between the two pass modes are that `//@ check-pass` invokes only
32+
`--emit=metadata`, whereas `//@ build-pass` will perform full crate metadata encoding and full
33+
codegen.
34+
35+
### Forcing a pass mode via `compiletest`'s `--pass=check` option, and certain PR CI jobs setting `--pass=check`
36+
37+
The reason you only observe the failure in certain PR CI jobs, such as
38+
[`x86_64-gnu-llvm-20`](https://github.com/rust-lang/rust/blob/864339abf952f07098dd82610256338520167d4a/src/ci/github-actions/jobs.yml#L125-L129)
39+
is that selected PR CI jobs have special `./x test` invocations that force pass mode to `check` to
40+
make sure that keeps working properly:
41+
42+
```sh
43+
../x.ps1 --stage 2 test tests/ui --pass=check --host='' --target=i686-unknown-linux-gnu
44+
#^----------
45+
# forced `check` pass mode
46+
```
47+
48+
See
49+
[`x86_64-gnu-llvm.sh`](https://github.com/rust-lang/rust/blob/864339abf952f07098dd82610256338520167d4a/src/ci/docker/scripts/x86_64-gnu-llvm.sh#L20).
50+
51+
52+
## Possible solutions
53+
54+
### The `//@ ignore-pass` directive
55+
56+
If it is *intentional* that the error is only emitted under `//@ build-pass`, then the test writer
57+
can opt the test out of forcing pass modes by using the `//@ ignore-pass` directive.
58+
59+
See also: the [*Directives*](../reference/directives.md) chapter.
60+
61+
62+
## Differential diagnosis
63+
64+
Note that there are other possible root causes, such as:
65+
66+
- Semantic conflicts, e.g. the `HEAD` your branch was based on locally is outdated, and since then,
67+
the remote `HEAD` contains commits that somehow causes your compiler error to be not emitted or
68+
suppressed. In this case, rebase against latest `main`.
69+
- Architecture or some other environmental conditions that might cause the error to not be emitted.
70+
For instance, if the error is only emitted for 64-bit architectures, but the PR CI might be
71+
explicitly testing against a 32-bit architecture.

src/__testing/explanations/intro.md

Whitespace-only changes.

src/__testing/how-to-guides/intro.md

Whitespace-only changes.

src/__testing/how-to-use.md

Whitespace-only changes.

src/__testing/prologue.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Testing in `rust-lang/rust`
2+
3+
TODO

src/__testing/reference/directives.md

Whitespace-only changes.

src/__testing/reference/intro.md

Whitespace-only changes.

0 commit comments

Comments
 (0)