Remove unnecessary stage2 host builds from cross-compiled dist builders #145874

Kobzol · 2025-08-26T07:21:15Z

This is a repeated regression (#138004, #138123) that was reintroduced in #145472. I thought that we have a test for it, but alas, the "correct" test required --disable-docs. I added the test in this PR, and re-added the dist::Std build optimization that solves this.

r? @jieyouxu

jieyouxu

Oof, thanks

View changes since this review

jieyouxu · 2025-08-26T11:45:06Z

@bors r+ rollup=never

bors · 2025-08-26T11:45:10Z

📌 Commit c7f90c1 has been approved by jieyouxu

It is now in the queue for this repository.

Kobzol · 2025-08-26T11:45:52Z

This is causing some unnecessary builds also on dist builders with docs (because dist docs steps unnecessarily build stage 2 rustc). I'll fix that with a more general approach later, once this lands.

jieyouxu · 2025-08-26T11:46:53Z

Looking at the queue, since it's rather tame, let's bump this to p=5.

@bors p=5 (unnecessary builds)

bors · 2025-08-26T12:03:55Z

⌛ Testing commit c7f90c1 with merge 5ab6924...

hkBst · 2025-08-26T13:38:00Z

src/bootstrap/src/core/build_steps/dist.rs

+        // Normally, to build stage N libstd, we need stage N rustc.
+        // However, if we know that we will uplift libstd from stage 1 anyway, building the stage N
+        // rustc can be wasteful.
+        // In particular, if we do a cross-compiling dist stage 2 build from T1 to T2, we need:


Does 'T' stand for target? I suggest spelling it out whatever it is. Alternatives: machine, arch(itecture)

hkBst · 2025-08-26T13:39:18Z

src/bootstrap/src/core/builder/tests.rs

-    // Enable dist cranelift tarball by default with `x dist` if cranelift is enabled in
-    // `rust.codegen-backends`.
+    /// Simulates e.g. the powerpc64 builder, which is fully cross-compiled from x64, but it does
+    /// not build docs. Crutically, it shouldn't build host stage 2 rustc.


Crucially or Critically?

Haha :D Thanks, I'll fix it in a followup PR.

bors · 2025-08-26T15:10:13Z

☀️ Test successful - checks-actions
Approved by: jieyouxu
Pushing 5ab6924 to master...

github-actions · 2025-08-26T15:12:48Z

What is this?

This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing 4356e83 (parent) -> 5ab6924 (this PR)

Test differences

Show 7 test diffs

Stage 0

core::builder::tests::snapshot::dist_all_cross: pass -> [missing] (J0)
core::builder::tests::snapshot::dist_all_cross_extended: [missing] -> pass (J0)
core::builder::tests::snapshot::dist_all_cross_extended_no_docs: [missing] -> pass (J0)

Additionally, 4 doctest diffs were found. These are ignored, as they are noisy.

Job group index

J0: aarch64-apple, aarch64-gnu, aarch64-gnu-llvm-19-2, aarch64-msvc-2, i686-gnu-2, i686-gnu-nopt-2, i686-msvc-2, x86_64-gnu, x86_64-gnu-llvm-19-1, x86_64-gnu-llvm-19-3, x86_64-gnu-llvm-20-1, x86_64-gnu-llvm-20-3, x86_64-gnu-nopt, x86_64-gnu-stable, x86_64-mingw-2, x86_64-msvc-2

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \
    test-dashboard 5ab69249f36678c0a770a08d3d1b28a8103349ff --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

dist-apple-various: 5477.2s -> 4224.8s (-22.9%)
dist-ohos-x86_64: 5338.3s -> 4195.8s (-21.4%)
dist-ohos-armv7: 5023.5s -> 4045.7s (-19.5%)
dist-riscv64-linux: 5929.9s -> 4790.4s (-19.2%)
dist-s390x-linux: 6032.7s -> 4950.6s (-17.9%)
aarch64-apple: 5109.4s -> 6024.4s (17.9%)
dist-aarch64-windows-gnullvm: 5473.1s -> 4498.1s (-17.8%)
dist-armhf-linux: 5919.6s -> 4894.4s (-17.3%)
dist-loongarch64-musl: 6069.7s -> 5062.2s (-16.6%)
dist-powerpc-linux: 6006.4s -> 5014.1s (-16.5%)

How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

rust-timer · 2025-08-26T18:32:21Z

Finished benchmarking commit (5ab6924): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results (primary 2.3%, secondary -3.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	2.3%	[2.3%, 2.3%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-3.0%	[-3.9%, -1.9%]	4
All ❌✅ (primary)	2.3%	[2.3%, 2.3%]	1

Cycles

Results (primary -2.2%, secondary 2.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.9%	[2.1%, 3.4%]	5
Improvements ✅ (primary)	-2.2%	[-2.2%, -2.2%]	1
Improvements ✅ (secondary)	-2.2%	[-2.2%, -2.2%]	1
All ❌✅ (primary)	-2.2%	[-2.2%, -2.2%]	1

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 468.126s -> 466.377s (-0.37%)
Artifact size: 391.18 MiB -> 391.17 MiB (-0.00%)

Avoid more rustc rebuilds in cross-compilation scenarios This is a continuation of #145874. It adds a `compiler_for_std` function, which is a slimmed down version of `compiler_for`, which is much simpler, and designed to be used only for the standard library. The build, dist and doc steps somtimes work with a stage2 std for a given target. That currently requires building a stage2 host compiler. However, if we uplift the stage1 libstd anyway, that is wasteful, in particular when we are cross-compiling. The last two commits progressively make the stage 2 host rustc build avoidance more and more aggressive. I think that if we decide that it is fine to ship stage1 libstd everywhere, then it makes sense to go all the way. When we ship stuff, we always build it with the stage 1 compiler (e.g. we ship stage 2 rustc which is built with stage 1 rustc). Libstd is the only component where stage N is built with the stage N compiler. So I think that shipping stage 1 libstd is "enough", and we could thus optimize what gets built on CI. r? `@jieyouxu`

Avoid more rustc rebuilds in cross-compilation scenarios This is a continuation of rust-lang/rust#145874. It adds a `compiler_for_std` function, which is a slimmed down version of `compiler_for`, which is much simpler, and designed to be used only for the standard library. The build, dist and doc steps somtimes work with a stage2 std for a given target. That currently requires building a stage2 host compiler. However, if we uplift the stage1 libstd anyway, that is wasteful, in particular when we are cross-compiling. The last two commits progressively make the stage 2 host rustc build avoidance more and more aggressive. I think that if we decide that it is fine to ship stage1 libstd everywhere, then it makes sense to go all the way. When we ship stuff, we always build it with the stage 1 compiler (e.g. we ship stage 2 rustc which is built with stage 1 rustc). Libstd is the only component where stage N is built with the stage N compiler. So I think that shipping stage 1 libstd is "enough", and we could thus optimize what gets built on CI. r? `@jieyouxu`

Kobzol added 2 commits August 26, 2025 08:52

Add snapshot test for cross-compiled dist without docs

598387b

Do not unnecessarily build stage2 host rustc in some dist builders

c7f90c1

rustbot assigned jieyouxu Aug 26, 2025

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Aug 26, 2025

jieyouxu approved these changes Aug 26, 2025

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 26, 2025

hkBst reviewed Aug 26, 2025

View reviewed changes

bors added the merged-by-bors This PR was explicitly merged by bors. label Aug 26, 2025

bors merged commit 5ab6924 into rust-lang:master Aug 26, 2025
11 checks passed

rustbot added this to the 1.91.0 milestone Aug 26, 2025

Kobzol deleted the fix-dist-builds branch August 26, 2025 15:14

Kobzol mentioned this pull request Aug 26, 2025

Avoid more rustc rebuilds in cross-compilation scenarios #145902

Merged

Remove unnecessary stage2 host builds from cross-compiled dist builders #145874

Remove unnecessary stage2 host builds from cross-compiled dist builders #145874

Uh oh!

Conversation

Kobzol commented Aug 26, 2025

Uh oh!

jieyouxu left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jieyouxu commented Aug 26, 2025

Uh oh!

bors commented Aug 26, 2025

Uh oh!

Kobzol commented Aug 26, 2025

Uh oh!

jieyouxu commented Aug 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bors commented Aug 26, 2025

Uh oh!

hkBst Aug 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hkBst Aug 26, 2025

Choose a reason for hiding this comment

Uh oh!

Kobzol Aug 26, 2025

Choose a reason for hiding this comment

Uh oh!

bors commented Aug 26, 2025

Uh oh!

Uh oh!

github-actions bot commented Aug 26, 2025

Test differences

Stage 0

Job duration changes

Uh oh!

rust-timer commented Aug 26, 2025

Overall result: no relevant changes - no action needed

Instruction count

Max RSS (memory usage)

Cycles

Binary size

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

jieyouxu left a comment •

edited by rustbot

Loading

jieyouxu commented Aug 26, 2025 •

edited

Loading

hkBst Aug 26, 2025 •

edited

Loading