Make cargo test work for bootstrap self test #146595

Shourya742 · 2025-09-15T14:07:32Z

This PR enables the bootstrap self-test to run via cargo test. I have removed the detect_src_and_out test for now, but it will be reintroduced in a follow-up PR where all bootstrap tests will be migrated to use testCtx.

r? @Kobzol

try-job: aarch64-apple

rustbot · 2025-09-15T14:07:35Z

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

Kobzol · 2025-09-15T14:19:09Z

It's really cool being able to debug bootstrap tests 🤩 However, the source directory is mounted as read-only on CI, I think that either we'll have to port more tests over to TestCtx first, or keep the old test hacks.

Kobzol

Thank you, this is a great cleanup. Left some comments.

View changes since this review

src/bootstrap/src/utils/tests/mod.rs

src/bootstrap/src/utils/helpers.rs

src/bootstrap/src/core/download.rs

src/bootstrap/src/core/config/config.rs

src/bootstrap/src/core/config/tests.rs

src/bootstrap/src/core/config/toml/mod.rs

…ia using the temp directory created via the testCtx

…ride

…n for current

…ssigned to CARGO_MANIFEST_DIR, so the config actually use the <src>/bootstrap.toml and the /tmp/bootstrap.toml

rustbot · 2025-09-21T15:29:29Z

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

src/bootstrap/src/core/config/config.rs

src/bootstrap/src/utils/tests/mod.rs

Kobzol

Thanks, that's pretty much how I imagined it.

View changes since this review

src/bootstrap/src/core/config/config.rs

src/bootstrap/src/core/builder/tests.rs

src/bootstrap/src/core/config/tests.rs

Kobzol · 2025-09-25T12:57:09Z

@bors2 delegate=try

rust-bors · 2025-09-25T12:57:12Z

✌️ @Shourya742, you can now perform try builds on this pull request!

You can now post @bors2 try to start a try build.

Kobzol · 2025-09-25T13:07:58Z

@bors try

…, r=<try> Make cargo test work for bootstrap self test try-job: aarch64-apple

rust-bors · 2025-09-25T15:07:58Z

☀️ Try build successful (CI)
Build commit: fb8a940 (fb8a940b60076e6c217f88f7416f8f71d58d1302, parent: bbcbc7818ba27c951f9fd881fd3ec4cd5123b741)

Kobzol · 2025-09-25T15:32:37Z

@bors try

…, r=<try> Make cargo test work for bootstrap self test try-job: aarch64-apple

Kobzol · 2025-09-25T16:54:06Z

@bors r+

Let's try again :)

bors · 2025-09-25T16:54:09Z

📌 Commit 191c7ed has been approved by Kobzol

It is now in the queue for this repository.

rust-bors · 2025-09-25T17:09:31Z

☀️ Try build successful (CI)
Build commit: 88604c7 (88604c718431b44109a5586a67c0b39b50df56d9, parent: 7cfd7d328b14b936c7ffede92cacebe8557c6388)

bors · 2025-09-26T12:55:19Z

⌛ Testing commit 191c7ed with merge a885811...

bors · 2025-09-26T16:08:31Z

☀️ Test successful - checks-actions
Approved by: Kobzol
Pushing a885811 to master...

github-actions · 2025-09-26T16:11:38Z

What is this?

This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing 5b9007b (parent) -> a885811 (this PR)

Test differences

Show 1 test diff

Stage 0

core::config::tests::detect_src_and_out: pass -> [missing] (J0)

Job group index

J0: aarch64-apple, aarch64-gnu, aarch64-gnu-llvm-20-2, aarch64-msvc-2, i686-gnu-2, i686-gnu-nopt-2, i686-msvc-2, x86_64-gnu, x86_64-gnu-llvm-20-1, x86_64-gnu-llvm-20-3, x86_64-gnu-nopt, x86_64-gnu-stable, x86_64-mingw-2, x86_64-msvc-2

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \
    test-dashboard a8858111044a9391ac7558f969d3bf62ef43222d --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

dist-aarch64-apple: 6465.7s -> 8584.1s (32.8%)
dist-x86_64-apple: 7302.6s -> 6007.1s (-17.7%)
aarch64-msvc-2: 4904.1s -> 5532.7s (12.8%)
dist-powerpc64le-linux-gnu: 5776.8s -> 5122.2s (-11.3%)
dist-various-2: 2136.7s -> 2362.4s (10.6%)
x86_64-gnu-llvm-20: 2187.7s -> 2415.8s (10.4%)
test-various: 4444.0s -> 4827.3s (8.6%)
dist-x86_64-windows-gnullvm: 5178.1s -> 4734.5s (-8.6%)
pr-check-2: 2325.1s -> 2156.7s (-7.2%)
aarch64-gnu-debug: 3796.7s -> 4060.1s (6.9%)

How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

rust-timer · 2025-09-26T17:55:54Z

Finished benchmarking commit (a885811): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results (primary 3.1%, secondary 2.3%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	3.1%	[3.1%, 3.1%]	1
Regressions ❌ (secondary)	2.3%	[2.3%, 2.3%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	3.1%	[3.1%, 3.1%]	1

Cycles

Results (primary -4.2%, secondary -6.7%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-4.2%	[-10.3%, -1.7%]	22
Improvements ✅ (secondary)	-6.7%	[-20.1%, -0.9%]	19
All ❌✅ (primary)	-4.2%	[-10.3%, -1.7%]	22

Binary size

Results (secondary 0.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.0%	[0.0%, 0.0%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Bootstrap: 478.827s -> 470.315s (-1.78%)
Artifact size: 388.10 MiB -> 388.10 MiB (0.00%)

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Sep 15, 2025

rustbot assigned Kobzol Sep 15, 2025

This comment has been minimized.

Sign in to view

Kobzol mentioned this pull request Sep 18, 2025

bootstrap: Move the tests::snapshot submodule into its own file #146703

Closed

Kobzol reviewed Sep 19, 2025

View reviewed changes

Shourya742 added 14 commits September 21, 2025 20:56

make cargo test work for bootstrap self test

1bc1993

walk up the ancestors

a12969e

add an API in ConfigBuilder to point to config file for toml parsing

6c79f54

let verify method run in test settings

d488d33

remove using default toml config for test in get_toml, we handle it v…

ce4604e

…ia using the temp directory created via the testCtx

allow symlinking during test

24ed1a0

move most of the test to new testCtx

05131bd

this is dicy, whether we have a method to explicitly enable_llvm_over…

a29474d

…ride

rename config_toml to with_default_toml_config

0c68c82

remove create_config_without_ci_llvm_override duplication

9189bf7

add dry_run flag in config builder and remove runtime test hacks

671aabd

initialize out with CARGO_TARGET_DIR and then go for manifest and the…

afe380d

…n for current

add explicit config assignment when running the test, as the src is a…

a48cd76

…ssigned to CARGO_MANIFEST_DIR, so the config actually use the <src>/bootstrap.toml and the /tmp/bootstrap.toml

remove prepare_test_specific_dir and update tests accordingly

33e262e

Shourya742 force-pushed the make-cargo-test-work-for-self-test branch from 9eb758a to 33e262e Compare September 21, 2025 15:29

Kobzol reviewed Sep 22, 2025

View reviewed changes

src/bootstrap/src/core/config/config.rs Outdated Show resolved Hide resolved

src/bootstrap/src/core/config/config.rs Outdated Show resolved Hide resolved

src/bootstrap/src/utils/tests/mod.rs Show resolved Hide resolved

Shourya742 added 3 commits September 22, 2025 18:06

remove explicit target assignment in config during rustc initialization

6adbb3a

add check for toml file

8a0e380

add comment explaining the test_build_dir

83b784f

Kobzol reviewed Sep 22, 2025

View reviewed changes

src/bootstrap/src/core/config/config.rs Outdated Show resolved Hide resolved

src/bootstrap/src/core/builder/tests.rs Outdated Show resolved Hide resolved

src/bootstrap/src/core/config/tests.rs Show resolved Hide resolved

move config check logic from get_toml to parse_inner

aaa82ae

bors added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 24, 2025

This comment has been minimized.

Sign in to view

Ensure that --build-dir is always specified in tests

079addf

This comment has been minimized.

Sign in to view

rust-bors bot added a commit that referenced this pull request Sep 25, 2025

Auto merge of #146595 - Shourya742:make-cargo-test-work-for-self-test…

fb8a940

…, r=<try> Make cargo test work for bootstrap self test try-job: aarch64-apple

This comment has been minimized.

Sign in to view

rust-bors bot added a commit that referenced this pull request Sep 25, 2025

Auto merge of #146595 - Shourya742:make-cargo-test-work-for-self-test…

88604c7

…, r=<try> Make cargo test work for bootstrap self test try-job: aarch64-apple

Kobzol mentioned this pull request Sep 25, 2025

Reduce space taken by workflow URLs in try build started comment rust-lang/bors#438

Merged

This comment has been minimized.

Sign in to view

Make cargo test work again

191c7ed

Kobzol force-pushed the make-cargo-test-work-for-self-test branch from 04dec14 to 191c7ed Compare September 25, 2025 16:05

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 25, 2025

bors added the merged-by-bors This PR was explicitly merged by bors. label Sep 26, 2025

bors merged commit a885811 into rust-lang:master Sep 26, 2025
11 checks passed

rustbot added this to the 1.92.0 milestone Sep 26, 2025

This was referenced Sep 26, 2025

Rename rust.use-lld to rust.bootstrap-override-lld #147046

Merged

Rollup of 11 pull requests #147073

Closed

Make cargo test work for bootstrap self test #146595

Make cargo test work for bootstrap self test #146595

Uh oh!

Conversation

Shourya742 commented Sep 15, 2025 • edited by Kobzol Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Sep 15, 2025

Uh oh!

This comment has been minimized.

Kobzol commented Sep 15, 2025

Uh oh!

Kobzol left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rustbot commented Sep 21, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Kobzol left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

Kobzol commented Sep 25, 2025

Uh oh!

rust-bors bot commented Sep 25, 2025

Uh oh!

Kobzol commented Sep 25, 2025

Uh oh!

This comment has been minimized.

rust-bors bot commented Sep 25, 2025

Uh oh!

Kobzol commented Sep 25, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

Kobzol commented Sep 25, 2025

Uh oh!

bors commented Sep 25, 2025

Uh oh!

rust-bors bot commented Sep 25, 2025

Uh oh!

bors commented Sep 26, 2025

Uh oh!

bors commented Sep 26, 2025

Uh oh!

Uh oh!

github-actions bot commented Sep 26, 2025

Test differences

Stage 0

Job duration changes

Uh oh!

rust-timer commented Sep 26, 2025

Overall result: no relevant changes - no action needed

Instruction count

Max RSS (memory usage)

Cycles

Binary size

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

Shourya742 commented Sep 15, 2025 •

edited by Kobzol

Loading

Kobzol left a comment •

edited by rustbot

Loading

Kobzol left a comment •

edited by rustbot

Loading