rustc: Stabilize options for pipelined compilation #62766
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
alexcrichton
added a commit
to alexcrichton/cargo
that referenced
this pull request
Jul 17, 2019
This commit enables pipelined compilation by default in Cargo now that the requisite support has been stablized in rust-lang/rust#62766. This involved minor updates in a number of locations here and there, but nothing of meat has changed from the original implementation (just tweaks to how rustc is called).
This was referenced Jul 17, 2019
7 tasks
|
Are |
drrlvn
reviewed
Jul 18, 2019
|
After mentioning my preference for "just adding to the JSON format without an opt-in", I have to say that I like this design! Using |
|
@jsgf no they can be combined and both affect how the |
3906a1f to
2e6a064
Compare
10000
This comment has been minimized.
This comment has been minimized.
|
Since this makes the Does the following proposal sound reasonable? I think the best option is where cargo and rustc share a rendering library. However, since it will likely be a very long time before that is done, perhaps we can plan to stabilize cache-messages with limited "short" support. It would cache whatever format was originally requested (human or short). Then when replaying, it would replay whatever was cached. This might be a little confusing if the user is switching between formats, but I suspect this should be extremely rare if ever happens. I'm happy with that plan, I just want to confirm that it sounds reasonable. |
|
@alexcrichton When adding |
|
@ehuss oh sorry to clarify I haven't forgotten about that I just figured that it wasn't something we were going to solve right out the gate (or would rather require further changes to the compiler to fix). I don't think what you mention is entirely unreasonable since switching between formats is likely rare, but in some sense rustc/cargo share at least part of "rendering library" by agreeing on what ansi color codes are. That still requires rustc to give Cargo exact text though which is a bit of a bummer. In any case though I think that this should at least be forwards compatible with a scheme where cargo/rustc agree on a short rendering scheme and Cargo can replay both messages given one invocation of rustc, and otherwise I just wanted to be sure to not couple the stabilizations too closely. |
This comment has been minimized.
This comment has been minimized.
2e6a064 to
a769554
Compare
This comment has been minimized.
This comment has been minimized.
a769554 to
2ad1335
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This commit stabilizes options in the compiler necessary for Cargo to enable "pipelined compilation" by default. The concept of pipelined compilation, how it's implemented, and what it means for rustc are documented in rust-lang#60988. This PR is coupled with a PR against Cargo (rust-lang/cargo#7143) which updates Cargo's support for pipelined compliation to rustc, and also enables support by default in Cargo. (note that the Cargo PR cannot land until this one against rustc lands). The technical changes performed here were to stabilize the functionality proposed in rust-lang#60419 and rust-lang#60987, the underlying pieces to enable pipelined compilation support in Cargo. The issues have had some discussion during stabilization, but the newly stabilized surface area here is: * A new `--json` flag was added to the compiler. * The `--json` flag can be passed multiple times. * The value of the `--json` flag is a comma-separated list of directives. * The `--json` flag cannot be combined with `--color` * The `--json` flag must be combined with `--error-format=json` * The acceptable list of directives to `--json` are: * `diagnostic-short` - the `rendered` field of diagnostics will have a "short" rendering matching `--error-format=short` * `diagnostic-rendered-ansi` - the `rendered` field of diagnostics will be colorized with ansi color codes embedded in the string field * `artifacts` - JSON blobs will be emitted for artifacts being emitted by the compiler The unstable `-Z emit-artifact-notifications` and `--json-rendered` flags have also been removed during this commit as well. Closes rust-lang#60419 Closes rust-lang#60987 Closes rust-lang#60988
2ad1335 to
1731233
Compare
|
Gonna send a ping to @rust-lang/compiler for this as well, it'd be great to get this in for the 1.38 release and it will take some time to land this, land the subsequent Cargo support, and then finally land the Cargo update in rust-lang/rust. |
|
@bors r+ |
|
📌 Commit 1731233 has been approved by |
bors
added a commit
that referenced
this pull request
Jul 30, 2019
…r=oli-obk rustc: Stabilize options for pipelined compilation This commit stabilizes options in the compiler necessary for Cargo to enable "pipelined compilation" by default. The concept of pipelined compilation, how it's implemented, and what it means for rustc are documented in #60988. This PR is coupled with a PR against Cargo (rust-lang/cargo#7143) which updates Cargo's support for pipelined compliation to rustc, and also enables support by default in Cargo. (note that the Cargo PR cannot land until this one against rustc lands). The technical changes performed here were to stabilize the functionality proposed in #60419 and #60987, the underlying pieces to enable pipelined compilation support in Cargo. The issues have had some discussion during stabilization, but the newly stabilized surface area here is: * A new `--json` flag was added to the compiler. * The `--json` flag can be passed multiple times. * The value of the `--json` flag is a comma-separated list of directives. * The `--json` flag cannot be combined with `--color` * The `--json` flag must be combined with `--error-format=json` * The acceptable list of directives to `--json` are: * `diagnostic-short` - the `rendered` field of diagnostics will have a "short" rendering matching `--error-format=short` * `diagnostic-rendered-ansi` - the `rendered` field of diagnostics will be colorized with ansi color codes embedded in the string field * `artifacts` - JSON blobs will be emitted for artifacts being emitted by the compiler The unstable `-Z emit-artifact-notifications` and `--json-rendered` flags have also been removed during this commit as well. Closes #60419 Closes #60987 Closes #60988
|
☀️ Test successful - checks-azure |
|
Thanks @oli-obk! |
|
@alexcrichton I guess nightly-2019-07-30 got released without the cargo change, as will future nightlies until cargo is updated? Right now compiling anything with |
|
That is correct, yes. Pipelining was always a nightly-only feature and this is the breakage that entails. The Cargo update, also mentioned in the description of this PR, is at rust-lang/cargo#7143. |
|
Cool. Thanks for confirming. |
alexcrichton
added a commit
to alexcrichton/cargo
that referenced
this pull request
Jul 31, 2019
This commit enables pipelined compilation by default in Cargo now that the requisite support has been stablized in rust-lang/rust#62766. This involved minor updates in a number of locations here and there, but nothing of meat has changed from the original implementation (just tweaks to how rustc is called).
bors
added a commit
to rust-lang/cargo
that referenced
this pull request
Jul 31, 2019
…=ehuss Enable pipelined compilation by default This commit enables pipelined compilation by default in Cargo now that the requisite support has been stablized in rust-lang/rust#62766. This involved minor updates in a number of locations here and there, but nothing of meat has changed from the original implementation (just tweaks to how rustc is called).
Centril
added a commit
to Centril/rust
that referenced
this pull request
Aug 21, 2019
Run Clippy without json-rendered flag Removed in rust-lang#62766 Replacing it with `--json=diagnostic-rendered-ansi` fails: ``` error: using `--json` requires also using `--error-format=json` ``` Running `./x.py clippy src/libstd` locally works fine (with colors) on Linux so I don't know if there is something to fix.
netbsd-srcmastr
pushed a commit
to NetBSD/pkgsrc
that referenced
this pull request
Oct 2, 2019
Pkgsrc changes: * Adapt to the move of the implementation of random numbers. * Remove patch which is no longer relevant (Signals.inc) * Cross-build currently fails due to the still unresolved rust-lang/rust#62558, so bootstrap kits for 1.38.0 have to be built natively, and will follow shortly. * Bump bootstrap requirements to 1.37.0 except for armv7-unknown-netbsd-eabihf which I've neither managed to cross-build nor build natively. Upstream changes: Version 1.38.0 (2019-09-26) ========================== Language -------- - [The `#[global_allocator]` attribute can now be used in submodules.][62735] - [The `#[deprecated]` attribute can now be used on macros.][62042] Compiler -------- - [Added pipelined compilation support to `rustc`.][62766] This will improve compilation times in some cases. For further information please refer to the [_"Evaluating pipelined rustc compilation"_][pipeline-internals] thread. - [Added tier 3\* support for the `aarch64-uwp-windows-msvc`, `i686-uwp-windows-gnu`, `i686-uwp-windows-msvc`, `x86_64-uwp-windows-gnu`, and `x86_64-uwp-windows-msvc` targets.][60260] - [Added tier 3 support for the `armv7-unknown-linux-gnueabi` and `armv7-unknown-linux-musleabi` targets.][63107] - [Added tier 3 support for the `hexagon-unknown-linux-musl` target.][62814] - [Added tier 3 support for the `riscv32i-unknown-none-elf` target.][62784] \* Refer to Rust's [platform support page][forge-platform-support] for more information on Rust's tiered platform support. Libraries --------- - [`ascii::EscapeDefault` now implements `Clone` and `Display`.][63421] - [Derive macros for prelude traits (e.g. `Clone`, `Debug`, `Hash`) are now available at the same path as the trait.][63056] (e.g. The `Clone` derive macro is available at `std::clone::Clone`). This also makes all built-in macros available in `std`/`core` root. e.g. `std::include_bytes!`. - [`str::Chars` now implements `Debug`.][63000] - [`slice::{concat, connect, join}` now accepts `&[T]` in addition to `&T`.][62528] - [`*const T` and `*mut T` now implement `marker::Unpin`.][62583] - [`Arc<[T]>` and `Rc<[T]>` now implement `FromIterator<T>`.][61953] - [Added euclidean remainder and division operations (`div_euclid`, `rem_euclid`) to all numeric primitives.][61884] Additionally `checked`, `overflowing`, and `wrapping` versions are available for all integer primitives. - [`thread::AccessError` now implements `Clone`, `Copy`, `Eq`, `Error`, and `PartialEq`.][61491] - [`iter::{StepBy, Peekable, Take}` now implement `DoubleEndedIterator`.][61457] Stabilized APIs --------------- - [`<*const T>::cast`] - [`<*mut T>::cast`] - [`Duration::as_secs_f32`] - [`Duration::as_secs_f64`] - [`Duration::div_duration_f32`] - [`Duration::div_duration_f64`] - [`Duration::div_f32`] - [`Duration::div_f64`] - [`Duration::from_secs_f32`] - [`Duration::from_secs_f64`] - [`Duration::mul_f32`] - [`Duration::mul_f64`] - [`any::type_name`] Cargo ----- - [Added pipelined compilation support to `cargo`.][cargo/7143] - [You can now pass the `--features` option multiple times to enable multiple features.][cargo/7084] Misc ---- - [`rustc` will now warn about some incorrect uses of `mem::{uninitialized, zeroed}` that are known to cause undefined behaviour.][63346] Compatibility Notes ------------------- - Unfortunately the [`x86_64-unknown-uefi` platform can not be built][62785] with rustc 1.39.0. - The [`armv7-unknown-linux-gnueabihf` platform is also known to have issues][62896] for certain crates such as libc. [60260]: rust-lang/rust#60260 [61457]: rust-lang/rust#61457 [61491]: rust-lang/rust#61491 [61884]: rust-lang/rust#61884 [61953]: rust-lang/rust#61953 [62042]: rust-lang/rust#62042 [62528]: rust-lang/rust#62528 [62583]: rust-lang/rust#62583 [62735]: rust-lang/rust#62735 [62766]: rust-lang/rust#62766 [62784]: rust-lang/rust#62784 [62785]: rust-lang/rust#62785 [62814]: rust-lang/rust#62814 [62896]: rust-lang/rust#62896 [63000]: rust-lang/rust#63000 [63056]: rust-lang/rust#63056 [63107]: rust-lang/rust#63107 [63346]: rust-lang/rust#63346 [63421]: rust-lang/rust#63421 [cargo/7084]: rust-lang/cargo#7084 [cargo/7143]: rust-lang/cargo#7143 [`<*const T>::cast`]: https://doc.rust-lang.org/std/primitive.pointer.html#method.cast [`<*mut T>::cast`]: https://doc.rust-lang.org/std/primitive.pointer.html#method.cast [`Duration::as_secs_f32`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.as_secs_f32 [`Duration::as_secs_f64`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.as_secs_f64 [`Duration::div_duration_f32`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.div_duration_f32 [`Duration::div_duration_f64`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.div_duration_f64 [`Duration::div_f32`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.div_f32 [`Duration::div_f64`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.div_f64 [`Duration::from_secs_f32`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.from_secs_f32 [`Duration::from_secs_f64`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.from_secs_f64 [`Duration::mul_f32`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.mul_f32 [`Duration::mul_f64`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.mul_f64 [`any::type_name`]: https://doc.rust-lang.org/std/any/fn.type_name.html [forge-platform-support]: https://forge.rust-lang.org/platform-support.html [pipeline-internals]: https://internals.rust-lang.org/t/evaluating-pipelined-rustc-compilation/10199
netbsd-srcmastr
pushed a commit
to NetBSD/pkgsrc
that referenced
this pull request
Oct 18, 2019
Pkgsrc changes: * Adapt to the move of the implementation of random numbers. * Remove patch which is no longer relevant (Signals.inc) * Cross-build currently fails due to the still unresolved rust-lang/rust#62558, so bootstrap kits for 1.38.0 have to be built natively, and will follow shortly. * Bump bootstrap requirements to 1.37.0 except for armv7-unknown-netbsd-eabihf which I've neither managed to cross-build nor build natively. Upstream changes: Version 1.38.0 (2019-09-26) ========================== Language -------- - [The `#[global_allocator]` attribute can now be used in submodules.][62735] - [The `#[deprecated]` attribute can now be used on macros.][62042] Compiler -------- - [Added pipelined compilation support to `rustc`.][62766] This will improve compilation times in some cases. For further information please refer to the [_"Evaluating pipelined rustc compilation"_][pipeline-internals] thread. - [Added tier 3\* support for the `aarch64-uwp-windows-msvc`, `i686-uwp-windows-gnu`, `i686-uwp-windows-msvc`, `x86_64-uwp-windows-gnu`, and `x86_64-uwp-windows-msvc` targets.][60260] - [Added tier 3 support for the `armv7-unknown-linux-gnueabi` and `armv7-unknown-linux-musleabi` targets.][63107] - [Added tier 3 support for the `hexagon-unknown-linux-musl` target.][62814] - [Added tier 3 support for the `riscv32i-unknown-none-elf` target.][62784] \* Refer to Rust's [platform support page][forge-platform-support] for more information on Rust's tiered platform support. Libraries --------- - [`ascii::EscapeDefault` now implements `Clone` and `Display`.][63421] - [Derive macros for prelude traits (e.g. `Clone`, `Debug`, `Hash`) are now available at the same path as the trait.][63056] (e.g. The `Clone` derive macro is available at `std::clone::Clone`). This also makes all built-in macros available in `std`/`core` root. e.g. `std::include_bytes!`. - [`str::Chars` now implements `Debug`.][63000] - [`slice::{concat, connect, join}` now accepts `&[T]` in addition to `&T`.][62528] - [`*const T` and `*mut T` now implement `marker::Unpin`.][62583] - [`Arc<[T]>` and `Rc<[T]>` now implement `FromIterator<T>`.][61953] - [Added euclidean remainder and division operations (`div_euclid`, `rem_euclid`) to all numeric primitives.][61884] Additionally `checked`, `overflowing`, and `wrapping` versions are available for all integer primitives. - [`thread::AccessError` now implements `Clone`, `Copy`, `Eq`, `Error`, and `PartialEq`.][61491] - [`iter::{StepBy, Peekable, Take}` now implement `DoubleEndedIterator`.][61457] Stabilized APIs --------------- - [`<*const T>::cast`] - [`<*mut T>::cast`] - [`Duration::as_secs_f32`] - [`Duration::as_secs_f64`] - [`Duration::div_duration_f32`] - [`Duration::div_duration_f64`] - [`Duration::div_f32`] - [`Duration::div_f64`] - [`Duration::from_secs_f32`] - [`Duration::from_secs_f64`] - [`Duration::mul_f32`] - [`Duration::mul_f64`] - [`any::type_name`] Cargo ----- - [Added pipelined compilation support to `cargo`.][cargo/7143] - [You can now pass the `--features` option multiple times to enable multiple features.][cargo/7084] Misc ---- - [`rustc` will now warn about some incorrect uses of `mem::{uninitialized, zeroed}` that are known to cause undefined behaviour.][63346] Compatibility Notes ------------------- - Unfortunately the [`x86_64-unknown-uefi` platform can not be built][62785] with rustc 1.39.0. - The [`armv7-unknown-linux-gnueabihf` platform is also known to have issues][62896] for certain crates such as libc. [60260]: rust-lang/rust#60260 [61457]: rust-lang/rust#61457 [61491]: rust-lang/rust#61491 [61884]: rust-lang/rust#61884 [61953]: rust-lang/rust#61953 [62042]: rust-lang/rust#62042 [62528]: rust-lang/rust#62528 [62583]: rust-lang/rust#62583 [62735]: rust-lang/rust#62735 [62766]: rust-lang/rust#62766 [62784]: rust-lang/rust#62784 [62785]: rust-lang/rust#62785 [62814]: rust-lang/rust#62814 [62896]: rust-lang/rust#62896 [63000]: rust-lang/rust#63000 [63056]: rust-lang/rust#63056 [63107]: rust-lang/rust#63107 [63346]: rust-lang/rust#63346 [63421]: rust-lang/rust#63421 [cargo/7084]: rust-lang/cargo#7084 [cargo/7143]: rust-lang/cargo#7143 [`<*const T>::cast`]: https://doc.rust-lang.org/std/primitive.pointer.html#method.cast [`<*mut T>::cast`]: https://doc.rust-lang.org/std/primitive.pointer.html#method.cast [`Duration::as_secs_f32`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.as_secs_f32 [`Duration::as_secs_f64`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.as_secs_f64 [`Duration::div_duration_f32`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.div_duration_f32 [`Duration::div_duration_f64`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.div_duration_f64 [`Duration::div_f32`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.div_f32 [`Duration::div_f64`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.div_f64 [`Duration::from_secs_f32`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.from_secs_f32 [`Duration::from_secs_f64`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.from_secs_f64 [`Duration::mul_f32`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.mul_f32 [`Duration::mul_f64`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.mul_f64 [`any::type_name`]: https://doc.rust-lang.org/std/any/fn.type_name.html [forge-platform-support]: https://forge.rust-lang.org/platform-support.html [pipeline-internals]: https://internals.rust-lang.org/t/evaluating-pipelined-rustc-compilation/10199
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit stabilizes options in the compiler necessary for Cargo to
enable "pipelined compilation" by default. The concept of pipelined
compilation, how it's implemented, and what it means for rustc are
documented in #60988. This PR is coupled with a PR against Cargo
(rust-lang/cargo#7143) which updates Cargo's support for pipelined
compliation to rustc, and also enables support by default in Cargo.
(note that the Cargo PR cannot land until this one against rustc lands).
The technical changes performed here were to stabilize the functionality
proposed in #60419 and #60987, the underlying pieces to enable pipelined
compilation support in Cargo. The issues have had some discussion during
stabilization, but the newly stabilized surface area here is:
--jsonflag was added to the compiler.--jsonflag can be passed multiple times.--jsonflag is a comma-separated list ofdirectives.
--jsonflag cannot be combined with--color--jsonflag must be combined with--error-format=json--jsonare:diagnostic-short- therenderedfield of diagnostics will have a"short" rendering matching
--error-format=shortdiagnostic-rendered-ansi- therenderedfield of diagnosticswill be colorized with ansi color codes embedded in the string field
artifacts- JSON blobs will be emitted for artifacts being emittedby the compiler
The unstable
-Z emit-artifact-notificationsand--json-renderedflags have also been removed during this commit as well.
Closes #60419
Closes #60987
Closes #60988