The `app_dirs` crate has become unmaintained and is broken on macOS with
Rust 1.25. The `directories` crate provides the same functionality and
is maintained, so switch to using that instead.
This change makes sccache run rustc with `--color=always`, removing any other
--color option from the compiler options so that it's not included in the
cache key.
A `color_mode` method is added to the `CompilerHasher` trait, and the value
is sent back to the client in the `CompileFinished` method, so the client can
determine whether specific --color options were passed without having to
do its own commandline parsing.
The client uses the new `strip-ansi-escapes` crate to remove escape codes
from the compiler output in situations where color output is not desired:
* When `--color=never` was passed on the compiler commandline.
* When `--color=auto` is in effect (the default) but the output is not
a terminal.
The existing cargo tests were lightly modified to run compile once with
`--color=never` and once with `--color=always` to validate that cache hits
are not affected by color options.
This cache module uses a Memcached cluster.
To make sccache use this, set SCCACHE_MEMCACHED
to a list of tcp://<hostname>:<port> addresses,
separated by whitespace.
This commit alters the main sccache server to operate and orchestrate its own
GNU make style jobserver. This is primarily intended for interoperation with
rustc itself.
The Rust compiler currently has a multithreaded mode where it will execute code
generation and optimization on the LLVM side of things in parallel. This
parallelism, however, can overload a machine quickly if not properly accounted
for (e.g. if 10 rustcs all spawn 10 threads...). The usage of a GNU make style
jobserver is intended to arbitrate and rate limit all these rustc instances to
ensure that one build's maximal parallelism never exceeds a particular amount.
Currently for Rust Cargo is the primary driver for setting up a jobserver. Cargo
will create this and manage this per compilation, ensuring that any one `cargo
build` invocation never exceeds a maximal parallelism. When sccache enters the
picture, however, the story gets slightly more odd.
The jobserver implementation on Unix relies on inheritance of file descriptors
in spawned processes. With sccache, however, there's no inheritance as the
actual rustc invocation is spawned by the server, not the client. In this case
the env vars used to configure the jobsever are usually incorrect.
To handle this problem this commit bakes a jobserver directly into sccache
itself. The jobserver then overrides whatever jobserver the client has
configured in its own env vars to ensure correct operation. The settings of each
jobserver may be misconfigured (there's no way to configure sccache's jobserver
right now), but hopefully that's not too much of a problem for the forseeable
future.
The implementation here was to provide a thin wrapper around the `jobserver`
crate with a futures-based interface. This interface was then hooked into the
mock command infrastructure to automatically acquire a jobserver token when
spawning a process and automatically drop the token when the process exits.
Additionally, all spawned processes will now automatically receive a configured
jobserver.
cc rust-lang/rust#42867, the original motivation for this commit