commit df73e91dee261ba91057ac390ebaf9501e6dbfd1
author David Tolnay <dtolnay@gmail.com> Fri Jan 05 13:18:18 2024 -0800
committer GitHub <noreply@github.com> Fri Jan 05 13:18:18 2024 -0800
tree 12cff25c9fd5be387b206f831b721a70ecbf98c9
parent 6b35cfdbd031864e2bd038c444a206c96e466d13

Change default Rust distribution downloads from .tar.gz to .tar.xz (#2399)

Rust infra folks requested that I send this change.

According to Rust's logs, currently 30% of bytes served by
static.rust-lang.org are .gz tarballs, and primarily that is attributed
to Bazel according to user-agent.

The .gz tarballs are needlessly large and consume more bandwidth for
Rust than we want to serve.

This PR changes rules_rust to default to .xz tarballs instead. Rustup
has been using xz downloads for the past 6.5 years.

The default set of tars downloaded by rules_rust (rustc, rust-std,
cargo, clippy, rustfmt, llvm-tools) is 215M in gz and 127M in xz, which
is 41% smaller. If adopted, this PR would save static.rust-lang.org more
than 12% of its bandwidth.

As a tradeoff, decompressing that set of 6 files with xz is slower: 3.6
seconds for gz and 8.4 seconds for xz. (Single-threaded. If Bazel
extracts different files in parallel than the magnitude of the
difference would be smaller but the ratio is similar.)

In https://github.com/rust-lang/infra-team/issues/89 we intend to sunset
gz downloads, or do something to push users away from them (such as
throttling), so gz will be the wrong choice no matter what. To make up
for decompression speed we are investigating providing zstd tarballs in
https://github.com/rust-lang/infra-team/issues/97 which decompress
significantly faster than gz while being not much larger than xz.