The goal of this change is to switch to python-zstandard for Docker
image compression so we can employ multi-threaded compression. This will
cut down the wall time it takes to compress images, decreasing end-to-end
times.
In order to use python-zstandard, I needed to write a Python script
for doing the compression. Since I was writing a Python script, I
figured I'd move Docker image downloading to that script as well.
This way, the raw Docker image never hits disk: it is streamed straight
from Docker into a zstandard compressor and that output is written to
disk. For large images, this will eliminate a few gigabytes of disk
writes.
The one extra complication about this I don't care for is you need a
special Python package to teach the "requests" package how to download
from UNIX domain sockets.
MozReview-Commit-ID: EufaRzR6A4Y
--HG--
extra : rebase_source : 2143bfee729bdc075c3a87a1e607eff2f0c164d2
Gregory Szorc