When docker images use setup_packages.sh, they add apt sources. While we
currently do run apt-get update to pick those new sources, if a package
provided by them is already installed and not explicitly listed in
subsequent apt-get install, they're not going to be upgraded.
Differential Revision: https://phabricator.services.mozilla.com/D26100
--HG--
extra : moz-landing-system : lando
We however leave moving the packages building to a script for another
day.
Differential Revision: https://phabricator.services.mozilla.com/D19624
--HG--
rename : taskcluster/docker/debian-base/cloud-mirror-workaround.sh => taskcluster/docker/debian-raw/cloud-mirror-workaround.sh
rename : taskcluster/docker/debian-base/setup_packages.sh => taskcluster/docker/debian-raw/setup_packages.sh
to give time to docker images and toolchains to build.
--HG--
rename : taskcluster/docker/debian-raw/cloud-mirror-workaround.sh => taskcluster/docker/debian-base/cloud-mirror-workaround.sh
rename : taskcluster/docker/debian-raw/setup_packages.sh => taskcluster/docker/debian-base/setup_packages.sh
We however leave moving the packages building to a script for another
day.
Differential Revision: https://phabricator.services.mozilla.com/D19624
--HG--
rename : taskcluster/docker/debian-base/cloud-mirror-workaround.sh => taskcluster/docker/debian-raw/cloud-mirror-workaround.sh
rename : taskcluster/docker/debian-base/setup_packages.sh => taskcluster/docker/debian-raw/setup_packages.sh
We however leave moving the packages building to a script for another
day.
Differential Revision: https://phabricator.services.mozilla.com/D19624
--HG--
rename : taskcluster/docker/debian-base/cloud-mirror-workaround.sh => taskcluster/docker/debian-raw/cloud-mirror-workaround.sh
rename : taskcluster/docker/debian-base/setup_packages.sh => taskcluster/docker/debian-raw/setup_packages.sh
We however leave moving the packages building to a script for another
day.
Differential Revision: https://phabricator.services.mozilla.com/D19624
--HG--
rename : taskcluster/docker/debian-base/cloud-mirror-workaround.sh => taskcluster/docker/debian-raw/cloud-mirror-workaround.sh
rename : taskcluster/docker/debian-base/setup_packages.sh => taskcluster/docker/debian-raw/setup_packages.sh
Eventually, workers will provide these variables directly
(https://bugzilla.mozilla.org/show_bug.cgi?id=1460015). But for now, this
ensures that TASKCLUSTER_ROOT_URL is set everywhere in production, and
TASKCLUSTER_PROXY_URL is set wherever the proxy is active.
The taskgraph Taskcluster utils module gets a `get_root_url()` that gets the
root URL for the current run, either from an environment variable in production
or, on the command line, defaulting to https://taskcluster.net for user
convenience. When the production instance's URL changes, we can simply change
that default.
Other changes to use this function are reserved for later commits.
This changes the docker build process propagate TASKCLUSTER_ROOT_URL into the
docker images where necessary (using %ARG), specifically to create URLs for
debian repo paths.
--HG--
extra : rebase_source : 4f50e9d066da62a1887baabd8603844c85a32ee6
extra : source : 5ea6f03f845e49d503f5d0283557f54561c41654
Eventually, workers will provide these variables directly
(https://bugzilla.mozilla.org/show_bug.cgi?id=1460015). But for now, this
ensures that TASKCLUSTER_ROOT_URL is set everywhere, and TASKCLUSTER_PROXY_URL
is set wherever the proxy is active.
The setup for the mach commands defaults to https://taskcluster.net for user
convenience. When the production instance's URL changes, we can simply change
that default.
This changes the docker build process propagate TASKCLUSTER_ROOT_URL into the
docker images where necessary (using %ARG), specifically to create URLs for
debian repo paths.
--HG--
extra : rebase_source : 0626ebdb66a9f4078cb75ab71be256c334297363
Eventually, workers will provide these variables directly
(https://bugzilla.mozilla.org/show_bug.cgi?id=1460015). But for now, this
ensures that TASKCLUSTER_ROOT_URL is set everywhere, and TASKCLUSTER_PROXY_URL
is set wherever the proxy is active.
The setup for the mach commands defaults to https://taskcluster.net for user
convenience. When the production instance's URL changes, we can simply change
that default.
This changes the docker build process to propagate TASKCLUSTER_ROOT_URL into
the docker images, and for good measure includes some code to use that value to
generate debian repo paths.
Differential Revision: https://phabricator.services.mozilla.com/D14196
--HG--
extra : moz-landing-system : lando
If a packaging task ended up being retried for any reason, the downstream
docker tasks that depended on them would fail, since they were hard-coding the
artifacts from the initial run.
Differential Revision: https://phabricator.services.mozilla.com/D7364
If a packaging task ended up being retried for any reason, the downstream
docker tasks that depended on them would fail, since they were hard-coding the
artifacts from the initial run.
Differential Revision: https://phabricator.services.mozilla.com/D7364
For tasks that only need python, the base images are sufficient. The in-tree
code verifies that the caches are setup as volumes in the image being used,
so set the caches in the base images as well.
Differential Revision: https://phabricator.services.mozilla.com/D10148
--HG--
extra : moz-landing-system : lando
If a packaging task ended up being retried for any reason, the downstream
docker tasks that depended on them would fail, since they were hard-coding the
artifacts from the initial run.
Differential Revision: https://phabricator.services.mozilla.com/D7364
--HG--
extra : moz-landing-system : lando
Let's install python-zstandard for both Python 2 and Python 3 in
all our Debian-based images so it is readily available for use.
MozReview-Commit-ID: 1L8zDc5MYXA
--HG--
extra : rebase_source : db718891dd31d4feceff76fbce753b63049e20b1
The python3-minimal package provides /usr/bin/python3 on Debian.
This commit installs this package so a `python3` executable is
provided.
This required backporting the package to wheezy. The final patch
is trivial. But I wasted a bit of time figuring out why `mk-build-deps`
wasn't working. It would no-op and exit 0 and then the build would
complain about missing dependencies!
glandium's theory is that the ":any" multiarch support on wheezy
isn't complete. Removing ":any" seems to make things "just work."
MozReview-Commit-ID: FBicpK4SmkQ
--HG--
extra : rebase_source : a28ce731024e8ed6a43fb30e2ed57da2abb50d0f
In preparation for making it usable on Windows, after which point
having it in a directory with "docker" in it doesn't make much sense.
MozReview-Commit-ID: Hgu0buFyJwF
--HG--
rename : taskcluster/docker/recipes/run-task => taskcluster/scripts/run-task
extra : rebase_source : 3c0b502d28b5aad54bd04069efbfda88e25bbb20
We want Python 3.5+ to be available everywhere so various processes
can start using it.
The debian-base Dockerfile is shared by Debian 7 and 9 images.
Debian 9 ships with Python 3.5 and after the previous commit, we
have a Python 3.5 package for Debian 7. So we simply install the
"python3.5" package to get Python on all the Debian images.
MozReview-Commit-ID: 9ZmoSxtHWTZ
--HG--
extra : rebase_source : be4e62e7d731a3c39ee9ce205d75f1e525192acc
When running setup_packages in a docker image that derives from another,
we're currently overwriting the file that contains the apt sources for
the package artifact repositories that were used for the parent docker
image.
This doesn't cause practical problems for the existing docker images,
but in some cases where a user gets a one-click loaner, it might cause
problems when they try to install a package that has a dependency that
can't be fulfilled once those sources are overwritten.
To give a practical example, installing the gdb package from wheezy
requires libpython2.7, but if you try to do that on a derivative of the
debian7-base image, you don't have the deb7-python artifact repository
in your sources.list, and would fail to install gdb because apt can't
install a version of libpython2.7 that can be installed alongside the
python2.7 that is installed.
By putting easy repository in a separate file, named after the task id
of the corresponding package task, we ensure each an every one of them
is uniquely represented in /etc/apt/sources.list.d.
--HG--
extra : rebase_source : efb83b8c292a28c43ede24d5a3879dfbbfe94af7
We've observed apt failures multiple times where it apparently fails to
get a file in full from snapshot.debian.org. Making it retry
automatically rather than retriggering tasks seems better.
--HG--
extra : rebase_source : f3ffb415ccc30b7e7c44e6a48b29eb20e69efdd5
The apt in Debian stretch doesn't allow repositories with a Release file
not being GPG signed. Setting up GPG signatures on the
taskcluster-artifact-based repositories is a tricky process, and not
strictly necessary. It turns out not creating a Release file at all
works just as well, and works across all current Debian versions. The
packages priorities remain the same, such that packages from those
repositories are still prefered over the ones from the main Debian
repository (as long as versions are higher).
See comment in cloud-mirror-workaround.sh for that part.
--HG--
extra : rebase_source : df5af330859a314285a6c1922d899489997d2f19
That image is used to derive all the debian7-* images, and its
definition is parametrized, which will allow to create other images
based on other versions of Debian, from the same definition.
XZ_OPT is kept in each of those because we don't want to automatically
set it in all further derived images.
--HG--
extra : rebase_source : 7f4597c1ea4af83627a9373dbdc7945d20b7d996
Tom Prince
Mike Hommey
Dustin J. Mitchell
Sebastian Hengst
Margareta Eliza Balazs
Dorel Luca
Gregory Szorc