Given that we have a universal unpack method now do not keep 'unzip' in method names.
Also adapt arguments to be better understandable.
MozReview-Commit-ID: ClDB5mSVcI2
--HG--
extra : rebase_source : 5bfee9d3c56436dd3a9f7c279517642ac70bb179
Get rid of external unpack tools (unzip and tar) and use the Python internal classes instead.
This patch only changes this behavior for the base script class but not for custom code in other
test scripts and modules, which would make it too complex. A follow-up bug will be filed instead.
MozReview-Commit-ID: L0eoITlqTdC
--HG--
extra : rebase_source : 5c9f04c29eddea09f2bbac18d9fc491671b1ccdf
Now that the Firefox UI tests are in the tree, this is possible and less
of a pain. Unfortunately, due to bug 1228446, this test is disabled for
e10s.
MozReview-Commit-ID: A16EVJ8eYyB
--HG--
extra : rebase_source : 3d7b1401a4bafcd7c4bcb0804fe8c98a804a2611
Because it is now possible for options.app to get set after 'parse_args' time, we need to make sure
the argument validation happens later. To accomplish this we pass in the parser instance to
'run_test_harness' and do parser.validate there. This unfortunately requires some minor uses of
global to accomplish easily due to how mach handles parsers.
MozReview-Commit-ID: s3Js1aZlSE
--HG--
extra : rebase_source : 3a94debda3dbed839074094707cadf32e7f7337c
My one concern is that this change could increase the amount of processing
time spent on telemetry initialization, causing the runtime of the robocop
test suite to increase. Checking my try push [1] against other try pushes,
it doesn't seem to have made a significant difference, but the change
in runtime between pushes can be large (e.g. > 5min) so it's hard to
tell.
[1]: https://treeherder.mozilla.org/#/jobs?repo=try&revision=2017843315fe&selectedJob=24641374
MozReview-Commit-ID: LeeGgNEp74h
--HG--
extra : rebase_source : 21b01fa8a5357de19046fc946b4098cfd0f7b823
extra : amend_source : 457f229e6b92b8834ddd6dfef5837753f47d570b
Under some circumstances Marionette currently fails to stop the application in case of socket issues. To
ensure that the application always gets closed - in the case when Marionette started it - the check for crashes
decorator gets updated to do a full process check.
MozReview-Commit-ID: DAiF2ZjAjT5
--HG--
extra : rebase_source : 9e959b4187ef959ee9b7262e8438a5aa84396723
Custom Marionette error classes should not re-invent the message property which already exists in the
Exception class. This is fixed by calling constructor appropriately.
MozReview-Commit-ID: 1oWjg7MnrSe
--HG--
extra : rebase_source : 81a63c496f6bfbfda2565583edd18cbe1944fd99
Whenever an exception is raised while tests are executed, the log error message should only be
printed once. As best this should happen in `cli()`, so that subclasses can better set their
own behavior, and we safe us from re-raising the exception.
MozReview-Commit-ID: 5NLBnJAjUMQ
--HG--
extra : rebase_source : 17e1574c8671037912d85c0575db493c96f972b2
note: this requires a clean work dir unfortunately. so you have to blow away
the fake build/hg-share and any repos in build/
MozReview-Commit-ID: 3TfNLdga9Dt
--HG--
extra : rebase_source : 25972c5b53eb1bddd490c7aea6a085b713ff7d03
extra : amend_source : 5841fb61e94ab9c4c0f43b344f1a68d589a5c356
this can be uplifted through 48 mozilla-release
MozReview-Commit-ID: KncTJ8hAgfO
--HG--
extra : rebase_source : 18dc6c107a138317f95b433e33bf3081166c4478
HSTS priming changes the order of mixed-content blocking and HSTS
upgrades, and adds a priming request to check if a mixed-content load is
accesible over HTTPS and the server supports upgrading via the
Strict-Transport-Security header.
Every call site that uses AsyncOpen2 passes through the mixed-content
blocker, and has a LoadInfo. If the mixed-content blocker marks the load as
needing HSTS priming, nsHttpChannel will build and send an HSTS priming
request on the same URI with the scheme upgraded to HTTPS. If the server
allows the upgrade, then channel performs an internal redirect to the HTTPS URI,
otherwise use the result of mixed-content blocker to allow or block the
load.
nsISiteSecurityService adds an optional boolean out parameter to
determine if the HSTS state is already cached for negative assertions.
If the host has been probed within the previous 24 hours, no HSTS
priming check will be sent.
(r=ckerschb,r=mayhemer,r=jld,r=smaug,r=dkeeler,r=jmaher,p=ally)
The robustcheckout Mercurial extension does a clone+checkout optimally.
Read the bug for more on it.
robustcheckout is already used by mozharness automation. It has resulted
in a significant reduction in I/O usage and utilization in automation.
This commit replaces tc-vcs with the robustcheckout equivalent.
We replace the existing tc-vcs scope and cache with a new one.
Because Dustin and I are paranoid, we maintain separate caches per
SCM level - even though we could arguably share the same cache. Defense
in depth.
Robustcheckout (when used with --sharebase) pools storage for related
repos automatically. i.e. changesets from inbound and central will
be in the same store. This means you likely only have one copy of
each changeset per cache. This can result in significant space savings.
And, since there are fewer copies floating around, hg.mozilla.org
and various network appliances are working less too!
Since tc-vcs is no longer used, we stop it from being installed.
While we're here, we also change the images to execute as the
"worker" user. This happens automatically as a result of using
the "checkout-and-run" script.
MozReview-Commit-ID: EDeebuP7TkT
--HG--
extra : rebase_source : 2bec5dd9d6fe5565831bb35f195859aa12dd0bf2
extra : intermediate-source : 06481d97a485f6566554b087bc3880d76361e8ec
extra : source : d368700c93ef085325a081219d7aeb8512bc54a1
extra : histedit_source : c07505273fc8f10acf8e8d3ee01e327afd0aa63d
The script will be used as the main command in task YAML files.
It changes ownership of caches. Then switches to the "worker" user.
Then performs a Gecko checkout. Then executes whatever command was
requested via its arguments.
The script has been added to the shared recipes directory so it can
eventually be used by other Docker images. This means if we e.g. want
to add Git support, we only need to update one file in the tree.
MozReview-Commit-ID: Fuy1VrdSGYn
--HG--
extra : rebase_source : 407b2c584d56c95e9d9b23781539f2979a775893
extra : histedit_source : bd8b7fd541ed27da31082730ad3054b68b06544b
Like we do for the decision image, we install Mercurial 3.8.4 from deb
files hosted on tooltool. This provides more control and determinism
than installing via apt.
As part of this change, Mercurial is upgraded from whatever was hosted
in apt to 3.8.4.
Since the deb packages don't provide a global hgrc, we create one
ourselves. This is effectively copied from the decision image.
Most of the work is being done in a new, standalone
install-mercurial.sh script. This script is part of the
newly-established testing/docker/recipes directory. The intent of this
directory is to hold common files referenced by multiple images. Our
custom Dockerfile syntax to include files from outside the directory
with the Dockerfile is used to add these files to the build context.
MozReview-Commit-ID: K7gVm2Geihj
--HG--
extra : rebase_source : 6d1089ac34e43d399c7cf608d09eaaf405df00f7
extra : histedit_source : 656a4cea33ef913102b03238475461884c2749a0
Using our special Dockerfile syntax to include arbitrary files, we
include the previously vendored tooltool.py file in the image build
context and add it directly from there. No github.com communication
needed.
MozReview-Commit-ID: J42iXj87LEu
--HG--
extra : rebase_source : 90845e6793629b56998bf2fae2985913ee49c4eb
extra : histedit_source : 1fd5e64e40ae700efcf78b54e2a865b0594e0955
Visual aligning makes diffs harder to read. Use line continuations
to avoid this. Also make the package list alphabetical.
MozReview-Commit-ID: KqT4aqYyZfH
--HG--
extra : rebase_source : 08d2e4f61860bf6183ec3afaf598be158cd182be
extra : histedit_source : ff450a22617425214e90d42a6f1b530da8682847
Changes to the decision Docker image have been compelted. We're ready to
use the new image.
We tag the image, update version references, change the task caches
so the new Mercurial pooled storage from the robustcheckout extension is
used, and convert the decision tasks to run as the "worker" user.
MozReview-Commit-ID: 61v9Ivy59zG
--HG--
extra : rebase_source : 640318a87660950c5e0680867a1bfdd68e35f127
extra : histedit_source : ec53fc576c00e5f2053167b37544ac7afccaecb5
When we switch to use robustcheckout for version control foo, we'll
also be taking the opportunity to have the decision and action tasks
execute as the "worker" user.
Since caches are mounted and owned by root and since tasks initially
run as root, this makes defining the container command in YAML a bit
difficult because we have to do some work as root then switch users
and continue executing. Rather than shoehorning all that complicated
logic into YAML, we introduce bash scripts that do it. These will
be plugged into the task YAML when we formally switch the tasks
to use the new Docker image.
We provide one script for running Gecko decision tasks. We provide
another for running action tasks. These are the two consumers of
the decision image we care about.
We also sneak in a change to add the executable bit to checkout-gecko.
MozReview-Commit-ID: CXlyHZJSHcP
--HG--
extra : rebase_source : 80621d4833a9d745eaff7da4641dfd4ace8ae1db
extra : histedit_source : e6ce7de5d14c8781d8dd94a8eff76c3227cd18b5
Wes Kocher
Henrik Skupin
Mike Conley
Andrew Halberstadt
Carsten "Tomcat" Book
Mark Hammond
Xidorn Quan
Jeremy Chen
Michael Comella
Ben Kelly
Edgar Chen
Daisuke Akatsuka
Iris Hsiao
Sotaro Ikeda
Rail Aliiev
Jordan Lund
Kate McKinley
Andreas Tolfsen
Gregory Szorc