Previously the [DEFAULT] section of a manifest would simply overwrite whatever
values were passed down from the parent. This patch ensures we use
'combine_fields' so things like 'skip-if' and 'support-files' are properly
merged.
Differential Revision: https://phabricator.services.mozilla.com/D57410
--HG--
extra : moz-landing-system : lando
There aren't any manifests using '[parent:<manifest>]' in mozilla-central.
Depends on D57406
Differential Revision: https://phabricator.services.mozilla.com/D57407
--HG--
extra : moz-landing-system : lando
This was probably a remnant from Mozmill. I don't see any uses of it in mozilla-central
anywhere (or even comm-central for that matter).
Depends on D57405
Differential Revision: https://phabricator.services.mozilla.com/D57406
--HG--
extra : moz-landing-system : lando
Changes:
Fix syntax in the test file to work under python3.
Assorted changes to the code style, and eliminating unnecessary warnings.
Differential Revision: https://phabricator.services.mozilla.com/D56803
--HG--
extra : moz-landing-system : lando
Previously, when running |mach mochitest path/to/manifest.ini|, any tests in
manifests that that one includes would not be run.
This fixes that behaviour.
Differential Revision: https://phabricator.services.mozilla.com/D57164
--HG--
extra : moz-landing-system : lando
Changes:
Import the appropriate version of `StringIO` instead of `BytesIO` depending on the version of python, and use it in `manifestparser.py`.
This is required for `test_convert_directory.py` to pass on both python versions. Changes to the test was not required.
Differential Revision: https://phabricator.services.mozilla.com/D56865
--HG--
extra : moz-landing-system : lando
We've long handled chunks by defining the total number of chunks in our CI
configuration, and then passing that value down into the test harnesses at task
runtime (via the '--this-chunk' and '--total-chunks' parameters). The test
harness then runs an algorithm to determine which tests should be run in "this"
chunk.
There are several problems with this approach, but by far the biggest is that
we can't use test information in our scheduling algorithms. The information
simply isn't available yet. This patch switches things around such that we
determine which tests go in which tasks during the taskgraph generation. This
means we have perfect information around which tasks are running which tests,
and if e.g a ccov or machine learning algorithm deems a particular test
important, we can make sure to *only* schedule the tasks that contain that
test.
I'm planning to enable this a couple suites at a time so we don't accidentally
stop running tests. This specifically only enables this mode for
'mochitest-media', 'mochitest-browser-chrome' and 'mochitest-devtools-chrome'.
I chose these suites because they are the ones that are already using the
'chunk_by_runtime' algorithm.
Differential Revision: https://phabricator.services.mozilla.com/D52729
--HG--
extra : moz-landing-system : lando
Ensures child (aka included) manifests always run in the same chunk as their parent.
Differential Revision: https://phabricator.services.mozilla.com/D55284
--HG--
extra : moz-landing-system : lando
Also rename the key from 'ancestor-manifest' to 'ancestor_manifest' to be
consistent with other keys.
Differential Revision: https://phabricator.services.mozilla.com/D55283
--HG--
extra : moz-landing-system : lando
Otherwise, Valgrind is liable to see false positives from mismatched
`new` where the `delete` has been inlined to `free` or vice versa.
Differential Revision: https://phabricator.services.mozilla.com/D55553
--HG--
extra : moz-landing-system : lando
Simple addition of one item to list of granted permissions, for Android P+; see bug 1553515.
Differential Revision: https://phabricator.services.mozilla.com/D55561
--HG--
extra : moz-landing-system : lando
Ensures child (aka included) manifests always run in the same chunk as their parent.
Differential Revision: https://phabricator.services.mozilla.com/D55284
--HG--
extra : moz-landing-system : lando
Also rename the key from 'ancestor-manifest' to 'ancestor_manifest' to be
consistent with other keys.
Differential Revision: https://phabricator.services.mozilla.com/D55283
--HG--
extra : moz-landing-system : lando
Changes:
Replace `<list>.items()` calls with `six.iteritems()`.
Remove `try/except` handling of `unittest` import as we have standardized on python2.7 in CI.
Use `six` to handle metaclass changes while python2/3 intercompatibility is required.
Differential Revision: https://phabricator.services.mozilla.com/D54376
--HG--
extra : moz-landing-system : lando
As a side-effect this will also update runtime data for all suites using
'--chunk-by-runtime'.
This change simultaneously:
1. Stores runtime data from all suites
2. Stores runtime data from all tests (no more percentile)
3. Stores distinct data for android, unix (osx/linux) and windows
4. Reduces the size of 'testing/runtimes' from 408k -> 168k
The chunks look more balanced from my unscientific glance (especially on Windows).
Differential Revision: https://phabricator.services.mozilla.com/D53702
--HG--
extra : moz-landing-system : lando
Build flavors are defined in 'python/mozbuild/mozbuild/testing.py'.
This change is needed by D52729 but it's also a good way to tell which suites
are integrated into the TestManifestBackend in the build system. So I'm landing
it here instead.
Depends on D53030
Differential Revision: https://phabricator.services.mozilla.com/D53698
--HG--
extra : moz-landing-system : lando
Whereas:
- desktop tests don't make this check;
- the check for directory existence has been troublesome and almost never useful;
- bug classification of this condition has been troublesome;
- if a startup crash actually did occur before crashreporter init, there would still be an indication in logcat and possibly a tombstone, and the "No test summary found" check would definitely be triggered;
Let's stop checking for minidumps directory creation.
Differential Revision: https://phabricator.services.mozilla.com/D54755
--HG--
extra : moz-landing-system : lando
This function works on all GeckoProcessTypes, not just those for child
processes.
Differential Revision: https://phabricator.services.mozilla.com/D54375
--HG--
extra : moz-landing-system : lando
Changes:
These files do not appear to be referred to by anything as searched on searchfox.
I believe we are safe to remove them.
Differential Revision: https://phabricator.services.mozilla.com/D54201
--HG--
extra : moz-landing-system : lando
As a side-effect this will also update runtime data for all suites using
'--chunk-by-runtime'.
This change simultaneously:
1. Stores runtime data from all suites
2. Stores runtime data from all tests (no more percentile)
3. Stores distinct data for android, unix (osx/linux) and windows
4. Reduces the size of 'testing/runtimes' from 408k -> 168k
The chunks look more balanced from my unscientific glance (especially on Windows).
Differential Revision: https://phabricator.services.mozilla.com/D53702
--HG--
extra : moz-landing-system : lando
Build flavors are defined in 'python/mozbuild/mozbuild/testing.py'.
This change is needed by D52729 but it's also a good way to tell which suites
are integrated into the TestManifestBackend in the build system. So I'm landing
it here instead.
Depends on D53030
Differential Revision: https://phabricator.services.mozilla.com/D53698
--HG--
extra : moz-landing-system : lando
This patch fixes a minor issue with manifestparser when it is used in python 3. The problem was that dict.items() returns a generator in python 3 instead of a list.
Differential Revision: https://phabricator.services.mozilla.com/D53953
--HG--
extra : moz-landing-system : lando
Changes:
- rename `test.py` to a more descriptive `test_moznetwork.py` and change associated names in the manifest
- added `r` specifier to strings as per PEP warning
- bump version to 1.0.0 pending release to pypi
Differential Revision: https://phabricator.services.mozilla.com/D52107
--HG--
extra : moz-landing-system : lando
This moves the parts of toolkit/library/rust/shared/lib.rs related to
panic hooking to a new mozglue subdirectory, which will be used for
things that can be statically linked to e.g. libxul, rather than in
a "shared library".
The panic hook is disabled when building spidermonkey via the mozjs_sys
crate.
Differential Revision: https://phabricator.services.mozilla.com/D52793
--HG--
extra : moz-landing-system : lando
This moves the parts of toolkit/library/rust/shared/lib.rs related to
panic hooking to a new mozglue subdirectory, which will be used for
things that can be statically linked to e.g. libxul, rather than in
a "shared library".
The panic hook is disabled when building spidermonkey via the mozjs_sys
crate.
Differential Revision: https://phabricator.services.mozilla.com/D52793
--HG--
extra : moz-landing-system : lando
This moves the parts of toolkit/library/rust/shared/lib.rs related to
panic hooking to a new mozglue subdirectory, which will be used for
things that can be statically linked to e.g. libxul, rather than in
a "shared library".
The panic hook is disabled when building spidermonkey via the mozjs_sys
crate.
Differential Revision: https://phabricator.services.mozilla.com/D52793
--HG--
extra : moz-landing-system : lando
Simplified and streamlined the logic in the Linux side of things, to address 1595147.
Only attempt to import and use either `platform` or `distro`, not both.
Perform sanitization of the output string since `distro.linux_distribution()` has a slightly different output than the `platform` equivalent.
Minor version bump to 1.2.1 denoting a bugfix for release to pypi.
Differential Revision: https://phabricator.services.mozilla.com/D52440
--HG--
extra : moz-landing-system : lando
While not strictly necessary for this series, this patch allows:
./mach test path/to/manifest.ini
Which will be especially useful to have now that we are switching to running
tasks by manifest.
Depends on D52241
Differential Revision: https://phabricator.services.mozilla.com/D52242
--HG--
rename : testing/mozbase/moztest/tests/data/srcdir/apple/mochitest.ini => testing/mozbase/moztest/tests/data/srcdir/apple/a11y.ini
extra : moz-landing-system : lando
This gives us the ability to retrieve all browser-chrome tests (no flavor) but
not devtools-chrome (have a flavor).
Differential Revision: https://phabricator.services.mozilla.com/D52241
--HG--
extra : moz-landing-system : lando
Allows 'paths' passed into the pathprefix filter to be manifests. Any path that
ends with '.ini' is considered a manifest.
Depends on D51899
Differential Revision: https://phabricator.services.mozilla.com/D51900
--HG--
extra : moz-landing-system : lando
This loader uses 'reader.find_variables_from_ast' to parse all *_MANIFESTS variables from
moz.build files using the abstract syntax tree. This means it will find all such variables
regardless of the current buildconfig.
Differential Revision: https://phabricator.services.mozilla.com/D51834
--HG--
extra : moz-landing-system : lando
Test test data looks like it was pulled from a live all_tests.pkl file. There
are way more path components than necessary.
This simplifies the test paths so they are easy to manipulate/add/inspect. It
will also make it easier to craft a fake "sourcedir" to test the
TestManifestLoader in the next commit such that the data from both matches.
I decided to use a fruit theme for directories because:
1. Using real directories will pollute grep/searchfox/etc queries with junk.
2. Using a 'dirA', 'dirB', 'dirC' scheme is hard to read.
3. Why not?
This change does not functionally modify what is being tested.
Depends on D51832
Differential Revision: https://phabricator.services.mozilla.com/D51833
--HG--
extra : moz-landing-system : lando
When I updated the dependencies and the `mozinfo.py` code I forgot to bump up the version denoting some changes. This will bump up the version from 1.1.0 to 1.2.0 and permit me to upload a new version to pypi.
Differential Revision: https://phabricator.services.mozilla.com/D51898
--HG--
extra : moz-landing-system : lando
I'm making this refactor now because a future commit is going to completely re-write the
test data (so that it matches the data that a future TestManifestLoader is going to use).
With this method, updating the data will be a lot less tedious.
Differential Revision: https://phabricator.services.mozilla.com/D51278
--HG--
extra : moz-landing-system : lando
This patch adds GPU and CPU clock frequencies as well as Watts used to the metrics that are gathered into perfherder from mozpower.
Differential Revision: https://phabricator.services.mozilla.com/D50522
--HG--
extra : moz-landing-system : lando
Other places take care to handle this case, but not running the tooltool
script. Let's fix that.
Differential Revision: https://phabricator.services.mozilla.com/D51266
--HG--
extra : moz-landing-system : lando
This is needed to release a new mozlog with the PRECONDITION_FAILED
test and subtest status for use in web-platform-tests.
Update all in-tree dependencies on mozlog to >=5.0. These were found
with `hg grep 'mozlog.*[0-9]'`.
Only testing/web-platform/tests/tools/wptrunner/requirements.txt
remains on 4.2.0, and it will be updated in upstream wpt after mozlog
5.0 has been released.
Differential Revision: https://phabricator.services.mozilla.com/D50456
--HG--
extra : moz-landing-system : lando
Previously there was a somewhat strange setup where we had both TestResolver
and TestMetadata classes. Both had 'resolve_tests' function and the separation
of concerns between the two were not clear.
With this change, all of the logic that is related to manipulating and
resolving the loaded tests has been moved to the TestResolver class. Also, the
TestMetadata class has been renamed to TestLoader, and it is solely responsible
for loading the metadata (from the build backend).
Future commits will add other types of TestLoaders.
Differential Revision: https://phabricator.services.mozilla.com/D49768
--HG--
extra : moz-landing-system : lando
A minor cleanup. Re-write paths will now automatically be joined to
self.topobjdir.
Depends on D49766
Differential Revision: https://phabricator.services.mozilla.com/D49767
--HG--
extra : moz-landing-system : lando
Similar to the vcs change, the MozbuildObject already has a reader attribute
available. So we can re-use that instead of creating our own.
Depends on D49765
Differential Revision: https://phabricator.services.mozilla.com/D49766
--HG--
extra : moz-landing-system : lando
Encapsulates all the logic around generating and loading the build backend
metadata on the TestMetadata class. Previously the TestResolver would trigger
the generation if necessary, and TestMetadata would load it. Now both
generation and loading happens in TestMetadata.load_tests.
This change also adds some convenience properties to make it easier to query
the loaded data.
Differential Revision: https://phabricator.services.mozilla.com/D49765
--HG--
extra : moz-landing-system : lando
This prevents us from adding the puppeteer tests over and over again. It
follows the wpt example.
Differential Revision: https://phabricator.services.mozilla.com/D49790
--HG--
extra : moz-landing-system : lando
Since TestResolver is a subclass of MozbuildObject, there's no need to create
separate repository object. It already has one.
Depends on D49761
Differential Revision: https://phabricator.services.mozilla.com/D49764
--HG--
extra : moz-landing-system : lando
This works around a bug in Python:
https://bugs.python.org/issue32745
Null characters aren't allowed in 'c_wchar_p' types anymore, but we can get around
the issue by allocating a buffer in memory and casting it after the fact. This was
discovered via trial and error and I'm not really sure why it works.. But it does.
This also enables the tests under Python 3 on Windows (which thankfully all
seem to pass).
Differential Revision: https://phabricator.services.mozilla.com/D48113
--HG--
extra : moz-landing-system : lando
This patch adds fullscreen and windowed youtube tests for the V9 and H264 encoding at 1080p30 and 1080p60. Each subtest runs for 20 page cycles which amounts to about 5 minutes each. It also begins adding these to power test tasks running on the macosx-1014 reference hardware.
Differential Revision: https://phabricator.services.mozilla.com/D45067
--HG--
extra : moz-landing-system : lando
This implementation speaks the ADB wire protocol over TCP/IP. This is
in constrast to the Python implementation, which generally invokes adb
on the command line. In thousands of runs across multiple devices,
this implementation has proved surprisingly robust.
Differential Revision: https://phabricator.services.mozilla.com/D44895
--HG--
extra : moz-landing-system : lando
Bug 1572563 wanted to be extra sure that `ls` could be found. To do
that it tries to run various `ls` variants in a loop until one such
variant exits with exit code 0. However, `ls` can be present and
functional and still exit with code 1. For many unrooted devices, bare
`ls` will exit with code 1 because there are files and directories for
which the shell user doesn't have read permission.
This works around by trying to list only the file that the loop is
trying to execute.
Differential Revision: https://phabricator.services.mozilla.com/D45770
--HG--
extra : moz-landing-system : lando
This API returns the data from get_info() along with a handful
list of dumpsys services data.
Differential Revision: https://phabricator.services.mozilla.com/D45317
--HG--
extra : moz-landing-system : lando
Depending on mozprofile by relative path makes it hard to publish
mozrunner to crates.io because cargo is unable to work out the
version range the crate needs.
By specifying both a path and a semver range we ensure mozrunner
uses the in-tree version of mozprofile when building locally,
and the upstream crates.io version when published and used elsewhere.
This means this version number must be bumped every time a new
(backwards incompatible) mozversion is released.
It is debatable whether the version range should be exact, i.e. "0.6.0",
but I opted to go with "0.6" which is how I would normally define
a crate dependency. As long as mozprofile continues to follow the
principles of semantic versioning this should be fine.
Differential Revision: https://phabricator.services.mozilla.com/D45511
--HG--
extra : moz-landing-system : lando
The moz.build file for the mozrunner crate is not included by the
build system, and the Bugzilla categorisation it attempts to define
is already provided by testing/mozbase/moz.build.
Differential Revision: https://phabricator.services.mozilla.com/D45408
--HG--
extra : moz-landing-system : lando
When there is an input of either &self or &mut self, its lifetime is automatically assigned to the elided output lifetimes
Depends on D45355
Differential Revision: https://phabricator.services.mozilla.com/D45356
--HG--
extra : moz-landing-system : lando
The main motivation here is to get these subsections out of the main index.
While changing the heading to an 'h2' or lower would suffice, I opted to remove
them entirely. I did this because the links in this section is for the overall
global documentation. E.g, the "Indices and tables" section under /tools/lint,
has nothing to do with linting.
It is still possible to access the index and search page from the root:
https://firefox-source-docs.mozilla.org/#indices-and-tables
Differential Revision: https://phabricator.services.mozilla.com/D43653
--HG--
extra : moz-landing-system : lando
With bug 1566778 we started searching the system path on BSDs
similarly to how we had done it on Linux. The patch forgot to
adapt the fallback implementation of the platform module, causing
a compile issue on BSDs since they are now covered by the same
implementation as what used to be Linux-specific.
Differential Revision: https://phabricator.services.mozilla.com/D42625
--HG--
extra : moz-landing-system : lando
webextensions now uses "browser_specific_settings" instead of "applications" in
the manifest file. This patch make mozprofile look for both places.
Differential Revision: https://phabricator.services.mozilla.com/D42457
--HG--
extra : moz-landing-system : lando
These changes will fix the API that read values out of ADB commands
for Python 3.
Differential Revision: https://phabricator.services.mozilla.com/D41921
--HG--
extra : moz-landing-system : lando
Makes it possible for mach to resolve test paths for Puppeteer,
so that individual tests can be run from the command line using
"./mach test", as such:
% ./mach test remote/test/puppeteer/test/screenshot.spec.js
As the Puppeteer test suite is imported from upstream and we cannot
change this directory at will (i.e. to add test manifest files),
we take the same approach as for WPT and populate the manifest by
recursively walking the remote/test/puppeteer/test/**/*.spec.js file tree.
Differential Revision: https://phabricator.services.mozilla.com/D37013
--HG--
extra : moz-landing-system : lando
Makes it possible for mach to resolve test paths for Puppeteer,
so that individual tests can be run from the command line using
"./mach test", as such:
% ./mach test remote/test/puppeteer/test/screenshot.spec.js
As the Puppeteer test suite is imported from upstream and we cannot
change this directory at will (i.e. to add test manifest files),
we take the same approach as for WPT and populate the manifest by
recursively walking the remote/test/puppeteer/test/**/*.spec.js file tree.
Differential Revision: https://phabricator.services.mozilla.com/D37013
--HG--
extra : moz-landing-system : lando
We don't expect to use any unsafe and doing this helps static analysis tools
determine that the library is low risk from the point of view of memory unsafety,
and makes it less likely someone will add in some unsafe code in the future without
appropriate care.
Differential Revision: https://phabricator.services.mozilla.com/D41730
--HG--
extra : moz-landing-system : lando
Bug 1539437 indtroduced a new ChromiumProfile class in mozprofile
that's imported into mozrunner. This means that the 7.5.0 release of
mozrunner should have been accompanied by a corresponding mozprofile
release and requirements version bump.
Differential Revision: https://phabricator.services.mozilla.com/D41167
--HG--
extra : moz-landing-system : lando
This patch implements the MozPower class in the Mozpower module which is the interface for the user to use the power measurement tooling. It primarily detects the user's OS and CPU combination and provides the appropriate methods for them.
Differential Revision: https://phabricator.services.mozilla.com/D39194
--HG--
extra : moz-landing-system : lando
This patch implements the MacIntelPower class in the Mozpower module. It is used for power measurement testing on Mac Intel-based machines using Intel Power Gadget.
Differential Revision: https://phabricator.services.mozilla.com/D39193
--HG--
extra : moz-landing-system : lando
This patch implements IntelPowerGadget and IPGResultsHandler for the Mozpower module. These classes handle data generation with Intel Power Gadget as well as cleaning the data and formatting it into the PERFHERDER_DATA format.
Differential Revision: https://phabricator.services.mozilla.com/D39192
--HG--
extra : moz-landing-system : lando
This patch initializes the Mozpower module and implements the PowerBase class. This class is used as a base for subclasses that implement power measurement tooling for various OS and CPU combinations.
Differential Revision: https://phabricator.services.mozilla.com/D39191
--HG--
extra : moz-landing-system : lando
Various cleanup:
- remove extraneous calls to grant_runtime_permissions
- remove unused legacy jimdb support code
- remove "This may take a while" logging
- emphasize x86/x86_64 capabilities of emulator
Differential Revision: https://phabricator.services.mozilla.com/D40302
--HG--
extra : moz-landing-system : lando
Other harness code intermittently complains if the directory is absent. Maybe this will help.
Differential Revision: https://phabricator.services.mozilla.com/D39621
--HG--
extra : moz-landing-system : lando
The missing directory seems to happens ~20 or so times per test run. I hope this
can help with investigations.
Differential Revision: https://phabricator.services.mozilla.com/D39613
--HG--
extra : moz-landing-system : lando
Add internal method _wait_for_bootcompleted to wait until sys.boot_completed and dev.bootcomplate are set.
Add internal method _initialize_boot_state to encapsulate the steps required to initialize device state.
Make sure to set device state after rebooting.
Normalize optional arguments for _test_path calls.
Add verbose logging for shell_bool.
Differential Revision: https://phabricator.services.mozilla.com/D39876
--HG--
extra : moz-landing-system : lando
The current comment processing code strips whitespace from a line, calculates
comment offsets based on the unstripped version, and then strips those offsets
from the stripped version. That means that, for multi-line directives, which
typically have two spaces at the front, the offsets are wrong and lines with
comments end up with a trailing "# " that the expression parser doesn't
understand.
This patch fixes the comment parser to correctly use the stripped line for
offset calculations instead.
Differential Revision: https://phabricator.services.mozilla.com/D38724
--HG--
extra : rebase_source : 9f19314ccab3fb2fa68642ff0aef978cb5c3e13c
Similar to bug 1552672, very rarely is_zipfile() can return True on a
tarfile if the tarfile happens to contain a magic four byte sequence at
the right location in the file. By checking for is_zipfile() first, we
try to unzip the tar file and then fail. Instead we can call
is_tarfile() first, which is more robust, and call is_zipfile() last.
Differential Revision: https://phabricator.services.mozilla.com/D37765
--HG--
extra : moz-landing-system : lando
Changes:
- for macosx1014, in the tests where `subprocess` is called and the process relies on `six` being present, ensure that `six` is available in the PYTHONPATH by explicitly inserting the contents of `sys.path` to the modified environment variable dict
- change how the command line call to run the suite is being built in `taskcluster/taskgraph/transforms/job/mach.py`
- migrate source-tests from macosx1010 to macosx1014
Differential Revision: https://phabricator.services.mozilla.com/D36681
--HG--
extra : moz-landing-system : lando
I don't know if this covers all the things that use mozinfo (probably not)
but it covers all the suites that use mozinfo and have webrender conditions
in the test manifests (i.e. mochitest and wpt variants).
Differential Revision: https://phabricator.services.mozilla.com/D35869
--HG--
extra : moz-landing-system : lando
The mozlog documents have been updated to reflect the addition of a `known_intermittent`
parameter in `test_status` and `test_end` included in the `StructuredLogger`, and across
multiple mozlog handlers and formatters.
As this was a major change, the version has been bumped to 4.2 for the next pypi release.
Web-platform-tests will eventually require this update in order for future patches related
to `known_intermittent` statuses to merge upstream.
Differential Revision: https://phabricator.services.mozilla.com/D36029
--HG--
extra : moz-landing-system : lando
This effectively avoids the mkdir failures I see with the 29.0.11 emulator
on packet.net with Android 7.0 x86_64. I hate to add this sort of complication
but it really helps clear the way for an otherwise useful upgrade.
Differential Revision: https://phabricator.services.mozilla.com/D34740
--HG--
extra : moz-landing-system : lando
Use the geckoview TestRunnerActivity, org.mozilla.geckoview.test, by default
for all types of mochitests, reftests, and web-platform tests. TRA is already
the default for gtest and geckoview-junit. Fennec, based on ANDROID_PACKAGE_NAME,
remains the default for robocop and marionette-test and I have no plans to
change those. There is a related issue for xpcshell-test -- not the package
name, but the default apk -- but I am reluctant to handle that until bug 1553225
is resolved.
Differential Revision: https://phabricator.services.mozilla.com/D35479
--HG--
extra : moz-landing-system : lando
In particular:
* trait objects without an explicit `dyn` are deprecated
* `...` range patterns are deprecated
I think these shouldn't really warn by default and should be clippy / opt-in
lints, but anyway, doesn't hurt.
Differential Revision: https://phabricator.services.mozilla.com/D35135
--HG--
extra : moz-landing-system : lando
Improve support for Python 3 by running modernize over adb.py and a few other related changes.
Differential Revision: https://phabricator.services.mozilla.com/D19306
--HG--
extra : moz-landing-system : lando
This patch adds a `known_intermittent_statuses` attribute to the `StatusHandler`
class, allowing it to keep a count of expected intermittents for future use.
Additionally, known intermittents are not recorded as `unexpected_statuses` but
are recorded as `expected_statuses`.
testing/mozharness/mozharness/mozilla/structuredlog.py is directly affected by
this change and has been updated to also reflect `known_intermittent_statuses`.
However, it may require a test to be written to check this addition.
The `StatusHandler` test has been added to, ensuring this patch works as expected.
Differential Revision: https://phabricator.services.mozilla.com/D33086
--HG--
extra : moz-landing-system : lando
This patch adds support for the known_intermittent field in the html,
and grouping formatters. Tests have been added to check the new
field in grouping and tbpl formatter tests.
Differential Revision: https://phabricator.services.mozilla.com/D32773
--HG--
extra : moz-landing-system : lando
mozalloc_abort and related abort functions are the top frame for many
different, unrelated crashes because they happen to be the standard way to
abort execution. That makes it difficult to properly classify and deal with
intermittent failures.
This patch changes our crash handling behavior so that we try to skip any
frames at the top of the stack that are in generic abort functions, and use
the topmost frame which is actually relevant to the crash reason instead.
Differential Revision: https://phabricator.services.mozilla.com/D33051
--HG--
extra : moz-landing-system : lando
As the number of tests have grown considerably, for clarity, this patch
moves the formatter tests in test_structured.py to test_formatters.py.
html.py and its corresponding test were also amended to account for
discrepancies in the way Python 2 and 3 handle strings (bytes vs text type).
Differential Revision: https://phabricator.services.mozilla.com/D33048
--HG--
extra : moz-landing-system : lando
As the number of tests have grown considerably, for clarity, this patch
moves the formatter tests in test_structured.py to test_formatters.py.
I am striking an error on one of the tests since the move.
`test_base64_other`, which tests that ability to read different characters
in the HTMLformatter. It appears that whilst moving the tests, `test_end`
now returns a unicode string, which fails the assertion. Am looking into
Python 2 v 3 discrepancies, but any guidance would be most welcome. Error
message will be attached below.
Differential Revision: https://phabricator.services.mozilla.com/D32732
--HG--
extra : moz-landing-system : lando
There is no real need to use mozprocess from this code, so switched to subprocess.
Differential Revision: https://phabricator.services.mozilla.com/D32897
--HG--
extra : moz-landing-system : lando
The in-tree log formatters have been updated to reflect the new `known_intermittent`
field, ensuring that a status matching a `known_intermittent` status is not logged as an
unexpected failure. A message is printed when there is a test status that matches this.
A test for known intermittents has been added to the test_formatters, following the
same testing style for pass or fail.
Differential Revision: https://phabricator.services.mozilla.com/D32174
--HG--
extra : moz-landing-system : lando
Currently, some of the raw JSON logs for mochitest and marionette, et al, include
empty dictionaries, None values and other unremarkable values that are marked
as optional. This fix aims to remove these unnecessary items from being
passed to the raw log.
A method has been added to the log_actions class which removes defaults if they
are marked as optional and the value is included in the default list. This is
called on the kwargs returned by the convert_known method, before being
propagated to the log_raw method for StructuredLogger.
Differential Revision: https://phabricator.services.mozilla.com/D25081
--HG--
extra : moz-landing-system : lando
Currently, some of the raw JSON logs for mochitest and marionette, et al, include
empty dictionaries, None values and other unremarkable values that are marked
as optional. This fix aims to remove these unnecessary items from being
passed to the raw log.
A method has been added to the log_actions class which removes defaults if they
are marked as optional and the value is included in the default list. This is
called on the kwargs returned by the convert_known method, before being
propagated to the log_raw method for StructuredLogger.
Differential Revision: https://phabricator.services.mozilla.com/D25081
--HG--
extra : moz-landing-system : lando
Added a new optional field on the test_end and test_status actions called expected_intermittents
which accepts a List of expected intermittent statuses. The default is None. As it is an optional
field, upon landing of D25081, it will only show if there is an expected intermittent.
A test was added to check that expected_intermittents are logged.
Differential Revision: https://phabricator.services.mozilla.com/D31809
--HG--
extra : moz-landing-system : lando
`./mach run` doesn't work since `_get_host_platform` returns None. So we should
return `win32` on Windows.
Differential Revision: https://phabricator.services.mozilla.com/D30608
--HG--
extra : moz-landing-system : lando
`mach run` as it is doesn't really parallel `mach run` on Desktop;
this makes it a little closer more fully featured. The underlying
functionality is all there in layers of mozharness; let's make it
easier to get to.
Differential Revision: https://phabricator.services.mozilla.com/D18292
--HG--
extra : moz-landing-system : lando
If there is still value to this -- if you know of anyone using jimdb via mach, etc -- I am
happy to throw this away, but otherwise, would like to clean it up.
Differential Revision: https://phabricator.services.mozilla.com/D30943
--HG--
extra : moz-landing-system : lando
The Mozlog version number needed to be updated to 4.1 for the new
release on PyPI. The dependencies of other libraries that require
mozlog were updated to reflect the new version. (mozversion,
mozrunner, mozprofile)
Differential Revision: https://phabricator.services.mozilla.com/D30550
--HG--
extra : moz-landing-system : lando
Uninstalling the test app before re-installing ensures a clean, predictable
starting state for gtest.
Differential Revision: https://phabricator.services.mozilla.com/D30407
--HG--
extra : moz-landing-system : lando
I think a more robust solution would involve disabling the checksum behavior in tooltool.py;
I will leave-open for us to consider later. In the meantime, this will get things working again.
Differential Revision: https://phabricator.services.mozilla.com/D30403
--HG--
extra : moz-landing-system : lando
tempdir has been deprecated for a while. I've set tempfile to 3.0.2 on
purpose to avoid duping rand for now (but this is one more step in the
direction of getting everything to rand 0.6 as tempfile 3 is on it
already).
Differential Revision: https://phabricator.services.mozilla.com/D28278
--HG--
extra : moz-landing-system : lando
In addition to the inconvenience of geckoview installation, there are other reasons to
re-install, notably to test against new changes; let's always offer to install, regardless
of which app is requested and regardless of whether it is installed.
I've also removed the looping that was here: I think prompting once is sufficient.
Differential Revision: https://phabricator.services.mozilla.com/D29880
--HG--
extra : moz-landing-system : lando
Previously we would silently change the value of "e10s" from False to True.
This can cause confusion and lead people to falsely think mochitest-chrome/a11y
work with e10s (they do not).
Now we explicitly error out in this case. This might be slightly less
convenient for the developer (e.g they might need to re-run the command), but
the downside of needing to rerun a test command is less than the risk of
misunderstanding what is being tested.
Note: when running |mach test| or |mach mochitest| on a directory that contains
both chrome/a11y and another suite, we'll still do the right thing and
implicitly set "e10s=False".
Differential Revision: https://phabricator.services.mozilla.com/D28538
--HG--
extra : moz-landing-system : lando
This officially makes 'moztest.resolve' the source of truth when it comes to
suite names. It aligns that file with the names used in both the
desktop_unittest and android_emulator_unittest scripts.
Differential Revision: https://phabricator.services.mozilla.com/D27555
--HG--
extra : moz-landing-system : lando
mozdebug has code to locate Visual Studio using vswhere.exe, but it only
works if vswhere is in PATH. We have a copy vendored in the source tree,
so this change adds support for using that binary when available. Additionally
the code parsing the output of vswhere appeared to be using a key that
does not exist in the JSON output, so this change fixes that to match the
documentation: https://github.com/Microsoft/vswhere/wiki#examples .
Differential Revision: https://phabricator.services.mozilla.com/D16542
--HG--
extra : moz-landing-system : lando
Since e10s is the default configuration, we shouldn't explicitly mark things
with the "-e10s" suffix. Instead we should mark things that *don't* run with
'e10s. This patch removes '-e10s' from all treeherder group symbols and task
labels, adds the "-1proc" suffix to tasks that are non-e10s.
Differential Revision: https://phabricator.services.mozilla.com/D25958
--HG--
extra : moz-landing-system : lando
If mozbuild parsing fails due to a missing file (eg: a file not existing
in UNIFIED_SOURCES), then no Makefiles are written out, but
config.status exists. This would cause mozbuild to think that configure
doesn't need to run, and rely on make to perform the backend-out-of-date
check in rebuild-backend.mk. Unfortunately since no Makefiles were
written, the make command fails immediately and no attempt is made to
re-create the backend. Note that this is only a problem if the first
mozbuild parsing from a clobber build fails, otherwise there is
typically a top-level Makefile from a previous build to call into (at
which point make can determine it is out-of-date, and re-invoke itself).
The fix is to have the RecursiveMake backend re-use the same logic that
was introduced into mozbuild for alternate backends, and remove
rebuild-backend.mk. This way, mozbuild can always determine if the
backend needs to be regenerated, even if the initial parsing failed.
Test code was also relying on rebuild-backend.mk to generate the
TestBackend, but moving backend_out_of_date() into MozbuildObject allows
this code to be shared.
Differential Revision: https://phabricator.services.mozilla.com/D26262
--HG--
rename : build/gen_test_backend.py => python/mozbuild/mozbuild/gen_test_backend.py
extra : moz-landing-system : lando
If mozbuild parsing fails due to a missing file (eg: a file not existing
in UNIFIED_SOURCES), then no Makefiles are written out, but
config.status exists. This would cause mozbuild to think that configure
doesn't need to run, and rely on make to perform the backend-out-of-date
check in rebuild-backend.mk. Unfortunately since no Makefiles were
written, the make command fails immediately and no attempt is made to
re-create the backend. Note that this is only a problem if the first
mozbuild parsing from a clobber build fails, otherwise there is
typically a top-level Makefile from a previous build to call into (at
which point make can determine it is out-of-date, and re-invoke itself).
The fix is to have the RecursiveMake backend re-use the same logic that
was introduced into mozbuild for alternate backends, and remove
rebuild-backend.mk. This way, mozbuild can always determine if the
backend needs to be regenerated, even if the initial parsing failed.
Test code was also relying on rebuild-backend.mk to generate the
TestBackend, but moving backend_out_of_date() into MozbuildObject allows
this code to be shared.
Differential Revision: https://phabricator.services.mozilla.com/D26262
--HG--
rename : build/gen_test_backend.py => python/mozbuild/mozbuild/gen_test_backend.py
extra : moz-landing-system : lando
Restore old "FAIL" tests color to red. Restore heading of mochitest. Fix
marking of unexpected failures.
Differential Revision: https://phabricator.services.mozilla.com/D25996
--HG--
extra : moz-landing-system : lando
This patch introduces a new marionette media test along
with a Youtube test.
To run the Youtube streaming test locally:
./mach marionette-test dom/media/test/marionette/test_youtube.py -vv --gecko-log -
Differential Revision: https://phabricator.services.mozilla.com/D23644
--HG--
extra : moz-landing-system : lando
This patch introduces a new marionette media test along
with a Youtube test.
To run the Youtube streaming test locally:
./mach marionette-test dom/media/test/marionette/test_youtube.py -vv --gecko-log -
Differential Revision: https://phabricator.services.mozilla.com/D23644
--HG--
extra : moz-landing-system : lando
mitmproxy appears to return an error code of 1 if we use the `--no-upstream-cert`. The actual playback works fine. This patch adds an option to reduce the log level to `info` in order to prevent tests from being marked orange due to a shutdown error message.
Differential Revision: https://phabricator.services.mozilla.com/D25000
--HG--
extra : moz-landing-system : lando
TaskCluster tasks use the `TOOLTOOL_CACHE` env var to specify which directory to use for the `tooltool` cache. This patch falls back to that value if `TOOLTOOLCACHE` is not found.
Differential Revision: https://phabricator.services.mozilla.com/D25305
--HG--
extra : moz-landing-system : lando
Other than the MediaPipelineFactory references, nothing else is obsolete, so this is a
very minor cleanup, just to improve readability a bit.
Differential Revision: https://phabricator.services.mozilla.com/D25124
--HG--
extra : moz-landing-system : lando
Some of these were working with the '<flavor>-<subsuite>' mechanism that was
previously being used, but better to be explicit wherever possible.
Depends on D25077
Differential Revision: https://phabricator.services.mozilla.com/D25078
--HG--
extra : moz-landing-system : lando
We should use filter functions instead of regexes here. Would be a lot more robust.
Depends on D25076
Differential Revision: https://phabricator.services.mozilla.com/D25077
--HG--
extra : moz-landing-system : lando
Make sure ProcessHandlerMixin.Process works also under 3.6.
3.6 introduced a _wait() method in its Popen class, that
conflicts with the one in mozprocess. The patch renames it
and also adds timeout where possible.
Differential Revision: https://phabricator.services.mozilla.com/D22157
--HG--
extra : moz-landing-system : lando
Currently, tests in mozsystemmonitor are skipped because the manifest is missing
from mozbase/moz.build. This has now been added.
For the moment, this patch also skips the test if run against Python 3. This is
not ideal - a better fix may be setting the unittest.skipIf condition to skip
if the there is simply a failure when run with Python 3.
Differential Revision: https://phabricator.services.mozilla.com/D22426
--HG--
extra : moz-landing-system : lando
This addition allows formatters to be configured by sending a message.
For example, the show_logs attribute has been added to the GroupingFormatter
to allow switching on/off the display of all log messages at runtime.
A generic handle_message method was added to the BaseFormatter to allow
attributes to be configured, with optional arguments for those that require
it.
An additional log(data) method has been implemented on the GroupingFormatter
that checks self.show_logs, and if True, displays all messages regardless of
failure or success, with component, level and stack.
A test may be required to ensure this is behaving as expected.
Differential Revision: https://phabricator.services.mozilla.com/D23726
--HG--
extra : moz-landing-system : lando
The default color dictionary is implemented, to avoid formattor specifying the colors discretely
Differential Revision: https://phabricator.services.mozilla.com/D24196
--HG--
extra : moz-landing-system : lando
This patch will remove the very long wait on start and stop,
should be down to one second.
Differential Revision: https://phabricator.services.mozilla.com/D23668
--HG--
extra : moz-landing-system : lando
The regex should match '-' characters as well as \w to properly trim the
error message if the device string contains a dash.
Differential Revision: https://phabricator.services.mozilla.com/D22823
--HG--
extra : source : 26031d3623333bbdedd1fb388f0867bc9f7cc23c
The regex should match '-' characters as well as \w to properly trim the
error message if the device string contains a dash.
Differential Revision: https://phabricator.services.mozilla.com/D22823
--HG--
extra : moz-landing-system : lando
The default color dictionary is implemented,to avoid hardcode color information. The functionality for overridding the values in formatter, is hitherto partially
implemented
Differential Revision: https://phabricator.services.mozilla.com/D23134
--HG--
extra : moz-landing-system : lando
Add debug() for the process handler and process reader classes, and add a few
debug() calls that I am interested in.
Differential Revision: https://phabricator.services.mozilla.com/D23946
--HG--
extra : moz-landing-system : lando
I don't know why we're sleeping around `certutil` invocations at all,
but let's at least not spend 10s of seconds doing nothing locally.
Differential Revision: https://phabricator.services.mozilla.com/D17398
--HG--
extra : moz-landing-system : lando
This patch also removes custom_script and stops
executing start() in the constructor.
Differential Revision: https://phabricator.services.mozilla.com/D23288
--HG--
extra : moz-landing-system : lando
The ability to capture the parent process' stdio is suggested to be a useful feature
to move from web-platform/tests into mozlog. To do so, I have created a new capture.py
file within mozlog/mozlog. This includes the CaptureIO class and its dependencies,
including the LoggingWrapper and LogThread classes. These have been removed from their
original location, to avoid duplication, and the files depending on them updated
accordingly.
It would be useful to add unittests testing the CaptureIO enter and exit methods, and
the original_stdio, logging_queue and logging_thread properties. I have begun such a
file with test_capture.py in mozlog/tests. This is a work in progress, however I may
need some guidance, please, in regards to creating appropriate mock data to assert.
Differential Revision: https://phabricator.services.mozilla.com/D22166
--HG--
extra : moz-landing-system : lando
Adds archive extraction support to download_file_from_url (and also make it py3
compatible)
Differential Revision: https://phabricator.services.mozilla.com/D22662
--HG--
extra : moz-landing-system : lando
Make sure we shutdown the mitmproxy process in case the setup fails.
Differential Revision: https://phabricator.services.mozilla.com/D22467
--HG--
rename : testing/mozbase/mozproxy/tests/test.py => testing/mozbase/mozproxy/tests/test_proxy.py
extra : moz-landing-system : lando
This patch moves testing/raptor/raptor/playback into its own testing package in testing/mozbase/mozproxy
so we can use the proxy in other places than Raptor.
Differential Revision: https://phabricator.services.mozilla.com/D21200
--HG--
rename : testing/raptor/raptor/playback/__init__.py => testing/mozbase/mozproxy/mozproxy/__init__.py
rename : testing/raptor/raptor/playback/base.py => testing/mozbase/mozproxy/mozproxy/backends/base.py
rename : testing/raptor/raptor/playback/mitmproxy.py => testing/mozbase/mozproxy/mozproxy/backends/mitm.py
rename : testing/raptor/raptor/playback/mitmproxy-rel-bin-linux64.manifest => testing/mozbase/mozproxy/mozproxy/backends/mitmproxy-rel-bin-linux64.manifest
rename : testing/raptor/raptor/playback/mitmproxy-rel-bin-osx.manifest => testing/mozbase/mozproxy/mozproxy/backends/mitmproxy-rel-bin-osx.manifest
rename : testing/raptor/raptor/playback/mitmproxy-rel-bin-win.manifest => testing/mozbase/mozproxy/mozproxy/backends/mitmproxy-rel-bin-win.manifest
rename : testing/raptor/raptor/playback/mitmproxy_requirements.txt => testing/mozbase/mozproxy/mozproxy/backends/mitmproxy_requirements.txt
extra : moz-landing-system : lando
The HTML formatter now references the correct link and does not produce a 404 error
Differential Revision: https://phabricator.services.mozilla.com/D21207
--HG--
extra : moz-landing-system : lando
The HTML formatter now references the correct link and does not produce a 404 error
Differential Revision: https://phabricator.services.mozilla.com/D21207
--HG--
extra : moz-landing-system : lando
On the Android 7.0 x86_64 emulator, pidof occasionally returns
no results for a running process. To guard against this case, mozdevice
retries exactly once.
Differential Revision: https://phabricator.services.mozilla.com/D20898
--HG--
extra : moz-landing-system : lando
Improve support for Python 3 by running modernize over adb.py and a few other related changes.
Differential Revision: https://phabricator.services.mozilla.com/D19306
--HG--
extra : moz-landing-system : lando
This bit of code annoyingly runs before we know whether we really need an
emulator or not, so the error blocks testing even with a real device.
Differential Revision: https://phabricator.services.mozilla.com/D19700
--HG--
extra : moz-landing-system : lando
Test manifests may be included by multiple other manifests, optionally
with additional variables below the `[include:...]` section header.
These additional variables are specific to the manifest that contained
the "include" section, and should not inadvertently be shared with other
manifests that also happen to include this manifest.
To achieve that, store the defaults for included manifests in a (path to
parent manifest, path to included manifest) tuple instead of just the
included manifest.
Differential Revision: https://phabricator.services.mozilla.com/D18086
--HG--
extra : moz-landing-system : lando
Sebastian Hengst
Thinker Li
Andrew Halberstadt
Edwin Takahashi
Florin Strugariu
Geoff Brown
Gijs Kruitbosch
Mike Hommey
Nathan Froyd
Mihai Alexandru Michis
Botond Ballo
Andrew McCreight
Cosmin Sabou
Gregory Mierzwinski
Bob Clary
Ciure Andrei
Gurzau Raul
J.C. Jones
Philip Jägenstedt
Sylvestre Ledru
Razvan Maries
Dorel Luca
Wes Kocher
Chris Manchester
Christian Holler
shindli
alexandru.ionescu
Henrik Skupin
championshuttler
James Graham
b4hand
William Lachance
Nick Alexander
Tarek Ziadé
Andreas Tolfsen
David Heiberg
Matt Woodrow
Maja Frydrychowicz
Andreea Pavel
Marian Raiciof
Kris Maglione
Tom Prince
Eric Rahm
Mike Shal
Victor Porof
Kartikaya Gupta
Nikki Sharpley
Emilio Cobos Álvarez
Alexandre Poirot
Nikki S
Makoto Kato
Narcis Beleuzu
Bastien Orivel
Andrew Erickson
Joel Maher
Bogdan Tara
Ted Mielczarek
Ionut Goldan
Mike Shal
Brendan Dahl
Ian Moody
Bob Clary
Chris AtLee
vrinda
Noemi Erli
Coroiu Cristina
Sam Sneddon
pgadige
Nupur Baghel
Kriti Singh
Florian Quèze
Asfiya
Dave Hunt
AndreiH
Jan Henning
A. Wilcox
Rob Wu