This will make sure that when running |mach python-test --python 3| locally,
we only run the tests that also run in CI with python 3 (and therefore pass
presumably).
MozReview-Commit-ID: 3OBr9yLSlSq
--HG--
extra : rebase_source : 456340d0ecdddf1078f2b5b4ebb1eddf3813b26a
This patch allows executing |mach python-test| against Python 3 by specifying the optional |--three| command line option. When this option is present, pipenv will be used to manage a virtual environment using Python 3 and attempt to run the tests. When it is not present, pipenv will not be used, and everything will work as it did before this patch.
My original plan was to use pipenv regardless of the target version of Python, however I encountered several issues running some of the tests against our Python packages. Once all tests have been patched to run against Python 3, then we should be able to use pipenv when running them against Python 2.
Note that this patch allows tests to run against Python 3, but there are plenty of issues preventing them from passing. With this patch in place we can start to add Python 3 support to our packages and have the tests running in CI to ensure we don't regress back to just supporting Python 2.
MozReview-Commit-ID: BuU5gZK83hL
IHG: changed taskcluster/ci/source-test/python.yml
--HG--
extra : rebase_source : ca2b15d905f7a5c895a2fd8916144841f5d205de
This patch allows executing |mach python-test| against Python 3 by specifying the optional |--three| command line option. When this option is present, pipenv will be used to manage a virtual environment using Python 3 and attempt to run the tests. When it is not present, pipenv will not be used, and everything will work as it did before this patch.
My original plan was to use pipenv regardless of the target version of Python, however I encountered several issues running some of the tests against our Python packages. Once all tests have been patched to run against Python 3, then we should be able to use pipenv when running them against Python 2.
Note that this patch allows tests to run against Python 3, but there are plenty of issues preventing them from passing. With this patch in place we can start to add Python 3 support to our packages and have the tests running in CI to ensure we don't regress back to just supporting Python 2.
MozReview-Commit-ID: BuU5gZK83hL
IHG: changed taskcluster/ci/source-test/python.yml
--HG--
extra : rebase_source : e1a64c0ffa8fe5cce71a041579601d4a72e37779
This is a new issue that gets linted with flake8 3.5.0. Basically you should
never use a blank except: statement.
This will catch all exceptions, including KeyboardInterrupt and SystemExit
(which is likely not intended). If a catch all is needed, use
`except: Exception`. If you *really* mean to also catch KeyboardInterrupt et
al, use `except: BaseException`.
Of course, being specific is often better than a catch all.
MozReview-Commit-ID: FKx80MLO4RN
--HG--
extra : rebase_source : 7c74a7d0d81f2c984b47aff3a0ee3448b791177b
Currently, marking a logger as a structured logger will require a
subsequent function call in order for the logger to be hooked up
to active handlers. This behavior is not intuitive and makes it
easy to not have handlers for newly-registered loggers. This means
messages may not be logged anywhere.
In addition, we have to manually specify which named loggers to
enable structured logging for. This can be annoying.
We change the behavior of register_structured_logger() to
automatically add existing terminal and json handlers to the
logger being marked as structured.
We also introduce an API to enable structured logging for all
loggers. Existing consumers of registered_structured_logger()
in mozbuild have been updated to use this API. A new consumer
has been added for the `mach configure` command because it should
have been there before.
We stop short of making enable_all_structured_loggers() the default.
This is because various commands interact with the log manager in
ways that will result in duplicate logging of messages and
dropping of structured messages. There is a bit of a rabbit hole
here and addressing it can be done as a follow-up.
MozReview-Commit-ID: 1aU6eJvTSMP
--HG--
extra : rebase_source : 2a0a569b378cc3083b55fc7076b291abdfb7453f
We don't use this attribute outside of this function. Besides, you can
easily obtain a handle on the logger by calling
``logging.getLogger('mach')``.
MozReview-Commit-ID: 41vdn6McowW
--HG--
extra : rebase_source : dca383e5f5b770fa3adb09d753897633258302a1
This passes `python3 -mcompileall`. Changes:
* use `0o` prefix for octal literals
* print as a function
* except .. as
* use six.reraise to replace a multi-argument raise statement
* use six.string_types and six.moves.configparser
* remove uses of `L` suffix for long integers
MozReview-Commit-ID: KLaYRNHGpay
--HG--
extra : rebase_source : 6ca1b5447cd28eff8d9f2805add6a0f07e8b4c63
This preserves ./mach settings' --list option. If --list is passed in, we call splitlines()
on the description and print only the first line. The full multi-line description will be
printed otherwise.
This also displays the type and/or choices of the option as appropriate.
MozReview-Commit-ID: 7UMsN9qslWt
--HG--
extra : rebase_source : 4bc9554d8652e02e290c6a190634f1a72cdbadc3
We currently raise if we detect a command has both the `parser` attribute and a subcommand
at the same time, but as far as I can tell, there's no good reason to do this. Handling a
parser + subcommands seems to work exactly how you would expect. Furthermore, it isn't an
error to have subcommands + @CommandArgument, so it doesn't make sense that we're allowing
one but not the other.
This change solves an (admittedly unique) use case I'm trying to build into |mach try|. There
are N subcommands that all support a --save and --load style argument. So, e.g, we might have:
./mach try syntax --save foo
./mach try fuzzy --save bar
The main command will have the ability to detect which subcommand a saved value was generated
from and automatically dispatch to it. So this will work:
./mach try --load foo # dispatches to the syntax subcommand
./mach try --load bar # dispatches to the fuzzy subcommand
In order to share the --save/--load arguments across the main command + subcommands, we need
to set the parser attribute.
MozReview-Commit-ID: KmXRj8TBvYK
--HG--
extra : rebase_source : cbf1e402a080913709a34430274ae3191821dd72
This will facilitate handling this error condition specially.
Some unused imports were also removed.
MozReview-Commit-ID: 4zxVDgE7NxU
--HG--
extra : rebase_source : c4d0dd96629e028ce8a83215564a8d865e4b4b3d
Prefs can be set with `./mach run --setpref foo=bar` or in the ~/.mozbuild/machrc file as:
[runprefs]
foo=bar
MozReview-Commit-ID: HO3tdFi9ffi
--HG--
extra : rebase_source : 3c2e6f33567448c19defafb656e6cb9f3a729391
This allows subcommands to use external argument parsers.
MozReview-Commit-ID: 7TkbTff0Tv8
--HG--
extra : rebase_source : a1c245efa7ac7b28b974534b4cd2727c96f9219d
After this patch, the following will all display the subcommand help where they previously displayed
the command help:
$ mach help <command> <subcommand>
$ mach <command> --help <subcommand>
$ mach <command> <subcommand> --help
The command help will still be shown for:
$ mach help <command>
$ mach <command> --help
MozReview-Commit-ID: 7EsVblnCaFM
--HG--
extra : rebase_source : 2a1d289d56164366ce140fa653adec93f56be067
The sgr0 terminal capabity defines the byte sequence needed to reset
terminal text to its default state. For reasons documented inline in
this commit, we now print this sequence after every line printed by
mach's terminal logger.
MozReview-Commit-ID: 3RukP0QXtqy
--HG--
extra : rebase_source : 5e4b7d001300ec1059b53423b310ac9fdd514c72
This deprecates PYTHON_UNIT_TESTS and replaces it with PYTHON_UNITTEST_MANIFESTS.
In the build system, this means python unittests will be treated the same as all
other test suites that use manifestparser. New manifests called 'python.ini' have
been created for all test directories containing python unittests.
MozReview-Commit-ID: IBHG7Thif2D
--HG--
extra : rebase_source : 11a92a2bc544d067946bbd774975140147458caa
The way it works now, `mach` commands often invoke subprocesses where
the subprocesses' stdio file descriptors are pipes so the mach command
can e.g. parse output.
Processes like clang, gcc, and cargo determine if they can send color
codes to {stderr, stdout} by seeing if those file descriptors are TTYs.
When e.g. `make` is executed via `mach`, this test fails because those
descriptors are pipes (even though they eventually end up on a TTY).
We can't wire the file descriptors to the TTY because `mach` needs
to analyze output. We don't want users defining process flags to force
color in their mozconfigs because color codes would still be sent
if stdout was not a TTY.
This patch sets the MACH_STDOUT_ISATTY environment variable in all mach
commands when stdout is a TTY. Subsequent processes can then look for
this variable to determine whether to override color settings, print
terminal control codes, etc.
MozReview-Commit-ID: GxXP2mQssjC
--HG--
extra : rebase_source : 4b99547b453cb7dd5cb590a71ed554ce2bc4759d
Currently, environment variables set when running mach commands will
propagate after the command is finished. This can allow unwanted state
to bleed through.
This likely isn't an issue today, but isolating state during code
execution is generally a good practice. So do that.
MozReview-Commit-ID: AdaomGub5EF
--HG--
extra : rebase_source : ce81987a1f6de3a16bce6a9e45b9dc8e8eb29b4b
Before, mach would only detect MINGW32 (used with msys). Now it also detects MINGW64 (used with msys2).
MozReview-Commit-ID: 6IrNNmkLKgB
--HG--
extra : rebase_source : e84ee16b5bbc1b6b113163ba5b61759f763645c1
MozReview-Commit-ID: AQ3w2oCPQeN
--HG--
extra : topic : bar
extra : rebase_source : 2e0de1688fca11bd62c88bd66ef9cf9ad97206f8
extra : amend_source : 45c025594d645a9252989c8f13387de05e49d4dd
These config options can be defined in ~/.mozbuild/machrc or topsrcdir/machrc.
Aliases work similar to the identically named option in an hgrc.
For example:
[alias]
browser-test = mochitest -f browser
mochitest = mochitest -f plain
MozReview-Commit-ID: CnOocEslRUI
--HG--
extra : rebase_source : 2a6fa154aca7fea8f159ed840728951a37bc52ec
Runtime configs have been implemented for awhile, but disabled. This patch
enables configuration. Config files will be loaded in the following order
(later files override earlier ones):
1a. $MACHRC
1b. $MOZBUILD_STATE_PATH/machrc (if $MACHRC is unset)
2. topsrcdir/machrc
3. CLI via --settings
Note: .machrc may be used instead of machrc if desired.
MozReview-Commit-ID: IntONAZLGML
--HG--
extra : rebase_source : ff79b129eaea7cca5064d30fa6ddc76fceb9669b
This adds a |mach settings locale-gen| subcommand to automatically generate locale
specific documentation for settings. It also refactors |mach settings-create| to
|mach settings| and moves |mach settings| to |mach settings -l|. Finally it performs
some misc cleanup mostly related to locales.
MozReview-Commit-ID: 1VWLcb9ehAH
--HG--
extra : rebase_source : 8f580217123d79e66323ca4be948a3297ae4ced3
Some sections should support user-defined options. For example, in an [alias] section, the option names
are not well-defined, rather specified by the user. This patch allows user-defined option names for any
section that has a 'section.*' option defined. Even with 'section.*', option types are still well-defined.
MozReview-Commit-ID: L34W9v9Fy28
--HG--
extra : rebase_source : 9333f552edead9bf1cf464e28ef8fbbb9bed5597
Defining settings was a little complicated. First it required overriding a '_register_settings'
method, and then it required making N calls to a second 'register_setting' method within that.
This patch simplifies the process of defining settings by only requiring a
'config_settings' attribute. This attribute should be a list of tuples:
[
('<section>.<option>', '<type>', <default>, set(<choices)),
]
`default` and `choices` are optional. Alternatively, 'config_settings' can be a callable
that returns a list of tuples in the same format. This still allows for greater flexibility
for more advanced cases.
MozReview-Commit-ID: F4DTTNJdJsa
--HG--
extra : rebase_source : e3dd455ba559cd3992c9c1b3eaf021c9e0707cc1
For example, say there is a command 'foo' that has a subcommand 'bar'. Prior to this, it was not
possible to run:
./mach foo
as its own independent command. The above would instead print the subcommand help for 'bar'.
MozReview-Commit-ID: JU4dXoxnCyu
--HG--
extra : rebase_source : bb15532ad39456b270071bc60d7b15e15af04e48
This is currently preventing a command from having both args and subcommands at the same
time.
MozReview-Commit-ID: 66frAqamGjv
--HG--
extra : rebase_source : 3bbd1cb508e5aab7ba7f9936c0c486ed8f626fe9
This adds a post_dispatch_handler hook that will be called after each
mach invocation and uses it to submit data to telemetry.
--HG--
extra : rebase_source : 1dedea568b7ccec17fa58eb1841425b165d8a644
Mach has special handling for remainder arguments which previously
only worked with the decoration form. Extend this to also work when
a command passes in an explict parser
--HG--
extra : commitid : ICnYk764Xbc
extra : rebase_source : 1518edcb7bef375cbc2931b8da54b015045c8ce1
This will give us the ability to execute custom code at command dispatch
time. We can use this for global tests before dispatch.
--HG--
extra : commitid : 5vfmSqOis4W
extra : rebase_source : c285b16007c4b604b164d079a275198c8760e480
Without this, we throw UnrecognizedArgumentError when running commands
such as `./mach mochitest-plain test`, which causes an error message
such as the below to be emitted:
It looks like you passed an unrecognized argument into mach.
The mochitest-plain command does not accept the arguments: test
This patch will fix the above command to instead print the corresponding
deprecation message.
This is a regression from bug 1176620 that results in all Firefox mach commands showing up in the help for B2G, even though they don't work there.
--HG--
extra : rebase_source : e779b866a82c2df1dbf913d24b93a0dc9838ff9f
This type is now redundant with our new rich type for capturing all mach
command metadata. Eliminate it and using _MachCommand instead.
--HG--
extra : commitid : 7pXhf6V7F8m
extra : rebase_source : 343615096f23d3acf23f7487c7b8c7137c85337e
Chris Manchester
Andrew Halberstadt
Dave Hunt
Andreea Pavel
David Major
Tarek Ziadé
Gregory Szorc
Steve Armand
Dustin J. Mitchell
Jonas Finnemann Jensen
Brian Grinstead
Michael Layzell
Nathan Hakkakzadeh
Steve Fink
Dan Minor
Robert Longson
Gijs Kruitbosch
Xidorn Quan
James Graham
Supradeep T R
Ehsan Akhgari