This patch makes BuildEnvironmentNotFoundException a subclass of AttributeError as well, because hasattr in
python3 no longer catches all tracebacks but only AttributeError, and we use both hasattr and
BuildEnvironmentNotFoundException to guard against a handful of buildconfig variables in a few places
where it is OK to not have a buildenvironment.
We also use universal_newlines in real_host in init.configure (since I found
that fix while working on the AttributeError one) so that we get the right string type back from the process call
Lastly this patch also uses BytesIO for calling into a ReducedConfigureSandbox as its stdout and stderr pipes,
This ensures that related code handling the sandbox doesn't complain about getbuffer() missing in StringIO in py3.
Differential Revision: https://phabricator.services.mozilla.com/D36605
--HG--
extra : moz-landing-system : lando
The build docker images need python-dev installed to build psutil, used
by the build resources monitor.
Differential Revision: https://phabricator.services.mozilla.com/D40781
--HG--
rename : python/mozbuild/mozbuild/resources/html-build-viewer/index.html => python/mozbuild/mozbuild/resources/html-build-viewer/build_resources.html
extra : moz-landing-system : lando
The longer version is that this changes how the HTML viewer looks up
files. Instead of looking at a list file that contains a list of
build resources data, it now looks at a build_resources.json file that
either contains data directly, or a list of files containing data.
Differential Revision: https://phabricator.services.mozilla.com/D40780
--HG--
extra : moz-landing-system : lando
This patch adds a global lint that only runs when a file or directory
that matches their configuration (via `extensions` and `exclude`) has
been modified or specified.
Global lints never shard into chunks; they are, by definition global
(i.e., across the entire source tree) and act on all inputs in a
single invocation. It's up to the global lint to manage command line
sizes, etc. Since batching is handled by the lint type but sharding
is handled by the lint roller, there's a little abstraction leak so
that the lint type can control how its invocation is sharded: the
existing `batch` member is generalized from the existing `True` and
`False` to add a new `"global"` value which disables sharding.
Differential Revision: https://phabricator.services.mozilla.com/D35275
--HG--
extra : moz-landing-system : lando
This allows lints to "condition" themselves on having a build
environment or a specific build application. It also adds the "name"
parameter, so that setup functions can be shared across lints.
`MozbuildObject` cannot be used as parameters to functions distributed
via multiprocessing, since they cannot be pickled (due, currently, to
internal terminal handles). Therefore we extract just a few key
parts of the environment to expose.
Differential Revision: https://phabricator.services.mozilla.com/D35274
--HG--
extra : moz-landing-system : lando
On Windows in Python 2, the subprocess module requires the use of bytes with
the 'env' argument. For that reason, we would sometimes use byte strings with
'os.environ' like so:
os.environ[b"FOO"] = b"bar"
However, this is a failure with Python 3 as 'os.environ' must only be used with
the text type. This patch creates a new 'setenv' helper that ensures we create
new environment with 'bytes' on Python 2, and 'text' on Python 3.
Differential Revision: https://phabricator.services.mozilla.com/D38237
--HG--
extra : moz-landing-system : lando
It causes confusion because it's the mode used to _read_ the overwritten
file. Make that more obvious by renaming to `readmode`.
Differential Revision: https://phabricator.services.mozilla.com/D39445
--HG--
extra : moz-landing-system : lando
There is less incentive to keep things building with older versions of
clang for OSX builds, and we're going to require an objective-C feature
that was added in clang 5.
Differential Revision: https://phabricator.services.mozilla.com/D38581
--HG--
extra : moz-landing-system : lando
The preprocessed install manifest code uses a Makefile dependency file
to list inputs. If one of the inputs was missing (due to a file removal
or rename), an incremental build would fail until clobbered. We should
treat a missing input as simply being out-of-date and continue
processing.
Differential Revision: https://phabricator.services.mozilla.com/D38576
--HG--
extra : moz-landing-system : lando
Rather than relying on the mar-channel-id set in the `mar` binary, set the channel
explicitly from taskcluster. This allows us to re-use the `mar` binary between
builds/channels.
Differential Revision: https://phabricator.services.mozilla.com/D37481
--HG--
extra : moz-landing-system : lando
Credit: Callek for figuring out an issue in 'make check' making the binary
absolute in mozbuild.base.
Differential Revision: https://phabricator.services.mozilla.com/D37319
--HG--
extra : moz-landing-system : lando
When building Gecko/Android/aarch64 on Windows, `--target` parameter may not be incorrect value. Although `check_compiler`'s `info` is target compiler, clang on Windows is always detected as `clang-cl`, not `clang`.
```
c:/Users/mkato/.mozbuild/clang/bin/clang.exe -E -dM - < /dev/null
...
#define _MSC_VER 1916
```
So even if using clang on Windows, not clang-cl, we should detect as 'clang' correctly
Differential Revision: https://phabricator.services.mozilla.com/D36422
--HG--
extra : moz-landing-system : lando
By globally importing PackageFrontend from the globe-analysis module we break the logger for
the PackageFrontend package.
Differential Revision: https://phabricator.services.mozilla.com/D36887
--HG--
extra : moz-landing-system : lando
The SDK headers may not be installed in /usr/include. The usual response
has been to have people run e.g. `open
/Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg`
which is not really sustainable.
This makes builds that happen on a macOS host try to detect their SDK
and use that as a default for --with-macos-sdk, which has the side
effect of enabling the SDK version check in that configuration.
Differential Revision: https://phabricator.services.mozilla.com/D36558
--HG--
extra : moz-landing-system : lando
This test is working with minimal effort. Let's get it running to prevent
future regressions.
Differential Revision: https://phabricator.services.mozilla.com/D36098
--HG--
extra : moz-landing-system : lando
Configure should just be able to find the right one. If it doesn't, that
should be fixed in configure rather than with suggestions in bootstrap.
Differential Revision: https://phabricator.services.mozilla.com/D36562
--HG--
extra : moz-landing-system : lando
Configure should just be able to find the right one. If it doesn't, that
should be fixed in configure rather than with suggestions in bootstrap.
Differential Revision: https://phabricator.services.mozilla.com/D36562
--HG--
extra : moz-landing-system : lando
Bug 1554987 made `mach try` use a transient remote, but that causes
problems with existing setups that happen to use the same remote name,
because of a combination of not-quite-as-documented-as-it-should
behavior of git.
- `git -c foo.bar=qux` doesn't override the value of `foo.bar` from the
git configuration when `foo.bar` is an item that can take several
values.
- `remote.$remote.url` and `remote.$remote.pushurl` take several values,
allowing to give several URLs.
The combination of both means that if the git configuration already has
`remote.try.url` set, that value takes precedence (because git push
tries them one after the other, and takes the one from the command line
last)
One way we could increase the chances of things working out fine would
be to use `remote.try.pushurl`, which if already set, is more likely to
be right than an existing `remote.try.url`.
OTOH, it turns out, after more investigation, that bug 1554987 requires
a footgunny setup to happen in the first place. Namely, it requires
having run `git lfs install` from a git-cinnabar clone.
so we just go back to the previous status quo.
Differential Revision: https://phabricator.services.mozilla.com/D36149
--HG--
extra : moz-landing-system : lando
See https://stackoverflow.com/a/24026735.
Adding the `docs` package requirement is not ideal, but it's not worth
the effort to install it only in automation (or in the relevant task),
and it's not *that* large: 1.0G on my macOS installation.
Differential Revision: https://phabricator.services.mozilla.com/D35834
--HG--
extra : moz-landing-system : lando
4.3.3 is the strict minimum required for v-c-t's config wizard, but it
is preferable people use more modern versions. We could go with 5.0, but
it feels like people still using 4.8 and 4.9 don't really need to be
bugged to update to a more recent version, they are kind of modern
enough. OTOH MozillaBuild comes with 4.5.x, and this will force an
upgrade for those.
Differential Revision: https://phabricator.services.mozilla.com/D35756
--HG--
extra : moz-landing-system : lando
This makes the bootstrap behavior wrt. Mercurial consistent on all
platforms, making Windows bootstrap only upgrade Mercurial if the
version is older than MODERN_MERCURIAL_VERSION.
As a side effect, this avoids upgrading to version 5.0.1, which doesn't
come with wheels at the moment.
Differential Revision: https://phabricator.services.mozilla.com/D35754
--HG--
extra : moz-landing-system : lando
The `mach resource-usage` page supports a list of build resource data,
but only ever displays the first one. Nothing actually creates a list
with multiple items automatically, but one might want to do that
manually to explore data from multiple builds more conveniently.
So if such a list exists, we expose a dropdown list of all the data
available, and switch the graph when a different item is chosen from the
list.
Differential Revision: https://phabricator.services.mozilla.com/D35757
--HG--
extra : moz-landing-system : lando
The git version shipped in some versions of OSX is patched by apple in a
way such that doing `git push hg::ssh://...` fails with an error message
like `Invalid remote name "hg::ssh://...`.
So instead, we define a named remote via inline configuration, and use
that remote's name for the push.
Differential Revision: https://phabricator.services.mozilla.com/D35351
--HG--
extra : moz-landing-system : lando
The git version shipped in some versions of OSX is patched by apple in a
way such that doing `git push hg::ssh://...` fails with an error message
like `Invalid remote name "hg::ssh://...`.
So instead, we define a named remote via inline configuration, and use
that remote's name for the push.
Differential Revision: https://phabricator.services.mozilla.com/D35351
--HG--
extra : moz-landing-system : lando
This makes running without mach more consistent. e.g. running
`make -C $objdir/toolkit/library/rust target` makes the cargo log
verbose, and adding `-s` makes it less verbose.
Differential Revision: https://phabricator.services.mozilla.com/D35521
--HG--
extra : moz-landing-system : lando
When the `MOZ_RUST_TIER` environment variable is set, we enable a separate
tier that builds rust code only. This is useful to measure build times for
rust code separately from other compilations.
Differential Revision: https://phabricator.services.mozilla.com/D35501
--HG--
extra : moz-landing-system : lando
NSS_ALLOW_SSLKEYLOGFILE no longer has issues upstream, we can allow it again.
Differential Revision: https://phabricator.services.mozilla.com/D34915
--HG--
extra : moz-landing-system : lando
Justin Wood
rdalal
Mike Hommey
Nick Alexander
Zibi Braniecki
Andreea Pavel
Moritz Birghan
Bryce Seager van Dyk
Joel Maher
Andrew Halberstadt
Johan Lorenzo
Bastien Abadie
Andi-Bogdan Postelnicu
James Willcox
Edwin Gao
Makoto Kato
Mike Shal
Simon Sapin
Agi Sferro
Chris Manchester
Tom Prince
Victor Porof
Ed Lee
Francesco Lodolo (:flod)
Ryan VanderMeulen
Jeane Carlos
Sylvestre Ledru
Samuel Thibault
Kartikaya Gupta
championshuttler
Cosmin Sabou
Rok Garbas
Dorel Luca
prathiksha
L. David Baron
Eric Rahm
Bob Clary