Essentially:
FilterPath("docs").contains("docshell")
was returning True because "docshell" startswith "docs".
Differential Revision: https://phabricator.services.mozilla.com/D64661
--HG--
extra : moz-landing-system : lando
Windows programs run via Wine don't like Unix absolute paths (they look
like command line arguments), so we need to use relative paths.
Mingw already run fxc2 via wine, but for some reason it doesn't care
about the Unix absolute paths. genshaders does need some adjustements to
run properly with the real fxc.
Now, on actual Windows, because the temporary directory where
tempfile.NamedTemporaryFile creates files by default is not necessarily
on the same drive as where the command runs from, a relative path can't
be constructed. So we also force the temporary file to be created in the
current (obj) directory.
There is no similar concern for other files because we only go from
objdir to srcdir, and the build system already doesn't support both
being on a separate drive.
While here, flush stdout when the genshared script writes to it, so that
the messages are printed out immediately rather than randomly, later,
after output from subprocesses.
Differential Revision: https://phabricator.services.mozilla.com/D64294
--HG--
extra : moz-landing-system : lando
Makepkg has a commented config in /etc/makepkg.conf to set the path for packages to be built:
```
#PKGDEST=/home/packages
```
When you uncommit this line, packages will be built in the specified path, instead of cwd. This causes the bootstrap script to fail.
This commit will override the PKGDEST config so that package is built in cwd and could be located by the following lines of script.
Differential Revision: https://phabricator.services.mozilla.com/D64195
--HG--
extra : moz-landing-system : lando
The motivating change here was wpt-sync try pushes failing because
`libgraphitewasm.dylib` wasn't getting packaged in Mac artifact builds.
We add `libosclientcerts.dylib` because packaging was complaining about
that library as well.
Differential Revision: https://phabricator.services.mozilla.com/D63705
--HG--
extra : moz-landing-system : lando
Without PYTHON3 defined, we can't actually run any GENERATED_FILES
scripts in the fastermake backend.
Differential Revision: https://phabricator.services.mozilla.com/D63437
--HG--
extra : moz-landing-system : lando
Without PYTHON3 defined, we can't actually run any GENERATED_FILES
scripts in the fastermake backend.
Differential Revision: https://phabricator.services.mozilla.com/D63437
--HG--
extra : moz-landing-system : lando
Install moz-phab using the correct command for the current operating system as
per moz-phab's documentation.
Differential Revision: https://phabricator.services.mozilla.com/D59139
--HG--
extra : moz-landing-system : lando
GENERATED_FILES now defaults to python3 unless py2=True is specified as
an argument. All existing GENERATED_FILES scripts and GeneratedFile
templates have the py2=True attribute added, so this patch should
effectively be a no-op.
Going forward, individual scripts can be converted to python3 and their
corresponding py2=True attribute can be deleted. In effect, this patch
will be backed out in pieces until all scripts run in python3, at which
point the py2 attribute itself can be removed.
Differential Revision: https://phabricator.services.mozilla.com/D60919
--HG--
extra : moz-landing-system : lando
We're going to enable this on Mac, and it won't do to have configure
assert when we actually do so.
Differential Revision: https://phabricator.services.mozilla.com/D62799
--HG--
extra : moz-landing-system : lando
This fixes str-vs-bytes in code paths that are only executed
with unapplied migrations, where we validate some parts of the
generated commits.
Differential Revision: https://phabricator.services.mozilla.com/D63049
--HG--
extra : moz-landing-system : lando
NodeJS 8.x is End-of-Lifed and is no longer receiving security fixes. 10.19.0 is now the oldest Long Term Support version of NodeJS, and it has just been released with several HTTP security fixes.
Differential Revision: https://phabricator.services.mozilla.com/D62781
--HG--
extra : moz-landing-system : lando
GENERATED_FILES now defaults to python3 unless py2=True is specified as
an argument. All existing GENERATED_FILES scripts and GeneratedFile
templates have the py2=True attribute added, so this patch should
effectively be a no-op.
Going forward, individual scripts can be converted to python3 and their
corresponding py2=True attribute can be deleted. In effect, this patch
will be backed out in pieces until all scripts run in python3, at which
point the py2 attribute itself can be removed.
Differential Revision: https://phabricator.services.mozilla.com/D60919
--HG--
extra : moz-landing-system : lando
We're going to enable this on Mac, and it won't do to have configure
assert when we actually do so.
Differential Revision: https://phabricator.services.mozilla.com/D62799
--HG--
extra : moz-landing-system : lando
GENERATED_FILES now defaults to python3 unless py2=True is specified as
an argument. All existing GENERATED_FILES scripts and GeneratedFile
templates have the py2=True attribute added, so this patch should
effectively be a no-op.
Going forward, individual scripts can be converted to python3 and their
corresponding py2=True attribute can be deleted. In effect, this patch
will be backed out in pieces until all scripts run in python3, at which
point the py2 attribute itself can be removed.
Differential Revision: https://phabricator.services.mozilla.com/D60919
--HG--
extra : moz-landing-system : lando
This patch works around the removal of the platform.linux_distribution() function in Python 3.8 and later. It uses the in-tree third party distro package to provide linux distribution information to build system bootstrap.
Differential Revision: https://phabricator.services.mozilla.com/D60359
--HG--
extra : moz-landing-system : lando
This change adds `-ferror-limit=0` to the "command" lines in the
compile_commands.json file that's built by mach.
Differential Revision: https://phabricator.services.mozilla.com/D61392
--HG--
extra : moz-landing-system : lando
Changes:
Build string using the `format()` instead of `%` and relocate the `expandtabs` call to not trigger a AttributeError exception.
Differential Revision: https://phabricator.services.mozilla.com/D60939
--HG--
extra : moz-landing-system : lando
Changes:
Use compatibility layer provided by six for `iteritems` and `itervalues`.
Make `urlparse` import compatible with both 2/3.
Differential Revision: https://phabricator.services.mozilla.com/D60944
--HG--
extra : moz-landing-system : lando
This is a ride-along commit that would have made debugging the problem solved
by this bug easier.
Depends on D60688
Differential Revision: https://phabricator.services.mozilla.com/D60689
--HG--
extra : moz-landing-system : lando
1.9.0 is outdated, and the changes from bug 1609832 actually require 1.20.0.
1.21.1 is the latest current release.
Differential Revision: https://phabricator.services.mozilla.com/D60630
--HG--
extra : moz-landing-system : lando
Bug 1210157 added `unicode_literals` to the script but one use of
literals needed to stay a bytes string.
Differential Revision: https://phabricator.services.mozilla.com/D60628
--HG--
extra : moz-landing-system : lando
Instead of showing percentages, show actual numbers for memory usage,
with caveats:
- On macOS, psutil doesn't seem to be getting anything that would sum up
to the total it reports, so we keep the "percent".
- While the sum of all the fields shown on Linux does sum up to the total,
the visible usage doesn't quite match what the "percent" look like for
the same dataset.
- On Windows, only "used" and "free" are available. They do sum up to
the total.
Differential Revision: https://phabricator.services.mozilla.com/D60226
--HG--
extra : moz-landing-system : lando
This was happening because we had two exclude paths that were run through the 'collapse' algorithm:
editor/libeditor/tests/browserscope/lib/richtext
editor/libeditor/tests/browserscope/lib/richtext2
The problem was that in order to determine the base directory, the algorithm called
'os.path.commonprefix'. This function just returns the common string prefix, which
is the '.../lib/richtext' path. Even though in the above, the base *should* have
been '.../lib'.
To fix the problem, we add a check to ensure the computed base doesn't have any
sibling directories with the same prefix. If there are, it means 'commonprefix' was
too greedy.
Differential Revision: https://phabricator.services.mozilla.com/D60173
--HG--
extra : moz-landing-system : lando
The original code had unfinished code to switch between different
categories, but I found it more useful to be able to see both CPU and
memory at the same time.
Differential Revision: https://phabricator.services.mozilla.com/D60111
--HG--
extra : moz-landing-system : lando
Two of these no longer have any misc:: rules associated with them, and
can be removed outright.
The remaining misc:: rule in toolkit/components/telemetry is already
traversed because it contains EXTRA_COMPONENTS, which are processed
during misc in the RecursiveMake backend. So we can remove HAS_MISC_RULE
entirely even though we still have a custom misc rule to process addons
(until bug 988938 is fixed).
Differential Revision: https://phabricator.services.mozilla.com/D59425
--HG--
extra : moz-landing-system : lando
Legacy extensions are no longer loaded, so we can drop the build config for it. We
still need flags for handling experimental APIs since what we require differs between builds
and distributions.
Differential Revision: https://phabricator.services.mozilla.com/D57413
--HG--
extra : moz-landing-system : lando
Top Sites must be disabled for this to work correctly. This is because we haven't defined the behaviour for when two restricting providers are active. Bug 1607797 must be resolved before this is preffed on.
Differential Revision: https://phabricator.services.mozilla.com/D59351
--HG--
extra : moz-landing-system : lando
We renamed the toolchain task in bug 1608428 and we forgot to update
bootstrap along with the renaming.
Differential Revision: https://phabricator.services.mozilla.com/D59699
--HG--
extra : moz-landing-system : lando
This gets rid of the last use of the 'which' module left in the tree. So not
only will this help 'configure' become a little more Python 3 compatible, but
we can now remove 'third_party/python/which'.
Differential Revision: https://phabricator.services.mozilla.com/D37427
--HG--
extra : moz-landing-system : lando
This gets rid of the last use of the 'which' module left in the tree. So not
only will this help 'configure' become a little more Python 3 compatible, but
we can now remove 'third_party/python/which'.
Differential Revision: https://phabricator.services.mozilla.com/D37427
--HG--
extra : moz-landing-system : lando
Since we have a limit of the number of arguments length that the shell can support we have to
pass the files for the analysis to `run-clang-tidy.py` in chuncks, calculated in proportion
to the number or logical cpu cores.
Differential Revision: https://phabricator.services.mozilla.com/D58971
--HG--
extra : moz-landing-system : lando
Many of the utility classes very heavily rely on multiple inheritance which is broadly unncessary and made debugging difficult, so I ripped it out wherever it was necessary. The indented_repr() function is also more or less a re-implementation of pprint, so to support Python 3 I've instead made it a small wrapper around pprint. (This results in the output of the function being slightly different than we're used to, as pprint does sorting/indentation/line breaking differently than indented_repr() currently does, but it'll be nice to not have to maintain this code that is slightly different than pprint for no real reason.)
Differential Revision: https://phabricator.services.mozilla.com/D58306
--HG--
extra : moz-landing-system : lando
Make the choice of VCS work via an interactive prompt rather than by
command line argument, just like we do for all other choices.
Differential Revision: https://phabricator.services.mozilla.com/D58614
--HG--
extra : moz-landing-system : lando
Warning flags might have been generated for a different compiler, so we
can't use them here. The same argument applies to coverage flags.
Depends on D58070
Differential Revision: https://phabricator.services.mozilla.com/D58071
--HG--
extra : moz-landing-system : lando
The optimization flags that may have been chosen for our target compiler
are not necessarily appropriate for our wasm compiler.
Differential Revision: https://phabricator.services.mozilla.com/D58070
--HG--
extra : moz-landing-system : lando
There was only a single use of 'install-to-subdir' in all our manifests (which
was removed in the previous commit). All the comments around it suggest that it
was a giant hack. Removing it cleans up a lot of complexity and saves me the
trouble of needing to support it in the new sourcedir based TestResolver.
Differential Revision: https://phabricator.services.mozilla.com/D57170
--HG--
extra : moz-landing-system : lando
By default we create both the RecursiveMake and FasterMake backends, but
building.py is only checking if the RecursiveMake backend is out of date
to decide if the backend needs to be updated. Since the FasterMake
backend may have different input files, we also need to check if that's
out of date, otherwise renaming certain files can cause './mach build
faster' to break.
Differential Revision: https://phabricator.services.mozilla.com/D58010
--HG--
extra : moz-landing-system : lando
Change import order.
Enable python3 testing for `test_copier.py`.
In the test methods, ensure the returned file names are decoded as UTF-8 by default.
In the test methods, check for byte-string by default; in python2, this would be treated like the standard string.
Differential Revision: https://phabricator.services.mozilla.com/D55187
--HG--
extra : moz-landing-system : lando
This undoes the mistaken move of this directory into third_party that happened
as part of bug 1346025.
The .flake8 changes are required because lldbutils used to be excluded from
linting by virtue of being inside third_party, but would otherwise be subject
to it after this move. Fixing the existing lint errors will happen in:
- https://bugzilla.mozilla.org/show_bug.cgi?id=1605144
- https://bugzilla.mozilla.org/show_bug.cgi?id=1605145
Differential Revision: https://phabricator.services.mozilla.com/D57141
--HG--
rename : third_party/python/lldbutils/README.txt => python/lldbutils/README.txt
rename : third_party/python/lldbutils/lldbutils/__init__.py => python/lldbutils/lldbutils/__init__.py
rename : third_party/python/lldbutils/lldbutils/content.py => python/lldbutils/lldbutils/content.py
rename : third_party/python/lldbutils/lldbutils/general.py => python/lldbutils/lldbutils/general.py
rename : third_party/python/lldbutils/lldbutils/gfx.py => python/lldbutils/lldbutils/gfx.py
rename : third_party/python/lldbutils/lldbutils/layout.py => python/lldbutils/lldbutils/layout.py
rename : third_party/python/lldbutils/lldbutils/utils.py => python/lldbutils/lldbutils/utils.py
extra : moz-landing-system : lando
We recently updated the version of pip we use, and it no longer supports
--use-wheels.
Differential Revision: https://phabricator.services.mozilla.com/D57404
--HG--
extra : moz-landing-system : lando
This patch causes the newest version of NodeJS 8.17.0 to be used to build the mozilla toolchains installed on the CI machines as well as by `mach bootstrap`, which in turn gets an important security fix by updating to npm 6.13.4.
This patch also causes mach commands which call npm to abort if the version in use is older than 6.13.4 to avoid (especially) exploits on local developers that exfiltrate their data.
Differential Revision: https://phabricator.services.mozilla.com/D57674
--HG--
extra : moz-landing-system : lando
If mercurial is already installed via pip (or possibly other means),
then mach bootstrap will fail. We should check if we already have a
sufficient mercurial before forcing it to be installed with
ports/homebrew.
Differential Revision: https://phabricator.services.mozilla.com/D57566
--HG--
extra : moz-landing-system : lando
Currently BuildReader._relevant_mozbuild_finder attempted to exclude objdirs by
ignoring the pattern 'obj*'. However this can cause problems for developers who
don't prefix their objdirs with the string 'obj'. This attempts to ignore all
objdirs by looking for 'config.status' files in the topsrcdir.
This wasn't a huge issue before because BuildReader._relevant_mozbuild_finder
was only used in a handful of obscure edge cases. But now it's being invoked by
the taskgraph and consequently |mach try|. So we'll see this issue pop up for
people more frequently.
Differential Revision: https://phabricator.services.mozilla.com/D57113
--HG--
extra : moz-landing-system : lando
This change modifies all tests that use key3/cert8 to use the new files. It
removes test_sdr_upgraded_with_password, as without the upgrade part that is now
the same test as test_sdr_preexisting_with_password.
Differential Revision: https://phabricator.services.mozilla.com/D55708
--HG--
rename : security/manager/ssl/tests/unit/test_sdr_preexisting/key4.db => security/manager/ssl/tests/unit/test_broken_fips/key4.db
extra : moz-landing-system : lando
This change modifies all tests that use key3/cert8 to use the new files. It
removes test_sdr_upgraded_with_password, as without the upgrade part that is now
the same test as test_sdr_preexisting_with_password.
Differential Revision: https://phabricator.services.mozilla.com/D55708
--HG--
rename : security/manager/ssl/tests/unit/test_sdr_preexisting/key4.db => security/manager/ssl/tests/unit/test_broken_fips/key4.db
extra : moz-landing-system : lando
This change removes the legacy libnssdbm database that we migrated away from since Firefox 60.
This change modifies all tests that use key3/cert8 to use the new files. It
removes test_sdr_upgraded_with_password, as without the upgrade part that is now
the same test as test_sdr_preexisting_with_password. It otherwise removes support for libnssdbm everywhere in Gecko.
Differential Revision: https://phabricator.services.mozilla.com/D55708
--HG--
rename : security/manager/ssl/tests/unit/test_sdr_preexisting/key4.db => security/manager/ssl/tests/unit/test_broken_fips/key4.db
extra : moz-landing-system : lando
This optionally uses the rust-based `dump_syms` toolchain if it has been
installed, otherwise it uses the locally built version.
Differential Revision: https://phabricator.services.mozilla.com/D53320
--HG--
extra : moz-landing-system : lando
Bug 1295542 added empty chrome.manifest files to prevent malware from abusing
them. This workaround is no longer necessary because Firefox stopped reading
chrome.manifest outside omni.ja since bug 1543761.
Differential Revision: https://phabricator.services.mozilla.com/D55954
--HG--
extra : moz-landing-system : lando
Make the $PYTHON3 build var point to a full virtualenv bootstrapped with
the same libraries as the $PYTHON Python 2 build var. This allows us to
upgrade build tasks from $PYTHON to $PYTHON3.
This patch adds some debug logging and documentation to the Python
2 virtualenv so that it is easier to diagnose issues that may arise
from running two different Python interpreters in re-entrant
multiprocess routines.
Differential Revision: https://phabricator.services.mozilla.com/D50819
--HG--
extra : moz-landing-system : lando
This optionally uses the rust-based `dump_syms` toolchain if it has been
installed, otherwise it uses the locally built version.
Differential Revision: https://phabricator.services.mozilla.com/D53320
--HG--
extra : moz-landing-system : lando
In 1-tier PGO builds that shared the objdir between the instrumented and
profile-use builds, the instrumented build objects used a different
suffix (.i_o) to separate them from the profile-use build (which uses
the default .o suffix). These builds are now always in separate objdirs,
and don't need special suffix rules anymore.
As a bonus, this helps fix an issue with buildid.cpp continually
rebuilding because libxul_so.list always lists the inputs as *.o, which
don't exist if we're using a .i_o suffix. Make would always re-create
buildid.cpp and therefore libxul.so in the instrumented build even when
nothing has changed.
Differential Revision: https://phabricator.services.mozilla.com/D56115
--HG--
extra : moz-landing-system : lando
Allow running the jsshell using ./mach run. This copies the debugger
parameter handling from desktop configurations.
Differential Revision: https://phabricator.services.mozilla.com/D56296
--HG--
extra : moz-landing-system : lando
Local PGO builds now use 3-tier machinery under the hood. Instead of a
single object directory that gets cleaned in between the instrumented
and profile-use builds, now the instrumented build uses a separate
'${objdir}/instrumented' directory. This makes it easier to handle
within mach since we can drive the two builds with environment variables
and keep all build artifacts separate, without needing to do manual
cleanup in between.
Differential Revision: https://phabricator.services.mozilla.com/D50098
--HG--
extra : moz-landing-system : lando
This patch addresses the following issues which prevent building the
documentation on a Windows system:
1. check_jsdoc attempts to launch a file by the name of jsdoc using
subprocess.check_output(). But, the file jsdoc is a Bash script,
so it can't be launched this way on Windows. We should find the
correct file name to use instead (likely jsdoc.cmd).
2. The Windows CopyFile API will fail if the destination directory does
not exist. The procedure that creates the staging directory for Sphinx
attempts to use this API before any directories are created, so most calls
to it fail and most files don't get staged. These errors are ignored, so
the result is that Sphinx succeeds in generating a set of documentation
that is mostly empty.
3. Several instances where manually constructed paths that contain both back
and forward slashes are compared to or otherwise used with normalized paths,
resulting in incorrect behaviors.
Differential Revision: https://phabricator.services.mozilla.com/D54549
--HG--
extra : moz-landing-system : lando
Some of these were obvious typos. Others probably reflect once-correct components
that have been combined, split, or otherwise obsoleted; for these I've tried to
use the component associated with the bugs for recent changes to the affected files.
Differential Revision: https://phabricator.services.mozilla.com/D55756
--HG--
extra : moz-landing-system : lando
Remove the "clever" logic from patch D53070, which was meant to reduce code duplication, and replace with easier-to-understand logic. Copy-and-paste the previous version of the CompileFlags class before patch D53070 to ensure that we stamp out the regression.
Differential Revision: https://phabricator.services.mozilla.com/D55472
--HG--
extra : moz-landing-system : lando
This change does not fully enable C++17, as we still need standard
library support from some platforms.
Differential Revision: https://phabricator.services.mozilla.com/D54202
--HG--
extra : moz-landing-system : lando
If Mercurial (hg) is not installed on a Centos/Fedora system, execution
of:
$ ./mach bootstrap
will be failed as the bootstrap script tries to execute:
# dnf update mercurial
This commit adds 'dnf install' command if the mercurial wasn't
installed on the system. This command will try to execute installation
of mercurial.
In other case there will be a try to upgrade to the latest possibly version
of mercurial.
Differential Revision: https://phabricator.services.mozilla.com/D55639
--HG--
extra : moz-landing-system : lando
Andrew Halberstadt
shindli
Andreea Pavel
Narcis Beleuzu
Mike Hommey
Ricky Stewart
Chris Manchester
Mihai Alexandru Michis
Mohammad H Sekhavat
Andi-Bogdan Postelnicu
Brindusan Cristian
Botond Ballo
Noemi Erli
Dorel Luca
Ben Hearsum
Jonathan Watt
Nathan Froyd
Emilio Cobos Álvarez
Mike Shal
Csoregi Natalia
Steve Fink
Lifan Zeng lifanfzeng
Bastien Abadie
byron jones
Zibi Braniecki
Francesco Lodolo (:flod)
Axel Hecht
Dan Mosedale
Henrik Skupin
Cosmin Sabou
Coroiu Cristina
Julian
Daniel Varga
Masatoshi Kimura
Artem
Bogdan Tara
Jan Sonntag
Ciure Andrei
Edwin Takahashi
Gijs Kruitbosch
Gabriele Svelto
Shane Caraveo
Harry Twyford
Matt Singleton
James Graham
Brian Grinstead
Sylvestre Ledru
Geoff Lankow
Razvan Maries
Oana Pop Rus
J.C. Jones
Eric Rahm
Māris Fogels
Ted Campbell
David Major
Molly Howell
Florin Strugariu
Geoff Brown
Gurzau Raul
Sebastian Hengst
Alexander Kuleshov