https://github.com/rust-lang/rust/issues/52694 is a miscompilation I
found in rust when it uses system llvm 5 or 6, that was fixed 5 months
ago in the llvm rust bundles. This may or may not affect clang, but
considering it was also reported to upstream llvm independently of rust,
it's better to side with caution.
It doesn't affect 3.9, and bug 1478919 got rid of the last use of clang
5 (except for clang-tidy, but that's not used to compile).
The patches come from the llvm trunk from 5 months ago, so they're
already in our clang 7 snapshots.
Windows static analysis builds are still using an old trunk, but are
stuck on bug 1427808. They're "only" for static analysis, though.
--HG--
extra : rebase_source : f4fce69eb7c69b6245518a1bad37e04236c7075b
This was cargo-culted from the asan/tsan mozconfigs, but is not necessary
for builds without sanitizers.
--HG--
extra : rebase_source : 41bad4761f424410cb7a099ecaecce8a86becf59
JS, at least, doesn't need bindgen at this point. If JS does start
requiring bindgen under certain build configurations, we'll have to
figure out what to do about the bindgen dependency at that point:
require bindgen always, or just require bindgen for the
configuration(s).
Summary:
This adds a clangcl flag to gyp.mozbuild and then uses that to set the same
warnings flags for clang-cl as for clang.
Reviewers: dmajor
Tags: #secure-revision
Bug #: 1478000
Differential Revision: https://phabricator.services.mozilla.com/D2427
--HG--
extra : rebase_source : 6137d2f59e5504d448d4fb32a7e734756ff76afa
Summary:
We're currently returning a fake value on all of our automation builds. Might
as well not run the script at all.
Tags: #secure-revision
Bug #: 1371485
Differential Revision: https://phabricator.services.mozilla.com/D1803
--HG--
extra : rebase_source : 20717372c15547b35759ff69088aea1ca92b7198
This adds just enough host shared library support for this one use case,
but also takes shortcuts, because fully supporting host shared library
is a deep rabbit hole I'm not ready to take just to fix --enable-lto
--enable-clang-plugin on mac builds.
One downside is that one my machine the plugin now takes > 80s to build,
instead of 15s before, thanks to the lack of unified sources.
--HG--
extra : rebase_source : bf52a72a01d4e3eb77cf52b646b19734b9273075
Currently, --enable-lto just implies --enable-linker=lld, which
essentially only works on Linux, and assumes one can't do lto with
anything other than lld. Which is not true. As a matter of fact, even
ld.bfd can do lto, as long as the gold plugin for llvm is available,
or when doing lto with GCC instead of clang.
Anyways, to allow more lto setups, we adapt the --enable-linker setup
to:
- work on macOS, which it currently doesn't, and add support for the mac
linker (ld64), which, unfortunately, doesn't have a clean way to be
detected, so work around that.
- default to lld if lto is enable, no linker was explicitly given, the
compiler is clang *and* the build target is not macOS.
--HG--
extra : rebase_source : 1dab2ad6230d18e7f4285943e76352e23b987d4e
Because we have no linking configure test in python configure (yet), we
just make old-configure tests use LINKER_LDFLAGS, and make those flags
added to LDFLAGS by old-configure at the same time.
--HG--
extra : rebase_source : 80ab7c5021c7ddd1b53d58ef76cd4372a524d3cb
The intent is for the build system to soon require Node.js to build
Firefox. But we aren't ready to make Node.js a build requirement
just yet.
The goal of this commit is to implement configure detection for
Node.js so that we can a) work out detection bugs b) give people a
means to validate system compatibility *before* we throw the switch to
require Node.js.
This commit introduces configure logic for finding a Node.js
executable, resolving its version, and validating its suitability.
By default, if Node.js cannot be found or there is an error resolving
its version, we print some warning messages and move on.
If --enable-nodejs is used (not the default), errors are raised
if Node.js cannot be found or its version isn't suitable.
Once we require Node.js, the added code can likely be simplified.
When writing the code, I went out of my way to make failures as
non-fatal as possible. e.g. normally we'd say that failures to run
`node --version` would be fatal. I'm purposefully trying to not have
this configure check break anyone's environment, even if failure
occurs. Again, the goal is to introduce the configure checks first
in a non-fatal way such that we can debug failures so the flag day
transition is simpler.
Differential Revision: https://phabricator.services.mozilla.com/D1818
--HG--
extra : moz-landing-system : lando
Added `./mach python-safety`, distinct from python-test so it doesn't have
to be run on every CI job - its errors may not depend on the area the push has changed.
Added the python/safety directory to ensure a different Pipfile is used, avoiding
conflicts with python-test.
Differential Revision: https://phabricator.services.mozilla.com/D1825
--HG--
extra : moz-landing-system : lando
We're well overdue for an upgrade of the rust compiler requirements.
Now that we're building with 1.28 (albeit a beta, due to be bumped when
it's released), we can bump the requirement away from 1.24 which is now
old. 1.27 is too new, though, so settle for the older 1.26.
--HG--
extra : rebase_source : a17aa496bf3d4af4d1349d69a637c686c6817d0f
This will make sure that when running |mach python-test --python 3| locally,
we only run the tests that also run in CI with python 3 (and therefore pass
presumably).
MozReview-Commit-ID: 3OBr9yLSlSq
--HG--
extra : rebase_source : 456340d0ecdddf1078f2b5b4ebb1eddf3813b26a
The media/libpng/moz.build file overrides the C standard used via
-std=c89, per bug 1371266, which conflicts with the use of the
arm_neon.h header: compilation fails on the inline keyword, which didn't
exist in C89. We thus "bump" to the GNU89 standard, which is C89+GNU
extensions, including inline.
--HG--
extra : rebase_source : fe93a13e3bef8888e1874d2e94a6d8ef396aaf83
When I originally implemented bug 1458161, this is how it was done, but
it was suggested to use a configure-time check. This turned out to not
be great, because the rust compiler changes regularly, and we don't run
the configure tests when the version changes. When people upgraded their
rust compiler to 1.27, the code subsequently failed to build because the
features were still set for the previous version they had installed.
--HG--
extra : rebase_source : 1b5f7a02ad8495d68cd29289f7beea59b8912183
When a binary has a PT_GNU_RELRO segment, the elfhack injected code
uses mprotect to add the writable flag to relocated pages before
applying relocations, removing it afterwards. To do so, the elfhack
program uses the location and size of the PT_GNU_RELRO segment, and
adjusts it to be aligned according to the PT_LOAD alignment.
The problem here is that the PT_LOAD alignment doesn't necessarily match
the actual page alignment, and the resulting mprotect may end up not
covering the full extent of what the dynamic linker has protected
read-only according to the PT_GNU_RELRO segment. In turn, this can lead
to a crash on startup when trying to apply relocations to the still
read-only locations.
Practically speaking, this doesn't end up being a problem on x86, where
the PT_LOAD alignment is usually 4096, which happens to be the page
size, but on Debian armhf, it is 64k, while the run time page size can be
4k.
--HG--
extra : rebase_source : 5ac7356f685d87c1628727e6c84f7615409c57a5
We're well overdue for an upgrade of the rust compiler requirements.
Now that we're building with 1.28 (albeit a beta, due to be bumped when
it's released), we can bump the requirement away from 1.24 which is now
old. 1.27 is too new, though, so settle for the older 1.26.
--HG--
extra : rebase_source : c788ef4f7da9949b81df2f0577af6f6039ea63d8
When we're running using pipenv, we have more than one virtual environment. This means the current check to see if python matches the initial virtual environment gives a false positive when we're in a secondary virtual environment. This patch changes the condition to check if the current python path exists within the virtual environments root directory.
MozReview-Commit-ID: AAONwLWsigL
--HG--
extra : rebase_source : c0ac94448ee4545417b5116e58b51c6187cdb175
We used to do that before bug 1455767. Taking a step back, what we want
(and what this change implements) is to:
- allow to specify a linker to use explicitly
- check that using the corresponding linker flags makes us use that
linker
- if no linker is specified, and developer options are enabled, we want
to try to use gold if it's available, otherwise, we want to detect
what kind of linker the default linker is
- if --disable-gold is explicitly given, we don't want to automatically
use it.
--HG--
extra : rebase_source : 8d89c54bd5e555984d815beb8fdd3f5f75dae31e
Instead of clang 4, which they were the last to use, so remove the
clang 4 toolchain.
--HG--
extra : rebase_source : d03a083e9217aeb6c1d2c91decb978426f0e8d1a
The linux64-clang toolchain alias is currently clang 5. Switch it to
clang 6, but keep the spidermonkey tsan builds on clang 5 because of
bug 1467673.
The LLVM headers that come with clang 6 contain a DEBUG define that
conflicts with our DEBUG define and breaks the clang-plugin build,
so force unset ours.
--HG--
extra : rebase_source : aae88f1166108f003b06c022f14d5f4c61fc1ed9
Also work around https://bugs.llvm.org/show_bug.cgi?id=37746 by
explicitly handling ObjC interface variables separately. This actually
allows the searchfox macosx build to go much further than it used to (it
now fails during make package with apparently no output for rust code)
--HG--
extra : rebase_source : 354981ca9deebed5c60d3684f5c3abc554422393
We used to do that before bug 1455767. Taking a step back, what we want
(and what this change implements) is to:
- allow to specify a linker to use explicitly
- check that using the corresponding linker flags makes us use that
linker
- if no linker is specified, and developer options are enabled, we want
to try to use gold if it's available, otherwise, we want to detect
what kind of linker the default linker is
--HG--
extra : rebase_source : 655089f29c6403f6c31a977dcf5c6fd1e9941121
--enable-release not being passed means developer options are enabled,
which is generally speaking not desirable for builds meant to be
shipped. This is somewhat alleviated for Firefox by MOZILLA_OFFICIAL
implying --enable-release (as well as MOZ_AUTOMATION), but that doesn't
apply to e.g. standalone js builds (even some of the standalone js jobs
on our automation don't set MOZ_AUTOMATION for some reason).
A reasonable thing to do is just to default builds for release/beta
milestones to --enable-release, but still allow --disable-release
to enable the developer options.
--HG--
extra : rebase_source : 770d51b10a9cd17c63972435bb61eed10345ea71
Currently, many tasks fetch content from the Internets. A problem with
that is fetching from the Internets is unreliable: servers may have
outages or be slow; content may disappear or change out from under us.
The unreliability of 3rd party services poses a risk to Firefox CI.
If services aren't available, we could potentially not run some CI tasks.
In the worst case, we might not be able to release Firefox. That would
be bad. In fact, as I write this, gmplib.org has been unavailable for
~24 hours and Firefox CI is unable to retrieve the GMP source code.
As a result, building GCC toolchains is failing.
A solution to this is to make tasks more hermetic by depending on
fewer network services (which by definition aren't reliable over time
and therefore introduce instability).
This commit attempts to mitigate some external service dependencies
by introducing the *fetch* task kind.
The primary goal of the *fetch* kind is to obtain remote content and
re-expose it as a task artifact. By making external content available
as a cached task artifact, we allow dependent tasks to consume this
content without touching the service originally providing that
content, thus eliminating a run-time dependency and making tasks more
hermetic and reproducible over time.
We introduce a single "fetch-url" "using" flavor to define tasks that
fetch single URLs and then re-expose that URL as an artifact. Powering
this is a new, minimal "fetch" Docker image that contains a
"fetch-content" Python script that does the work for us.
We have added tasks to fetch source archives used to build the GCC
toolchains.
Fetching remote content and re-exposing it as an artifact is not
very useful by itself: the value is in having tasks use those
artifacts.
We introduce a taskgraph transform that allows tasks to define an
array of "fetches." Each entry corresponds to the name of a "fetch"
task kind. When present, the corresponding "fetch" task is added as a
dependency. And the task ID and artifact path from that "fetch" task
is added to the MOZ_FETCHES environment variable of the task depending
on it. Our "fetch-content" script has a "task-artifacts"
sub-command that tasks can execute to perform retrieval of all
artifacts listed in MOZ_FETCHES.
To prove all of this works, the code for fetching dependencies when
building GCC toolchains has been updated to use `fetch-content`. The
now-unused legacy code has been deleted.
This commit improves the reliability and efficiency of GCC toolchain
tasks. Dependencies now all come from task artifacts and should always
be available in the common case. In addition, `fetch-content` downloads
and extracts files concurrently. This makes it faster than the serial
application which we were previously using.
There are some things I don't like about this commit.
First, a new Docker image and Python script for downloading URLs feels
a bit heavyweight. The Docker image is definitely overkill as things
stand. I can eventually justify it because I want to implement support
for fetching and repackaging VCS repositories and for caching Debian
packages. These will require more packages than what I'm comfortable
installing on the base Debian image, therefore justifying a dedicated
image.
The `fetch-content static-url` sub-command could definitely be
implemented as a shell script. But Python is readily available and
is more pleasant to maintain than shell, so I wrote it in Python.
`fetch-content task-artifacts` is more advanced and writing it in
Python is more justified, IMO. FWIW, the script is Python 3 only,
which conveniently gives us access to `concurrent.futures`, which
facilitates concurrent download.
`fetch-content` also duplicates functionality found elsewhere.
generic-worker's task payload supports a "mounts" feature which
facilitates downloading remote content, including from a task
artifact. However, this feature doesn't exist on docker-worker.
So we have to implement downloading inside the task rather than
at the worker level. I concede that if all workers had generic-worker's
"mounts" feature and supported concurrent download, `fetch-content`
wouldn't need to exist.
`fetch-content` also duplicates functionality of
`mach artifact toolchain`. I probably could have used
`mach artifact toolchain` instead of writing
`fetch-content task-artifacts`. However, I didn't want to introduce
the requirement of a VCS checkout. `mach artifact toolchain` has its
origins in providing a feature to the build system. And "fetching
artifacts from tasks" is a more generic feature than that. I think
it should be implemented as a generic feature and not something that is
"toolchain" specific.
I think the best place for a generic "fetch content" feature is in
the worker, where content can be defined in the task payload. But as
explained above, that feature isn't universally available. The next
best place is probably run-task. run-task already performs generic,
very-early task preparation steps, such as performing a VCS checkout.
I would like to fold `fetch-content` into run-task and make it all
driven by environment variables. But run-task is currently Python 2
and achieving concurrency would involve a bit of programming (or
adding package dependencies). I may very well port run-task to Python
3 and then fold fetch-content into it. Or maybe we leave
`fetch-content` as a standalone script.
MozReview-Commit-ID: AGuTcwNcNJR
--HG--
extra : source : 0b941cbdca76fb2fbb98dc5bbc1a0237c69954d0
extra : histedit_source : a3e43bdd8a9a58550bef02fec3be832ca304ea93
After this change, we consistently import GPG keys from files in
the GCC build scripts.
MozReview-Commit-ID: BcyvCQoGbMS
--HG--
extra : source : 5fce34a460b51e45ac280a9f0cb8bad896fbcff1
extra : histedit_source : 01621ea8111315c251a9493a11efca72c2ba3c7d
Currently, many tasks fetch content from the Internets. A problem with
that is fetching from the Internets is unreliable: servers may have
outages or be slow; content may disappear or change out from under us.
The unreliability of 3rd party services poses a risk to Firefox CI.
If services aren't available, we could potentially not run some CI tasks.
In the worst case, we might not be able to release Firefox. That would
be bad. In fact, as I write this, gmplib.org has been unavailable for
~24 hours and Firefox CI is unable to retrieve the GMP source code.
As a result, building GCC toolchains is failing.
A solution to this is to make tasks more hermetic by depending on
fewer network services (which by definition aren't reliable over time
and therefore introduce instability).
This commit attempts to mitigate some external service dependencies
by introducing the *fetch* task kind.
The primary goal of the *fetch* kind is to obtain remote content and
re-expose it as a task artifact. By making external content available
as a cached task artifact, we allow dependent tasks to consume this
content without touching the service originally providing that
content, thus eliminating a run-time dependency and making tasks more
hermetic and reproducible over time.
We introduce a single "fetch-url" "using" flavor to define tasks that
fetch single URLs and then re-expose that URL as an artifact. Powering
this is a new, minimal "fetch" Docker image that contains a
"fetch-content" Python script that does the work for us.
We have added tasks to fetch source archives used to build the GCC
toolchains.
Fetching remote content and re-exposing it as an artifact is not
very useful by itself: the value is in having tasks use those
artifacts.
We introduce a taskgraph transform that allows tasks to define an
array of "fetches." Each entry corresponds to the name of a "fetch"
task kind. When present, the corresponding "fetch" task is added as a
dependency. And the task ID and artifact path from that "fetch" task
is added to the MOZ_FETCHES environment variable of the task depending
on it. Our "fetch-content" script has a "task-artifacts"
sub-command that tasks can execute to perform retrieval of all
artifacts listed in MOZ_FETCHES.
To prove all of this works, the code for fetching dependencies when
building GCC toolchains has been updated to use `fetch-content`. The
now-unused legacy code has been deleted.
This commit improves the reliability and efficiency of GCC toolchain
tasks. Dependencies now all come from task artifacts and should always
be available in the common case. In addition, `fetch-content` downloads
and extracts files concurrently. This makes it faster than the serial
application which we were previously using.
There are some things I don't like about this commit.
First, a new Docker image and Python script for downloading URLs feels
a bit heavyweight. The Docker image is definitely overkill as things
stand. I can eventually justify it because I want to implement support
for fetching and repackaging VCS repositories and for caching Debian
packages. These will require more packages than what I'm comfortable
installing on the base Debian image, therefore justifying a dedicated
image.
The `fetch-content static-url` sub-command could definitely be
implemented as a shell script. But Python is readily available and
is more pleasant to maintain than shell, so I wrote it in Python.
`fetch-content task-artifacts` is more advanced and writing it in
Python is more justified, IMO. FWIW, the script is Python 3 only,
which conveniently gives us access to `concurrent.futures`, which
facilitates concurrent download.
`fetch-content` also duplicates functionality found elsewhere.
generic-worker's task payload supports a "mounts" feature which
facilitates downloading remote content, including from a task
artifact. However, this feature doesn't exist on docker-worker.
So we have to implement downloading inside the task rather than
at the worker level. I concede that if all workers had generic-worker's
"mounts" feature and supported concurrent download, `fetch-content`
wouldn't need to exist.
`fetch-content` also duplicates functionality of
`mach artifact toolchain`. I probably could have used
`mach artifact toolchain` instead of writing
`fetch-content task-artifacts`. However, I didn't want to introduce
the requirement of a VCS checkout. `mach artifact toolchain` has its
origins in providing a feature to the build system. And "fetching
artifacts from tasks" is a more generic feature than that. I think
it should be implemented as a generic feature and not something that is
"toolchain" specific.
I think the best place for a generic "fetch content" feature is in
the worker, where content can be defined in the task payload. But as
explained above, that feature isn't universally available. The next
best place is probably run-task. run-task already performs generic,
very-early task preparation steps, such as performing a VCS checkout.
I would like to fold `fetch-content` into run-task and make it all
driven by environment variables. But run-task is currently Python 2
and achieving concurrency would involve a bit of programming (or
adding package dependencies). I may very well port run-task to Python
3 and then fold fetch-content into it. Or maybe we leave
`fetch-content` as a standalone script.
MozReview-Commit-ID: AGuTcwNcNJR
--HG--
extra : rebase_source : 4918b8c3bac53d63665006802054038bfbca0314
After this change, we consistently import GPG keys from files in
the GCC build scripts.
MozReview-Commit-ID: BcyvCQoGbMS
--HG--
extra : rebase_source : 657ccce8e242cabdfaff396fd0d6439754a3f364
python-zstandard's 0.9.1 source distribution contains a debian/
directory.
On Squeeze, producing a Debian package is straightforward.
On Wheezy, we need to hack up Build-Depends because Wheezy doesn't
have a package for the Hypothesis fuzzing library. This package is
only used for testing and our package building disables testing,
so we don't even need to further hack up the packaging to disable
tests.
MozReview-Commit-ID: 6raXjdzggCH
--HG--
extra : rebase_source : 672492a40d65df8430eb17ba033bcb1c0890b7df
This build target doesn't have LTO enabled on it (yet)
MozReview-Commit-ID: 56tAHMyvH7o
--HG--
extra : rebase_source : 90039cd8e97332e2ef8aad7908b8a04b2869f4a5
The crash reporter symbol files are the easiest cross-platform way to
find static initializers. While some types of static initializers (e.g.
__attribute__(constructor) functions) don't appear there in a notable
way, the static initializers we do care the most about for tracking do
(static initializers from C++ globals). As a matter of fact, there is
only a difference of 2 compared to the currently reported count of 125
on a linux64 build, so this is a good enough approximation. And allows
us to easily track the count on Android, OSX and Windows builds, which
we currently don't do.
The tricky part is that the symbol files are in
dist/crashreporter-symbols/$lib/$fileid/$lib.sym, and $fileid is hard to
figure out. There is a `fileid` tool in testing/tools, but it is a
target binary, meaning it's not available on cross builds (OSX,
Android).
So the simplest is just to gather the data while creating the symbol
files, which unfortunately requires to go through some hoops to make it
happen for just the files we care about.
--HG--
extra : rebase_source : 458fed1ffd6f9294eefef61f10ff7a284af0d986
for L10n jobs should run per-push based on the corresponding builds
Differential Revision: https://phabricator.services.mozilla.com/D1403
--HG--
extra : rebase_source : b7b7f34204bd06ba2baf92ae6d103f8c207214fb
for L10n jobs should run per-push based on the corresponding builds
Differential Revision: https://phabricator.services.mozilla.com/D1403
--HG--
extra : rebase_source : fc669df941b70b720a6eb4118ad36d6679a28d48
This relies on the fact that providing multiple --version-script
combines them all, so we effectively create a new symbol version
that has no global symbol, but hides the std:🧵:_M_start_thread
symbols.
This version script trick happens to work with BFD ld, gold, and lld.
The downside is that when providing multiple --version-script's, ld
doesn't want any of them to have no version at all. So for the libraries
that do already have a version script (through SYMBOLS_FILE), we use a
version where there used to be none, using the library name as the
version. Practically speaking, this binds the libraries a little closer
than they used to be, kind of non-flat namespace on OSX (which is the
default there), meaning the dynamic linker will actively want to use
symbols from those libraries instead of a system library that might
happen to have the same symbol name.
--HG--
extra : rebase_source : a7f672c35609d993849385ddb874ba791b34f929
Version 2.25.1's libiberty can choke on some symbols. That was fixed in
2.27. As of writing, the last version is 2.30. Conservatively go with
2.28.1, which is the same major version (2.28) as what is currently in
Debian stable.
--HG--
extra : rebase_source : 9e5ba94421a1568f662cfd98af7168ea1c890934
This relies on the fact that providing multiple --version-script
combines them all, so we effectively create a new symbol version
that has no global symbol, but hides as much std::* stuff as possible.
The added symbol script could use `extern "C++"` syntax and demangled
symbols but there is no guarantee the demangled symbols won't change.
Plus, it's not possible to match demangled symbols that have a return
type: they contain a space, and the only way to match that is to use
double quotes, which doesn't allow globs at the same time.
This version script trick happens to work with BFD ld, gold, and lld.
The downside is that when providing multiple --version-script's, ld
doesn't want any of them to have no version at all. So for the libraries
that do already have a version script (through SYMBOLS_FILE), we use a
version where there used to be none, using the library name as the
version. Practically speaking, this binds the libraries a little closer
than they used to be, kind of non-flat namespace on OSX (which is the
default there), meaning the dynamic linker will actively want to use
symbols from those libraries instead of a system library that might
happen to have the same symbol name.
--HG--
extra : rebase_source : 78adb64b90e75ebad203b8a647b305c9d7198d16
In the span of one week, both gmplib.org and multiprecision.org,
respective home of gmp and mpc have gone down. The latter is still down.
It turns out that all versions of gmp and mpfr we need are mirrored on
ftp.gnu.org, so we can just use that instead. For mpc, versions > 1.0
are on ftp.gnu.org, but not earlier versions.
The one mpc version <= 1.0 we do need is 0.8.2, and a copy of the exact
same archive, as per its sha256, which we're already checking per the
gcc build scripts, can be found on snapshot.debian.org. We lose gpg
validation on the way, but since we're already checking the sha256,
that's a fine tradeoff.
At least this unblocks changes to toolchains until multiprecision.org
comes back online.
--HG--
extra : rebase_source : f2bc67f8757655d99bfd9735b80d7205f7bfe47b
Can be used:
ac_add_options --enable-linker=lld-7
or
ac_add_options --enable-linker=lld
MozReview-Commit-ID: GfDevGeooU4
--HG--
extra : rebase_source : ee2aad5f8f721a6fe5840affde6b29a3b940fb91
The python3-minimal package provides /usr/bin/python3 on Debian.
This commit installs this package so a `python3` executable is
provided.
This required backporting the package to wheezy. The final patch
is trivial. But I wasted a bit of time figuring out why `mk-build-deps`
wasn't working. It would no-op and exit 0 and then the build would
complain about missing dependencies!
glandium's theory is that the ":any" multiarch support on wheezy
isn't complete. Removing ":any" seems to make things "just work."
MozReview-Commit-ID: FBicpK4SmkQ
--HG--
extra : rebase_source : a28ce731024e8ed6a43fb30e2ed57da2abb50d0f
This option does nothing if it's provided, and things will probably
break in some most peculiar manner if somebody uses --without-pthreads.
Given that neither outcome is useful, we should remove the option.
It's the *compile*SdkVersion that needs to match the installed Android SDK plat-
form in order to be able to build an app, whereas the *target*SdkVersion is
merely a compatibility flag.
Since the received wisdom is that targetSdkVersion should be <= compileSdk-
Version and Android Studio is also showing a warning to that effect if you
modify the build.gradle of a small sample app accordingly, I've also added a
corresponding configure check of our own to enforce this.
MozReview-Commit-ID: F2RZemChFrm
--HG--
extra : rebase_source : cf4f6256baa4446d673b94d97f9497f93d7917ff
This serves two purposes:
1) It makes web-platform-tests pref downloading/handling a little more robust. When
run externally, it now downloads the entire testing/profiles directory. When loading
prefs it will look for both prefs_general.js (to support older versions of Firefox)
and profiles.json (for support moving forward).
This way we can add/remove/rename pref files under these directories without needing
to worry about breaking upstream wpt.
2) It provides developers an overview of which harnesses are using which base profiles.
Instead of hunting through test harness code to find this information, they can glance
at profiles.json.
MozReview-Commit-ID: AMzdnD8aGA2
--HG--
extra : rebase_source : 6fa0a802680417e49fcef99f3d03de7458a8fcba
svn revert requires a path, and does not take a revision. This isn't an
issue on build machines because we do a fresh checkout every time.
But if you're trying to run build-clang locally, with existing checkouts,
it will:
1) successfully svn update
2) run svn revert, saying "Skipped <rev>"
(except you don't see it because of -q)
3) svn revert returns a successfull eror code
4) patch fails because the file was never reverted and it attempts
to re-apply the patch
Also I think the revert command needs to come first.
MozReview-Commit-ID: 4OfrJNZwJNU
--HG--
extra : rebase_source : b3474e8048b3110f3f5948c3351923c02735ca4d
This moves testing/profiles/prefs_general.js to
testing/profiles/common/user.js. It also adds an 'extensions' folder to the
common profile. Dropping extension files here will get them installed in all
test harnesses (useful for testing on try).
The idea is that all test harnesses will eventually use this 'common' profile.
We'll also create some new per harness profiles, e.g testing/profiles/mochitest
and testing/profiles/reftest. This way there will be a single location
developers can go to set preferences, both for a specific harness, and across
all harnesses.
MozReview-Commit-ID: 8sqBqLiypgU
--HG--
rename : testing/profiles/prefs_general.js => testing/profiles/common/user.js
extra : rebase_source : 72a4a4b691e93c77479c7e70647b0ca373862c51
OOM rust crashes are currently not identified as such in crash reports
because rust libstd handles the OOMs and panics itself.
There are unstable ways to hook into this, which unfortunately are under
active changes in rust 1.27, but we're currently on 1.24 and 1.27 is not
released yet. The APIs didn't change between 1.24 and 1.26, so it's
fine-ish to use them as long as we limit their use to those versions.
As long as the Firefox versions we ship (as opposed to downstream) use
the "right" version of rust, we're good to go.
The APIs are in their phase of stabilization, so there shouldn't be too
many variants of the code to support.
--HG--
extra : rebase_source : 08a85aa102b24380b1f6764effffcc909ef3191b
This commit also removes "DEFAULT_APP", which is unused.
MozReview-Commit-ID: 5YYaC5LJqUn
--HG--
extra : rebase_source : 45f0f8f11698890fae0dcca71174f88dbdb412c8
Since it is run with `mach python`, it uses the environment defined by
`build/virtualenv_packages.txt`. Thus, we don't need to create a separate
virtualenv.
Differential Revision: https://phabricator.services.mozilla.com/D1015
--HG--
extra : rebase_source : 023095f82d7219a10dffb3579276c5285db37cfc
extra : source : a2999a5b9b7aa08a7c5c2ba6384724853d14b9c5
We previously did not require Python 3.5+ everywhere because we failed
to detect Python 3.5 on Windows in CI. That's because CI isn't using
start-shell.bat and it hasn't yet updated PATH to include
%MOZILLABUILD%\python3.
We shouldn't need to teach CI to have PATH contain everything in
MozillaBuild. This commit teaches moz.configure to automatically use
MozillaBuild's Python 3 if we're running in MozillaBuild.
Since we can now detect Python 3 everywhere in CI, we make Python 3.5+
required on all build configurations.
MozReview-Commit-ID: BwgWGeYMyPM
--HG--
extra : rebase_source : b52f604b01c73b0493b142f3f71aa26c99a42b91
for: taskgraph: Make update tasks support esr60
MozReview-Commit-ID: GUmAq3sBXGB
--HG--
extra : rebase_source : 0eaeb17809fede07f6b9fc4ee5d856d0078f83be
But only if we are:
a) not running in CI
b) running in CI on Linux
We will ideally make the requirement global. But Python 3.5 is not
yet available in CI on macOS. And we're not finding the MozillaBuild
copy in configure.
This was previously announced in November at
https://groups.google.com/d/msg/mozilla.dev.platform/rJrPh1QYXrQ/hqRrQsJ_BgAJ.
MozReview-Commit-ID: IyPCAcL3gop
--HG--
extra : rebase_source : f9e3db043a1ce9c1a903c943663f22245febf101
checksums.py is now run after upload.py, as part of the "upload" make
target.
As part of the refactor, checksums.py now takes as arguments a list of
directories containing files to checksum. It will recursively checksum
all files in listed directories.
This means we no longer have to pass an explicit list of files into
checksums.py. Instead, we can pass the artifact directory and
automatically checksum everything within. This will allow us to
generate files directly into the artifact directory instead of
having to run upload.py to copy files there.
MozReview-Commit-ID: 6ntnXU2Pp0E
--HG--
extra : rebase_source : 7e019b366f14b3692ec702ff331a1e0306dc3805
.write() is the preferred mechanism to write to a file object.
MozReview-Commit-ID: 1uhNeFayoxV
--HG--
extra : rebase_source : 5244e5548cefb8ebf98cb2415ab139e96e967433
This makes the logic in process_files() simpler.
MozReview-Commit-ID: KdphRJZLinx
--HG--
extra : rebase_source : 14a89fc9ba85d1dd4c2d2d18d0383552bb2d8e69
Garbage in garbage out. Let's make this code simpler.
MozReview-Commit-ID: Gv0ZcMacyOX
--HG--
extra : rebase_source : 2e8131bc1926b6ad7f186d0778fe4b339c6ba834
A missing or unimportable hashlib is only possible on very old or very
busted Pythons. Remove code for conditionally handling a failure to
import it.
MozReview-Commit-ID: EXVlmttcwUa
--HG--
extra : rebase_source : 30177c86dc5e92dd3234177f383681f9aa70682e
The argument is never used in the local file copying function. So let's
nuke the argument.
MozReview-Commit-ID: JeU4LaVvGD7
--HG--
extra : rebase_source : 0e305c5bff0e3871fd11a554d75e7f53ceea4bdd
This was only used for the remote server case. That code was recently
deleted.
MozReview-Commit-ID: 7usqWZ7CuBR
--HG--
extra : rebase_source : f27b37391802a2980feffa5d88c92228d137824f
extra : source : 46b8a00288c8d7b88fd9b7aee20e88ddfd96d57b
With the transition to Taskcluster, "uploads" are artifacts in a local
directory. So we don't need to support uploading to a remote server
using SCP.
This commit removes all the code to support uploading to a remote
server.
And since property files were only written out for the remote case, all
that code can be deleted as well.
Since UPLOAD_HOST no longer means anything, we no longer set it in
mozharness configs.
MozReview-Commit-ID: 66gkM8erkGk
--HG--
extra : rebase_source : ee85bb927cfb98e1ca383ab2591febfc1f0ce5cd
We add a minimal Python script to run a process and prefix all its
output with a string. We change the automation tiers to evaluate all
make targets using this script.
MozReview-Commit-ID: 79g5KUd5ked
--HG--
extra : rebase_source : 63388a71b51e5abc05ca8bd48e180af72bf799e6
The argument is never used in the local file copying function. So let's
nuke the argument.
MozReview-Commit-ID: JeU4LaVvGD7
--HG--
extra : rebase_source : a898814f4084288200928f062fbb06ffbbae60bd
This was only used for the remote server case. That code was recently
deleted.
MozReview-Commit-ID: 7usqWZ7CuBR
--HG--
extra : rebase_source : 39c33057aab6773f8f85b1294e4965965954d9e6
With the transition to Taskcluster, "uploads" are artifacts in a local
directory. So we don't need to support uploading to a remote server
using SCP.
This commit removes all the code to support uploading to a remote
server.
And since property files were only written out for the remote case, all
that code can be deleted as well.
Since UPLOAD_HOST no longer means anything, we no longer set it in
mozharness configs.
MozReview-Commit-ID: 66gkM8erkGk
--HG--
extra : rebase_source : 15143a18598370eeaaa98f8f0ac8458d758db51f
The previous commit moved creating installers to be side effect of creating
packages. This makes the installer step not actually do anything. So remove the
step from automation.
Differential Revision: https://phabricator.services.mozilla.com/D864
--HG--
extra : rebase_source : 75b67a6e57031ae189dafe1d9854e4105aa22621
extra : source : fcb756834abbdbe0ae6e59a8cfdf39024f50a226
The previous commit moved creating installers to be side effect of creating
packages. This makes the installer step not actually do anything. So remove the
step from automation.
Differential Revision: https://phabricator.services.mozilla.com/D864
--HG--
extra : rebase_source : 174a366890da295634ef8971d0608ea60979f110
extra : histedit_source : 06fdd0e2ccb93f061ba5a40c2a4137eed9898916
(This also fixes Bug 879740 and Bug 1204543.)
build/pgo/certs contains an NSS database set that has a bunch of hand-generated
certificates, and many of these hand-generated certificates are specifically
depended upon for a variety of unit tests. This patch changes all of these to
use the "pycert.py" and "pykey.py" utilities that produce deterministic keys
and certificates.
The naming convention here is new, and defined in the README. It is based on
the mochitest runtest.py naming convention that imports .ca and .client
PEM-encoded certificates.
Unfortunately, the updates to build/pgo/genpgocert.py to generate these files
depends on OpenSSL in order to produce PKCS12 archives for pk11tool to import
into NSS. This could be done with pure-NSS tooling, but it'd require some new
command line functionality, which is out-of-scope for this change.
Note that build/pgo/genpgocert.py no longer takes arguments when run. It's not
run automatically anywhere that I can see, but could (reasonably) be, now.
Differential Revision: https://phabricator.services.mozilla.com/D971
--HG--
extra : amend_source : bc389b9b0a807a4889feb14db439daa28635dfe9
The existing suppression tries to be specific about the callstack that
arrives at the function, but that breaks when we rejigger the machinery
up the callstack. In practice, the existing suppressions cover most of
the ways we would arrive at selector parsing, and so I think the
specificity here is more trouble than it's worth.
MozReview-Commit-ID: 2k52xq64SQX
This makes the changes necessary to use TestRunnerActivity when geckoview
is installed and requested, but we do not yet attempt to run any such
test tasks in automation.
And adapt the build-gcc.sh script for the changes to
contrib/download_prerequisites.
--HG--
rename : taskcluster/scripts/misc/build-gcc-6-linux.sh => taskcluster/scripts/misc/build-gcc-7-linux.sh
extra : rebase_source : b1d785777b8c141c0eb0f52a73734abd2db21b94
This forces the linker ot use libatomic with clang on x86 (not only Android)
and forwards the intent to use libatomic to the shell's moz.build.
--HG--
extra : rebase_source : 0c803a3e11efcad3f17a462c2d38e85ec6cb556a
Now that XPT files are not loaded from files at runtime, code for
packaging XPT files can be removed.
This means that a couple of test XPIDL interfaces will get shipped in
builds to users that weren't before, but I don't think that matters
much.
This also puts XPT files into the local objdir for the XPIDL makefile,
instead of dist/bin, because they are no longer part of the
distribution.
MozReview-Commit-ID: 7gWj8KWUun3
--HG--
extra : rebase_source : 65bac47c2cd1a20b3c675a01b44a25a1d2d3ab7a
Debian 7 ships Python 3.2 by default. That's too old for our
upcoming build requirement of Python 3.5.
This commit adds a Python 3.5 package for wheezy that backports
the Python 3.5 from a much later Debian version.
The patch was inspired by the existing patch for Python 2.7.
However, it needed additional work. The changes and reasons
should all be documented in the changelog file as part of the
package diff we apply.
I'm a bit disappointed we had to disable PGO. But it was
reliably segfaulting during the build. I didn't feel like going
down that rabbit hole.
MozReview-Commit-ID: ABpHW1KYFQP
--HG--
extra : rebase_source : 02dbd13236fe741cb33f07c803218fda339c214e
Now that XPT files are not loaded from files at runtime, code for
packaging XPT files can be removed.
This means that a couple of test XPIDL interfaces will get shipped in
builds to users that weren't before, but I don't think that matters
much.
This also puts XPT files into the local objdir for the XPIDL makefile,
instead of dist/bin, because they are no longer part of the
distribution.
MozReview-Commit-ID: 7gWj8KWUun3
--HG--
extra : rebase_source : 6f7d4fd1d6cdea2c14866705a2dc972eb5f43382
This patch adds suppressions as needed on automation, for Servo-in-Gecko as
compiled by rustc 1.25. It appears there are only three false error sites
fun:*style*values*specified*color*Color*style*parser*Parse*parse*
fun:*selectors*parser*SelectorList*Impl*parse*
fun:*style*properties*shorthands*
but there are a number of different paths leading to them, especially the
first, hence the use of 17 suppressions in total, so as to remain specific.
Following some investigation of the machine code involved, I think these are
all Valgrind/Memcheck false positives, unfortunately, and probably to do
with (legitimate) operand swapping in && or || expressions by rustc+LLVM.
MozReview-Commit-ID: EpDmb4PEyoy
The check for whether an implicit constructor is in a third-party
directory currently has a homegrown list of directories to ignore.
Let's move this over to the (more complete!) list of third-party
directories maintained elsewhere.
In nsFrame::ComputeSize and nsFrame::ComputeSizeWithIntrinsicDimensions, the
following expressions
isFlexItem && usingFlexBasisForISize
isFlexItem && !usingFlexBasisForISize
are sometimes compiled by recent gcc/clang in the opposite order, viz
[!]usingFlexBasisForISize && isFlexItem. In this case the transformation is
correct as can be seen by analysing code earlier in these functions.
Unfortunately this causes Valgrind/Memcheck to report a branch on undefined
data which, in this case, is a false positive.
A simple fix is simply to initialise usingFlexBasisForISize to false at its
declaration point.
--HG--
extra : rebase_source : 39877e4ea8ec678288e6b49af120445c96ef8c0a
It's much easier to find that way, specially for functions with tons of
argument.
MozReview-Commit-ID: i3PMjDOcH7
--HG--
extra : rebase_source : 2c79f0b4eb5c6022c3896ded3bd26ceda524ce5a
We want this on for "normal" developer builds, but we want to leave it
off when the build is for some kind of profiling, i.e. when the
optimization level is higher than "normal" builds.
Note that static analysis was the only remaining user of the 32-bit toolchain, so I've removed win32-clang-cl (or rather, renamed it to win32-clang-cl-st-an).
--HG--
rename : build/build-clang/clang-win32.json => build/build-clang/clang-win32-st-an.json
rename : build/build-clang/clang-win64.json => build/build-clang/clang-win64-st-an.json
rename : taskcluster/scripts/misc/build-clang32-windows.sh => taskcluster/scripts/misc/build-clang32-st-an-windows.sh
rename : taskcluster/scripts/misc/build-clang64-windows.sh => taskcluster/scripts/misc/build-clang64-st-an-windows.sh
Require the current stable Rust release so new features can
be used in development.
MozReview-Commit-ID: 4NQNk3RfBkF
--HG--
extra : rebase_source : 9d88e6fdb823bd2e2ca8ac9940b1fafd420eebdc
This patch does a few things:
1) It removes the symantecRoot and symantec_affected certs from build/pgo/certs'
DB.
2) It upgrades that DB from the old format to SQLite (and this 8/3 to 9/4).
3) It adds a new cert "imminently_distrusted" to that DB for the bc test.
4) It changes the Subject of the immient distrust test to only have the CN
field: this is because certutil reorders C to come after CN, and just like
with the real Symantec certs, I had put C first. So rather than deal with
importing the end entity for the pgo tests, I decided to just make things
simple and change the tested subject.
5) Finally, it re-enables the test that was disabled in Bug 1434300.
MozReview-Commit-ID: Bt2RKyInJje
--HG--
rename : build/pgo/certs/cert8.db => build/pgo/certs/cert9.db
rename : build/pgo/certs/key3.db => build/pgo/certs/key4.db
extra : rebase_source : efceb67ae16f0af617bbd8bec201d52eee0f467d
The Proguard dependency is now managed by Gradle.
MozReview-Commit-ID: EOvKSE5z28P
--HG--
extra : rebase_source : 760b117f500cc639cc8c24e9c02933990f358dd7
The moz.build Java JAR definitions are, of course, broken, but they
will be removed soon enough.
MozReview-Commit-ID: KIxqLDwd9I7
--HG--
extra : rebase_source : 8312b3f125793f73d3e835d1c0a5c7cabd4ebc0c
I choose to clean a bunch of ANDROID_* moz.build cruft here, too,
since it's just passing dependencies between moz.build and
Makefile.in. The replacement for all of this is to just use
GENERATED_FILES in moz.build, but it'll still take some work to get to
that. (Why does this stuff exist? GENERATED_FILES didn't exist and
was resisted when I built this stuff.)
MozReview-Commit-ID: D3GJqJNL0Ih
--HG--
extra : rebase_source : 07351f9d3702cfc42c58bd317885d07882c45c3a
In some cases we have SourceLocation objections that are isMacroId()
rather than isFileId() and so don't have a filename at all. In other
cases the filename is something clang-internal like "<scratch>". In
both of these situations we don't want to output any analysis data.
However, the code previously was taking the empty filename and resolving
it relative to the source folder, and then tried to write to the
source folder or create a file corresponding to a subfolder. This
resulted in general badness. This patch makes sure we ignore analysis
from places that don't have a corresponding file so that we don't
have this problem.
MozReview-Commit-ID: 7WluygY9Uy8
--HG--
extra : rebase_source : 35d7e71fc05bbd5a3357a57db9de25782178622a
This adds an RAII class and macro that can be quickly added in functions
to log entry/exit from the function. This is useful to debugging.
MozReview-Commit-ID: 4Ud8jLOxI0R
--HG--
extra : rebase_source : 518d30fe44dff67bffb186e23c1eb858c02280af
The indexer has paths handed to it on Windows with the backslash path separator.
However it currently hard-codes the forward-slash Unix/macOS path separator,
so we need to generify that code appropriately.
MozReview-Commit-ID: Iy8bImt2BXW
--HG--
extra : rebase_source : 4b88b44319c05ce816afc4e690d3d33d39b7e43c
The heuristic simply fails for clang as shipped in Android NDKs: those
clang binaries can't target any non-Android host.
MozReview-Commit-ID: 6AhOJxE3boW
--HG--
extra : rebase_source : 3716d4ff7ce2f1e796489e02e312d7554a5b1e6c
The heuristic simply fails for clang as shipped in Android NDKs: those
clang binaries can't target any non-Android host.
MozReview-Commit-ID: 6AhOJxE3boW
--HG--
extra : rebase_source : 2c8b59c15a7a7e9803f5447fc8f41fc843b2e647
The fedora project is migrating from pkgs.fedoraproject.org to
src.fedoraproject.org. We should use the newer URL, particularly because the
later has a globally trusted TLS certificate.
MozReview-Commit-ID: 7TducICRR0k
--HG--
extra : rebase_source : 88e58d064d1699e853527019200d6fbd4989d6b3
This change was made in the mozsearch/mozsearch repo at some point and wasn't
synced to m-c.
MozReview-Commit-ID: HnW80AIrGt7
--HG--
extra : rebase_source : f621d89cb712ac5c13e7636cf2d9d7222a6f422b
This is a new issue that gets linted with flake8 3.5.0. Basically you should
never use a blank except: statement.
This will catch all exceptions, including KeyboardInterrupt and SystemExit
(which is likely not intended). If a catch all is needed, use
`except: Exception`. If you *really* mean to also catch KeyboardInterrupt et
al, use `except: BaseException`.
Of course, being specific is often better than a catch all.
MozReview-Commit-ID: FKx80MLO4RN
--HG--
extra : rebase_source : 7c74a7d0d81f2c984b47aff3a0ee3448b791177b
This adds a `moz_temporary_class` annotation that can be used to indicate
a class is intended to only be used as a temporary.
--HG--
extra : rebase_source : 2c4d5f0946739eafba485053624199bd4a05107a
The GDB version in Debian wheezy doesn't handle the DWARF data that the
GCC version we use to build Firefox and toolchains produce. So we take
the GDB version from Debian stretch and backport it.
--HG--
extra : rebase_source : dae0e9dcd5dde5a7c74b6cefd560480fccd9c5fa
Android headers can have a <sys/quota.h> header but with its functions
ifdef'ed out. Check for that by checking for the presence of quotactl().
MozReview-Commit-ID: 26ILnbP7flO
--HG--
extra : rebase_source : fe78c114189a362cdad3d79ce67868b045328392
NDK headers are grouped into a "sysroot" directory, which doesn't
contain architecture-specific bits, and a "system" directory, which
contains only the architecture-specific bits. Previously, both
directories are the same, under platforms/android-*/arch-*/. However,
with unified headers in NDK r16, the two are different, so we need to
support that in the Android build scripts.
Unified headers also rely on the build system defining the
__ANDROID_API__ macro, so we add support for that as well.
MozReview-Commit-ID: 9zBNQC3BRFl
--HG--
extra : rebase_source : 9ded282343df64d9cc4abcf7d7c6b03ac3423ff0
In this patch, we remove 3 tests:
1. test for blocked domain
2. test for blocked sub-domain
3. test for non-blocked domain
Two nsIDOMWindowUtils APIs were added for adding/removing a mock domain to the
existing blocklist. They are only used in the Stylo blocklist tests, so we
remove them as well.
MozReview-Commit-ID: A0ETWlqwbpN
Such a shame that the functionDecl() matcher doesn't handle them. I didn't find
a cleaner way to handle them, but I'm a 100% noob with AST matchers, so there
may be a more elegant way to do this.
MozReview-Commit-ID: 3HJQdFpN4hy
--HG--
extra : rebase_source : 27e48e6fb264499fd99e75eb54a22276758ab3e4
Android headers can have a <sys/quota.h> header but with its functions
ifdef'ed out. Check for that by checking for the presence of quotactl().
MozReview-Commit-ID: 26ILnbP7flO
--HG--
extra : rebase_source : 147e4be9eb98326fd4946b7b3695c573c7108347
NDK headers are grouped into a "sysroot" directory, which doesn't
contain architecture-specific bits, and a "system" directory, which
contains only the architecture-specific bits. Previously, both
directories are the same, under platforms/android-*/arch-*/. However,
with unified headers in NDK r16, the two are different, so we need to
support that in the Android build scripts.
Unified headers also rely on the build system defining the
__ANDROID_API__ macro, so we add support for that as well.
MozReview-Commit-ID: 9zBNQC3BRFl
--HG--
extra : rebase_source : 36c9d4d5625fecbbe2485668811f85f9d94372d5
The first two are variants of existing ones, with a slightly different
signature, and tha last one is well known and found in glib-related
suppression files like https://github.com/flatpak/flatpak/blob/master/tests/glib.supp
--HG--
extra : rebase_source : 2a209cc5987405e7ff638bbf69e1b74202b0357e
The MACHTYPE bash variable is an odd thing that returns e.g.
x86_64-redhat-linux-gnu on a CentOS system, but x86_64-pc-linux-gnu
on a Debian system, and possibly something different on other distros.
mach valgrind-test is the only place actually relying on MACHTYPE.
Others rely on information from python modules. Uniformize that, and use
the more generic 'pc' rather than 'redhat'.
--HG--
rename : build/valgrind/i386-redhat-linux-gnu.sup => build/valgrind/i386-pc-linux-gnu.sup
rename : build/valgrind/x86_64-redhat-linux-gnu.sup => build/valgrind/x86_64-pc-linux-gnu.sup
extra : rebase_source : ad94ce69e8094d2b9ddae97a3d261945886c0a61
They were added in bug 1427344 to reduce the differences between builds
on CentOS docker images and builds on Debian docker images. We're now
entirely on Debian docker images, so we can back this out.
--HG--
extra : rebase_source : 672e389cb8d6dbf7860a989024690de18e5c320e
As of bug 1430036 it was only set when building on CentOS, and as of bug
1432398, we don't have CentOS-based docker images anymore.
--HG--
extra : rebase_source : 5ade9bee773bca3283cfdb9d69209033fe82253f
It's been more than two weeks since the 1.23 stable release, and
we're making official builds with that toolchain release, so begin
requiring that version so new language features can be used in
development.
MozReview-Commit-ID: E6WuP41ceTn
--HG--
extra : rebase_source : 75850dd9edbf8e3f9beab394e4af7fad76ce3b17
This will tell us which functions PGO is spending its time in,
which functions are being optimized, and other interesting
statistics. The output is a handful of pages of text in our
current PGO builds and it seems useful for understanding the PGO
black box.
MozReview-Commit-ID: HUbAhltQrdL
--HG--
extra : rebase_source : eb188ebf84ee9ce6c7fe8bf69b6b156ee9f14956
We want annotationProcessors to be compiled and archived into a JAR at
build time, ready to generate JNI wrappers. (That is, until we turn
the whole thing into a real annotation processor.) But even if we do
use a real annotation processor, we still need to generate SDK
bindings, which is less clearly expressed as an annotation processor.
(It's more of a build step.)
Gradle provides a huge number of ways to organize build logic to
achieve this: see
https://docs.gradle.org/current/userguide/organizing_build_logic.html.
Unfortunately, the best such way -- putting the code into
$topsrcdir/buildSrc -- has key disadvantages:
1) it pollutes the top-level $topsrcdir, and there's no way to change the
location of buildSrc (https://github.com/gradle/gradle/issues/2472);
2) it's complicated to have a dependent project
(mobile/android/annotations) expose its code via a buildSrc project;
3) using buildSrc at all appears to conflict with the Android-Gradle
plugin version that we are using.
Therefore, this commit does something much simpler: it adds a
Java-only project and uses the resulting Gradle "Jar" task and archive
output as input to the existing Gradle "generate JNI wrappers" task.
MozReview-Commit-ID: 2OyYLPneE1M
--HG--
extra : rebase_source : d99b74a0a1e0bb3e8f4d4540978328388e5c2e42
We add a wrapper for llvm-dsymutil for macosx CI builds such that when
it crashes, we attempt to get a reduced test case and upload it as a
build artifact. This will allow to more easily report such crashes
upstream.
--HG--
extra : rebase_source : be208e6a46b60659a4e51acbe2bd7c4081189d1c
By default, wget prints dots every 1k bytes. This can render a
lot of output for large files. We switch to the "mega" style, which
makes each dot represent 64k, thus reducing output by up to 64x.
We also force the use of dot display. By default, it uses "bar"
which attempts to use terminal formatting if possible. Since most
of this code executes in CI and terminal control characters can
interfere with logged output, we force the use of "dot." (Although
wget appears to automatically switch to dot in TC today. But
consistency is good.)
MozReview-Commit-ID: IpTWJdcauTV
--HG--
extra : rebase_source : 5c9aa1bbdcd78eaa0b31347ad026a2c1beaedc03
There were a few constraints in the choice of the version of dpkg to
backport:
- 1.17.20 is the first version that supports the debian source format
for that xz-utils package.
- versions >= 1.17.10 and <= 1.17.22 fail to build on wheezy.
- versions >= 1.17.21 depend on a version of patch not available on
wheezy.
All in all, the simpler choice was to go with version 1.17.20 with a
backport of the build failure fix.
That version of dpkg breaks the version of devscripts in wheezy, so the
version from wheezy-backports would be better to use, but we can't
unconditionally use it on all builds, because it happens that
mk-build-deps from that version is broken with the dpkg in wheezy.
In the end, it's simpler to build that backport and rely on package task
dependencies rather than selectively install the package from
wheezy-backports, so we do that. Except we can't use version
2.14.11~bpo70+1 because of bug 1419577.
--HG--
extra : rebase_source : 19ad1a44b770229fbc7e15bbcf01d3cb101315a8
In many cases, building docker images starts on machines that don't have
a cached checkout, and it often takes forever to get a full clone. It
used to be worsened when 3 jobs could run at the same time because the
worker would start up clean, and 3 jobs would be doing a mercurial clone
at the same time, thrashing I/O, but that part is fortunately fixed.
It is still, however, appreciable not to waste time in the mercurial
clone part of image creation.
--HG--
extra : rebase_source : 8c76bc91e1d5102f68c43e1050d61971fef32e9f
Will also address Bug 1377553 and part of Bug 1419607
MozReview-Commit-ID: AUCqBxEGpAl
--HG--
extra : rebase_source : 5547e2c8fbf4e2e87182b8720d8352c131e4ec65
This marks **/docs/** as exclusively docs, and code that is autodoc'd as
inclusively docs.
That means that a change that purely modifies documentation files will *only*
run `docs` tasks, while a change that modifies autodoc'd source code will
*additionaly* run `docs` tasks. The tasks do not run by default.
MozReview-Commit-ID: G9tOK0AwtrI
--HG--
extra : rebase_source : 8dd971e5c9b0eb5f47895664a4ea49442f303ecb
extra : source : 0881de9b2b5e36ec37cc866f1d4af109da57a919
This marks **/docs/** as exclusively docs, and code that is autodoc'd as
inclusively docs.
That means that a change that purely modifies documentation files will *only*
run `docs` tasks, while a change that modifies autodoc'd source code will
*additionaly* run `docs` tasks. The tasks do not run by default.
MozReview-Commit-ID: G9tOK0AwtrI
--HG--
extra : rebase_source : 8dd971e5c9b0eb5f47895664a4ea49442f303ecb
extra : source : 0881de9b2b5e36ec37cc866f1d4af109da57a919
In many cases, building docker images starts on machines that don't have
a cached checkout, and it often takes forever to get a full clone. It
used to be worsened when 3 jobs could run at the same time because the
worker would start up clean, and 3 jobs would be doing a mercurial clone
at the same time, thrashing I/O, but that part is fortunately fixed.
It is still, however, appreciable not to waste time in the mercurial
clone part of image creation.
--HG--
extra : rebase_source : bbe8b001849e59bb655bb0e9766a6071ad38a52c
The one available in Debian wheezy is 3.81, but we're explicitly using
4.0 on CentOS, most notably because of its --output-sync option which
helps make logs better in some ways.
This takes the package from Debian jessie and builds it for Debian
wheezy.
--HG--
extra : rebase_source : 20bb550703fec41ed0175ef7f78c5b9a394160f3
Switching to Debian build images will force a version bump from
pulseaudio 0.9.something to pulseaudio 2.0. Practically speaking, as
long as bug 1427150 is fixed (which it is), this is strictly better,
because this enables all the PA_CHECK_VERSION(2,0,0) code in libcubeb,
which handles port availability (whether output is plugged or not), and
with bug 1427150, it does so in a backward compatible manner.
Now, since this is a behavior change from what we're currently shipping,
this has the potential of triggering unexpected test failures, or break
sound for users. The likelyhood of the latter happening is rather low,
though, because Linux distros have been building with pulseaudio >= 2.0
for a long time and we haven't heard about port availability breaking
sound for them. But it's still better to decouple this change from the
switch to Debian.
We abuse the build-gtk3.sh script which installs gtk3 in the CentOS
build image to install pulseaudio as well.
--HG--
extra : rebase_source : eb4e4033c50d59117b5199d1653d85f871503b2f
The one available in Debian wheezy is 1.7.10.4, which is really old, and
on our centos images, we're using 2.8.0rc3, which, while old too, is
more modern. While we may want to go with a more recent version, I'd
rather avoid differing from what we currently use, so use the exact same
version.
--HG--
extra : rebase_source : dfdf75a635073c248faef8a67648b2a83e4a1d84
One of the last remaining differences when building Firefox on Debian
with all the changes we've done so far is that linking against the
system libc statically links some CRT objects. This causes massive
differences in the resulting binaries because of slight differences
in those objects (because they weren't compiled with the same compiler
and because they're not for the exact same glibc version)
In practice, their content difference don't cause any problem. If they
did, we wouldn't be able to run our builds on newer systems than those
we build them on. The only hypothetical risk would be to run on systems
with a glibc older than Debian 7's, but those already can't run Firefox
anyways (those systems don't have Gtk+3, which is a system requirement).
AFAICT, this is only an hypothetical problem anyways, even such systems
with Gtk+3 should be able to run those builds. Plus, this is a change
that will happen anyways when switching to Debian-based build images,
since they would be using the CRT objects from there. We're merely
making it happen earlier so that the differences from switching to
Debian-based build images are more tractable.
Note we only do this when building GCC on Debian, allowing to roll back
to CentOS-based toolchains by just switching back the toolchain jobs to
use the desktop-build docker image again.
When building on Debian (which we now are), this means we enable
.init_array/.fini_array.
When building on CentOS, this means no change. Which implies we could
roll back to CentOS-based toolchains by just switching back the
toolchain jobs to use the desktop-build docker image again.
This change causes massive differences in the resulting binaries because
of the offset differences, but practically speaking, there is no
difference. .init_array/.fini_array have been supported in glibc for 18
years.
We believe this is another spurious memcheck error triggered
by code from Rust 1.23.0. See also bug 1394696.
For some reason, this error occurs both with and without
the leading underscore on mangled std::sync::once methods,
so this change matches either with a wildcard.
MozReview-Commit-ID: 4upSAPqAtNA
--HG--
extra : rebase_source : 5f697aaa5e170369f08d385d10c1aac9d8c1e50b
This marks **/docs/** as exclusively docs, and code that is autodoc'd as
inclusively docs.
That means that a change that purely modifies documentation files will *only*
run `docs` tasks, while a change that modifies autodoc'd source code will
*additionaly* run `docs` tasks. The tasks do not run by default.
MozReview-Commit-ID: G9tOK0AwtrI
--HG--
extra : rebase_source : 8dd971e5c9b0eb5f47895664a4ea49442f303ecb
extra : source : 0881de9b2b5e36ec37cc866f1d4af109da57a919
We build packages of the same versions that were installed by
taskcluster/docker/recipes/install-cmake.sh and
taskcluster/docker/centos6-build/system-setup.sh in the desktop-build
image.
--HG--
extra : rebase_source : 843b89065daabd450f54ebf7a2cf55d00977e23a
Currently, the build can finish succesfully even when one of the
programs fails to build and is not included in the final artifact. The
macosx build then fails because of that, which is the wrong place to
be failing.
--HG--
extra : rebase_source : 4a41b2f96eea45d3eefa2734900603b6e6ee0ea5
There are multiple methods with the same name and that differ in their
arguments. They end up being ordered in the source file randomly,
despite there being some sorting done, because the sorting was only done
on the method name.
Now, when the method name matches, also compare the arguments.
--HG--
extra : rebase_source : a89b8c9dbad1d7506e0068119ba25cd34150bafc
The Python version validation in virtualenv.py is only called in 2
locations: `python -m mozbuild.virtualenv` and in moz.configure.
I believe that nobody calls `python -m mozbuild.virtualenv` any more.
That means that moz.configure is the only caller of
verify_python_version(). That means we can inline the logic into
moz.configure.
It makes sense for version checking to live in moz.configure because
the role of moz.configure is to evaluate the sanity of the environment.
So this commit does just that.
MozReview-Commit-ID: 7FLL0cGblFS
--HG--
extra : rebase_source : 4c2ecbe06399aad917f58ffb25a571993b736965
Recent refactoring made the wrapper pretty useless. Since I'm about
to look at this code once more, let's remove it while we're here.
MozReview-Commit-ID: GA9cKeLH7Iu
--HG--
extra : rebase_source : 9e08ed60eab694a9f20385aa5f85a9909809b199
The system binutils and gcc are built with that option on Debian, but
not on CentOS. That makes no practical difference, except for the fact
that when building GCC, we use our own-built binutils (as per bug
1427316), but use the system GCC. And a GCC built with --with-sysroot=/
doesn't work with a binutils built without. However, a GCC built without
--with-sysroot=/ works fine with a binutils built with it. So this
change is compatible with building our GCC on both CentOS and Debian.
We currently use a 32-bit Rust toolchain for win32 builds, but this can lead
to OOM situations. This patch makes win32 builds use a 64-bit Rust toolchain,
which requires a little bit of extra configuration because rustc needs to
be able to find a link.exe that produces 64-bit binaries for building
things like build scripts, which are host binaries.
We will now generate a batch file that sets LIB to the paths to 64-bit
libraries and invokes the x64-targeting link.exe, and add a section to the
.cargo/config file to instruct cargo to use that batch file as the linker
when producing 64-bit binaries.
MozReview-Commit-ID: 9vKBbm7Gvra
--HG--
extra : rebase_source : 273a99be71914167664482c2bdb26c840ec6867b
mk_add_options has this kind of awkward feature where
mk_add_options VAR=value
would set VAR for the build through client.mk, but not when running
make -C objdir target. But
mk_add_options "export VAR=value"
does.
We might want to change that on the long run, but the side effects would
have to be calculated first.
OTOH, we have automation jobs that run compilations during `make check`
(e.g. rusttests), which is not invoked through client.mk. So they
currently don't get the same PATH as the build part, meaning that
they're using system binutils instead of the one from the GCC toolchain
package.
--HG--
extra : rebase_source : aab7f221243c486cf70c7b0c91b9313231050ed8
We no longer support Android/armv6 and we requires NEON for Android/arm, so
we can remove armv6 support for Android.
MozReview-Commit-ID: Hh17BTyE0wR
--HG--
extra : rebase_source : 57e043ecb1bb57a026c0b656b82768b899ddae78
I was able to reproduce the failure to apply llvm-dsymutil on the last
mozilla-central revision before debug info was temporarily disabled for
rust, and validated that the problem was gone in clang 4.
After some bisection, r289565 was identified as having fixed the
problem, which, reading the commit message, makes sense.
It was however not possible to simply cherry-pick, because of multiple
code changes between 3.9 and 4. However, apart from the volume of
conflicting changes, it was more or less straightforward to backport.
Interestingly, the patch for r313872 was relying on changes from
r289565, which is why it required a variant specifically for 3.9, but
now we can use the same patch as for other versions. Well, except
there's a small difference in the context, and build-clang.py doesn't
allow fuzz, so we manually edit the patch to remove that line from the
context.
--HG--
extra : rebase_source : de0ab262d401c37c0e9300b0ef7923a07c009d87
Mike Hommey
Nathan Froyd
Kartikaya Gupta
Dan Minor
Jacek Caban
Robert Bartlensky
dvarga
David Heiberg
Ivan Pozdeev
Bogdan Tara
Marco Castelluccio
David Major
Chris Manchester
Gurzau Raul
Tiberius Oros
Ciure Andrei
Ms2ger
Tom Ritter
Gregory Szorc
Simon Fraser
Andreea Pavel
Dave Hunt
Geoff Brown
Henri Sivonen
Andrew Halberstadt
Xidorn Quan
Dorel Luca
Mike Hommey
Kris Maglione
Johan Lorenzo
Miko Mynttinen
shindli
arthur.iakab
Cosmin Sabou
Jonathan Watt
Chris AtLee
Emilio Cobos Álvarez
Panos Astithas
Christian Holler
Csoregi Natalia
Noemi Erli
Sylvestre Ledru
Justin Wood
Sofia Carrillo
Masatoshi Kimura
Andi-Bogdan Postelnicu
Brindusan Cristian
Rob Wood
Jan Henning
Aaron Klotz
Tom Prince
J.C. Jones
Bobby Holley
Jan Beich
Ryan VanderMeulen
Jeff Walden
Benjamin Bouvier
Andrew McCreight
Jeff Gilbert
Julian Seward
Narcis Beleuzu
Carl Corcoran
Sebastian Hengst
Mihai Tabara
Paolo Amadini
Matt Brubeck
Ralph Giles
Kim Moir
Nick Alexander
Chris Peterson
Rail Aliiev
Jesse Schwartzentruber
Ben Hearsum
J. Ryan Stinnett
Florian Quèze
Eric Rahm
Jim Chen
Jeremy Chen
Margareta Eliza Balazs
Dustin J. Mitchell
Matías Zúñiga
Jean-Luc Bonnafoux
Ted Mielczarek
Zibi Braniecki
Makoto Kato