Since bug 635961, building with relro makes elfhack try to use the bss
data for a temporary function pointer. If there is not enough space for
a pointer in the bss, elfhack will complain it couldn't find the bss.
In normal circumstances, this is most likely fine. Libraries with a bss
so small that it can't fit a pointer are already too small to be
elfhacked anyways. In Firefox, the two libraries with the smallest bss
have enough space for two pointers, and aren't elfhacked (libmozgtk.so
and libplds4.so).
However, the testcase that is used during the build to validate that
elfhack works doesn't have a large enough bss on x86-64, making elfhack
bail out, and the build fail as a consequence.
This, in turn, is due to the only non-thread-local zeroed data being an
int, which is not enough to fit a pointer on x86-64. We thus make it a
size_t.
--HG--
extra : rebase_source : bca2ddbf9d4a5e8786881fc524d642c38d610227
The PT_PHDR segment is optional, but the Android toolchain decides to
create one in some cases, and places it first. When that happens, the
work around for bug 1233963 fails, because the fake phdr section has not
been adjusted yet (it only happens when we see a PT_LOAD).
So we adjust the fake phdr section when we see a PT_PHDR segment (and
avoid re-updating it when we see a subsequent PT_LOAD).
--HG--
extra : rebase_source : 2190ec2f20ba9d144b8828874f9f8d70dd5ad2f6
Somehow, with the Android toolchain, we end up with
non-empty-but-really-empty DT_INIT_ARRAYs.
In practical terms, they are arrays with no relocations, and content
that is meaningless:
$ objdump -s -j .init_array libnss3.so
libnss3.so: file format elf32-little
Contents of section .init_array:
1086e0 00000000 ....
$ readelf -r libnss3.so | grep 1086e0
$ objdump -s -j .init_array libplugin-container-pie.so
libplugin-container-pie.so: file format elf32-little
Contents of section .init_array:
4479c ffffffff 00000000 ffffffff 00000000 ................
$ readelf -r libplugin-container-pie.so | grep 4479c
Because so far, elfhack expected meaningful DT_INIT_ARRAYs, it bailed out
early in that case.
--HG--
extra : rebase_source : 217aacb42fdfabb466ed1f8149dfaeb4a595eda8
When the clang plugin is used, building something during export needs to
happen after the plugin is built. But there is no dependency ensuring
this happens.
OTOH, these sources in elfhack/inject don't need to be built that early,
so we'll just leave to the build system to build it at a proper time.
--HG--
extra : rebase_source : a6bef8ec6eece3a1b0e45f84c907c2fbc0800863
We can just check the GPG signature for the upstream tarballs that are
GPG signed. We keep a copy of the relevant GPG keys in tree so that
we only use a controlled set of keys.
I validated the GPG keys by:
- Creating a fresh keyring.
- Importing the keys with gpg --receive-key.
- Importing my own GPG public key in that keyring.
- Importing the gpg keys that the PGP pathfinder told me were on the path
to those keys (which weren't directly in their keyring, so I had to
manually find some steps first).
- Using `gpg --check-sigs` to validate that the all those keys I got are
the right ones.
Then the relevant GPG keys were exported with `gpg --export --armor` and
stripped with https://github.com/glandium/pgpstrip/.
For MPC, the first GPG-signed version upstream was 0.8.2, while the GCC
script to download prerequisites downloads 0.8.1. So instead of using
0.8.1, we use 0.8.2, which we can verify.
For GMP, the GCC script downloads 4.3.2. The only web-of-trust path is
through a revoked key, which signs a revoked uid of the GMP key.
Releases newer than 5.1.0 are signed with a new key that can be
validated with the steps above. So instead of using 4.3.2, we use 5.1.3
(last of the 5.1.x line).
But MPFR 2.4.2, which the GCC script downloads, doesn't build against
GMP 5.1.3, so instead of that, we use MPFR 3.1.5.
Sadly, the remaining GCC prerequisites are not signed, so I had to:
- Download the files from ftp.gnu.org.
- Download the corresponding files from snapshot.debian.org.
- Compare the raw files when possible, or the uncompressed (not extracted)
files (when, thankfully, they matched).
- Validate those snapshot.debian.org files checksums against the
checksums in the corresponding Sources.bz2/xz files.
- Validate the Sources.bz2/xz checksums against the corresponding InRelease
files.
- Validate the InRelease files GPG signatures against the Debian
archives keyring.
With all those things we actually don't get through the GCC script, we
also change how we get those prerequisites, by diverting the commands
the script runs and making it output the urls instead of downloading and
extracting the files.
All downloaded files, GPG-validated or otherwise, have their SHA-256
digest checked against a list in build/unix/build-gcc/checksums.
--HG--
extra : rebase_source : e6809a6ac392e6c5f99801826e1d30bdeee7ddf5
CLOSED TREE
Backed out changeset 158233bce738 (bug 1197325)
Backed out changeset b5ac3fa0bbe7 (bug 1197325)
Backed out changeset 55a8ad127517 (bug 1197325)
This removes the unnecessary setting of c-basic-offset from all
python-mode files.
This was automatically generated using
perl -pi -e 's/; *c-basic-offset: *[0-9]+//'
... on the affected files.
The bulk of these files are moz.build files but there a few others as
well.
MozReview-Commit-ID: 2pPf3DEiZqx
--HG--
extra : rebase_source : 0a7dcac80b924174a2c429b093791148ea6ac204
Something similar was done in bug 1278718 for ASan builds, because of
indirect dependencies on libstdc++ being picked by the linker and
leading to linkage failure with the older binutils from the CentOS 6
image we use to do desktop builds.
Build slaves on automation are based on Centos 6, which doesn't have a
recent enough version of libstdc++ for our new requirements. But since
we're building with a recent GCC or clang with its own libstdc++, we do
have such a libstdc++ available somewhere, and the compiler picks it
when invoking the linker.
Problems start happening when we execute some of the built programs
during the build, like host tools (e.g. nsinstall), or target programs
(xpcshell, during packaging). In that case, we need the compiler's
libstdc++ to be used. Which required adding the GCC or clang library
directory to LD_LIBRARY_PATH.
Unconveniently enough, the clang 3.5 tooltool package we're using for
ASAN builds until we can update to at least 3.8 (bug 1278718) doesn't
contain libstdc++.so. So for those builds, pull the GCC package from
tooltool as well, and pick libstdc++ from there.
We have very few directories where we have SOURCES declared that are not
part of a library or program in some way. In fact, there is only one
where it is legitimate because we only use the object file
(build/unix/elfhack/inject). Others are the result of moz.build control
flow (see e.g. netwerk/standalone), and we end up building more objects
than we need to.
There are other cases where we need objects without actually linking
them anywhere, but there are other sources in the same directory, and a
corresponding Linkable is emitted. And in fact, the only case I knew
about (media/libvpx), doesn't use such objects since bug 1151175.
The unix mozconfig.rust is actually completely generic now that we're
using toolchains built with --enable-rpath in tooltool.
Move the mozconfig.rust fragment up a level to reduce confusion.
Write a mozconfig.rust fragment which makes the rust toolchain
provided by tooltool available for linux builds, similar to
what we do for MacOS X.
Include this in linux64 mozconfigs to enable rust for official
nightly builds of that target. These aren't used outside of automation
builds, so including rust there will verify code on check-in
without requiring developers to install rust.
We must whitelist the mozconfig fragment to pass the consistency
check since we're not ready to let this feature ride the trains
to beta and release.
The tooltool reference is to a custom build of rustc 1.4
with --enable-rpath to avoid having to add the rustc lib
directory to LD_LIBRARY_PATH which somehow conflicts with
the gtk3 build we also install through tooltool.
It is also built with --enable-llvm-static-stdcpp on a
rust-buildbot dist docker image (centos:5 + script updates)
to avoid issues with GLIBCXX and GLIBC symbol versions.
This adds a stages config option, which can be used to select 1, 2, and
3 stage builds. It also marks the default trunk configuration to do 3
stage builds, and defaults to that.
Since CMake generated build systems can run cmake if necessary, this
will make it possible to pick up changes from the source directory if
any and resume as much of the build as possible.
This builds the foundation for removing the need to blow away any of the
work done by the previous runs of the script.
We build gcc after clang, and extract libgcc libraries and libstdc++
headers from gcc and place them in the clang installation directory in a
way that clang favors before it searches the system for libraries and
includes.
Its only purpose is to disable PGO. Where that was not already explicitly done,
or irrelevant (because the directory only contains python), I disabled it in
moz.build.
As part of this move, HOST_NSPR_MDCPUCFG needed to be changed to get the quoting right.
--HG--
extra : commitid : J26MhSiPq9g
extra : rebase_source : 81c5b98371042803741ddace8d01b0097757dff3
When switching between Gtk+3 and Gtk+2, config.cache will contain a PKG_CONFIG
that may not be suitable for the build:
- after a Gtk+2 build, config.cache will point to the system pkg-config, which
doesn't like the pkg-config files in the Gtk+3 tooltool package.
- after a Gtk+3 build, config.cache will point to the Gtk+3 tooltool package's
pkg-config, which is likely not there in a Gtk+2 build.
Setting PKG_CONFIG avoids all config.cache considerations altogether, so set it
appropriately for both cases.
Some mozconfigs don't include mozconfig.linux*, and don't get gtk-related
definitions, so move them in a separate mozconfig. To avoid having two
files, one for 32-bit builds and one for 64-bit builds, rely on the
includer to set PKG_CONFIG_LIBDIR appropriately.
At the same time, move all the --enable-default-toolkit=cairo-gtk2 in that
new file in the case the gtk3 package wasn't pulled from tooltool.
Write a mozconfig fragment which makes the rust toolchain
provided by tooltool available for linux builds.
Use linux64 mozconfigs to enable rust for official builds of
that target. These aren't used outside of automation builds,
so including rust there will verify code on check-in without
requiring developers to install rust.
LTO changes in GCC5 make it break the assumptions made in the test-ctors
source. The simplest and more correct thing to do is to just force the
test files never to be built with LTO. They are meant to have a particular
layout that LTO might break at any time anyways.
There are, sadly, many combinations of linkage in use throughout the tree.
The main differentiator, though, is between program/libraries related to
Gecko or not. Kind of. Some need mozglue, some don't. Some need dependent
linkage, some standalone.
Anyways, these new templates remove the need to manually define the
right dependencies against xpcomglue, nspr, mozalloc and mozglue
in most cases.
Places that build programs and were resetting MOZ_GLUE_PROGRAM_LDFLAGS
or that build libraries and were resetting MOZ_GLUE_LDFLAGS can now
just not use those Gecko-specific templates.
'make check' is somewhat special in that we want to trigger testers
before it finishes. Since moving sendchange into mach is difficult and
'make check' may be going away in the future anyway, just pull it out
for now.
Also remove the MOZ_AUTOMATION_*_SENDCHANGE flags since we aren't using
them.
Incidentally, this makes the build fail because stdc++compat.o then appears
twice on the libxul linkage command line. This, in turn, is because
widget/xremoteclient creates both a program and an intermediate library for
libxul. The Program() template adds stdc++compat to the directory data,
which ends up being added to libxul as well. The same kind of issue arises
when linking the gtest-enabled libxul.
While eventually we'll be able to avoid those problems, it's not the case
yet, so work around the issue by making expand-libs skip .desc files that
appear multiple times.
- Linux ASAN nightly builds do not do update packaging
- Only set mar properties if the mar file was actually created
- Unset MOZ_SIGN_CMD for pretty-* steps
Many of the moved variables are likely not needed. moz.build should one
day validate the sandbox's output and error if "useless" variables are
present.
--HG--
extra : rebase_source : 3abdea056c18d00ede8c15b37db60532eca58630
Mike Hommey
Cameron McCormack
Wes Kocher
rforbes
Chris Manchester
Steve Fink
Lee Salzman
Ting-Yu Chou
Carsten "Tomcat" Book
Jesse Schwartzentruber
Justin Wood
Phil Ringnalda
Benjamin Smedberg
Nathan Froyd
Henri Sivonen
Enes Goktas
Tom Tromey
Edmund Wong
Mike Shal
Nick Thomas
Ehsan Akhgari
Ralph Giles
Ms2ger
Ted Mielczarek
Ryan VanderMeulen
Rail Aliiev
Birunthan Mohanathas
Masatoshi Kimura
Georg Koppen
Nicholas Nethercote
Christian Holler
Robert O'Callahan
Trevor Saunders
Frank Wein
Cykesiopka
Benjamin Peterson
Joey Armstrong
Ed Morley
Gregory Szorc
Brian O'Keefe