This means --enable-bootstrap now is the same as what
--enable-bootstrap=install currently does. --enable-bootstrap=install is at the
same time desupported.
We also remove --enable-bootstrap=update, which is not worth supporting.
Differential Revision: https://phabricator.services.mozilla.com/D105719
This changes things such that setting e.g. NASM=/usr/bin/nasm will avoid
bootstrapping nasm even when bootstrapping is enabled.
This is not applied to CC/CXX/HOST_CC/HOST_CXX because things are more
complicated.
This also simplifies how check_prog is called for a bootstrapped tool,
and avoids the repetition of when.
CBINDGEN handling needs the pattern being applied manually because it
currently doesn't use check_prog. Once --enable-bootstrap=install
becomes the default on developer builds, it will be possible to simplify
this.
Differential Revision: https://phabricator.services.mozilla.com/D105718
Because --with-ccache without a value fulfills a @depends_if, we need to
check for the length. And because we check the length, we can just use a
@depends.
Differential Revision: https://phabricator.services.mozilla.com/D106084
- Only bootstrap sccache when building with --with-ccache=sccache or
CCACHE=sccache
- Don't bootstrap dump_syms or nasm on builds that don't compile
Differential Revision: https://phabricator.services.mozilla.com/D105704
Instead of preemptively check for it, and then check if it's good enough to
build AV1, only check for (and bootstrap) nasm when building AV1 requires
it.
At the same time, we future-proof the code to be able to handle multiple
things requiring nasm, which we're going to add shortly.
Differential Revision: https://phabricator.services.mozilla.com/D105425
We want to find the full path to the correct `lldb-server` in the NDK.
We reference this variable in a later patch when preparing the device for
debugging.
Differential Revision: https://phabricator.services.mozilla.com/D94380
None of HAVE_NASM, HAVE_YASM, NASM_MAJOR_VERSION and NASM_MINOR_VERSION are
used. Also, the YASM variable is not necessary for old-configure anymore.
Differential Revision: https://phabricator.services.mozilla.com/D105269
* Puts the docs in order, so that contributors aren't jumping to the
middle of the page to install system tools, then back to the top to
clone Firefox.
* Removes docs on MacPorts since it's being removed in bug 1688263.
* Removes step to manually install brew packages since that happens
automatically in bootstrap now.
* Simplifies mercurial installation docs
* Removes unnecessary mozconfig-tweaking instructions
* Removes almost-always-unnecessary DEFINE and troubleshooting
information.
Differential Revision: https://phabricator.services.mozilla.com/D102973
Bug 1690930 added sysroots that can be bootstrapped. With this change,
we allow --enable-bootstrap=install to pull the right sysroot for the
configured target, and --enable-bootstrap to update it if it was already
there.
Differential Revision: https://phabricator.services.mozilla.com/D104797
We used to only push to opt-level=2 on --enable-release builds, to make
local builds faster with opt-level=1. Years later, it seems opt-level=2 makes no noticeable
difference in build times vs. opt-level=1, neither on my Threadripper
workstation at -j64 or my M1 Macbook Air at -j4.
That's one less difference to carry.
Differential Revision: https://phabricator.services.mozilla.com/D103266
2021-01-22 Kevin Jacobs <kjacobs@mozilla.com>
* automation/abi-check/previous-nss-release, lib/nss/nss.h,
lib/softoken/softkver.h, lib/util/nssutil.h:
Set version numbers to 3.62 Beta
[680ec01577b9]
2021-01-23 Kevin Jacobs <kjacobs@mozilla.com>
* tests/chains/scenarios/nameconstraints.cfg,
tests/libpkix/certs/NameConstraints.ipaca.cert,
tests/libpkix/certs/NameConstraints.ocsp1.cert:
Bug 1686134 - Renew two chains libpkix test certificates. r=rrelyea
[3ddcd845704c]
2021-01-25 Kevin Jacobs <kjacobs@mozilla.com>
* gtests/common/testvectors/hpke-vectors.h,
gtests/pk11_gtest/pk11_hpke_unittest.cc, lib/pk11wrap/pk11hpke.c,
lib/pk11wrap/pk11hpke.h, lib/pk11wrap/pk11pub.h:
Bug 1678398 - Update HPKE to draft-07. r=mt
This patch updates HPKE to draft-07. A few other minor changes are
included:
- Refactor HPKE gtests for increased parameterized testing.
- Replace memcpy calls with PORT_Memcpy
- Serialization tweaks to make way for context Export/Import (D99277).
This should not be landed without an ECH update, as fixed ECH test
vectors will otherwise fail to decrypt.
[e0bf8cadadc7]
* automation/abi-check/expected-report-libnss3.so.txt,
gtests/pk11_gtest/pk11_hpke_unittest.cc, lib/nss/nss.def,
lib/pk11wrap/pk11hpke.c, lib/pk11wrap/pk11pub.h:
Bug 1678398 - Add Export/Import functions for HPKE context. r=mt
This patch adds and exports two new HPKE functions:
`PK11_HPKE_ExportContext` and `PK11_HPKE_ImportContext`, which are
used to export a serialized HPKE context, then later reimport that
context and resume Open and Export operations. Only receiver
contexts are currently supported for export (see the rationale in
pk11pub.h).
One other change introduced here is that `PK11_HPKE_GetEncapPubKey`
now works as expected on the receiver side.
If the `wrapKey` argument is provided to the Export/Import
functions, then the symmetric keys are wrapped with AES Key Wrap
with Padding (SP800-38F, 6.3) prior to serialization.
[8bcd12ab3b34]
* automation/abi-check/expected-report-libssl3.so.txt,
gtests/ssl_gtest/libssl_internals.c,
gtests/ssl_gtest/libssl_internals.h,
gtests/ssl_gtest/ssl_extension_unittest.cc,
gtests/ssl_gtest/tls_ech_unittest.cc, lib/ssl/ssl3con.c,
lib/ssl/ssl3ext.c, lib/ssl/ssl3ext.h, lib/ssl/sslexp.h,
lib/ssl/sslimpl.h, lib/ssl/sslsecur.c, lib/ssl/sslsock.c,
lib/ssl/sslt.h, lib/ssl/tls13con.c, lib/ssl/tls13con.h,
lib/ssl/tls13ech.c, lib/ssl/tls13ech.h, lib/ssl/tls13exthandle.c,
lib/ssl/tls13exthandle.h, lib/ssl/tls13hashstate.c,
lib/ssl/tls13hashstate.h:
Bug 1681585 - Update ECH to Draft-09. r=mt
This patch updates ECH implementation to draft-09. Changes of note
are:
- Acceptance signal derivation is now based on the handshake secret.
- `config_id` hint changes from 32B to 8B, trial decryption added on
the server.
- Duplicate code in HRR cookie handling has been consolidated into
`tls13_HandleHrrCookie`.
- `ech_is_inner` extension is added, which causes a server to indicate
ECH acceptance.
- Per the above, support signaling ECH acceptance when acting as a
backend server in split-mode (i.e. when there is no other local
Encrypted Client Hello state).
[ed07a2e2a124]
2021-01-24 Kevin Jacobs <kjacobs@mozilla.com>
* cmd/selfserv/selfserv.c:
Bug 1681585 - Add ECH support to selfserv. r=mt
Usage example: mkdir dbdir && cd dbdir certutil -N -d . certutil -S
-s "CN=ech-public.com" -n ech-public.com -x -t "C,C,C" -m 1234 -d .
certutil -S -s "CN=ech-private-backend.com" -n ech-private-
backend.com -x -t "C,C,C" -m 2345 -d . ../dist/Debug/bin/selfserv -a
ech-public.com -a ech-private-backend.com -n ech-public.com -n ech-
private-backend.com -p 8443 -d dbdir/ -X publicname:ech-public.com
(Copy echconfig from selfserv output and paste into the below
command) ../dist/Debug/bin/tstclnt -D -p 8443 -v -A
tests/ssl/sslreq.dat -h ech-private-backend.com -o -N <echconfig> -v
[92dcda94c1d4]
Differential Revision: https://phabricator.services.mozilla.com/D102982
This adds a --enable-bootstrap build flag that will automatically update
cbindgen, node, clang, sccache, nasm, wine, lucetc, dump_syms, pdbstr,
and winchecksec if they are already installed in ~/.mozbuild.
Eventually, we'll want to allow to install toolchains that weren't
already install, but one step at a time.
This explicitly doesn't cover rustc, which is its own can of worms, or
android-{ndk,sdk}, which are not installed via toolchain artifacts
currently.
Differential Revision: https://phabricator.services.mozilla.com/D101723
Instead of adding all possible tool paths from ~/.mozbuild, we only
add the relevant paths for each of the tools we search for.
Differential Revision: https://phabricator.services.mozilla.com/D101718
The only thing that varies between toolchain_search_path and
host_toolchain_search_path is the path to the MSVC C/C++ compiler and
tools, because MSVC has a different compiler for each platform, and host
and target platforms may differ (when e.g. compiling for arm64 on
x86_64).
However, we don't use the MSVC compiler anymore, and the only thing we
use its path for is the assembler, which we don't use for host things
(and we don't have a HOST_AS), and to derive the path to some system
headers/SDK.
Differential Revision: https://phabricator.services.mozilla.com/D101714
There is only one place where it's used:
config/check_vanilla_allocations.py, which is only executed from
js/src/build/Makefile.in on the condition that the build is targeting
Linux and not LTO. But the LTO test is actually outdated, because we
don't build with `-flto`, but `-flto=thin`, so the exclusion doesn't
work anymore.
There is however no AC_CHECK_PROG, and we currently rely on NM to be
given, or fall back to "nm", which works in most cases, except LTO with
clang. It works on CI because in LTO builds we explicitly set NM to
llvm-nm (which can output symbols from LLVM bitcode objects), but we
could also do that automatically.
So we add a full detection of nm/llvm-nm to python configure, and limit
it to Linux, since we only ever use it there.
Differential Revision: https://phabricator.services.mozilla.com/D101681
js/src/aclocal.m4 contains includes starting with `../../`.
As explained in https://bugzilla.mozilla.org/show_bug.cgi?id=1680862#c8,
m4 will first try to resolve this path relative to the working directory
and only if that path doesn't exists, fall back to the location set by
`localdir` (from `-I`).
The working directory is usually MOZ_OBJDIR, an immediate subdirectory
of topsrcdir, so `../../` resolves to a location outside of topsrcdir.
Usually, that path does not exist, and m4 falls back to `localdir` that
was passed via `-I`.
But if that path existed and is incompatible with the current Gecko
checkout, then the build will fail (see bug report). To prevent this
from happening, this patch fixes the working directory to `localdir`,
so that m4 will immediately find the expected file.
Differential Revision: https://phabricator.services.mozilla.com/D101500
In addition to warning on regular methods overloading virtual functions,
GCC also watches for static functions doing such overloads.
:andi confirmed that this is not valuable, so the warning is being
disabled for GCC.
Differential Revision: https://phabricator.services.mozilla.com/D101367
Passing `-flto=thin` worked previously but the value passed was just ignored
and full lto was performed. On newer versions of gcc passing an unknown value
causes failure. So this commit checks if `-flto=thin` is passed and fails with
an error message if so, else full lto is enabled if any other value is passed.
Differential Revision: https://phabricator.services.mozilla.com/D100953
Some distros include flags when they specify the location of a binary,
such as: `XARGS=xargs -r`.
This was confusing in `configure`, since:
* We require that environment variables contain only path
overrides (without flags).
* The error message thrown when configure chokes was unclear: "why
would `$ xargs -r` fail?"
This patch should make our "path-only" requirement more clear.
Differential Revision: https://phabricator.services.mozilla.com/D100044
Old clang shakes its fist when `auto&& item : range` is used with a
range
that returns values instead of references.
Modern `clang` doesn't warn for this scenario, so we disable the
warning.
Also removes pragmas that manually disable this warning.
Differential Revision: https://phabricator.services.mozilla.com/D100155
I was waiting for a better reason to do this, because the cbindgen
changes from 0.15.0 to 0.16.0 don't break trunk builds. But since
downstream has updated (see bug 1684180) and there's no reason not to,
let's do this to avoid future churn.
Differential Revision: https://phabricator.services.mozilla.com/D100499
2020-12-11 Kevin Jacobs <kjacobs@mozilla.com>
* automation/abi-check/expected-report-libssl3.so.txt, automation/abi-
check/previous-nss-release, lib/nss/nss.h, lib/softoken/softkver.h,
lib/util/nssutil.h:
Set version numbers to 3.61 Beta
[f277d2674c80]
* gtests/<...>
Bug 1677207 - Update Google Test to release-1.10.0 r=bbeurdouche
./gtests/google_test/update.sh release-1.10.0 && hg remove -A && hg
add gtests/google_test/*
[89141382df45]
* gtests/<...>
Bug 1677207 - Replace references to TestCase, which is deprecated,
with TestSuite r=bbeurdouche
grep -rl --exclude-dir=google_test INSTANTIATE_TEST_CASE_P gtests |
xargs sed -i '' s/INSTANTIATE_TEST_CASE_P/INSTANTIATE_TEST_SUITE_P/g
grep -rl --exclude-dir=google_test SetUpTestCase gtests | xargs sed
-i '' s/SetUpTestCase/SetUpTestSuite/g
[e15b78be87fa]
* gtests/ssl_gtest/ssl_ciphersuite_unittest.cc,
gtests/ssl_gtest/ssl_debug_env_unittest.cc,
gtests/ssl_gtest/ssl_extension_unittest.cc,
gtests/ssl_gtest/ssl_loopback_unittest.cc,
gtests/ssl_gtest/ssl_renegotiation_unittest.cc,
gtests/ssl_gtest/ssl_resumption_unittest.cc,
gtests/ssl_gtest/ssl_version_unittest.cc,
gtests/ssl_gtest/tls_ech_unittest.cc:
Bug 1677207 - Use GTEST_SKIP in ssl_gtests. r=bbeurdouche
[0772f1bf5fd6]
2020-12-17 Robert Relyea <rrelyea@redhat.com>
* gtests/common/testvectors/ike-aesxcbc-vectors.h,
gtests/common/testvectors/ike-sha1-vectors.h,
gtests/common/testvectors/ike-sha256-vectors.h,
gtests/common/testvectors/ike-sha384-vectors.h,
gtests/common/testvectors/ike-sha512-vectors.h,
gtests/common/testvectors_base/test-structs.h,
gtests/pk11_gtest/manifest.mn, gtests/pk11_gtest/pk11_gtest.gyp,
gtests/pk11_gtest/pk11_ike_unittest.cc, lib/softoken/sftkike.c:
Bug 1682071 IKE Quick mode IPSEC give you incorrect keys if you are
asking for keys smaller than the hash size.
IKE Appendix B fixes.
This patch fixes 2 problems.
If you run either ike v1 App B or quick mode asking for a key with
length
mod macsize = 0, you will generate an extra block that's not used
and overwrites the end of the buffer.
If you use quick mode, the function incorrectly subsets the
existing key
rather than generating a new key. This is correct behavior for
Appendix B, where appendix B is trying to take a generated key and
create a new longer key (with no diversification, just transform the
key into something that's longer), so if you ask for a key less than
or equal to, then you want to just subset the original key. In quick
mode you are taking a base key and creating a set of new keys based
on additional data, so you want to subset the generated data. This
patch only subsets the original key if you aren't doing quickmode.
Full test vectors have now been added for all ike modes in this
patch as well (previously we depended on the FIPS CAVS tests to test
ike, which covers basic IKEv1, IKEv1_psk, and IKEv2 but not IKEv1
App B and IKE v1 Quick mode).
[f4995c9fa185]
2020-12-18 Robert Relyea <rrelyea@redhat.com>
* gtests/common/testvectors/rsa_pkcs1_2048_test-vectors.h,
gtests/common/testvectors/rsa_pkcs1_3072_test-vectors.h,
gtests/common/testvectors/rsa_pkcs1_4096_test-vectors.h,
gtests/freebl_gtest/Makefile, gtests/freebl_gtest/manifest.mn,
gtests/freebl_gtest/rsa_unittest.cc, gtests/manifest.mn,
gtests/pk11_gtest/pk11_rsaencrypt_unittest.cc,
gtests/pk11_gtest/pk11_rsaoaep_unittest.cc, lib/freebl/alghmac.c,
lib/freebl/alghmac.h, lib/freebl/rsapkcs.c:
Bug 1651411 New tlsfuzzer code can still detect timing issues in RSA
operations.
This patch defeats Bleichenbacher by not trying to hide the size of
the decrypted text, but to hide if the text succeeded for failed.
This is done by generating a fake returned text that's based on the
key and the cipher text, so the fake data is always the same for the
same key and cipher text. Both the length and the plain text are
generated with a prf.
Here's the proposed spec the patch codes to:
1. Use SHA-256 to hash the private exponent encoded as a big-
endian integer to a string the same length as the public modulus.
Keep this value secret. (this is just an optimisation so that the
implementation doesn't have to serialise the key over and over
again) 2. Check the length of input according to step one of
https://tools.ietf.org/html/rfc8017#section-7.2.2 3. When provided
with a ciphertext, use SHA-256 HMAC(key=hash_from_step1,
text=ciphertext) to generate the key derivation key 4. Use SHA-256
HMAC with key derivation key as the key and a two-byte big- endian
iterator concatenated with byte string "length" with the big- endian
representation of 2048 (0x0800) as the bit length of the generated
string.
- Iterate this PRF 8 times to generate a 256 byte string 5. initialise
the length of synthetic message to 0 6. split the PRF output into 2
byte strings, convert into big-endian integers, zero- out high-order
bits so that they have the same bit length as the octet length of
the maximum acceptable message size (k-11), select the last integer
that is no larger than (k-11) or remain at 0 if no integer is
smaller than (k-11); this selection needs to be performed using a
side-channel free operators 7. Use SHA-256 HMAC with key derivation
key as the key and a two-byte big-endian iterator concatenated with
byte string "message" with the big-endian representation of k*8
- use this PRF to generate k bytes of output (right-truncate last HMAC
call if the number of generated bytes is not a multiple of SHA-256
output size) 8. perform the RSA decryption as described in step 2 of
section 7.2.2 of rfc8017 9. Verify the EM message padding as
described in step 3 of section 7.2.2 of rfc8017, but instead of
outputting "decryption error", return the last l bytes of the
"message" PRF, when l is the selected synthetic message length using
the "length" PRF, make this decision and copy using side-channel
free operation
[fc05574c7399]
2020-12-22 Robert Relyea <rrelyea@redhat.com>
* gtests/freebl_gtest/rsa_unittest.cc,
gtests/pk11_gtest/pk11_rsaoaep_unittest.cc, lib/freebl/alghmac.c,
lib/freebl/rsapkcs.c:
Restore lost portion of the bleichenbacher timing batch that
addressed review comments. All the review comments pertained to
actual code comments, so this patch only affects the comments.
[fcebe146314e]
2020-12-22 Kevin Jacobs <kjacobs@mozilla.com>
* lib/dev/devslot.c:
Bug 1682863 - Revert nssSlot_IsTokenPresent to 3.58 after ongoing Fx
hangs with slow PKCS11 devices. r=bbeurdouche
This patch reverts the `nssSlot_IsTokenPresent` changes made in bug
1663661 and bug 1679290, restoring the version used in NSS 3.58 and
earlier. It's not an actual `hg backout` because the comment in
lib/dev/devt.h is worth keeping. While removing the nested locking
did resolve the hang for some (most?) third-party modules, problems
remain with some slower tokens after an even further relaxation of
the locking, which defeats the purpose of addressing the races in
the first place.
The crash addressed by these patches was caused by the Intermediate
Preloading Healer in Firefox, which has been disabled. We clearly
have insufficient test coverage for third-party modules, and now
that osclientcerts is enabled in Fx Nightly, any problems caused by
these and similar changes is unlikely to be reported until Fx Beta,
well after NSS RTM. I think the best option at this point is to
simply revert NSS.
[97ef009f7a78] [tip]
Differential Revision: https://phabricator.services.mozilla.com/D100401
The gyp flag logic in nICEr is supposed to ensure that the code is instrumented
for libFuzzer because we have a related fuzzing target. However, libFuzzer
instrumentation must be completely disabled for TSan due to incompatibility.
The current logic fails in doing so and incorrectly falls back to legacy
trace-pc instrumentation causing the TSan fuzzing build to fail on startup.
Differential Revision: https://phabricator.services.mozilla.com/D99351
Noemi Erli
Agi Sferro
Dorel Luca
Mike Hommey
Aaron Klotz
Mitchell Hentges
David Major
Kevin Jacobs
Rob Wu
Connor Sheehan
Sylvestre Ledru
Bogdan Tara
Emilio Cobos Álvarez
Sebastian Hengst
Dave Townsend
Christian Holler