Code review followups for 1675600: Restrict the use of new pass manager during LTO to builds where we're using the new pass manager in general, and (on Windows) where lld-link is new enough to understand the flag.
Differential Revision: https://phabricator.services.mozilla.com/D97372
Upstream clang supports --target=aarch64-apple-darwin, and that matches
what config.sub canonicalizes to, and thus what the default toolchain
prefix ends up being, so it's better to use aarch64-apple-darwin when
not compiling with Xcode clang.
Differential Revision: https://phabricator.services.mozilla.com/D97698
There is no earlier SDK that supports it. It seems Xcode clang doesn't
care (maybe it defaults to 11.0 is MACOSX_DEPLOYMENT_TARGET is too low?),
but upstream clang does.
Differential Revision: https://phabricator.services.mozilla.com/D97565
And don't set it via mozconfig. The default to /System/Library/PrivateFrameworks
may also not have matched the used SDK previously, so the new default is
better.
Differential Revision: https://phabricator.services.mozilla.com/D97564
2020-11-18 Kevin Jacobs <kjacobs@mozilla.com>
* lib/ssl/ssl3con.c, lib/ssl/tls13con.c, lib/ssl/tls13ech.c:
Bug 1654332 - Fixup a10493dcfcc9: copy ECHConfig.config_id with
socket r=jcj
A late review change for ECH was for the server to compute each
ECHConfig `config_id` when set to the socket, rather than on each
connection. This works, but now we also need to copy that config_id
when copying a socket, else the server won't find a matching
ECHConfig to use for decryption.
[3eacb92e9adf] [tip]
2020-11-17 Kevin Jacobs <kjacobs@mozilla.com>
* automation/abi-check/expected-report-libssl3.so.txt,
cmd/tstclnt/tstclnt.c, cpputil/tls_parser.h,
gtests/ssl_gtest/libssl_internals.c,
gtests/ssl_gtest/libssl_internals.h, gtests/ssl_gtest/manifest.mn,
gtests/ssl_gtest/ssl_auth_unittest.cc,
gtests/ssl_gtest/ssl_custext_unittest.cc,
gtests/ssl_gtest/ssl_extension_unittest.cc,
gtests/ssl_gtest/ssl_gtest.gyp,
gtests/ssl_gtest/ssl_tls13compat_unittest.cc,
gtests/ssl_gtest/tls_agent.cc, gtests/ssl_gtest/tls_agent.h,
gtests/ssl_gtest/tls_connect.cc, gtests/ssl_gtest/tls_connect.h,
gtests/ssl_gtest/tls_ech_unittest.cc,
gtests/ssl_gtest/tls_esni_unittest.cc,
gtests/ssl_gtest/tls_filter.cc, gtests/ssl_gtest/tls_filter.h,
lib/ssl/SSLerrs.h, lib/ssl/manifest.mn, lib/ssl/ssl.gyp,
lib/ssl/ssl3con.c, lib/ssl/ssl3ext.c, lib/ssl/ssl3ext.h,
lib/ssl/ssl3exthandle.c, lib/ssl/ssl3exthandle.h,
lib/ssl/ssl3prot.h, lib/ssl/sslencode.c, lib/ssl/sslencode.h,
lib/ssl/sslerr.h, lib/ssl/sslexp.h, lib/ssl/sslimpl.h,
lib/ssl/sslinfo.c, lib/ssl/sslsecur.c, lib/ssl/sslsock.c,
lib/ssl/sslt.h, lib/ssl/tls13con.c, lib/ssl/tls13con.h,
lib/ssl/tls13ech.c, lib/ssl/tls13ech.h, lib/ssl/tls13esni.c,
lib/ssl/tls13esni.h, lib/ssl/tls13exthandle.c,
lib/ssl/tls13exthandle.h, lib/ssl/tls13hashstate.c,
lib/ssl/tls13hashstate.h:
Bug 1654332 - Update ESNI to draft-08 (ECH). r=mt
This patch adds support for Encrypted Client Hello (draft-ietf-tls-
esni-08), replacing the existing ESNI (draft -02) support.
There are five new experimental functions to enable this:
- SSL_EncodeEchConfig: Generates an encoded (not BASE64) ECHConfig
given a set of parameters.
- SSL_SetClientEchConfigs: Configures the provided ECHConfig to the
given socket. When configured, an ephemeral HPKE keypair will be
generated for the CH encryption.
- SSL_SetServerEchConfigs: Configures the provided ECHConfig and
keypair to the socket. The keypair specified will be used for HPKE
operations in order to decrypt encrypted Client Hellos as they are
received.
- SSL_GetEchRetryConfigs: If ECH is rejected by the server and
compatible retry_configs are provided, this API allows the
application to extract those retry_configs for use in a new
connection.
- SSL_EnableTls13GreaseEch: When enabled, non-ECH Client Hellos will
have a "GREASE ECH" (i.e. fake) extension appended. GREASE ECH is
disabled by default, as there are known compatibility issues that
will be addressed in a subsequent draft.
The following ESNI experimental functions are deprecated by this
update:
- SSL_EncodeESNIKeys
- SSL_EnableESNI
- SSL_SetESNIKeyPair
In order to be used, NSS must be compiled with
`NSS_ENABLE_DRAFT_HPKE` defined.
[a10493dcfcc9]
* lib/ssl/ssl3con.c, lib/ssl/sslencode.c, lib/ssl/sslencode.h,
lib/ssl/tls13con.c, lib/ssl/tls13con.h:
Bug 1654332 - Buffered ClientHello construction. r=mt
This patch refactors construction of Client Hello messages. Instead
of each component of the message being written separately into
`ss->sec.ci.sendBuf`, we now construct the message in its own
sslBuffer. Once complete, the entire message is added to the sendBuf
via `ssl3_AppendHandshake`.
`ssl3_SendServerHello` already uses this approach and it becomes
necessary for ECH, where we use the constructed ClientHello to
create an inner ClientHello.
[d40121ba59ba]
2020-11-13 J.C. Jones <jjones@mozilla.com>
* automation/abi-check/expected-report-libnss3.so.txt, automation/abi-
check/expected-report-libnssutil3.so.txt, automation/abi-check
/previous-nss-release, lib/nss/nss.h, lib/softoken/softkver.h,
lib/util/nssutil.h:
Set version numbers to 3.60 Beta
[5e7b37609f22]
Differential Revision: https://phabricator.services.mozilla.com/D97492
Sometimes thread names bit rot in the list because there is no checker in place to ensure they are still active. The threads I removed have either:
-Been converted to use the background thread pool
-Been removed entirely
-Been renamed
And they no longer require entries in the list.
Differential Revision: https://phabricator.services.mozilla.com/D96846
The SEH unwind info problem no longer happens because nowadays bug 1631929 prevents the code pattern that led to it. I've confirmed that bug 1626951's bustage doesn't come back, both in regular and beta-simulation builds.
This cleanup will allow me to reverse the dependency in bug 1677742 and have `lto` depend on `new_pass_manager_flags`.
Differential Revision: https://phabricator.services.mozilla.com/D97362
Currently, we report "10.0.0.or.more" for the latest release of Xcode,
but we know it's 10.0.0 for sure, which is when the clang version
reported by Xcode is exactly 12.0.0. We know that in the past a bump of
LLVM has always been accompanied with a bump of the minor version, so we
expect no LLVM version bump until at least 12.0.1.
Differential Revision: https://phabricator.services.mozilla.com/D97241
Currently, in LTO builds, we use the new pass manager during the initial translation to bitcode but not for the final optimization during linking.
On Linux, we can enable the new pass manager during LTO with a plugin option. I've landed a patch upstream to allow it on Windows as well, which is included here.
Switching the pass manager brings speed improvements on its own, but it also reduces code size by ~6%, which we can use a portion of as budget to increase the import limit (via the hot multiplier) for even more speed improvements.
Differential Revision: https://phabricator.services.mozilla.com/D96108
Most of the deletions here come from bug 1481612, the `--with-windows-wheel` option to `mach vendor python`, which according to that commit message "is very single-purpose: it's intended to let us vendor an unpacked
wheel for psutil on Windows". Since vendoring `psutil` is something we're no longer doing, we can safely just delete that added code.
Differential Revision: https://phabricator.services.mozilla.com/D90919
Install `psutil` when setting up the `mach` `virtualenv`s and stop importing the in-tree version in the build.
Nothing in-tree currently assumes or mandates the installation of `psutil` (all uses of `psutil` are guarded with imports of the form `try : import psutil; except ImportError: psutil = None`), so there's no back-incompatibility concerns here. There will be an awkward period where telemetry will be lacking CPU/disk data for everyone until they re-run `mach bootstrap` or `mach create-mach-environment`, but that will come back as people gradually update their `virtualenv`s.
An alternative to circumvent that issue is REQUIRING that `psutil` be installed by adding an assertion in `mach` that `psutil` can be found (allowing us to remove all the conditional logic in-tree around whether `psutil` is installed), but I wouldn't claim that we're ready to do that and deal with whatever fallout might occur.
Differential Revision: https://phabricator.services.mozilla.com/D90914
CLOSED TREE
Backed out changeset baa1f7a615e4 (bug 1666345)
Backed out changeset b6646baa866d (bug 1661624)
Backed out changeset e4d550db6037 (bug 1667152)
Most of the deletions here come from bug 1481612, the `--with-windows-wheel` option to `mach vendor python`, which according to that commit message "is very single-purpose: it's intended to let us vendor an unpacked
wheel for psutil on Windows". Since vendoring `psutil` is something we're no longer doing, we can safely just delete that added code.
Differential Revision: https://phabricator.services.mozilla.com/D90919
Install `psutil` when setting up the `mach` `virtualenv`s and stop importing the in-tree version in the build.
Nothing in-tree currently assumes or mandates the installation of `psutil` (all uses of `psutil` are guarded with imports of the form `try : import psutil; except ImportError: psutil = None`), so there's no back-incompatibility concerns here. There will be an awkward period where telemetry will be lacking CPU/disk data for everyone until they re-run `mach bootstrap` or `mach create-mach-environment`, but that will come back as people gradually update their `virtualenv`s.
An alternative to circumvent that issue is REQUIRING that `psutil` be installed by adding an assertion in `mach` that `psutil` can be found (allowing us to remove all the conditional logic in-tree around whether `psutil` is installed), but I wouldn't claim that we're ready to do that and deal with whatever fallout might occur.
Differential Revision: https://phabricator.services.mozilla.com/D90914
Most of the deletions here come from bug 1481612, the `--with-windows-wheel` option to `mach vendor python`, which according to that commit message "is very single-purpose: it's intended to let us vendor an unpacked
wheel for psutil on Windows". Since vendoring `psutil` is something we're no longer doing, we can safely just delete that added code.
Differential Revision: https://phabricator.services.mozilla.com/D90919
Install `psutil` when setting up the `mach` `virtualenv`s and stop importing the in-tree version in the build.
Nothing in-tree currently assumes or mandates the installation of `psutil` (all uses of `psutil` are guarded with imports of the form `try : import psutil; except ImportError: psutil = None`), so there's no back-incompatibility concerns here. There will be an awkward period where telemetry will be lacking CPU/disk data for everyone until they re-run `mach bootstrap` or `mach create-mach-environment`, but that will come back as people gradually update their `virtualenv`s.
An alternative to circumvent that issue is REQUIRING that `psutil` be installed by adding an assertion in `mach` that `psutil` can be found (allowing us to remove all the conditional logic in-tree around whether `psutil` is installed), but I wouldn't claim that we're ready to do that and deal with whatever fallout might occur.
Differential Revision: https://phabricator.services.mozilla.com/D90914
Some of the surrounding lines recently changed so the patch refuses to apply. We can live with less context so I'd rather trim it than make a new clang-12 version of this patch.
Differential Revision: https://phabricator.services.mozilla.com/D96725
David Major
Mike Hommey
Kevin Jacobs
Kris Wright
Andi-Bogdan Postelnicu
Noemi Erli
Ricky Stewart
Razvan Maries
Mihai Alexandru Michis
Mitchell Hentges
smolnar
Sylvestre Ledru
Butkovits Atila
Emilio Cobos Álvarez