Changes to the build pipelines for the AzDO extension:
* Added CredScan and Semmle code security analysis tasks.
* Factored out the common steps between the PR and release build pipelines.
* Moved to use a custom agent pool.
Updating dependencies per npm audit.
Also updated MSBuildHelpers copied from azure-pipelines-task-lib to latest, which includes detection for MSBuild 17, and fixed project URL on package.json files.
1. opensslconf.h.cmake - moved to sub-directory openssl
2. bn_conf.h.cmake & dso_conf.h.cmake - moved to sub-directory openssl/crypto to align with openssl include directory structure from which these cmake are generated.
When we examine defination of async_posix.h it appears to exclude android platform from its definitaion by defining !defined(__ANDROID__) at the begining.
As a result async_posix.c definition will not be applicable for AOSP platform in that case it use async_null.c as default definition. So added conditional statement such that for AOSP it should consider async_null.c , for windows async_win.c and linux and ios async_posix.c source code respectively
ios issue - CMakeFiles/crypto.dir/openssl/crypto/dso/dso_dl.c.o has no symbols
Adjusted async code order such that arch/async_posix.c compiled first and then
async.c as some of the code definitation such as async_local_cleanup exists as part of platform specific code and is referred in async.c as a possible fix to error: undefined reference to 'async_local_cleanup on adroid specific builds.
as MSIX SDK code doesn't set explicitly any DSO macro and hence default DSO_METHOD was used as per old dso_openssl.c source code.
With the latest OpenSSL 1.1.1j - there is no default DSO as such inorder to use DSO_METHOD_null we have to set DSO_NONE macro.
- So, Crypto.cmake has been updated to set DSO_NONE to retain old behavior
- Removed dso_dl.c from compilation list as we DSO_NONE as default this is not going to consumed. Compiling this file without defining DSO_DL macro resulting failures on iOS x86 platform. It appears iOS x86 platform is too sensitive compiling a *.c file which endsup not having any content (due to entire content is wrapped on the existing of DSO_DL definition in the recent OpenSSL version)
* makemsix: Remove wrong help output
The help output for the bundle command referred to options the p, ep,
kt and kf options, that do not actually exist.
* makemsix: Minor fixes in bundle option descriptions
Some spaces were missing and / was used for some option names.
* makemsix: Split very long lines
The commit includes:
1. Updates to crypto_sources.cmake
a) to include all the neccessary sources/header file from the new OpenSSL 1.1.1j version
b) Removed sources files from crypto_sources.cmake that don't exist/deleted with the latest OpenSSL code
2. Updated OpenSSL version to 1.1.1 specificied in CMakeList.txt
3. Updates to SignatureValidator.cpp
a) added a local macro definition for CRYPTO_NUM_LOCKS - as this macro support has been taken out from OpenSSL but MSIX SDK code consumes it with std::lock constructs. It appears We should be fine using this constant, as code doesn't depend on OpenSSL Locking constructs which are out support.
b) Comment M_ASN1_OCTET_STRING_print code as this function has been removed in the latest OpenSSL version. Will have to discuss with experts for the alternatives.
c) Also update STACK_OF(X509_EXTENSION) *exts = cert->cert_info.extensions; "cert_info.extensions" instead "cert_info->extensions" as it appears this type has been updated else it would cause compile error
c) CRYPTO_THREADID_get_callback() == NULL) - comparison has been updated to use NULL else it would cause compile time error
[How Validated:]
- Compiled MSIX SDK code with the helps of
./makelinux.sh
- executed .vs/msixtest/msixtest and ensure all the tests passes
[Test Result:]
All tests passed (6268 assertions in 72 test cases)
The changes includes addition of:
1. bn_conf.h.cmake - generated with the help of running Makefile on "\msix-packaging\lib\openssl\include\crypto\bn_conf.h.in"
2. dso_conf.h.cmake - generated with the help of running Makefile on "\msix-packaging\lib\openssl\include\crypto\dso_conf.h.in"
3. opensslconf.h.cmake - generated with the help of running Makefile on "\msix-packaging\lib\openssl\include\openssl\opensslconf.h.in"
Above cmake file has neccessary cmake macro expansion flag which get resolved inside crypto.cmake when configure_file called on each of these
files
Additionally, crypto.cmake has some additional changes
- Copying the all the includes as per new OpenSSL source includes layout
- Additional flag settings to resolve certain compiler time definitions
* Invalid Memory Access in SignatureValidator
Problem:
* SignatureValidator::IsStoreOrigin() tries to read X.509 extensions
to determine if the origin of the signature matches the Windows Store OID.
* Extension data is converted from a raw buffer to an std::string for
comparision.
* The raw buffer is not null-terminated, and therefore, running
std::strlen() on it causes invalid memory access.
* This invalid access is caught by ASAN on macOS.
Solution:
* Null-terminate the raw buffer before trying to build an std::string
from it.
Tests:
* Ran app test suite that uses libmsix.dylib with ASAN on. No crashes
were reported.
* Invalid Memory Access in SignatureValidator
Problem:
* As @JohnMcPMS pointed out, writing "" with 1 byte size is null termination.
* A better solution would be to use bptr->length and avoid writing the null
byte altogether.
Tests:
* Ran app test suite that uses libmsix.dylib with ASAN on. No crashes
were reported.
Co-authored-by: Sayan Chaliha <sachalih@microsoft.com>
Adds new property on AppxBundleMetadata: ChildAppxPackages. This replaces the now [Obsolete] InternalAppxPackagesRelativePaths, as it has the file paths but also the metadata unique to each child package.
* Added option to AppxFactory to create package writer with file hash enabled in block map xml
* Changed PCSTR to LPCSTR as PCSTR is not defined in Linux and MacOS. Also updated test data about expected block map xml size
* Added CoCreateAppxFactoryWithOptions and CoCreateAppxFactoryWithHeapAndOptions to export list
* Test pull request
* Changed block map xml writer to add <FileHash> elements to BlockMap XML
* Update tests to reflect the change of block map xml and skip schema block map validation for Xerces parser
* Fix delete void pointer issue that some compilers do not allow
* Fixed a typo
Chacón
dependabot[bot]
Ruben Guerrero
Joakim
Madhusudhan-MSFT
ePirat
stephenk-msft
Sayan Chaliha
Union Palenshus
rhuang-msft