Separate guardian from Roslyn analyzers. We now use the roslyn analyzers from Microsoft in all builds as a part of the build. Guardian will be enabled when TOOLPATH_GUARDIAN variable exists.
Related work items: #2031143
Revert "Merged PR 719250: Use Microsoft.Artifacts.Authentication for cache/drop authentication"
This reverts commit dc8fb8ff65.
Reverting change that's breaking dev cache authentication for users without VPN.
- Microsoft.Artifacts.Authentication provides a wrapper layer for MSAL authentication. This will allow us to continue to keep BXL up to date with the latest MSAL changes without having to change too much on our end other than bumping up package versions.
Related work items: #1969711, #2063548
Turn on the (temporary) flag that enables the correct treatment of nuget dependencies when monikers are used to specify them.
This caused a bunch of issues that are fixed here as well, caused by packages now containing more dependencies than what they used to:
* Some dependencies were plainly missing, so they are included now
* Some packages were causing double-deployment errors, so new 'skipNuget..' clauses had to be added
* On the plus side, some specs got simplified and we don't need to pin particular qualifiers.
The plan is to make this temporary flag the default after this gets merged
This change adds a /logToKusto option to send all log lines to a kusto database (in the same way today this happens for DominoMessage on CB). The primary scenario for enabling this option is ADO builds.
In order to authenticate, the pipeline running the build needs to provide a managed identity that we need to authorize. 1eshp allows for associating a managed identitie to a pipeline, so that's pretty straightforward.
Docs (on EngHub, since the flow today is mostly tied to 1eshp) to follow.
*Note*: this PR needs a fix in the nuget generation logic that is already in, but it needs to become part of the LKG, so we'll get failures till that happens
Related work items: #2047667
Our public rolling validation tries to scrub this directory but fails with a warning because it is not a scrubbable mount. The result is we include unwanted packages in the produced artifact
This is the second PR that updates the dependencies in order to gert rid the dependencies to the stale keyvault nuget packages.
Related work items: #2011942
This is the first step towards fixing the Component Governance issue automatically created because of our dependency to a deprecated Azure.KeyVault package.
Related work items: #2011942
Migrate to the latest C# compiler that supports C# 11
The PR adds 'RequiredMemberAttribute' to support 'required' members from C# 11.
The latest compiler changed the behavior for '<<' operator making it checked. This PR addresses this by using 'unchecked' in the cases where the overflow is possible.
Related work items: #2017950
This PR adds .NET 7 RC1 support to BuildXL and Cache Service.
Please note, that the PR will be failing until the new LKG with net7 nuget support will be deployed.
Related work items: #1990373
This PR updates RocksDb packages to 20221219.1. The new version uses different logic for tracking the lifetime of logging adapter that prevents crashes due to inconsistent lifetime.
The original allowed having different lifetime for `RocksDbStore` (the actual wrapper around native rocksdb store) and `RocksDbLoggingAdapter` that was reponsible for marshaling lagging invocations between the native and the managed worlds.
Due to a race condition (that will be fixed separately) it was possible to leave `RocksDbStore` instance undisposed. That was cuasing runtime crashes because an undisposed rocksdb wrapper was trying to use finalized `RocksDbLoggingAdapter`.
In the design, the store tracks the lifetime of the logging adpater and the adapter itself is now hidden from the public API making the lifetime issues impossible.
This tool has been frequently failing during PR builds with:
```
##[error]DX0064 [Pip5DE6E748056C4CEF, BuildXL.Deployment - NugetPackages.winX64 - NuGet.exe [{configuration:"debug"}], d:\dbs\el\bxlint\Public\Src\Deployment\NugetPackages.dsc] - failed with exit code 57005,
##[error]Unhandled Exception: System.Threading.AbandonedMutexException: The wait completed due to an abandoned mutex.
##[error] at System.Threading.WaitHandle.ThrowAbandonedMutexException()
##[error] at System.Threading.WaitHandle.InternalWaitOne(SafeHandle waitableSafeHandle, Int64 millisecondsTimeout, Boolean hasThreadAffinity, Boolean exitContext)
##[error] at System.Threading.WaitHandle.WaitOne(TimeSpan timeout, Boolean exitContext)
##[error] at NuGet.Common.Migrations.MigrationRunner.Run() in D:\a\_work\1\s\src\NuGet.Core\NuGet.Common\Migrations\MigrationRunner.cs:line 33
##[error] at NuGet.CommandLine.Program.Main(String[] args) in D:\a\_work\1\s\src\NuGet.Clients\NuGet.CommandLine\Program.cs:line 65
```
This is an attempt to fix it.
This is Phase I hence only a warning is logged when a credential is detected in an env var. Depending on the results obtained from the logging information the implementation is modified accordingly.
Added the CredentialScanner class to handle the functionality related to credscan
Created a unit test to test the functionality of the scanner with various test cases
Modified the SetEnvironmentVariables method to call the credscan method from CredentialScanner class.
Added allowList mechanism and a unit test to test that
Related work items: #1975564
Revert "Merged PR 685164: Forward rocksdb logs to Kusto
Forward rocksdb logs to Kusto
Related work items: #1996361"
Reverted commit `60bffd1f`.
Related work items: #1996361
On newer kernel, copyfilerange no longer works cross-device. This is a known regression in the kernel.
The current BuildXL package has the Linux sandbox that doesn't contain this copyfilerange patch. AnyBuild requires this patch as it now has an updated Linux image for its agents. As a workaround, AnyBuild currently relies on two packages from BuildXL, i.e., buildxl package itself and runtime.ubuntu-linux-x64.buildxl. The latter is produced in an adhoc way.
With this PR, we can now rely solely on buildxl package.
This PR updates RuntimeContracts to 0.4.0 with the following important changes:
* All the assertions propagate caller expressions as string to make assertions more descriptive. For instance, `Contract.Requires(x != null)` will automatically embed `x != null` into a message.
* All the assertions support new interpolated string from C# 10 and allow lazy message construction. It means that now we don't need to use 'FluentAPI' and just create custom message with interpolated string syntax.
The following code is very efficient: `Contract.Assert(x > 0, $"x = {x}")`. The efficiency comes from the fact that the message is constructed only if the assertion is false and the exception will be thrown.
This PR also fixes cases when the messages were created eagerly causing excessive memory allocations that are now gone.
Related work items: #2007282
The custom loader is brittle that it hardcodes the assemblies to load. Moreover, the existing approach does not work with dotnetcore runtime due to some complicated way of MsBuild SDK resolution etc.
MsBuild team recommends that we should use Microsoft.Build.Locator for this purpose.
Because the locator can pin the MsBuild location if explicitly specified, all dotnetcore/full-framework unit tests work without any modification..
Related work items: #2006579
This change makes bxl consume an Ubuntu-built sandbox as a stop gap for the timeouts we are experimenting. The sandbox is the result of building the current prod bits (0.1.0-20221020.0.2) but changing the pool we use to an Ubuntu one.
Due to (possibly) kernel bug, copy_file_range no longer works when the file descriptors are not mounted on the same filesystems, despite what is said in the manual https://man7.org/linux/man-pages/man2/copy_file_range.2.html.
This bug breaks AnyBuild virtual filesystem (VFS) because the source file will be in the read-only (lower) layer of overlayfs, and this layer is mounted on AnyBuild FUSE, and the target file will be in the writable (upper) layer of overlayfs.
Without this PR AnyBuild cannot update its Linux image.
Related work items: #2000792, #2000797
Pasindu Gunasekara 🍣
Semih Okur
Serge Mera
Ignacio Alonso Battaglia
Sergey Tepliakov
Michael Pysson
Marcelo Lynch 🧉
Qi Wang
Julian Bayardo
Erik Mavrinac
Sahiti Chandramouli
Iman Narasamdya
