- Tests macos binaries on both amd64 and arm64.
- Updates rocksdb packages to pick up new changes to add arm64 support.
- Disables unit tests that rely on grpc core (these can be re-enabled when we switch to grpc-dotnet)
Related work items: #2195823
CaSaaS is currently on Net7 and it might need updated bxl bits while it's moving to Net8. This PR re-adds net7 qualifier. It's essentially a revert of `0a4d5b272900cfb6ad2c13795650e269500f9d17` and `5380d614bb98bfebb5a63900bb617ff51e93e56e`.
7.0.18 has a vulnerability, so this PR also includes a change that updates net7 bits to 7.0.19 to make the CG happy.
note: MsBuild is back to net7 (net7->net8 for msbuild was a part of net7 removal)
Reapply the original Net7 removal commit `4c5cc753`. The issue that we initially saw after it was merged was not caused by the commit. The commit forced a cache miss for a couple of pip and it uncovered the issue with vctip/lib.exe.
Must be merged after !784653
Related work items: #2174274
After net7 removal the minimal build fails in CB with 'child process survived' error.
Revert "Merged PR 784187: Remove net7 qualifier
Remove net7 qualifier and move msbuild to net8.
Related work items: #2174274"
Reverted commit `4c5cc753`.
Related work items: #2174274
- Removes usages of the `linux-x64` package
- Some changes in the deployment directory structure
- Also fixes a problem where we crash if the graph construction tool fails instead of being graceful
Related work items: #2000411
This caused some deployment issues in CB, and there is a hot fix that CB needs that got trapped in between cleanup waves. Reverting so CB can get unblocked.
* Removed now unnecessary 'dependentPackageIdsToSkip'
* Removed remaining unnecessary 'withQualifier' and the general netstandard2.0 forcing that happened on cache specs side
* Reorganized asptnetcore assembly references. This could be simplified further in the future, but now it is more self-contained.
* Some other minor cleanups
Next step is trying to make sure dependency versions are the right ones and try to turn on the flag that enforces this from the nuget resolver.
Revert "Merged PR 765049: DScript spec cleanup wave 2
* Removed now unnecessary 'dependentPackageIdsToSkip'
* Removed remaining unnecessary 'withQualifier' and the general netstandard2.0 forcing that happened on cache specs side
* Reorganized asptnetcore assembly references. This could be simplified further in the future, but now it is more self-contained.
* Some other minor cleanups
Next step is trying to make sure dependency versions are the right ones and try to turn on the flag that enforces this from the nuget resolver."
Reverted commit `2b01690d`.
* Removed now unnecessary 'dependentPackageIdsToSkip'
* Removed remaining unnecessary 'withQualifier' and the general netstandard2.0 forcing that happened on cache specs side
* Reorganized asptnetcore assembly references. This could be simplified further in the future, but now it is more self-contained.
* Some other minor cleanups
Next step is trying to make sure dependency versions are the right ones and try to turn on the flag that enforces this from the nuget resolver.
Mostly cache specs, but also touching some shared stuff:
* Unified some Nuget packages (System.Collections.Immutable, System.Memory, etc), and bumped versions to the latest. This triggered a bunch of other updates...
* Removed a lot of unnecessary 'withQualifier'
* Simplified some references by fixing the 'runtimes' folder behavior on the Nuget resolver (which forced a bunch of 'withWinRuntime' calls
More cleanup passes are possible, but those to come later
This PR forces:
- All gRPC clients in .NET 4.7.2 to use gRPC.Core (which is the only supported option)
- All gRPC clients in .NET >= 5 to use gRPC.NET
This is needed for:
- Simplify handling of HTTP clients (which we'll use to stop creating one HttpClient per request)
- Us to support encryption in the future.
The PR also fixes Microsoft.Extensions.Logging.Abstractions, which is a contentious nuget pkg for a variety of reasons and gets in the way of doing this. The change can't be submitted separately because it's hard to repro that way
I've recently updated ErrorProne.NET and added a few rules there.
This PR does the following:
1. Updates ErrorProne.NET to 0.6.1
2. Updates the compiler toolchain to the latest version
3. Fixes the following warnings:
## Structs with default equality and hashcode are used in hashtables
This is problematic since the default methods causes boxing and the hash function is very weak. For instance, hash code for `KeyValuePair<int, int>(1, 1)` and (2,2) are the same.
```
Public\Src\FrontEnd\Core\Incrementality\FrontEndPublicFacadeAndAstProvider.cs(316,20): warning EPC25: The default 'ValueType.Equals' is used in Content.Equals(other.Content)
Public\Src\Utilities\PackedTable\StringTable.cs(128,30): warning EPC24: A struct 'BuildXL.Utilities.PackedTable.CharSpan' with a default Equals and GetHashCode implementation is used as a key in a hash table
Public\Src\Engine\Scheduler\Caching\HistoricMetadataCache.cs(1048,17): warning EPC24: A struct 'BuildXL.Scheduler.Cache.HistoricMetadataCache.Expirable<BuildXL.Scheduler.Cache.HistoricMetadataCache.PublishedEntry>' with a default Equals and GetHashCode implementation is used as a key in a hash table
Public\Src\Tools\Execution.Analyzer\Analyzers.Partner\GraphDiffAnalyzer.cs(391,26): warning EPC24: A struct 'System.Collections.Generic.KeyValuePair<string, (string, bool)>' with a default Equals and GetHashCode implementation is used as a key in a hash table
Public\Src\Tools\Execution.Analyzer\Analyzers.Partner\GraphDiffAnalyzer.cs(94,13): warning EPC24: A struct 'System.Collections.Generic.KeyValuePair<BuildXL.Pips.PipId, BuildXL.Pips.PipId>' with a default Equals and GetHashCode implementation is used as a key in a hash table
Public\Src\Tools\SymbolDaemon\SymbolDaemon.cs(559,13): warning EPC24: A struct 'BuildXL.Ipc.ExternalApi.SealedDirectoryFile' with a default Equals and GetHashCode implementation is used as a key in a hash table
```
## Linear Enumerable.Contains instead of hashset.contains
```
Public\Src\Utilities\UnitTests\TestUtilities.XUnit\XAssert.cs(272,69): warning EPC23: Linear search via Enumerable.Contains is used instead of an instance Contains method
Public\Src\Engine\UnitTests\Processes\SandboxedProcessTest.cs(1601,17): warning EPC23: Linear search via Enumerable.Contains is used instead of an instance Contains method
Public\Src\Engine\Processes\ProcessDumper.cs(230,21): warning EPC23: Linear search via Enumerable.Contains is used instead of an instance Contains method
```
And a bunch of cases when we were using default `ToString` impl which is not helpful.
This PR reverts the revert and adds a fix to prevent the issue from happening.
Revert "Merged PR 757989: Remove Azure Storage SDK V9"
This reverts commit 0e95536756.
Linux build in rolling build pipeline started to fail after this change.
Reverts !758549
Revert "Merged PR 757989: Remove Azure Storage SDK V9"
This reverts commit 0e95536756.
Linux build in rolling build pipeline started to fail after this change.
We're now getting Component Governance alerts about our usage of this NuGet package. This PR removes it from everywhere that remains.
This PR is risky because it changes the API used by CASaaS across several subsystems.
BuildXL looks up cache miss data based on a priority list of keys configured by the user. In practice, the best general setup is to use a chain of git parent commits. This gets replicated in logic that invokes bxl.exe today for some of our customers. This PR provides this as a default experience without the end user needing to manually perform it.
The approach is heuristic, trying to pick candidate keys that might have been used to publish a fingerprint store for a build as "close" as the one running.
For this, the candidates are picked by
1. Get latest 5 commits from the current HEAD. We should get a match if this branch was built before, for example for a PR we expect one build per 'iteration'. Because an iteration can have multiple commits, we should not use a very small number (like, say, this commit and the one before), but let's not go overboard either (hence 5).
2. For any additional additional branches (indicated by the user in the argument), the assumption is that they run build against every commit (this should typically be the `main` branch, or the target branch for a PR).
a) We retrieve the merge-base from HEAD and that branch. We use as keys the hashes for the 3 commits immediately previous to the merge-base.
b) Finally, as a last resort, we use the latest 3 hashes from that branch as a last resort, assuming again that it was built recently
An example: for a PR (that has branched from `h5` and committed a number of times) merging against main, with this topology:
```
.--b5--b4--b3--b2--b1-- b0 [PR branch]
/
h8 -- h7 --h6--- h5 -- h4 -- h3 -- h2 -- h1 -- h0 [main]
```
The hashes for the keys would be (as described above)
1)`b0, b1, b2, b3, b4, b5`
2a) `h5, h6, h7`
2b) `h0, h1, h2`
Duplicates are removed, so something like this:
```
.--b1-- b0 [PR branch]
/
h8 -- h7 --h6--- h5 -- h1 -- h0 [main]
```
would result in the cadidates being `b0, b1, h5, h6, h7, h0, h1`.
Related work items: #2116194
Oleksii Kononenko
Camille Bruckmann
Pasindu Gunasekara 🍣
Michael Pysson
Julian Bayardo
Marcelo Lynch 🧉
Iman Narasamdya
Sahiti Chandramouli
Erik Mavrinac
Juan Carlos Guzman Islas
Serge Mera
Sergey Tepliakov
Qi Wang
John Erickson