Flushing changes to the filesystem are intended for just-produced outputs. Do not attempt to do that for the source of a copy file pip when tracking a potential symlink chain. That may conflict with an attempt to hash a source file and throw a sharing violation.
Related work items: #2203343
When we the schedule hits an internal error, we aggressively terminate the build (DX0249). As a result, service pips (and their shutdown pips) will / might not exit cleanly. If they do, do not log `ScheduleServicePipFailed` / `ScheduleServicePipShuttingDownFailed` errors.
Related work items: #2199314
Allow users to specify reclassifications on operations resolved by the observed input processor. Users can configure rules in their specification that individual pips can opt into, where some observation types are reclassified after being processed by the observed input processor.
The rule matching happens both after execution and when processing pathsets for the pip on cache lookup. Future optimizations might avoid some of the processings on cache lookup.
Note that any changes in these rules results in a weak fingerprint change for the pips, as there is no way to know if observations for the pip have to be reclassified with the new rules.
To learn more about the motivations for this change, refer to [this document](https://microsoft.sharepoint.com/:w:/t/1ES2/EXOegdiLQkRNt5bE7nhFic8BJBYKcETrqcY-gI2_jTN9zw?e=UEIxAg)
Related work items: #2185890
* Fix cache generation condition logic so it is triggered by the right data
* Establish a precedence order between different cache generation options
* Get rid of IAdoBuildRunnerService (there are no intentions to have other implementations for the service)
Modified cache config generator to ensure that the cache config resource name and cache config resource file are added used for cache config generation, for 1ES hosted pools.
Previous PR has missed some changes which is causing the unit test to fail.
Related work items: #2195544
This arg controls whether buildxl warns about spotlight (macOS's file indexer) potentially acting on build directories. Removing the arg and functionality for simplification.
There are a bunch of other discrepancies between BuildXL's command line and the names given to consts in strings.resx, which is what is used to generate Flags.md.
The script that generates Flags.md works directly off of strings.resx. I doubt a pass was ever down when that script was created to ensure strings.resx actually matched up with the command line. This change would be better if it included some guardrails to prevent the two from skewing again in the future, but I don't think the cost/benefit is there so I'm just doing a one-time cleanup pass.
- Prevents System.Text.Json from serializing MetadataContainer/ContentContainer/etc
- Allows Newtonsoft.Json deserialization into the same type
- Tests not added on purpose, we don't use Newtonsoft. But other repos do (ex CB).
Reapplies 790969 + takes care of the Windows case when process ids are reused
On Windows we never get ProcessExit events. At the same time, there is no process start duplication.
- Tests macos binaries on both amd64 and arm64.
- Updates rocksdb packages to pick up new changes to add arm64 support.
- Disables unit tests that rely on grpc core (these can be re-enabled when we switch to grpc-dotnet)
Related work items: #2195823
- Advertise that the build is finished from the orchestrator build runner, so if a worker is late then it can skip the BuildXL invocation altogether
- Various improvements to the test framework to be able to simulate multiple machines working together, in particular this PR adds a "harness" abstraction to encapsulate all the mocked resources for an agent, and the ability to test end-to-end execution with a mock "BuildXL launcher"
- Simplify some of the abstractions, in particular `BuildContext` was being thrown around a lot and we can avoid it - this makes its creation more consistent too, as it comes only from a single place (the BuildRunnerService)
Related work items: #2194244
Serge Mera
Oleksii Kononenko
Marcelo Lynch 🧉
Pasindu Gunasekara 🍣
Julian Bayardo
Sahiti Chandramouli
BuildXL-Pipelines
Camille Bruckmann
Michael Pysson
Qi Wang