Use TwoLevelCache to implement a bxl-side cache factory that allows to create a local cache + plain remote blob one. Existing knobs for both caches are preserved.
Previous change addressed the crash when loading cache after bxl cancelled. Crash still happen if load the cache after scheduler been cancelled.
To avoid crash, add access to scheduler cancellation token in PipTwoPhaseCache and check whether cancellation been requested before load cache.
Related work items: #2199303
Windows Terminal and other terminals support clickable hyperlinks. This change makes the link to the BuildXL log directory a clickable hyperlink to be opened by the operating system's file explorer of choice. This works on Windows and Linux.
I disabled it when run through an SSH session since the link would be a path on the host and the files may not exist on the SSH client. If run in a VSCode SSH session, VSCode has parsing for paths and does the right thing anyway.
This does change the behavior if you're running through a VSCode wrapped terminal locally. Previously, VSCode would attempt to show it through its file browser. Now it will use the file explorer of the OS instead. I didn't think this was a big deal and wasn't worthy of a config option.
Allow deduplicating content from ephemeral local CAS with CASaaS local content by always putting content into CASaaS first and then transferring hardlink to ephemeral local cas.
Flushing changes to the filesystem are intended for just-produced outputs. Do not attempt to do that for the source of a copy file pip when tracking a potential symlink chain. That may conflict with an attempt to hash a source file and throw a sharing violation.
Related work items: #2203343
When we the schedule hits an internal error, we aggressively terminate the build (DX0249). As a result, service pips (and their shutdown pips) will / might not exit cleanly. If they do, do not log `ScheduleServicePipFailed` / `ScheduleServicePipShuttingDownFailed` errors.
Related work items: #2199314
Allow users to specify reclassifications on operations resolved by the observed input processor. Users can configure rules in their specification that individual pips can opt into, where some observation types are reclassified after being processed by the observed input processor.
The rule matching happens both after execution and when processing pathsets for the pip on cache lookup. Future optimizations might avoid some of the processings on cache lookup.
Note that any changes in these rules results in a weak fingerprint change for the pips, as there is no way to know if observations for the pip have to be reclassified with the new rules.
To learn more about the motivations for this change, refer to [this document](https://microsoft.sharepoint.com/:w:/t/1ES2/EXOegdiLQkRNt5bE7nhFic8BJBYKcETrqcY-gI2_jTN9zw?e=UEIxAg)
Related work items: #2185890
* Fix cache generation condition logic so it is triggered by the right data
* Establish a precedence order between different cache generation options
* Get rid of IAdoBuildRunnerService (there are no intentions to have other implementations for the service)
Modified cache config generator to ensure that the cache config resource name and cache config resource file are added used for cache config generation, for 1ES hosted pools.
Previous PR has missed some changes which is causing the unit test to fail.
Related work items: #2195544
This arg controls whether buildxl warns about spotlight (macOS's file indexer) potentially acting on build directories. Removing the arg and functionality for simplification.
There are a bunch of other discrepancies between BuildXL's command line and the names given to consts in strings.resx, which is what is used to generate Flags.md.
The script that generates Flags.md works directly off of strings.resx. I doubt a pass was ever down when that script was created to ensure strings.resx actually matched up with the command line. This change would be better if it included some guardrails to prevent the two from skewing again in the future, but I don't think the cost/benefit is there so I'm just doing a one-time cleanup pass.
- Prevents System.Text.Json from serializing MetadataContainer/ContentContainer/etc
- Allows Newtonsoft.Json deserialization into the same type
- Tests not added on purpose, we don't use Newtonsoft. But other repos do (ex CB).
Reapplies 790969 + takes care of the Windows case when process ids are reused
On Windows we never get ProcessExit events. At the same time, there is no process start duplication.
Serge Mera
Qi Wang
BuildXL-Pipelines
Michael Pysson
Lance Collins
Oleksii Kononenko
Marcelo Lynch 🧉
Pasindu Gunasekara 🍣
Julian Bayardo
Sahiti Chandramouli
Camille Bruckmann