Add a LinuxPipDebugAnalyzer.
This Analyzer generates the required configuration for vscode to launch debugger. It takes the pip semi-stable hash as input, generates the configuration and writes it into a json file. Copy the configuration to vscode launch.json then start debugging
Related work items: #2207693
CancellationToken.CreateFailure() will assert the cancellation is requested. However, there are two cancellation tokens: Context.CancellationToken and Scheduler CancellationToken in HistoricMetaDataCache. Create the failure from wrong cancellation token throws the exception and crash the build.
In this pr, using the CancellationFailure constructor directly to avoid ContractException. It doesn't matter which cancellation is requested, the build will be cancelled.
Related work items: #2212685
Something in Office's dev build process tree may be intercepting and rendering the escape characters used by terminal hyperlink support. Reverting this until it can be debugged further to avoid ugly characters from showing up on the console
Related work items: #2211824
The remote cache cutoff feature stops querying the L3 after n number of consecutive misses (default today is n = 2).
If a pip is set to disabled cache, don't increment the UpstreamCacheMissLogestChain for its dependent pips to avoid result in a miss for the whole chain
Related work items: #2038931
Update engines.vscode to match @types/vscode. My suspicion is that newer versions of node are actually enforcing this (the error started to manifest on the new YAML release pipeline).
Original PR: !802728
After adding a number of logging statements to narrow down the issue, it had been determined that this could be a race condition with the garbageCollector in the unit test.
In the garbageCollector the concurrent map called m_exisitingContentEntries is populated accordingly which is required for serializing/ deserializing the cache entries.
The unit test iterates three times each times trying to load/reload the cache. Since the unit test is quick the garbageCollector is forced to abruptly terminate its operation before it can populate the map accordingly.
Once we hit closeAsync method. The CancellationToken is passed to garbageCollector forcing it to shut down its operations leaving the cache in an inconsistent state, but by awaiting the task we ensure that this does not happen.
By skipping the passing of CancellaitionToken to the gc we can avoid this race condition.
[Logs with Success and Failure.txt](https://dev.azure.com/mseng/9ed2c125-1cd5-4a17-886b-9d267f3a5fab/_apis/git/repositories/50d331c7-ea65-45eb-833f-0303c6c2387e/pullRequests/803195/attachments/Logs%20with%20Success%20and%20Failure.txt)
If we observe the logs we see that in the third iteration where we try to load the cache we do not see the garbage collector run at all(Can Collect print statement is missing in the failure run).
Related work items: #2191478
Use case-sensitive matching repo alias in pipeline definitions.
This is a workaround for bug #2210200
Without this, our pipeline run is neither tagged as 'Official' or 'Unofficial'; the tag is simply empty. This will fail our release pipeline.
Related work items: #2210200
The RunCheckinTests part of rolling build validation simulates some worker disconnect scenarions which generate warnings and an error. This change removed those expected errors from showing up and cluttering the Azure DevOps console.
If someone explicitly passes /fancyconsole in an AzureDevOps build, hyperlinks would be enabled. That's a weird configuration but some of the BuildXL repo automation was doing it. This additional check prevents that case.
Let users specify the total build timeout using `/buildTimeoutMins`. Before this PR the only way was via an engine env var (the CB runner uses this).
Deprecate /BuildWaitTimeout CLI. This flag (unrelated to the functionality being added here) is not being used and it has a confusing name given the new flag being added.
This pr is trying to address
1. Event dropping during the worker report execution log to orchestrator. The possible reasons are:
The execution log buffer is filled by another thread while we report it.
NotifyStream is deactivated before BinaryLogger completes all write events
2. CompareEvent trys to compare RecordFileForBuildManifest events
Changes in this pr:
1. Copy current content from stream and report to orchestrartor. Same as FlushManifestEvents
2. Track RecordFileForBuildManifest event in EventStatsExecutionLogTarget on worker.
3. Calculate total sent execution log size when reportExecutionLog succeed
4. Remove deactivate NotifyStream from Deactivate Execution Log. The stream will be disposed once the log target is disposed. Only deactivate it when we want to early cancel the sending action
5. Log Pending events into stats for manifest events and execution log reporting
Related work items: #2160034
Use TwoLevelCache to implement a bxl-side cache factory that allows to create a local cache + plain remote blob one. Existing knobs for both caches are preserved.
Previous change addressed the crash when loading cache after bxl cancelled. Crash still happen if load the cache after scheduler been cancelled.
To avoid crash, add access to scheduler cancellation token in PipTwoPhaseCache and check whether cancellation been requested before load cache.
Related work items: #2199303
Windows Terminal and other terminals support clickable hyperlinks. This change makes the link to the BuildXL log directory a clickable hyperlink to be opened by the operating system's file explorer of choice. This works on Windows and Linux.
I disabled it when run through an SSH session since the link would be a path on the host and the files may not exist on the SSH client. If run in a VSCode SSH session, VSCode has parsing for paths and does the right thing anyway.
This does change the behavior if you're running through a VSCode wrapped terminal locally. Previously, VSCode would attempt to show it through its file browser. Now it will use the file explorer of the OS instead. I didn't think this was a big deal and wasn't worthy of a config option.
Allow deduplicating content from ephemeral local CAS with CASaaS local content by always putting content into CASaaS first and then transferring hardlink to ephemeral local cas.
Qi Wang
Sahiti Chandramouli
Oleksii Kononenko
BuildXL-Pipelines
Serge Mera
Pasindu Gunasekara 🍣
Michael Pysson
Iman Narasamdya
Lance Collins
