Kyra encountered an issue where:
Drop downloads using a local cache. The local cache has a file that:
- Has the correct hash as file name
- Has the correct size as file size
- But is actually empty
This smells like this code caused the issue.
Reapply the original Net7 removal commit `4c5cc753`. The issue that we initially saw after it was merged was not caused by the commit. The commit forced a cache miss for a couple of pip and it uncovered the issue with vctip/lib.exe.
Must be merged after !784653
Related work items: #2174274
We now use different stores in different platforms. To simplify the things on the consumer, infra and the commandline side, I now lookup the given certificate name in both CurrentUser and LocalMachine stores.
There are a handful of isolated crashes on linux and codespaces where an IOException is thrown on Console.CursorLeft. It's unclear what's causing this. This change takes the approach of encapsulating all of the complicated console behavior for updating the same lines in place. If an exception is thrown anywhere inside of that, it will stop attempting that more complicated functionality and fall back on a simple Console.WriteLine for all subsequent messages. This should give a much better chance of the build being able to continue when something odd happens.
Related work items: #2138734
After net7 removal the minimal build fails in CB with 'child process survived' error.
Revert "Merged PR 784187: Remove net7 qualifier
Remove net7 qualifier and move msbuild to net8.
Related work items: #2174274"
Reverted commit `4c5cc753`.
Related work items: #2174274
Some instances of ValidateSuccessMatches errors on workers are related to this following scenario:
- Worker early-released and we call FinishAsync by capturing 'buildFailure' as null because HasFailed is false.
- We wait for worker to drain its pips
- Internal error occurs on the orchestrator
- Orchestrator terminates and tries to exit each worker.
- For early-released workers, there is already a pending FinishAsync due to the early release and the second 'FinishAsync' with the error message is ignored.
- Worker receives Exit message with no failure. ValidateSuccess condition fails because we expect the build to succeed, but there are errors logged due to pending/terminated pips on the worker itself.
Related work items: #2172783
Part of the sandbox cleanup. Root jail is not used by our customers, and the need for it for process remoting is not there now that AnyBuild is gone
Related work items: #2111125
Remove macOS sandbox / kernel extension code and examples. Trying to only remove dead code here, some refactoring is still possible to improve code structure but I will leave it for future PRs.
- Debugging some ptrace failures on 1JS
- Also disables report cache on fork/clone calls to fix a unit test that failed due to missed reports.
Related work items: #2171844
Soften build manifest warning when missing HistoricMetadataCache. With BlobL3 e2e build times are looking better than CASaaS in spite of this suboptimal behavior. Don't warn users when we plan on moving forward with this topology.
office decide to put source file into symbol drop so that source file can be retrieved from symbol store by a custom key. This will simply the implementation of the source link for office.
currently, symbolmetadata file which contains the debugentryData (including the clientKey to label the file and used for retrieving from sym store) is generated by buildxl before sending the symbol request. The key is generated based on the metadata of file by `SymstoreUtil`.
When office wants to index source files to symbol drop, source file extension may not be compatible with symstoreUtil nor do it meet requirements that key for downloading from symbolStore should be irrirelevant to content.
Office wants that `clientKey` is only relevant to path, commit or version that build can easily to compose download url for each indexed source file in symbol drop.
To do, office needs to control the generation of symbolMetaDataFile instead of letting bxl to do it.
Use a new common pool BuildXL-DevOpsAgents-Selfhost for both Windows and Linux runs. This is a pre-requisite step to be able to enable network isolation and use a common firewall for all runs.
The break is the change in StandardConsole.cs. That condition needs to be run any time the previous line was overwriteable to blank out the previously written text and move the cursor back to the appropriate position. The change was conditionalizing it when both the previous line was overwriteable and when the current line is.
I'm reverting the entire commit instead of just that one part because the error handling changes in the other files are related.
Related work items: #2138734, #2174322
There is logic added to prevent downstreams of succeedfast pips to be recursively dirtied. This can potentially cause underbuilds with incremental scheduling particularly when the upstreams of the pips in buildcone connected via succeedfast pip were dirtied.
The intent for the previous change is perf driven, to reduce the cache checks. However due to Eric's change which stops cache checks at the first up-to-date pip, the perf is retained and the change is not needed anymore. This PR removes the logic to stop downstreams of succeedfast pips to be dirtied.
Related work items: #2170157
Use GITHUB_USER env var to capture UserName for Codespaces.
Added an Engine Based test to test the three cases - custom value, ADO, codespaces.
Added a smaller unit test to specifically check the functionality of GetUserName.
Related work items: #2169835
Two fixes:
1) The new SandboxEvent constructors take `std::string&` as parameters but the lib functions pass `char*`s. If a tool passes null for a `char*`, the runtime crashes because it won't allow that implicit construction. We need to handle this gracefully because the tools can pass whatever to the libc functions we interpose. The fix is to create a special `SandboxEvent::Invalid` in these cases that we then ignore
2) We shouldn't report paths that are non-files. Add a check for this in `ResolveEventPaths`, which will report that the path isn't a file so we can ignore it
All occurrences of ReceivedReportFromUnknownPid from 1JS in the last week was about synthetic accesses (process requires ptrace, first allow write check in process). Let's not log in that case, we send these from native side before the process start event, and they are not real accesses in the end.
I'll check again in a week if that makes ReceivedReportFromUnknownPid reach 0, and in that case I'll turn it into a warning.
Revert the Lage builder to stop producing an error file. This change produced a DFA on office devmain (the lage graph builder is used directly as a tool in that codebase). Will roll forward again in a way that doesn't produce breaks.
Julian Bayardo
Oleksii Kononenko
BuildXL-Pipelines
Sahiti Chandramouli
Michael Pysson
Marcelo Lynch 🧉
Semih Okur
Serge Mera
Pasindu Gunasekara 🍣
Kai Chen
Qi Wang
Rahul Chamala