CaSaaS is currently on Net7 and it might need updated bxl bits while it's moving to Net8. This PR re-adds net7 qualifier. It's essentially a revert of `0a4d5b272900cfb6ad2c13795650e269500f9d17` and `5380d614bb98bfebb5a63900bb617ff51e93e56e`.
7.0.18 has a vulnerability, so this PR also includes a change that updates net7 bits to 7.0.19 to make the CG happy.
note: MsBuild is back to net7 (net7->net8 for msbuild was a part of net7 removal)
Reapply the original Net7 removal commit `4c5cc753`. The issue that we initially saw after it was merged was not caused by the commit. The commit forced a cache miss for a couple of pip and it uncovered the issue with vctip/lib.exe.
Must be merged after !784653
Related work items: #2174274
After net7 removal the minimal build fails in CB with 'child process survived' error.
Revert "Merged PR 784187: Remove net7 qualifier
Remove net7 qualifier and move msbuild to net8.
Related work items: #2174274"
Reverted commit `4c5cc753`.
Related work items: #2174274
Remove macOS sandbox / kernel extension code and examples. Trying to only remove dead code here, some refactoring is still possible to improve code structure but I will leave it for future PRs.
The inbox sdk source resolver fails for Ninja builds. Disable it explicitly in this case, to not require this configuration explicitly in the resolver settings.
Related work items: #2154307
Add minimal but self-contained documentation for using BuildXL with CMake and Ninja projects. Update the examples to reflect the recommended way to run CMake projects using Ninja generation, and marking the CMake resolver example as experimental and unsupported
Related work items: #2125879
This PR includes:
- Revamping core concept to more accessible explanation (for new members or for 3rd party).
- Add DScript walkthrough with runnable examples.
Be explicit about out and intermediate dirs in the msbuild project example (so thep predictors get that right). Update the platform toolset as well so it works with VS2022
Related work items: #2047654
The MsBuildGraphBuilder tool was loading `System.Collections.Immutable.dll` from the tool deployment folder. In newer versions of MSBuild a different version of the library is being referenced: loading this newer MSBuild and the old DLL caused a runtime error, (e.g. our example MsBuild project failed to build with the latest MsBuild with a `MissingMethodException`).
Related work items: #1807610
This PR implements the common case when deletion of shared opaque outputs can be postponed until right before the pip is executed.
The condition for postponing deletion of shared opaque outputs is:
- a sideband file exists for every process pip that produces at least one shared opaque directory and is not filtered out by the current filter
- the sideband is not corrupt (i.e., its `FileEnvelope` checksum matches)
- the pip's static fingerprint hasn't changed since the sideband file was recorded
When this condition is met
1. before the build:
- `EngineSchedule` skips over the step where shared opaque outputs are deleted
- any extraneous sideband files (i.e., sideband files found on disk whose corresponding pips are not present in the current pip graph) are deleted at this point, as well as all the files recorded in those sideband files
- all the other forms of scrubbing are still performed in `EngineSchedule` at this point
2. during the build:
- right before a pip is executed, its shared opaque outputs are read from its sideband file and deleted
- if any of those operations fail (i.e., reading the sideband file, deleting an output recorded in it, etc.) the pip is failed
Related work items: #1608015
- implement a handler for check_vnode_clone
- add a test (to the example build) that ensures that clone operations are properly detected
- add a test (to the example build) that ensure that obscure writes via sourcing csh scripts are detected
- publish and sign a new kext
AB#1607992
The implementation for Windows remains the same and continues to use magic timestamps.
On Mac, instead of timestamps extended attributes are used. Concretely, a file is a shared opaque output IFF it has a com.microsoft.buildxl:shared_opaque_output attribute and it's value is equal to 42 (at some point in the future we could insert a more meaningful value here, e.g., PipId)
AB#1607996
New classes:
- CsvFileLog: an ILog class that writes log messages to a CSV file and formats them according to a given schema
- KustoUploader: a class responsible for asynchronously receiving CSV files and ingesting them into a Kusto table
Special handling for directory rename.
Scenario: a pip writes a bunch of files into a directory (e.g., 'out.tmp') and then renames that directory (e.g., to 'out'). Once 'out.tmp' is renamed to 'out', we need to explicitly update all previously reported paths under 'out.tmp'. Since we cannot change the past and directly mutate previously reported paths, we simply enumerate the content of the renamed directory and report all the files in there as writes. This is exactly how this is done on Windows, except that it's implemented in the Detours layer.
AB#1362637
The idea is that all build outputs, no matter which pip produced them, have the special timestamps as long as they are under a shared opaque directory.
Before this change, pips that explicitly declare one or more shared opaque directories, their outputs that are under any of those directories are marked with the special timestamps.
What's possible is that there are some other pips (e.g., CopyFile or WriteFile) that produce additional files into those same shared opaque directories. Those files were not marked with the special timestamps; consequently, if those files become stale, the scrubber will fail to delete them before the build, which can cause problems downstream.
With this PR, all build outputs that end up being under any declared shared opaque output directory will have the special timestamps.
AB#1472511
* Restore all script exec permissions, update Readme.md with better macOS instructions
* Always run sandboxed when building internal
* Exec permission adjustment for all scripts
Oleksii Kononenko
Marcelo Lynch 🧉
Iman Narasamdya
Serge Mera
Ankur A Sharma
Kristijan Simic
Pasindu Gunasekara
Aleksandar Milicevic
Kristijan Šimić
Julián Bayardo
dannyvv
dannyvv
Kristijan Šimić
Erik Mavrinac
Microsoft Open Source