Before this PR, on every trie operation a leaf node corresponding to a given key is retrieved, creating any intermediate nodes along the way.
This is unnecessary for lookup operations. When the key does not exist in the trie, instead of traversing the trie until a leaf node is found and creating all intermediate nodes along the way, the lookup should fail as soon as an intermediate node is not found.
As a consequence of the previous implementation, if a process connects to the kext and then immediately disconnects, that alone causes the size of the `connectedClients_` dictionary to monotonically grow. This can become a problem because those nodes are released only when the kext is unloaded.
It's important to untrack /usr/lib instead of making it a source sealed directory because the set of dynamically loaded libraries (from /usr/lib) during an execution of a process is not necessarily deterministic. For example, when the same process---which itself is deterministic---is executed multiple times on the same inputs, the set of dynamically loaded libraries can vary from time to time.
Re-enabled tests:
- IntegrationTest.BuildXL.Scheduler.NonDeterminismProbeTests
- NonDeterminismOpaqueDirectoryOutputDifferentFiles
- NonDeterminismOpaqueDirectoryOutput
- Test.BuildXL.FingerprintStore.FingerprintStoreTests
- VerifyGarbageCollectWorks
- DontOverwriteExistingContentAddressableEntries
AB#1519677
AB#1513463
AB#1520150
In the past in elevated mode, we simply return the journal accessor without checking if we can query the journal. This causes a problem because then tests that require journal scan will run although the journal cannot be queried, e.g., due to failure in opening volume handle.
This change improves the typical dev loop on macOS by allowing 2 kexts to be loaded at the same time:
1. one that came with the LKG (which one is compiled as release), and
2. one compiled as debug from the current sources
The first one is typically used when compiling current sources, and the second one when running tests.
* More descriptive error message when finalizer fails for FileStream.
* Use two constructors on windows only
* Fix compilation issue for .net core
* Temporarily disabling the feature
* Switching back to TrackingFileStream.
* Closing the file
1. Make sure default source seal directories are added to the pip graph only if explicitly requested (by virtue of setting DependsOnCurrentOs).
2. Propagate errors if creating default source seal directories fails.
AB#1517833
BuildXL appeared to hanged upon disposing temp cleaner. The dump file only shows that the dispose method was stuck on Thread.Join.
These changes attempt at fixing this issue by doing two things:
Add cancellation token during directory deletion.
Add timeout to Thread.Join (temp cleaner is best effort)
Special handling for directory rename.
Scenario: a pip writes a bunch of files into a directory (e.g., 'out.tmp') and then renames that directory (e.g., to 'out'). Once 'out.tmp' is renamed to 'out', we need to explicitly update all previously reported paths under 'out.tmp'. Since we cannot change the past and directly mutate previously reported paths, we simply enumerate the content of the renamed directory and report all the files in there as writes. This is exactly how this is done on Windows, except that it's implemented in the Detours layer.
AB#1362637
The current EventCount table has columns whose names start with a digit; the AriaV2 static library for macOS doesn't seem to support this.
This commit renames `EventCount` to `EventCounts` and makes sure that no table column starts with a digit (when it does, `e_` is automatically prepended to it)
AB#1511704
The change is mainly needed so drop pips would not materialize all the files from inside drop artifacts. For example, if customers want to upload only small subset of files from a directory, in the past we would materialize all the files (even though we did not need them) and then upload only those files that match a filter. With this change, we would not materialize directory artifacts at all; instead, dropdaemon will only materialize those files that match the filter.
AB#1487719
This change introduces a so-called sandboxed process executor tool that takes a sandboxed process info as an input and outputs a sandboxed process result containing details of file accesses.
The tool will be used to run process pips that require admin privilege, and the tool will run inside a VM. Traditionally, BuildXL in SandboxedProcessPipExecutor will create a detoured child process and communicate with the child process by means of pipes. For process pips that require admin, SandboxedProcessPipExecutor will (1) serialize sandboxed process info, (2) launch the sandboxed process executor tool, and (3) deserialize sandboxed process result produce in (2).
The sandboxed process executor tool will either replace QuickBuild's Tracker.exe or be called by QuickBuild's Tracker.exe. The cutting layer allows the two scenarios to be done, but the latter is the easiest.
* Cleanup: Usings and headers in cache/verticalstore folder
* Cleanup: Usings and headers in cache/memoizationstore folder
* Cleanup: Usings and headers in cache/contentstore folder
* Cleanup: Usings and headers in cache/distcache.host folder
Aleksandar Milicevic
Iman Narasamdya
Kristijan Šimić
Huihui Nora Huang
Kevin Bazan Williams
Serge Mera
dannyvv
CloudDev Build Account
Anirudh Santhiar
David Wright
Sergey Teplyakov
Semih Okur
Lance Collins
Annie Fu
olkononenko
Michael Pysson
Titi Phan