Added buildEntity property to gather information about the build queue(CB env) and build pipeline(ADO env).
In CB this information is obtained from "cloudBuildQueue" name passed via traceInfo
In ADO this information is obtained from the ADO predefined variable "SYSTEM_DEFINITIONID".
Added two unit test cases, one to test if the changes are effective in ADO environment.
The other is to test the changes for a CB environment and also to check if the user passed value overrides the ADO set value or not.
Used the name "buildEntity" to represent both build queue and build pipeline. But suggestions are welcomed.
Another thought here is that. Since we already have **cloudBuildQueue** as a part of the env string in the telemetry. Would it be enough to just add another property called **pipeline** to represent this information for ADO. That way we can avoid duplication of information if it is CloudBuild.
Related work items: #1948523
Make sure we produce the same special sandbox event on both Windows and Linux when a source rewrite is detected. Enable related unit tests for Linux as well.
Currently copy speed is calculated based on the number of bytes copied in a certain interval. This interval includes both network and disk copy. We want to cancel a copy if network is slow, not if disk is busy. To achieve that we need to compute network copy speed.
Related work items: #1959303
Untracks the nuget downloader executable because for some reason the tool seems to have a different hash on different LKGs without any changes.
Related work items: #1965295
Add eviction strategy where one partition of content is preferred for eviction in a particular time interval. This essentially throttles the removal of replicated content from the universe so that all replicas will not be evicted within a short time window.
Some BXL remoting users cannot install AnyBuild through BXL because powershell cannot be found; for some reason.
Running it within cmd will use cmd itself to look for the path.
Added codebase property to gather information about the branch triggering the build.
In CB this information is passed through traceinfo.
In ADO this information is extracted from the predefined variable "BUILD_REPOSITORY_NAME".
Added two unit test cases, one to test if the changes are effective in the ADO environment.
The other is to test the changes for a CB environment and also to check if the user passed value overrides the ADO set value or not.
Related work items: #1954489
Don't invalidate cache files when failing to disable inheritance.
When validating changes in Ring_0, saw that we were invalidating a bunch of files because we were failing with this:
SelfCheck: Found invalid entry in cache. Hash=VSO0:38EE1C131611B85315AB98. Failed to disable inheritance for file D:\dbs\Cache\ContentAddressableStore\Shared\VSO0\38E\38EE1C131611B85315AB98EC185A3BCB4571DC8A19B23BD86262630637FFEAA200.blob, Error=System.IO.FileNotFoundException: D:\dbs\Cache\ContentAddressableStore\Shared\VSO0\38E\38EE1C131611B85315AB98EC185A3BCB4571DC8A19B23BD86262630637FFEAA200.blob
at System.Security.AccessControl.NativeObjectSecurity.CreateInternal(ResourceType resourceType, Boolean isContainer, String name, SafeHandle handle, AccessControlSections includeSections, Boolean createByName, ExceptionFromErrorCode exceptionFromErrorCode, Object exceptionContext)
at System.IO.FileSystemAclExtensions.GetAccessControl(FileInfo fileInfo, AccessControlSections includeSections)
at BuildXL.Native.IO.Windows.FileUtilitiesWin.IsAclInheritanceDisabled(String path) in \.\Public\Src\Utilities\Native\IO\Windows\FileUtilities.Win.cs:line 2378
at BuildXL.Cache.ContentStore.Stores.FileSystemContentStoreInternalChecker.ValidateFileAsync(Context context, ContentHash expectedHash, FileInfo fileInfo) in \.\Public\Src\Cache\ContentStore\Library\Stores\FileSystemContentStoreInternalChecker.cs:line 373. Evicting the file...
Make some logging in daemons use pooled StringBuilders to cut down on string allocations that are causing high mem usage.
Related work items: #1962567, #1965658
Decouple IPC interfaces and common classes from IPC providers. This reduces the dependency on ASP.NET packages to only our modules that really need to instantiate an IPC provider.
RpcException is thrown by grpc.net tasks when some of the new features are used (keepalive, load-balancer). The grpc.net is fixing some of them, but it will probably take some time.
One example:
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Status(StatusCode="DeadlineExceeded", Detail="Deadline Exceeded"))
---> Grpc.Core.RpcException: Status(StatusCode="DeadlineExceeded", Detail="Deadline Exceeded")
--- End of inner exception stack trace ---
Added a buildProperty called org to capture the information related to the organization triggering the build.
Added unit tests to check if the right org information is being extracted from the URL or not.
Added some other changes to remove repetitive code.
Related work items: #1954481
When pips are executed in VM, some tools cannot be executed with `net use`, and thus those tools need to be copied to a temporary location in the VM. The pip can use its specified temp folder as this temporary location. But since pips's temp folder is usually unique, it means that the same tool will be copied multiple times to the VM if there are multiple pips needing that tool. Not only that it will degrades performance, the VM will run out of space quickly because it has many pips running there at the same time, and all of them need a large tool.
With this shared temp folder, one pip can copy the tool once, and other pips can simply reuse it.
* SkipExtraPins now avoid pinning to L3 for WriteFilePip
* ImplicitPin on L3 pins right before calling GET, so it's not necessary
* Add Selector's ContentHash to the expiry cache
With these changes, I get GET-only L3 consumption when `/historicMetadataCache-` is specified.
Aleksandar Milicevic
Sahiti Chandramouli
Serge Mera
Shuvrajit Mukherjee
Pasindu Gunasekara 🍣
CloudDev Build Account
Lance Collins
Julian Bayardo
Iman Narasamdya
Sergey Tepliakov
Juan Carlos Guzman Islas
Oleksii Kononenko
Semih Okur
Marcelo Lynch 🧉
John Erickson