Граф коммитов

7648 Коммитов

Автор SHA1 Сообщение Дата
BuildXL-Pipelines a606b74a82 Merged PR 817536: Update BuildXL LKG Version to 0.1.0-20241127.1
[skip ci]
2024-12-01 16:03:02 +00:00
Marcelo Lynch 91feaf4284 Merged PR 817279: Log GrpcTrace when the async request is completed
Log GrpcTrace when the async request is completed. This can let us know if there is a delay in the processing:

![image.png](https://dev.azure.com/mseng/9ed2c125-1cd5-4a17-886b-9d267f3a5fab/_apis/git/repositories/50d331c7-ea65-45eb-833f-0303c6c2387e/pullRequests/817279/attachments/image.png)
2024-11-27 20:29:03 +00:00
Marcelo Lynch 70dd4a26bf Merged PR 816035: Add GetLogDrop script to our common scripts
This script is useful to me sometimes, it might be to others
2024-11-26 21:56:44 +00:00
Sahiti Chandramouli 676e72325a Merged PR 817081: Fix for TwoPhaseFailedToStoreMetadataForCacheEntry error bucket influx
Fix for TwoPhaseFailedToStoreMetadataForCacheEntry error bucket influx.
When the CancellationToken is requested, we log the TwoPhaseFailedToStoreMetadataForCacheEntry error and return a null entry.

But we do not want CancellationToken related issues to go under this bucket.  Also in the parent method we have a Contract which checks for the error to be logged if it receives a null entry. Handled this by adding an enum to include the three different states of the operation to store two-phase cache entry.

Related work items: #2231114
2024-11-26 20:20:15 +00:00
BuildXL-Pipelines ab14f5363c Merged PR 817060: Update NOTICE.txt
[skip ci]
2024-11-26 00:32:39 +00:00
BuildXL-Pipelines fe106e815e Merged PR 817061: Update Packages
Update packages: Microsoft.ComponentDetection.Contracts 5.1.8 -> 5.1.11
2024-11-25 23:08:38 +00:00
Marcelo Lynch d3861e9129 Merged PR 816539: Use property instead of Task<bool> method to guard against new tasks being created inadvertently
This complements !816459. A deadlock was caused by the method `EnsureLoadedAsync()` queuing tasks (by virtue of awaiting async methods) into a dedicated task scheduler that was asynchronously waiting for `EnsureLoadedAsync` itself. That PR solved the issue by ensuring this method does not await and just returning `Task`s, and this PR makes it tighter by switching it to a property, which will never allow awaits
2024-11-22 20:06:54 +00:00
Kunal Karoth b73df14087 Merged PR 814234: Make accommodations for shardsets in BXL
Make provisions for shardset domains. This is a side car change to the changes being made in the CB repo
https://mseng.visualstudio.com/Domino/_git/CloudBuild/pullrequest/815932

The idea behind this change is to accommodate for arbitrary values that a domain id maybe. The data type of domain id sort of leaked through into BXL and CB. That is implementation detail and `DomainIdFactory.Create` is responsible for managing that translation and should be used instead.

With this change, I expect CB and BXL to be able to pass in arbitrary string values to drop service.
2024-11-22 16:28:09 +00:00
BuildXL-Pipelines 410608c9f6 Merged PR 816803: Update BuildXL LKG Version to 0.1.0-20241120.7
[skip ci]
2024-11-22 01:52:56 +00:00
Iman Narasamdya 3b3facd2c1 Merged PR 816754: Release notes for 0.1.0-20241115.2.1
Release notes for 0.1.0-20241115.2.1
2024-11-22 00:51:42 +00:00
BuildXL-Pipelines 2183404984 Merged PR 816756: Update BuildXL LKG Version to 0.1.0-20241115.2.1
[skip ci]
2024-11-21 22:32:18 +00:00
Iman Narasamdya dda226519b Merged PR 816564: Add NRE into first chance exception for debugging NRE
Add NRE into first chance exception for debugging NRE

Related work items: #2142530, #2232707
2024-11-21 00:52:55 +00:00
Marcelo Lynch b2f1b123a2 Merged PR 816378: Avoid automatic installation of BuildXL for selfhost builds using the 1ESPT workflow
Automatic installation is becoming the default for the workflow, but we don't want to use it in our selfhost builds, where we intentionally want to run BuildXL from custom locations for testing and such
2024-11-21 00:40:36 +00:00
Pasindu Gunasekara 1568e6122e Merged PR 816349: Disable system call forward logging for putc
- This one gets logged too often causing performance issues

Related work items: #2186390
2024-11-20 22:56:19 +00:00
Marcelo Lynch a6d75aaab4 Merged PR 816394: TaskSourceSlim: cut allocations by 50%
The constructor creates an object that will be immediately discarded - the compiler can't optimize this away:

| Method | Mean      | Error     | StdDev    | Gen0      | Gen1      | Gen2     | Allocated |
|------- |----------:|----------:|----------:|----------:|----------:|---------:|----------:|
| Old    | 17.564 ms | 0.1729 ms | 0.1444 ms | 1875.0000 | 1093.7500 | 343.7500 |  18.31 MB |
| New   |  5.434 ms | 0.1077 ms | 0.2600 ms |  757.8125 |  750.0000 |        - |   9.16 MB |
2024-11-20 22:22:05 +00:00
Ben Witman e2bbe81d09 Merged PR 816459: EnsureLoadedAsync is synchronously waited on and must not queue up more async tasks
A deadlock can occur when the current context has all active threads running EnsureLoadedAsync which queues up an async task to that context.
To ensure this doesn't happen ensure we do not do any async/await in this method and ensure we return task result(s) that either are static or are from the singleton lazy initialization task.

This was subtly introduced with !797423

This should address #2229869 based on the dumps we have analyzed
2024-11-20 20:05:13 +00:00
Serge Mera c0f70ead88 Merged PR 816117: Integrate build cache to the developer cache flow
Use the recently deployed build cache endpoint `subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.CloudTest/buildcaches/{buildCacheId}/access` so the build cache client can retrieve the build cache info.

The end-user configuration looks like this:

```json
{
    "Assembly": "BuildXL.Cache.MemoizationStoreAdapter",
    "Type": "BuildXL.Cache.MemoizationStoreAdapter.BlobWithLocalCacheFactory",
    "RemoteCache": {
      "CacheId": "remoteexamplecache",
      "CacheLogPath": "[BuildXLSelectedLogPath].log",
      "DeveloperBuildCacheResourceId": "/subscriptions/7d3b156b-1072-4060-97f6-42057dc19952/resourceGroups/buildxlexperiments/providers/Microsoft.CloudTest/buildcaches/buildcache",
      "IsReadOnly" : true
    },
    "LocalCache": {
      "CacheId": "localexamplecache",
      "MaxCacheSizeInMB": 40480,
      "CacheLogPath": "[BuildXLSelectedLogPath].local.log",
      "CacheRootPath": "[BuildXLSelectedRootPath]"
    }
  }
```
2024-11-20 17:36:10 +00:00
Sahiti Chandramouli 24aa2399ce Merged PR 816097: Fix failing unit tests in Ubuntu 24.04
Resolving the issues related to Ubuntu 24.04
![Screenshot (22).png](https://dev.azure.com/mseng/9ed2c125-1cd5-4a17-886b-9d267f3a5fab/_apis/git/repositories/50d331c7-ea65-45eb-833f-0303c6c2387e/pullRequests/816097/attachments/Screenshot%20%2822%29.png)

Related work items: #2227479
2024-11-19 18:42:57 +00:00
Pasindu Gunasekara 7ac3188a10 Merged PR 815790: Add target platform to vscode extension vsixmanifest
Allows us to publish platform specific extensions.
Reference from vsce: 1939270129/src/package.ts (L1484)
2024-11-18 23:12:25 +00:00
Sahiti Chandramouli ae9116838f Merged PR 815070: Fix version parsing of single digit versions in linux distro systems.
In the crash we find that the version is a single digit number which is causing the Version.ParseVersion method within TryParse to fail. It expects a "." to be present within the version. Hence converting the version from say 11 to 11.0 to create a valid version for TryParse to parse.

Related work items: #2215963
2024-11-18 22:50:20 +00:00
Michael Pysson 9666c0cfbb Merged PR 815797: Remove mono from PR build base template
Remove mono from PR build base template. The apt-get is a noop anyway and it occasionally fails due to some file lock issue.
2024-11-18 22:41:18 +00:00
BuildXL-Pipelines aa54a06682 Merged PR 815964: Update BuildXL LKG Version to 0.1.0-20241115.2
[skip ci]
2024-11-18 18:42:24 +00:00
BuildXL-Pipelines d4e47426e5 Merged PR 815962: Update Packages
Update packages: Microsoft.ComponentDetection.Contracts 5.1.7 -> 5.1.8
2024-11-18 17:58:57 +00:00
Oleksii Kononenko bee94380cc Merged PR 815690: Increase timeout for manifest parsing in MaterializationDaemon
Increase timeout for manifest parsing in MaterializationDaemon
2024-11-15 19:35:57 +00:00
Marcelo Lynch 84f33184c2 Merged PR 815436: Add resilience to build tools installer's HttpClient
Add resilience to build tools installer's HttpClient
2024-11-15 00:23:01 +00:00
Serge Mera cccfef92b5 Merged PR 815223: Turn on NuGet version dependency enforcements
Turn on NuGet version dependency enforcements.
2024-11-14 20:50:37 +00:00
Michael Pysson e596d85f14 Merged PR 814946: Delete more linux outputs before running SDL tasks
Delete more linux outputs before running SDL tasks
2024-11-14 19:16:59 +00:00
BuildXL-Pipelines 754eb7d483 Merged PR 815385: Update BuildXL LKG Version to 0.1.0-20241113.5
[skip ci]
2024-11-14 19:07:45 +00:00
Michael Pysson af2847cd14 Merged PR 815379: Demote nuget download failure to user error
Demote nuget download failure to user error
2024-11-14 18:55:42 +00:00
Qi Wang 3d62533bbb Merged PR 814349: Expose warning/error regex to the JS resolver family
Expose warning/error regex to the JS resolver family

Related work items: #2224422
2024-11-14 01:43:48 +00:00
Marcelo Lynch abad034138 Merged PR 815225: Use default upstream feed in installer validation
The upstream is now set up, we don't need a manual feed override.
Also change properties to camelCase to standardize.
2024-11-14 01:18:28 +00:00
Qi Wang f3a723fd0d Merged PR 814881: Update process default timeout and warning timeout document
Update process default timeout and warning timeout document
2024-11-14 00:10:43 +00:00
Serge Mera 9ee9d9c919 Merged PR 814384: Update nuget references
Update nuget references so the whole closure is within the specified version ranges. After we get the new LKG we can turn off `doNotEnforceDependencyVersions: true`
2024-11-13 22:23:35 +00:00
Marcelo Lynch 7a0cc56aee Merged PR 814368: ConfigCop for AdoBuildConfig
Add a tool that validates the configuration to be deployed to AdoBuildConfig. This will be used to gate changes in that repository so that we always deploy configurations compatible with the installer.

Usage: `1ES.BuildToolsInstaller.Exe configcop --path <path-to-config-to-validate>`
2024-11-13 22:17:34 +00:00
BuildXL-Pipelines 741e0a2f18 Merged PR 814505: Update NOTICE.txt
[skip ci]
2024-11-13 20:57:56 +00:00
Michael Pysson 4353b3016f Merged PR 814918: Disable dev cache while backend moves
Disable dev cache while backend moves
2024-11-13 00:44:19 +00:00
Michael Pysson 12c81035b5 Merged PR 814921: Stop dev cache publishing build
Stop dev cache publishing build
2024-11-13 00:44:17 +00:00
Michael Pysson 9c1219816e Merged PR 814900: Delete more outputs of linux build to speed up 1ESPT
Delete more outputs of linux build to speed up 1ESPT
2024-11-12 23:45:54 +00:00
Marcelo Lynch 0b8697e1db Merged PR 814892: Minor logging tweaks in AdoBuildRunner
- Stop logging some messages from the era when the "worker pipeline" was the default
- Add a message to show progress when retrieving build properties
2024-11-12 23:17:29 +00:00
Julian Bayardo Spadafora 8f0a39cd88 Merged PR 814606: Make sure we dispose all Azurite instances
Make sure we dispose all Azurite instancex
2024-11-12 23:14:52 +00:00
Serge Mera 4600965f85 Merged PR 814620: [Nuget] Do not complain about dependencies out of range when the dependency is skipped or ignored
Do not complain about dependencies out of range when the dependency is skipped/ignored
2024-11-12 17:31:31 +00:00
Michael Pysson de78a3dd6c Merged PR 814612: Fix crash in assert message
PathAtom.ToString() is unimplemented. It needs a StringTable
2024-11-12 00:47:15 +00:00
BuildXL-Pipelines 276b356ebe Merged PR 814553: Update BuildXL LKG Version to 0.1.0-20241109.1
[skip ci]
2024-11-12 00:22:32 +00:00
BuildXL-Pipelines f5e82e8eaa Merged PR 814568: Update BuildXL LKG Version to 0.1.0-20241109.1
[skip ci]
2024-11-11 21:29:46 +00:00
Michael Pysson 9dbcdba78f Merged PR 814385: Clean up output stream handling to prevent running out of memory
Generally, output streams are handled in chunks to prevent them from reading everything into memory at once. But there was a check added back in commit 0167f82124 which violated this for sake of doing some checks for whether the output was truncated. This change reworks that handling to prevent it from calling ReadValueAsync() which reads in the entire stream.

I also changed ReadValueAsync to truncate rather than OOM as it is called in a number of places where we probably don't want to bother having elaborate chunking behavior.

Related work items: #2229870
2024-11-11 17:56:26 +00:00
Oleksii Kononenko d6c433bc26 Merged PR 812315: Add a simple example for graph fragments
Add a simple example for graph fragments
2024-11-11 17:25:54 +00:00
Michael Pysson 04e8421b7c Merged PR 813656: Add ImmediateWorkerRelease mode for AB testing worker counts
We'd like to run some worker count experiments for Windows builds. We don't currently have a way to have CloudBuild do this, so a workaround is to utilize the engine's abtest flag with a mode that immediately releases a configurable number of workers.

This is wasteful since it will waste some machine time preparing a worker that will eventually get lost, but it is much easier to implement at this level. Moreover, this translates to ADO based distributed builds.
2024-11-09 00:31:18 +00:00
Serge Mera 58e3a46f2f Merged PR 814357: Nuget group dependencies should consider all compatible frameworks
Nuget packages can specify group dependencies that are conditioned by a given target framework. E.g.:

```xml
<dependencies>
    <group targetFramework="net6.0">
        <dependency id="jQuery" version="1.10.2" />
        <dependency id="WebActivatorEx" version="2.2.0" />
    </group>
</dependencies>
```

The semantic of the condition is 'any framework compatible with net6.0 and the target framework' and not just net6.0. However, our nuget spec generation logic was adding this dependency **only** for net6.0. This was causing some dependent Nuget packages to be missing at deployment time. So far, we've been compensating for that manually adding the missing packages as explicit imports. But this is a risky practice since some packages might be required just at runtime and this might become evident only under particular code paths.

Modify the dependency generated logic to include all compatible frameworks wrt the specified framework dependency. This is a relatively safe change because at most we will be deploying *more* libraries than before. It should also allow us to clean up specs considerably after this reaches the LKG.
2024-11-08 23:23:21 +00:00
Marcelo Lynch 5d3a2d4256 Merged PR 813911: Rings and overrides for the build tools installer
Rings and overrides for the build tools installer
2024-11-08 23:06:43 +00:00
Michael Pysson 19f6495e6a Merged PR 814312: Quick mitigation for OOM handling output streams
Quick mitigation for OOM handling output streams

Related work items: #2229870
2024-11-08 20:33:26 +00:00