Граф коммитов

180 Коммитов

Автор SHA1 Сообщение Дата
Chad Su fc58516301 Fix BuildXL Integration extension issue with Visual Studio 2022 17.2.0 Preview 2.1 or above
```
SetSite failed for package [BuildXLVsPackage]Source: 'BuildXLVsPackageDev17' Description: Unable to cast object of type 'Microsoft.VisualStudio.CommonIDE.Solutions.CSlnUpdate' to type 'Microsoft.Internal.VisualStudio.Shell.Interop.IVsSolutionBuildManagerPrivate'. System.InvalidCastException: Unable to cast object of type 'Microsoft.VisualStudio.CommonIDE.Solutions.CSlnUpdate' to type 'Microsoft.Internal.VisualStudio.Shell.Interop.IVsSolutionBuildManagerPrivate'. at BuildXL.VsPackage.VsProject.ProjectFlavorFactory..ctor(BuildXLVsPackage package) in \.\Public\Src\IDE\VsIntegration\BuildXLVsPackageShared\VsProject\ProjectFlavorFactory.cs:line 59 at BuildXL.VsPackage.BuildXLVsPackage.<InitializeAsync>d__15.MoveNext() in \.\Public\Src\IDE\VsIntegration\BuildXLVsPackageShared\BuildXLVsPackage.cs:line 137
```
The problem is that the extension has its own definition of Microsoft.Internal.VisualStudio.Shell.Interop.IVsSolutionBuildManagerPrivate copied from Microsoft.Internal.VisualStudio.Shell.Interop.10.0.DesignTime.dll. The solution is to use the interface defined in Microsoft.Internal.VisualStudio.Interop.dll.
2022-04-08 16:26:43 +00:00
Aleksandar Milicevic f0a82776e6 Merged PR 656467: Use cross-plat Aria logger on Linux/Mac
Changes:
- add a new NuGet dependency on `Microsoft.Applications.Events.Server` (which is the cross-plat Aria SDK)
- use that on Linux and Mac
- delete the native SDK previously used on Mac
2022-04-06 19:51:25 +00:00
Aleksandar Milicevic 1fe619f579 Merged PR 656279: Initial steps toward selfhosting on Linux
Changes:
- fix various file name capitalization errors
- fix various nuget package name capitalization errors
- create and publish `Bond.CSharp.linux-x64` nuget package
- add `/etc` to default untracked scopes
- untrack `$HOME/.dotnet` when running the `Downloader` tool
- consistently spell `App.config`

With these changes, all of the following succeed for me in Ubuntu 20.04 WSL running in Windows 11:
- `./bxl.sh --minimal`
- `./bxl.sh --minimal --internal`
- `./bxl.sh --minimal --internal --shared-comp`

Once the changes make it to LKG, the next step will be to set up a Linux pipeline to build minimal selfhost.  Later, that pipeline can be expanded to run unit tests etc.
2022-04-05 19:44:47 +00:00
Iman Narasamdya af2e5b41a8 Merged PR 655366: [AnyBuild] Update AnyBuild SDK to 0.1.13
Update AnyBuild SDK
2022-03-29 19:21:58 +00:00
Iman Narasamdya 6a41aac535 Add a way to specify minimal specified installation
Add a way to specify minimal specified installation.

Plus enable configurable agent wait timeout.
2022-03-23 22:41:16 +00:00
Iman Narasamdya bb19f91246 Merged PR 651684: Enable server mode when remoting is enabled
Previously, the server mode wasn't compatible with process remoting. The root cause of the issue is because we use the singleton pattern to get-or-create process remoting manager. Thus, that manager is used across build sessions, but was actually disposed after the very first session.

This PR stops the use of singleton pattern, and simply creates a remoting manager for each build session. That creation happens inside the scheduler, and the obtained instance will be disposed when the scheduler is disposed.

Related work items: #1926109
2022-03-04 18:54:42 +00:00
Iman Narasamdya 76adb87e41 Merged PR 647318: Pipeline exercising BuildXL remoting capability
Currently it only remotes process pip with "compile" tag.

The remoting configuration is still not optimal currently, particularly for the beefy pool we have (32 cores).

In this run, https://dev.azure.com/mseng/Domino/_build/results?buildId=16779538&view=results, with remoting multiplier 1.5, and num of leases is assumed to be 2*maxproc, there are 42 processes going to AnyBuild, but 22 of them failed to find agents because AnyBuild is over saturated.

In this run, https://dev.azure.com/mseng/Domino/_build/results?buildId=16779745&view=results, with remoting multiplier 1.25 and num of leases 24, there are only 17 processes going to AnyBuild, and all of them successfully execute remotely.

More work is still needed for optimal configuration, e.g.,
- Feedback from AnyBuild about available leases, etc.

Related work items: #1912875
2022-02-02 21:27:11 +00:00
Iman Narasamdya 6725095c11 Merged PR 644381: Integrate BuildXL with AnyBuild SDK
Integrate BuildXL with AnyBuild SDK.

The design is to isolate AnyBuild use so that the `#if FEATURE_ANYBUILD_PROCESS_REMOTING` doesn't need to appear everywhere.

This PR depends on AnyBuild.SDK package that should be published after this PR !644510 is pushed.

Related work items: #1907310
2022-01-13 23:51:44 +00:00
Juan Carlos Guzman Islas 627dfeb24a Merged PR 644087: Upgrade ADO packages and fix breaking changes
Upgrade ADO packages and fix breaking changes
2022-01-10 21:25:02 +00:00
Marcelo Lynch 🧉 1d0dc08a64 Merged PR 643482: Fix runtime failures in DropDaemon's GetSbomPackages
- Remove the `__ENABLE_SBOM_PACKAGE_CONVERSION` env variable. This was meant to guard against using this feature while the library had a bug that is already resolved, so not necessary anymore
- Microsoft.SBOM.Adapters and some other packages reference Newtonsoft.Json 13.0.1, while we were using 12.0.3 throughout. This was causing a failure when the runtime tried to load the assembly after calling the library. Version 13.0.1 is incompatible with earlier ones so a downgrading binding redirect is not possible. Instead, we deploy the drop daemon with Newtonsoft.Json version 13.0.1 and use "forward" binding redirects.
- Add some extra logging

With these changes, an end-to-end build where CG is run in the build runner and the packages are retrieved and added to the SPDX SBOM was successful: https://cbtest.microsoft.com/build/1d85303e-322e-4ddf-af1f-585ecce96079
2022-01-06 01:28:25 +00:00
Marcelo Lynch 🧉 8889e2ec89 Merged PR 643161: Remove SBOMUtilities.ComponentDetectionConverter in favor of the Microsoft.SBOM.Adapters library
- Remove ComponentDetectionConverter in favor of the Microsoft.SBOM.Adapters library
- Update SBOM packages to version 2.0.99

Related work items: #1902188
2022-01-04 21:30:09 +00:00
Pasindu Gunasekara 🍣 a7f80c722b Merged PR 641108: Migrate daemon tools to netcore
- Update qualifier for daemons to NetCoreAppQualifier
- Update VsoCrendentialHelper calls for AAD
- Upgrade Microsoft.Windows.Debuggers.SymstoreInterop to a version that is compatible with netstandard2.0
2021-12-21 18:04:54 +00:00
Pratik Lade ddcb8d44d1 Merged PR 642128: QTest: Add mstest_retail option
Add option to provide the vstest.console.exe tool to be used to run tests.

Update nuget to incorporate changes that update how Fakes and Code Coverage are handled for TestPlatform.

Test build: https://mseng.visualstudio.com/Domino/_build/results?buildId=16562693&view=results

Related work items: #1874080
2021-12-17 22:32:37 +00:00
Pasindu Gunasekara 🍣 7a98a99f79 Merged PR 640680: Upgrade SBOM packages to 2.0.90 2021-12-08 21:35:32 +00:00
Pasindu Gunasekara 🍣 988bb1954f Merged PR 639286: Add SBOM packages to SBOM generation step in DropDaemon
- Calls the new Microsoft.Sbom.Adapters library to get a list of SBOMPackage objects from the component detection output.

Related work items: #1882259
2021-12-01 22:11:29 +00:00
Michael Pysson dcc2149904 Merged PR 640210: Revert sbomApiVersion from 1.2.36 to 1.2.31 due to serialization error
Revert sbomApiVersion from 1.2.36 to 1.2.31 due to serialization error
2021-12-01 20:13:43 +00:00
Marcelo Lynch 🧉 7109de92b0 Merged PR 638996: Optionally disable CloudBuildV1 SBOM Generation
Add an environment variable to suppress this SBOM format
2021-11-19 23:07:07 +00:00
Marcelo Lynch 🧉 68382b6612 Merged PR 638117: Update SBOM API version to address scale issues
Previous versions were not flushing the JSON while generating, causing memory issues

Related work items: #1882256
2021-11-13 05:17:03 +00:00
Marcelo Lynch 🧉 e0177cf78b Merged PR 637924: Bump SBOM API version and add logging
This version fixes a bug in the generation issue. Add a logging implementation wrapping the daemon logger.

Related work items: #1882256
2021-11-12 01:52:08 +00:00
Marcelo Lynch 🧉 fd879b0886 Merged PR 635704: Use the new common API to generate SBOMs from the DropDaemon 2021-11-11 19:33:21 +00:00
Marcelo Lynch 🧉 a37db4c037 Merged PR 637192: Reinstate change: add a helper library to extract SBOM Metadata from BuildSessionInfo to BuildXL
Revert the revert and remove problematic dependencies.
2021-11-08 18:59:59 +00:00
Oleksii Kononenko 9de4b4915a Merged PR 637024: Revert "Merged PR 636560: Add a helper library to extract SBOM Metadata from...
Revert "Merged PR 636560: Add a helper library to extract SBOM Metadata from BuildSessionInfo to BuildX...

Add a helper library to extract SBOM Metadata from BuildSessionInfo to BuildXL.Utilities

Related work items: #1882251"

Reverted commit `c7a683ce`.

Related work items: #1882251
2021-11-05 20:06:27 +00:00
Marcelo Lynch 🧉 c7a683ce3f Merged PR 636560: Add a helper library to extract SBOM Metadata from BuildSessionInfo to BuildX...
Add a helper library to extract SBOM Metadata from BuildSessionInfo to BuildXL.Utilities

Related work items: #1882251
2021-11-04 19:28:00 +00:00
Pratik Lade 8e83f1dc98 Merged PR 636263: QTest: Add new parameters for Blame Collector and VSTest Console logger options
QTest: Add new parameters for Blame Collector and VSTest Console logger options
2021-11-03 20:20:15 +00:00
Sergey Tepliakov 7e0366c2c1 Merged PR 633367: Remove SQLite from Bxl sdk
Remove SQLite from Bxl sdk
2021-11-02 17:31:50 +00:00
Sergey Tepliakov ff8315980b Merged PR 635626: Upload mdm metrics when the launcher is used
Upload mdm metrics when the launcher is used

Related work items: #1881914
2021-11-01 22:14:46 +00:00
Pratik Lade f3d28ce72f Merged PR 634435: QTest: Fix coverage upload issue
QTest: Update nuget

A loose async call caused coverage results to not get uploaded. This has been fixed in [PR 634426](https://dev.azure.com/mseng/Domino/_git/CloudBuild/pullrequest/634426)
2021-10-20 19:54:51 +00:00
Pratik Lade 9765c2c43a Merged PR 634151: QTest: Update nuget to latest
Changes include:

[PR 632424](https://dev.azure.com/mseng/Domino/_git/CloudBuild/pullrequest/632424) : Ensure that QTest sets correct state in Azure DevOps when "DoNotFailForZeroTestCases" is set
2021-10-18 20:39:54 +00:00
Pratik Lade 3a0fd74957 Merged PR 631163: Qtest: allow JUnit parser to read results from custom location
Qtest: Update nuget to incorporate following:
1. Allow JUnit parser to read results from custom location
2. Search only current directory for JUnit.xml to avoid recursion in search
2021-09-26 21:02:15 +00:00
Juan Carlos Guzman Islas c46e7e0854 Merged PR 630483: Update ADO package version
Update unified ADO package version
2021-09-21 18:03:52 +00:00
Danny van Velzen 🁴 0d12037eea Merged PR 630020: Update Symbol tooling version
Update Symbol tooling version

Related work items: #1844922
2021-09-17 17:26:22 +00:00
Aleksandar Milicevic 0e297ddca7 Merged PR 629512: Publish new osx-x64 runtime with kext and libBuildXLInterop.dylib reverted 2021-09-14 15:53:50 +00:00
Pratik Lade bd181a50cb Merged PR 628457: QTest: Enable coverage upload for JS projects
Fixed the coverage upload issue and reverted the previous revert.

Confirmed coverage upload is working for both .NET and JS targets and INT tests don't fail.
https://cloudtest.visualstudio.com/CloudTest/_build/results?buildId=52521&view=ms.vss-test-web.build-test-results-tab
2021-09-10 20:08:38 +00:00
Pratik Lade 3710029aa7 Merged PR 628376: Revert two PRs for QTest since Aug 24
Revert "Merged PR 626875: QTest: Support coverage upload for JS"

    This reverts commit 7e0d311d9e.

    Revert "Merged PR 627798: QTest: Update nuget to incorporate Gradle fix and allow env var for zipsandbox"

    This reverts commit 93570650e6.
2021-09-02 19:15:20 +00:00
Pratik Lade 93570650e6 Merged PR 627798: QTest: Update nuget to incorporate Gradle fix and allow env var for zipsandbox
QTest: Update nuget to incorporate Gradle fix

Related work items: #1870030
2021-08-31 19:38:55 +00:00
Pratik Lade 7e0d311d9e Merged PR 626875: QTest: Support coverage upload for JS
Validation Build: https://dev.azure.com/mseng/Domino/_build/results?buildId=15832911&view=codecoverage-tab

Related work items: #1852110
2021-08-24 23:33:43 +00:00
Pratik Lade 1107ad37bc Merged PR 625214: QTest: Update nuget
QTest: Update nuget

Following new updates were made:
- Bug fixes and improvements for Torch
- Upgrade Magellan version to new signed version
- Coverage collection with vsinstr in CloudBuild
- Support AnyCPU for VsInstr
- More QTest Bug Fixes
2021-08-17 04:46:19 +00:00
Kristijan Simic c3d6616c5c Merged PR 625705: Improve SandboxedProcess resource tracking for Unix
This PR improves resource tracking when executing processes using `SandboxedProcessUnix`.

Related work items: #1861113, #1861122
2021-08-12 22:13:29 +00:00
Pasindu Gunasekara 8d04598d33 Merged PR 616796: [Github Issue] Building bxl fails with missing VisualCppTools NuGet package
- Update MSVC package to 14.16.27034
- Update internal feed to use VisualCppTools.Internal.VS2017Layout from devdiv feed.
- Update external build instructions to get user to download visual studio build tools manually.
- Add Qspectre flag to msvc.
- Ignore some newer warnings being hit on windows sdk source files.

Related work items: #1846018
2021-07-14 21:21:37 +00:00
Pratik Lade 0f9f2ca816 Merged PR 612159: QTest: Update nuget to revert auto enabling trace logging
https://dev.azure.com/mseng/Domino/_git/CloudBuild/pullrequest/611690

Reverts auto enabling trace logging for heisenbug queues.
2021-05-07 18:05:14 +00:00
Lance Collins 15f742ffa0 Merged PR 612132: Add initial support for distributed "content metadata store"
Add initial support for distributed "content metadata store" (store which stores information currently stored in Redis i.e. content locations, memoization, and blobs) by using a gprc request to the master machine.
2021-05-07 02:24:18 +00:00
Juan Carlos Guzman Islas 24bb7c1551 Merged PR 612011: Upgrade ADO packages
Doing this while ignoring the new dependency `DotNetFxRefAssemblies.Corext`, which they added to add the `netstandard.dll` references we add during runtime.
2021-05-06 16:13:11 +00:00
Pratik Lade 2b4400588c Merged PR 609970: QTest: Update nuget
QTest: Update nuget
2021-04-23 22:59:30 +00:00
Juan Carlos Guzman Islas da885bff90 Merged PR 607192: Update ADO packages
Update ADO packages. Also, added a script because, for now, we still need to pull and push packages manually to our feed. Hopefully this can go away sometime in the near future.
2021-04-13 17:48:44 +00:00
Juan Carlos Guzman Islas 95ec4ae048 Merged PR 606254: Add microsoft-internal Microsoft.Caching.Redis NuGet package
Add microsoft-internal Microsoft.Caching.Redis NuGet package. This is a fork of the StackExchange.Redis repo. There should be no breaking changes, other than renaming the namespace `StackExchange.Redis` to `Microsoft.Caching.Redis`

Related work items: #1822916
2021-04-13 17:40:21 +00:00
Michael Pysson aa0231ee3c Merged PR 599382: Update Microsoft.Azure.Storage.Common to version matching ADO dependency update
Updates Microsoft.Azure.Storage.Common. Also reverts the LKG update in the BuildXL.Internal repo since that LKG contains the dependency issue which causes the cache to break
2021-02-23 21:51:37 +00:00
Wei Lim 18070f6f29 Merged PR 595461: Uptake latest ADO build
Related work items: #1806400
2021-02-19 21:57:41 +00:00
Suresh Thummalapenta d63d4f31c4 Merged PR 597177: Upgrading QTest version to 21.2.5.231426 2021-02-12 00:59:20 +00:00
Pratik Lade dccaaec1f1 Merged PR 595314: QTest: Add LogUploadMode option and update nuget to change parser name
1. Add qTestAzureDevOpsLogUploadMode option to dictate log upload behavior.
2. Update nuget to change parser name from Jest to JUnit.

JestParser has been renamed to JUnitParser for accuracy. This new nuget incorporates the change.
Builds using the value "Jest" as "QTestParserType" will have to replace the value to "Junit" for builds to pass. Currently we only have one customer who is running test builds, and they have been made aware of the change.

Validation run with newer BuildXL bits: https://dev.azure.com/mseng/Domino/_build/results?buildId=14224851&view=results
2021-01-29 20:09:40 +00:00
Pratik Lade 0db167a937 Merged PR 594290: QTest: Update Nuget. Use TestSourceDir to identify test job
Use TestSourceDir to identify test job
2021-01-22 20:49:15 +00:00