Reverting the revert, plus handling interactive auth for local builds:
* build.cmd 'primes' the auth token cache by launching the specified cred provider (for internal+interactive builds)
* The cred provider used by bxl is allowed to read/write to the auth token cache. This implies dealing with the IsRetry pattern.
I'll have somebody (besides me) check that the local case works before merging.
Same commit as before plus:
* Force check the nuget package fp on disk (apparently this was off by default because dotnet core didn't support nuget at that time) (that should fix the DFA we saw, which is a layout difference between a public vs internal package when an agent gets reused)
* Capture stderr when executing the cred provider, for better error diagnostics.
From the issues we saw, there is still one that I was not able to repro (https://dev.azure.com/mseng/Domino/_build/results?buildId=17296398&view=logs&j=cfa20e98-6997-523c-4233-f0a7302c929f&t=144c797d-20c9-5763-9672-cca1a82cd366). I'll keep trying, but the stderr should be able to tell us more if that happens again. Hopefully we can spot this before I merge this.
Revert "Merged PR 661589: Reapply: nuget resolver schedules real pips"
This reverts commit 53d8270164.
This change is still causing some breaks in downstream validation
Reverting the revert to merge the original PR: eebb9c136a?refName=refs%2Fheads%2Fdev%2Fsmera%2Frelease509.1.1
Plus, a couple minor tweaks to make this work for linux and mac. Those changes are grouped in a separate commits so they are easier to spot.
In parallel I'm working on the rolling build cred provider configuration changes so this PR can pass.
Revert "Merged PR 659816: Nuget resolver schedules real pips
Purpose: to unblock rolling builds.
This PR changes the NuGet resolver logic so instead of downloading packages at evaluation time, download pips are scheduled so the package download and extraction happens as part of the build. The high-level flow is the following:
- The nuget resolver downloads enough of each NuGet package just to understand its layout (usually <5k) + the nuspec file. This is taking ~10s on my machine. The inspection process is still using the local + bxl cache caching layer, in the same way the fully downloaded packages were using before. So hits are also possible.
- The spec generation process now generates calls into the (internal) nuget downloader SDK. The specs are still generated on disk. We can consider doing this in memory (as other resolvers do), but that can be left for a second iteration.
- A new nuget downloader tool is now deployed as part of bxl binaries, with a small DScript SDK that schedules it. The generated specs use it. Nuget.exe is not used anymore, the nuget api is used instead. This makes the whole process less prone to machine-depending state that nuget.exe reads by default and doesn't allow us to block.
Related work items: #1942615"
Reverted commit `eebb9c13`.
Related work items: #1942615
This PR changes the NuGet resolver logic so instead of downloading packages at evaluation time, download pips are scheduled so the package download and extraction happens as part of the build. The high-level flow is the following:
- The nuget resolver downloads enough of each NuGet package just to understand its layout (usually <5k) + the nuspec file. This is taking ~10s on my machine. The inspection process is still using the local + bxl cache caching layer, in the same way the fully downloaded packages were using before. So hits are also possible.
- The spec generation process now generates calls into the (internal) nuget downloader SDK. The specs are still generated on disk. We can consider doing this in memory (as other resolvers do), but that can be left for a second iteration.
- A new nuget downloader tool is now deployed as part of bxl binaries, with a small DScript SDK that schedules it. The generated specs use it. Nuget.exe is not used anymore, the nuget api is used instead. This makes the whole process less prone to machine-depending state that nuget.exe reads by default and doesn't allow us to block.
Related work items: #1942615
Serge Mera
Michael Pysson
Iman Narasamdya