- Make the public validation public end-to-end, by building a public engine and then using that to run tests (also without passing `--internal`)
- Use the 1ESPT build job for the distributed build test. We're still keeping the old YML for backup, but it becomes unused.
- Note that using 1ESPT for our own selfhost is not possible due to how we invoke `bxl.sh`, that's why that one still adds the worker stage manually.
- Adds two new npmrc files:
- .internal-npmrc can be used with cloudbuild or vsts-npm-auth to generate a token locally
- .ci-npmrc is used for ADO builds using an environment variable to store the token
- vsts-npm-auth does not support non-Windows platforms, so we need to manually do what it does on bxl.sh
- TODO: once merged, enable npm authentication on CloudBuild for bxl PR and rolling queues.
Related work items: #2115724
Introduced a flag to enable/disable explicit setting of the execute permissions bit for the root process in linux builds.
Added a condition to explicitly set this bit for node and dotnet in the extractor.
This is done to obtain more information about the linux permissions bug.
Related work items: #2104538
Add salt to both Windows and Linux PR to address a cache poisoning issue
----
## AI-Generated Description
This change adds a new condition to the **bxl.ps1** and **bxl.sh** scripts that checks if the user-provided arguments contain the `/p:BUILDXL_FINGERPRINT_SALT` parameter. If not, it adds this parameter with the value `casingPR` to the arguments passed to the BuildXL executable. This is done to force a salt for the cache fingerprinting, because a previous PR introduced some casing issues that polluted the cache. This change can be removed once the poisoned content is evicted from the cache.
Add a --release option to bxl.sh. This works as an orthogonal configuration option for all the existing build options (e.g. ' --minimal --release' , or '--deploy-dev --release'. Default is still debug.
Make sure we produce the same special sandbox event on both Windows and Linux when a source rewrite is detected. Enable related unit tests for Linux as well.
Changes:
- add a new NuGet dependency on `Microsoft.Applications.Events.Server` (which is the cross-plat Aria SDK)
- use that on Linux and Mac
- delete the native SDK previously used on Mac
Changes:
- fix various file name capitalization errors
- fix various nuget package name capitalization errors
- create and publish `Bond.CSharp.linux-x64` nuget package
- add `/etc` to default untracked scopes
- untrack `$HOME/.dotnet` when running the `Downloader` tool
- consistently spell `App.config`
With these changes, all of the following succeed for me in Ubuntu 20.04 WSL running in Windows 11:
- `./bxl.sh --minimal`
- `./bxl.sh --minimal --internal`
- `./bxl.sh --minimal --internal --shared-comp`
Once the changes make it to LKG, the next step will be to set up a Linux pipeline to build minimal selfhost. Later, that pipeline can be expanded to run unit tests etc.
Various small changes that allow building selfhost with BuildXL on Linux.
What's missing:
- downloading nugets
- remote telemetry
- bond.csharp tool (`gbc`) for Linux (needs to be built from sources and published as a new nuget)
- rush stuff
If (1) nugets are first downloaded manually, and (2) `gbc` is manually replaced with a Linux binary, building minimal selfhost on Linux works.
As of now, to run tests on Mac one must sequentially do the following:
1. run a BuildXL build that only compiles all test assemblies and creates a *standalone* test deployment
1. run another BuildXL build from the previously created standalone test deployment that actually executes the tests
Some drawbacks of this approach:
1. have to run 2 steps in sequence
1. tests that are executed on Mac must be explicitly opted-in (see [tests.osx.dsc](https://dev.azure.com/mseng/Domino/_git/BuildXL.Internal?path=%2FPublic%2FSrc%2FDeployment%2FTests.Osx%2Ftests.osx.dsc&version=GBmaster&line=15&lineEnd=15&lineStartColumn=14&lineEndColumn=27&lineStyle=plain))
- this makes it easy to forget to opt in any newly added test assemblies
1. impossible to execute just a single test class or test method at once
1. some SDKs (e.g., `xunit.dsc`) are duplicated between the main build and the build used for executing standalone test deployments.
The main reason for this setup was the fact that the main build and the unit tests themselves might require different versions of the MacSandbox kext to be loaded.
This PR makes it possible to compile and execute tests in a single build, matching the experience on Windows. For example, running
```bash
./bxl.sh --internal
```
on a Mac will execute the full selfhost build, including executing all the tests. Individual tests can still opt out of Mac (by using the existing mechanisms like `[(Class|Theory|Fact)IfSupported[requiresWindowsBasedOperatingSystem: true]`).
We already have different IDs for the release and debug configurations of the MacSandbox kext, which means we can already have different versions of the kext loaded as long as one is a 'release' and the other is 'debug'. That makes it sufficient for the most common scenario of using a 'release' version of BuildXL to run a build in which tests are executed in 'debug' configuration. It is also possible to execute both 'release' and 'debug' tests in a single build, but for that, a dev version of BuildXL must be built first and then used for subsequent test execution.
**Important Note**: the user is still responsible for ensuring that the necessary kexts are loaded (this needs to be done only when the kext changes; once loaded, the kexts don't need to be touched/reloaded between builds/test executions). To load the debug version of the kext:
```bash
./bxl.sh --minimal # produces out/bin/debug/osx-x64/*
out/bin/debug/osx-x64/bxl.sh --load-kext
```
Main changes:
- annotate Windows-only tests as such (so that they are skipped over when running on Mac)
- fix various small platform-dependent issues (like using an appropriate directory separator char)
- fix DScript SDK tests
- add `--test-method` and `--test-class` switches to `bxl.sh` to correspond to `-TestMethod` and `-TestClass` in `bxl.ps1`
Once this change is merged, I'll update the PR validation pipelines and remove the code for producing the standalone test deployment.
Merged PR 526181
The new LKG contains changes to how nuget/cgmanifest.json is generated. To avoid breaking our validation, the LKG and the generated nuget/cgmanifest.json must be updated together (this is a one-off manual update).
* Adjust tooling to work out of the box with the latest public LKG on macOS
* Update the distributed test commit hash for the CI pipeline Part B
* Only add the deployment root on non-windows runs
* Restore all script exec permissions, update Readme.md with better macOS instructions
* Always run sandboxed when building internal
* Exec permission adjustment for all scripts
Pasindu Gunasekara 🍣
Oleksii Kononenko
Marcelo Lynch 🧉
Sahiti Chandramouli
Serge Mera
Michael Pysson
Aleksandar Milicevic
Kristijan Simic
Kristijan Šimić
James Swineson
dannyvv
Kristijan Šimić
Microsoft Open Source