Component governance still complains about the NPM in node v17.1.0. I upgrade the node to v17.2.0, which has just released yesterday. That one contains the npm 1.8.4.
The main work was done by @<Pasindu Gunasekara 🍣> and I was integrating the new approach into our sdk.
This PR adds `LogGenerator` project that provides C# `SourceGenerator` for creating logs.
The benefit of this approach is that we don't need another compiler invocation and separate dlls are not produced if the project has generated logs. Instead those logs are produced at one step. So it should reduce the size of the output directories and speed up the self host build.
We don't change the way we generate logs, instead the solution is based on the existing log gen logic but it is called by the comipler.
**Considerations**
To make the transition simpler we now support two ways of generating logs via `generateLogs` argument and `generateLogsInProc`. The first one is the old one and the second one is the new one.
Also there is an environment variable that allows switching the mode globally from the old one to the new one.
Revert "Merged PR 636560: Add a helper library to extract SBOM Metadata from BuildSessionInfo to BuildX...
Add a helper library to extract SBOM Metadata from BuildSessionInfo to BuildXL.Utilities
Related work items: #1882251"
Reverted commit `c7a683ce`.
Related work items: #1882251
This should eliminate this warning in our various validation builds
`The file 'd:\dbs\el\bxlint\.CloudBuild\DominoDropConfig.json' is being used as a source file, but is not under a defined mountpoint. This file is thus 'untracked', and changes to it will not impact incremental builds`
- Update MSVC package to 14.16.27034
- Update internal feed to use VisualCppTools.Internal.VS2017Layout from devdiv feed.
- Update external build instructions to get user to download visual studio build tools manually.
- Add Qspectre flag to msvc.
- Ignore some newer warnings being hit on windows sdk source files.
Related work items: #1846018
The new packages contain updated native binaries for Linux which are compatible with a broader range of Linux distros.
The RocksDB package was updated in !617830
The core change is in `AzureEventHubClient.cs`, where we can now authenticate with an EventHub instance using a Managed Identity.
Related work items: #1826212, #1827238
Expose the analyzers support from the Bxl sdk.
Bxl supported Roslyn analyzers for a long time, but the list of supported analyzers was built in into Bxl Sdk.
This PR exposes `analyzers` properties and every project may add extra analyzers if needed:
```
@@public
export const dll = BuildXLSdk.library({
assemblyName: "BuildXL.Cache.ContentStore.Distributed",
sources: globR(d`.`,"*.cs"),
analyzers: [importFrom("protobuf-net.BuildTools").pkg]});
```
When moving or removing directory, Detours needs to enumerate it to report file writes on its members. During enumeration the path can go beyond MAX_PATH.
Tested on failed QuickBuild repo.
Related work items: #1836781
To get an absolute path for a file descriptor, we used to call `readlink("/proc/self/fd/{fd}")`. That's perfectly fine and still the best way to do it; the problem is that it is slow. (for example, it is not uncommon that a C compiler calls `putc` millions of times to produce a build output).
This PR implements a file descriptor table where where a mapping from a file descriptor to an absolute path is cached.
A new entry is computed and inserted into the table whenever a path for a file descriptor is needed but it hasn't already been computed.
Because file descriptors may be reused for different files over time, whenever a file descriptor is closed (i.e., the `close` syscall is intercepted) the corresponding entry in the table is invalidated.
Related work items: #1833259
Consider the following scenario:
1. Absent probe on path A/B/C/file.txt
2. Reparse point created on C in path A/B/C, where now A/B/C -> D/E/F and D/E/F/file.txt exists.
3. Present probe on path A/B/C/file.txt (by virtue of 2)
Step 1) populates the reparse point cache with A/B/C/file.txt as not needing resolution (since the path is absent). 2) should invalidate A/B/C, and now A/B/C does need resolution because C is a reparse point. But A/B/C/file.txt does not get invalidated, and therefore when 3) happens, A/B/C/file.txt is not resolved. This manifests in a cache miss when file.txt is a produced output, since we don't compensate for the absence of the file on cache lookup when the unresolved path is stored.
This PR adds:
* A conservative invalidation on reparse point creation (we don't detour the ioctl call, but we see a handle being opened for write and reparse point flags as the immediately previous operation). We invalidate the path in that case to stay on the safe side.
* A tracking mechanism for all descendants of a given path that the reparse point cache knows about. When invalidating a path, all descendants stored in the cache are also invalidated. This accounts for the above scenario, where when C is turned into a reparse point, both A/B/C and A/B/C/file.txt are invalidated
* A lightweight C++ unit test infrastructure (using boost) with unit tests for the new structure.
Related work items: #1824112
Semih Okur
Sergey Tepliakov
Julian Bayardo
Michael Pysson
Lance Collins
Pasindu Gunasekara 🍣
Marcelo Lynch 🧉
Oleksii Kononenko
Serge Mera
Rijul Luman
Aleksandar Milicevic
Kevin Bazan Williams
Iman Narasamdya
Juan Carlos Guzman Islas