- fix getting/setting extended attributes on Linux
- untrack `/proc`, `/sys`, `/lib/x86_64-linux-gnu` folders on Linux
- handle `mkdir`, `mkdirat`, and `fdopendir`
- ignore operations on file descriptors that are not files
- deny file accesses when so configured
- generic tracing of all intercepted syscalls
Related work items: #1706995
- eagerly call `dlsym` to resolve all sycall symbols upfront (instead of lazily once we are already handling a syscall)
- call the original syscall implementation before reporting accesses (helps avoid potential deadlocks when there are multiple libraries intercepting syscalls)
- capture the value of `errno` right after calling the original implementation, and restore it right before returning
Related work items: #1706182
Move to fluent assertions when the message is constructed.
Looking into this, I don't really like the new API as well.
Few other ideas:
```cshar
Contract.AssertWith(x > 0)?.Message($"x > 0, x=[{x}]");
Contract.AssertThat(x > 0)?.WithMessage($"x > 0, x=[{x}]");
Contract.RequiresThat(x > 0)?.IsTrue($"x > 0, x=[{x}]");
```
Related work items: #1689124
- update RocksDB nuget to a new version that is more portable than the previous one
- hack for choosing the correct SQLite.Interop.dll when running on Linux
- implement some more interop stuff for Linux
- update RocksDB to a newer version that contains native binaries for Linux
- enable KeyValueStore unit tests on Linux
- delete the `statx` p-invoke from `IO.Linux.cs` because it doesn't work with Ubuntu 18.04
This PR:
- Adds support for using NLog
- Adds an NLog target that logs to Azure Blob Storage
- Adds support for using host-provided values for telemetry
- Integrates the changes with the DistributedService verb
- Integrates the changes with CloudBuild-side telemetry (see associated PR in CB repository: https://dev.azure.com/mseng/Domino/_git/CloudBuild/pullrequest/532118?_a=overview)
Note that this is under a "feature flag", so nothing should fail out of the box. Ingestion delay has been roughly 5 minutes as far as I have tested.
Currently WIP:
- Improve upload strategy (currently creates too many small files!). Kusto recommends between 100MB and 5GB per ingestion
Missing features (might/will be tackled in future PRs):
- Split Message into CorrelationId, Role, Component, Operation, Arguments, Duration, Result and Exception columns (likely hard / very long work item because of legacy tracing code)
- Report ConfigurationId (should be easy)
- Support multiple Azure Blob targets. This could be useful for routing/duplicating output to different tables depending on runtime conditions.
Related work items: #1662908, #1669623
In public builds, skip downloading the `Redis-osx-x64` nuget on all platforms.
This will create an empty stub for this package, meaning we won't be able to run any Redis unit tests in public builds (which is fine)
Merged PR 529351: Update Grpc and Protobuf versions
Update Grpc and Protobuf versions.
Checked out the codegen and the generated code is almost identical:
* A few case changes of members in the generated protobuf cs file
* Comment changes
* Service classes now have a BindServiceMethod attribute
* The generated class now takes a `ChannelBase` rather than `Channel`
* A new overload of `serviceBinder.AddMethod` is used but passing the same data.
Updated XNU security header files to latest available, adjusted project / compilation settings to compile without issues and getting dependencies via our DownloadResolver.
Revert "Merged PR 529351: Update Grpc and Protobuf versions"
This reverts commit 46f0589b8e.
Cloudubild tried to deploy 2 times and it seems there are too many nuget dependency problems with version mismatches of transitive dependencies... I don't have the bandwidth to keep hinting this down as all the validations run seem to pass.
Update Grpc and Protobuf versions.
Checked out the codegen and the generated code is almost identical:
* A few case changes of members in the generated protobuf cs file
* Comment changes
* Service classes now have a BindServiceMethod attribute
* The generated class now takes a `ChannelBase` rather than `Channel`
* A new overload of `serviceBinder.AddMethod` is used but passing the same data.
Two main changes:
* CodeAnalysis classes claim to always give back absolute paths, but that's not the case. Manually resolve relative paths before reporting accessess
* Handle the case of switches for csc/vbc that are supported by the compiler in use but not for the VBCS logger.
Make sure augmented paths get canonicalized as absolute paths, similarly to what detours does. Some tools may report relative paths, or paths containing '..'.
Additionally, add two missing inputs for VBCSCompiler (sorry, this should have been a separate PR, but the change is small enough)
* Update RuntimeContracts to support non-nullable types.
* Roll back nuget end point
* Enable non-nullable refernece types in cache interfaces.
* Fix pragma
* Exclude non-nullable attributes for .NET Core.
This avoids generating package-lock.json upon every build; given that npm install without a lock file is not necessarily deterministic, every build can in theory generate a new package-lock.json. Instead, when the lock file is explicitly provided, the behavior of npm install is fixed and deterministic.
* Initial version of content placement analyzer
* added core project
* Added contentplacement.core and contenplacementanalyzer
* build download
* half of the pipeline is done
* First piece of the pipeline is done
* Pipeline [1]: Build download and sampling done
* Addressing PR comments
* Initial version of ContentPlacementAnalysisTools.ML
* WIP: ML pipeline 60%
* pipline 65%
* Pipeline 80%
* (WIP) tree parser
* (WIP) Evaluating random forest
* (WIP) 99% ML backend
* Done
* Tested and working
* removing csproj file
* fixed missing comments
* Adding scripts for extraction
* remove references to resx file
* Done with readme files
* Fix merge
* Refactor
* Add RocksDbBackingStore
* Move Store to Distributed
* Initial version of content placement analyzer
* added core project
* Added contentplacement.core and contenplacementanalyzer
* build download
* half of the pipeline is done
* First piece of the pipeline is done
* Pipeline [1]: Build download and sampling done
* Addressing PR comments
* Initial version of ContentPlacementAnalysisTools.ML
* WIP: ML pipeline 60%
* pipline 65%
* Pipeline 80%
* (WIP) tree parser
* (WIP) Evaluating random forest
* (WIP) 99% ML backend
* Done
* Tested and working
* removing csproj file
* fixed missing comments
* Adding scripts for extraction
* remove references to resx file
* Done with readme files
* Fix merge
* Refactor
* Add RocksDbBackingStore
* Move Store to Distributed
* WIP integrate blob
* Fix contract exception
* WIP download blob
* First version of OfflineMapper
* Compress and Uncompress
* Inital working version
* Added flag to distinguish CBTest from Prod
* Squash to be compatible with CodeFlow
AB#1574994
AB#1574153
AB#1574993
Populated the rest of the XLG events, and updated the protobuf file accordingly. Now the DB is written to immediately rather than let the events buffer up in memory first.
New classes:
- CsvFileLog: an ILog class that writes log messages to a CSV file and formats them according to a given schema
- KustoUploader: a class responsible for asynchronously receiving CSV files and ingesting them into a Kusto table
* Start implementation of BuildXL VFS cas process. Currently just proxies to a backing content server. Next step is to actually add virtualization.
* PR feedback.
* Only VFS build on full framework.
* PR feedback.
SDKs for CoreRT native compilation on macOS.
For now, only AppHostPatcher is compiled to native (and that executable is subsequently used to patch binaries throughout the rest of the build).
Future PRs can enable compiling other executables to native.
Aleksandar Milicevic
Danny van Velzen
Sergey Tepliakov
John Erickson
Serge Mera
Julian Bayardo
Juan Carlos Guzman Islas
Lance Collins
Kristijan Simic
Michael Pysson
Kristijan Šimić
Iman Narasamdya
Rijul Luman
Sergey Teplyakov
Mihai Codoban
Kevin Bazan Williams
Sahil Gandhi
David Federman
Semih Okur
Annie Fu
dannyvv
Microsoft Open Source