Fixes#8684Fixes#8273
Context
After #8275, we delete any destination file as part of the Copy task if we determine that we really should copy onto it. Unfortunately, if we try to copy a file onto itself, we delete it before we can copy onto itself, which just means it's gone. Fortunately, we have a check earlier that ensures that we skip any copy operation from a location to the same location. Unfortunately, it's a direct string comparison that doesn't evaluate to full paths first, so it misses slightly more complicated examples.
Changes Made
Take into account full paths
Testing
Unit tests + manual test that it doesn't delete the file anymore
Notes
This implementation tries to remove now-unnecessary full path derivations downstream, hence some added complexity, but it still means extra computation on the happy path if we end up creating a hard/symbolic link. An alternate direction eliminating any full path derivations on the happy path would save about 4% of Copy's execution time, per a quick perf test. (With how many samples I used, "no change" is within a standard deviation.)
Fixes#8634
Context
For SDK/workload references, the SDK is currently already passing relevant metadata such as AssemblyVersion and PublicKeyToken, so there is no need for RAR to open these files and parse their .NET metadata tables to get this information. Also, barring corrupted installs, these files are guaranteed to exist on disk so there is no need for RAR to do any I/O on them (file-exists, get-time-stamp, ..)
Changes Made
Tweaked RAR to trust the item metadata passed by the SDK. The feature is gated on the presence of the FrameworkReferenceName metadatum which is not documented and is used only by the relevant SDK tasks, to my best knowledge.
SDK does not specify the Culture component of assembly names so it's assumed to be neutral. This has passed all relevant validation so far. If non-neutral assemblies are ever used as references, we'll need to define a metadatum for SDK to set.
Testing
Existing and new unit tests.
Experimental insertion with an assert that assembly names calculated from item metadata are equivalent to those extracted from assembly files (verifies that the Culture=neutral assumption is correct).
Checked all assemblies specified in all FrameworkList.xml files shipped in the SDK (+ workloads) and verified that all of them are culture neutral.
Perf results
RAR micro-benchmark where the task is invoked with parameters corresponding to building a simple AspNetCore app:
13.13 ms -> 3.27 ms per invocation without StateFile cache.
15.03 ms -> 5.13 ms per invocation with StateFile cache.
RAR task duration as reported with /clp:PerformanceSummary when building a simple AspNetCore app with MSBuild server enabled:
20 ms -> 10 ms.
Notes
The change is behind a 17.8 changewave.
No changes have been made to per-project caching (via the StateFile parameter) to reduce scope. Changes to the per-project cache file will be done in another PR.
Fixes#8671
Context
New layout as agreed offline
specs (content remains same, we'll move here rar-core-scenarios from documentation/design folder)
|-----proposed (the secrets metadata and packages sourcing PRs would get routed here. In future - all in-progress work specs)
|-----archive (the rar-as-service.md from documentation/design will get moved here. In future - all not dismissed but not planned feature spec goes here)
Moving facilitated via git mv to preserve history and make diffs conscise
At least one person skimmed over the section we wanted to emphasize (`-bl`) and focused on `MSBUILDDEBUGENGINE`, sharing lower-fidelity logs that are harder to understand.
Remove the "Preferred way" callout--it's preferred in that section but not in general. Add a section header for command-line builds. Add some samples there.
Relates to:
#4795#8405
Context
Documenting some aspect of tailoring behavior of references. Capturing as well resolution/workarounds for the listed comunity reported issues
---------
Co-authored-by: Rainer Sigwald <raines@microsoft.com>
System.Security.Principal.Windows is inbox since net6.0
System.Net.Http is inbox since netcoreapp2.0
System.Reflection.Metadata is inbox since netcoreapp2.0
System.Threading.Tasks.Dataflow is inbox since netcoreapp2.0
Remove System.Net.Http package references which aren't needed as they underlying assembly is inbox on both .NETFramework and .NETCoreApp.
By avoiding the dependencies, we minimize the dependency graph and with that the attack surface.
cc @MichaelSimons (removes netstandard1.x dependencies)
This document aims to capture the core functionality provided by the ResolveAssemblyReference task when building .NET (Core - pun intended) projects. The goal is to rationalize and optimize the task, ultimately achieving substantially better performance and crossing out RAR from the list of notoriously slow build tasks.
Fixes#8441
* Clarify what .default means
.default (in MSBuild code) followed an earlier spec that referred to it as <default> and omitted some details. This clarifies the meaning of .default.
* Replace <default> with .default
Previously, when we saw the Item was empty, we'd jump out early. Fortunately, we already special-cased Count, and it turns out we can do the same with AnyHaveMetadataValue. Fixes#5113.
Fixes#6303
Related: KirillOsenkov/MSBuildStructuredLog#653
Context
Assembly loading logging for:
Task runs - including loads in separate app domain for AppDomainIsolatedTask
Except for tasks runs in OutOfProcessRaskHostNode - as neither LoggingService nor LoggingContext are available there - not sure if not supporting those is a problem.
Sdk resolving (explicitly skipping resolvers defined in Microsoft.Build assembly)
Loggers initialization (explicitly skipping resolvers defined in Microsoft.Build assembly orwithin Microsoft.Build.Logging namespace) - Note - loggers initialiation reordered - Binlog is the first one - as assembly load events are emited at the time of each logger initialization, currently LoggerService dispatches events immediately after construction, so as a result order of logger definitions matter (only the earlier initialized gets messages about latter ones initialization assembly loads). Alternative would be to change the LoggerService contract to start dispatching on explicit request (easy to implement and preserving current MSBuild behavior - however might be potentially breaking for API users?) or to cache those events and log them later (doable, but convoluted - probably not worth the efforts now)
Evaluation
Samples
task run:
Assembly loaded during TaskRun: System.Diagnostics.Debug, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a (location: C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Diagnostics.Debug\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Diagnostics.Debug.dll, MVID: bc6b825d-5f99-464e-a06d-e3ae4b860a34, AppDomain: [Default])
evaluation:
Assembly loaded during Evaluation: System.Collections.Immutable, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a (location: C:\Program Files\dotnet\shared\Microsoft.NETCore.App\7.0.2\System.Collections.Immutable.dll, MVID: 5a4c54a3-2092-428e-89cc-a391fd9d398a, AppDomain: [Default])
logger initialization:
Assembly loaded during LoggerInitialization (MyLoggers.BasicFileLogger): System.Reflection.Metadata, Version=7.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a (location: C:\Program Files\dotnet\shared\Microsoft.NETCore.App\7.0.2\System.Reflection.Metadata.dll, MVID: c602a8ce-3890-40fc-aa9b-fa3c23f81aab, AppDomain: [Default])
custom sdk resolver (no sample - similar to logger initialization)
Perf impact
No observable impact.
MSBUILDLOGALLASSEMBLYLOADS is used to turn on loggin of all assembly loads
build command: msbuild.exe <proj> /nodeReuse:false /bl
Scenario Mean Duration
Orchard - MSBuild main 00:00:59.25
Orchard - curr branch 00:00:58.30
Orchard - curr branch, all loads 00:00:58.30
Console - MSBuild main 00:00:01.08
Console - curr branch 00:00:01.09
Console - curr branch, all loads 00:00:01.10
Context
As discussed on triage - binlogs captured through VS project system extension tooling are less preferrable to logs captured via environment opt-in. Let's make it more explicit in our docs that we are often linking externally to our customers.
Fix other broken links.
Fixes#8438
Context
The primary branch for msbuild (and a number of other projects) has been migrating from 'master' to 'main'. A number of links reference the old 'master' branch name and are broken.
Changes Made
This PR updates those that have changed to 'main', along with updating a couple of other broken links.
Testing
Relying on the CI tests :)
Notes
It's my first MSBuild PR, apologies if there I've missed anything!
Enriched transcript of a talk with @rainersigwald about MSBuild nodes orchestration and scheduling.
I added some extra interpretations and links that were not voiced in the talk - so I'll be happy for review for corectness of the doc.
Context
The docs are hard to navigate
Changes Made
Added README.md and updated the table of contents in documentation folder
https://github.com/dotnet/msbuild/blob/main/documentation/specs/rar-as-service.md - moved to design folder
merged 2 pages on SDK resolvers
"debugging on Mono" is moved to deprecated and not included to contents
* Added FancyLogger
Fixes #
Context
For the average user, the current ConsoleLogger does not provide the required amount of information about the build, as it either shows too little information (with low verbosity levels) or too much information that it is very difficult to find (with high verbosity levels).
A better approach would be having the log update to show the necessary amount of information for the current performing action (be it a project, target or task) and hiding additional data for actions that completed successfully; thus reducing the amount of irrelevant info.
Likewise the addition of formatting (bold, italics, color) using ANSI escape codes provides the user with a much better experience.
A progress tracker for the build is also considered.
This PR focuses only on the opt-in/out mechanism for the feature.
Changes Made
Created a new FancyLogger class.
Created a ANSIBuilder static class with helper methods for ANSI formatting and graphics
Added the /fancylogger and /flg command line switches for enabling the FancyLogger
Added the $MSBUILDFANCYLOGGER environment variable for enabling the FancyLogger
Testing
Notes
Co-authored-by: Rainer Sigwald <raines@microsoft.com>
Fixes#5773
Context
Put a property inside a target without an enclosing PropertyGroup and build. Error informing me that is an invalid child element
Changes Made
Add one condition when parse the element under target. If the element has inner text and no other child elements except text, then this should be a property and throw invalid child element, the error is "MSB4067: The element <{0}> beneath element <{1}> is unrecognized. If you intended this to be a property, enclose it within a <PropertyGroup> element"
Tests
ReadInvalidPropertyUnderTarget
* Change all of the inferences to bootstrap except on bootstrapped_msbuild.sh files, though it's likely also not needed there
* Remove the tautology
* Remove suggestion to install .net sdk
* Mention MSBUILDDEBUGENGINE in Tips&Tricks
Some folks were looking for this and found this page, but wound up IMing me because DEBUGENGINE wasn't here.
* Casing!
- Changed http:// links to https:// where that was the case (a handful
of http:// links remain), but all changed ones were checked
- Where GitHub links changed from `master` to `main`
- Where source lines were referenced it now points to a given revision,
because pointing to a branch name means the line number may be
incorrect. NB: so this was very much intentional!
- Linked to the archived page for dead links
- Updated old MSDN and docs.microsoft.com and blog links to point to the
current location (even if it's the archive on learn.microsoft.com)
- Remove en-us from canonical links to learn.microsoft.com ...
Co-authored-by: Rainer Sigwald <raines@microsoft.com>
Mitigates https://github.com/dotnet/msbuild/issues/2064
Context
This is an alternate way to fix the same problem. This is incomplete: it would still need to replace "dwproj" with the !projectsKnownToBeMSBuild and add a warning.
Getting type information for a task that was supposed to be loaded in a TaskHost involves loading that assembly today. The point of TaskHost nodes is that they don't lock the assembly, and they disappear quickly, releasing the lock from when we actually need to load the assembly to execute it. This will mean that we don't lock the assembly to read its type information, actually complying with the user's likely intent.
If the user asks for a TaskHost, use MetadataLoadContext to get its information rather than loading the assembly fully.
Fixes#6461
Jan Krivanek
Rainer Sigwald
Forgind
Ladi Prosek
Jared Parsons
Viktor Hofer
Dmitriy Shepelev
Ladi Prosek
adr26
Vlada Shubina
Eduardo Villalpando Mello
Jan Krivanek
Eduardo Villalpando Mello
Jenny Bai
Noah Gilson
Ben Villalobos
Forgind
Oliver Schneider
Shreyas Jejurkar
AR-May
MichalPavlik