This allows the CI to run ALL the tests that the project has in
parallel. This is divided in two main changes:
1. Xharness - We move away from using boolenas to use a flag that states
the tests to run.
2. yaml - We have move the code to use a template per label. This new
jobs all run in parallel and the results are later collected by a
funel job
3. pwsh - Added a new class that understands that we have several mark
downs with the tests results. The classes parses them and them writes
a single comment (and example can be found here: https://github.com/xamarin/xamarin-macios/pull/15201#issuecomment-1162366240
The changes gives the following advantages vs how we used to run tests:
1. The CI run for all tests moves from taking 13 hours to 3/4 hours
(depending on the number of bots in the pool).
2. The download needed to verify the results on a case of failure is
smaller. Rather than downloading several GBs we now just download
that part of the html that we are interested in.
3. Better bot utlization. Bots are just used to a max of 2 hours, this
means that we can use the bots better since they are fragmented.
4. Less VMs. VSDrops has added support for macOS and Linux, we take
advanges of that here.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
This makes it not necessary to check for the currently selected Xcode in our
system dependency check. It also means it'll become much easier to work with
multiple branches simultaneously where each branch needs its own Xcode.
Make our local .NET the default .NET (in the root's global.json), and then if
a directory wants to use the system .NET, then that directory would have to
opt-in (using its own global.json).
This way we don't have to copy global.json/NuGet.config files around to run
tests with the correct .NET setup.
Ask ditto to thin native libraries and frameworks when copying them to the app
bundle to remove slices for architectures we're not building for.
Also add tests.
Fixes https://github.com/xamarin/xamarin-macios/issues/13081.
Co-authored-by: TJ Lambert <50846373+tj-devel709@users.noreply.github.com>
Also rename DOTNET_VERSION to SYSTEM_DOTNET_VERSION to make it clear what it's
referring to (and to not clash with DOTNET6_VERSION which has now been renamed
to DOTNET_VERSION).
.NET 7 is right around the corner.
This makes it line up with our other make variables (SYSTEM_MSBUILD,
SYSTEM_MONO, SYSTEM_CSC, etc.), and also prepares for removing the version
from the DOTNET6 variable (so that it becomes just DOTNET).
The advantage of writing the label after each test is that the test result will also
be printed, which means it's possible to see if any tests failed during the test
run, as opposed to having to wait until the entire test run is completed (which can
take a while) to realize that pretty much every test failed with some silly mistake
which could have been quickly fixed before re-running the tests.
Verifies that:
* There shouldn't be any (IntPtr) or (IntPtr, bool) constructors.
* The (NativeHandle) or (NativeHandle, bool) constructors should not be public.
* The constructors correctly chain to the base constructor according to our usual
pattern: the (IntPtr) ctor must chain to the self/base (IntPtr, bool) constructor
passing 'false' for the owns parameter.
* The constructors don't have any extra code in them (barring a few exceptions).
This makes it easier to iterate over all the *_SDK_VERSION variables in
template code, because they're all named using the standard platform names we
use elsewhere.
We either use the html web page, or for .NET tests there are custom makefile
logic.
This avoids having to maintain the makefile generation to keep up with our new
tests, and it fixes numerous warnings like this:
Makefile-mac.inc:56: warning: overriding recipe for target 'build-mac-modern-macOS'
Makefile-mac.inc:41: warning: ignoring old recipe for target 'build-mac-modern-macOS'
Makefile-mac.inc:59: warning: overriding recipe for target 'clean-mac-modern-macOS'
Makefile-mac.inc:44: warning: ignoring old recipe for target 'clean-mac-modern-macOS'
Makefile-mac.inc:62: warning: overriding recipe for target 'exec-mac-modern-macOS'
Makefile-mac.inc:47: warning: ignoring old recipe for target 'exec-mac-modern-macOS'
Makefile-mac.inc:65: warning: overriding recipe for target 'run-mac-modern-macOS'
Makefile-mac.inc:50: warning: ignoring old recipe for target 'run-mac-modern-macOS'
On CI we'll collect all the binlogs in the repository and make them available
for post-build analysis if need be, so this will make it easier to diagnose
build problems.
* Remove a lot of dead code.
* Use TestBase.CreateTask<T> to create tasks (and set required properties for all
tasks) instead of instantiating tasks directly. This required subclassing TestBase
in a few places, as well as making a few helper methods instance methods instead
of static methods.
* Bump to net472.
* A few other misc simplifications.
Split the iOS msbuild tests in two:
* Xamarin.MacDev.Tasks.Tests: contains in-process unit tests for tasks.
* Xamarin.MacDev.Tasks: contains larger tests that either invoke targets or a complete
build. These are currently in-process, but will become out-of-process soon to make
it possible to run them with dotnet.
Also make the new projects non-iOS-specific, because the macOS msbuild tests will
be moved here as well soon.
There is some duplicated code between these two test projects now (all files
that show up as new are copies of existing files), this will be cleaned up in
later pull requests.
* [msbuild] Move msbuild/tests to tests/msbuild to put all the tests together.
* [tests] Move test projects for Xamarin.Mac to tests/common/TestProjects
* [tests] Move test projects for Xamarin.iOS to tests/common/TestProjects
Make the bgen tests pass in the path to the attribute library, platform
assembly and all the .NET reference assemblies to bgen. This way we execute
these tests using the .NET version of everything.
Automatically detect which .NET version to use based on the global.json reachable from the project directory.
Due to how things are designed, some parts of the code wants the dotnet
executable before the code in question knows the project directory, which
isn't possible anymore. So there's a minor change to pass around a lambda that
can calculate the path to the executable instead of the executable itself.
Also add a very simple net5 test app (which doesn't build yet).
Current (expected) build result:
> xamarin-macios/tests/dotnet/MySingleView $ dotnet build
Microsoft (R) Build Engine version 16.7.0-preview-20258-02+26f6d1d87 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.
Determining projects to restore...
xamarin-macios/tests/dotnet/MySingleView/MySingleView.csproj : error NU1102: Unable to find package Microsoft.NETCore.App.Runtime.ios-x64 with version (= 5.0.0-preview.6.20264.1)
xamarin-macios/tests/dotnet/MySingleView/MySingleView.csproj : error NU1102: - Found 1 version(s) in nuget.org [ Nearest version: 5.0.0-preview.4.20251.6 ]
xamarin-macios/tests/dotnet/MySingleView/MySingleView.csproj : error NU1102: - Found 1 version(s) in Nuget Official [ Nearest version: 5.0.0-preview.4.20251.6 ]
xamarin-macios/tests/dotnet/MySingleView/MySingleView.csproj : error NU1102: - Found 0 version(s) in local-dotnet-feed
xamarin-macios/tests/dotnet/MySingleView/MySingleView.csproj : error NU1102: - Found 0 version(s) in Dotnet arcade
Failed to restore xamarin-macios/tests/dotnet/MySingleView/MySingleView.csproj (in 951 ms).
This fails because the .NET build logic isn't being told that it should look
for the mono runtime packs instead of the .NET Core runtime packs.
Create the various NuGet packages to support .NET 5+. The packages are
currently empty (and not very useful), but the actual content will come later.
The current set of NuGet packages are (this list is duplicated for each
platform: iOS, tvOS, watchOS and macOS):
* Microsoft.iOS.Sdk: currently contains the basic MSBuild targets files for an
MSBuild Project SDK. Will eventually contain all the build logic. Might also
eventually contain other tools (mlaunch, bgen, etc.), but these might also
end up in a different package.
* Microsoft.iOS.Ref: will contain the Xamarin.iOS.dll reference assembly.
* Microsoft.iOS.Runtime.[RID]: will contain architecture-specific files
(libxamarin*.dylib, the Xamarin.iOS.dll implementation assembly, etc.):
The NuGets built on CI are automatically published to a NuGet feed.
The versioning for the NuGet packages required a few changes: OS bumps are now
changed in Make.versions instead of Make.config (this is explained in the
files themselves as well).
Rolf Bjarne Kvinge
Manuel de la Pena
Alex Soto
Sebastien Pouliot