Split the BCL project generation in two: one part that figures out which
projects to generate, and which is done synchronously (because we need it to
compute the list of tests), then split out the actual generation and run it on
a background thread (since that doesn't have to happen until we want to
execute those tests).
This speeds up launching xharness in server mode significantly (from ~2s to
~0.2s).
* [xharness] Don't generate into the same directory/files for macOS Modern and macOS Full.
This meant that we were overwriting some generated files, which meant that we
were executing the Modern set of tests instead of the Full set of tests for at
least some configurations.
* Add a few ignored tests to the System.Configuration test now that we actually run it.
When a process times out, we try to print a stack trace for all threads. This
involves executing "lldb --source <script>", where the script contains:
process attach --pid 1234
thread list
thread backtrace all
detach
quit
Basically we attach to the project, and ask lldb to print stack traces.
The problem:
16:09:02.9522580 Printing backtrace for pid=25276
16:09:02.9528060 /usr/bin/lldb --source /var/folders/q7/mkzwrzcn7bzf3g2v38f3c1cw0000gn/T/tmp58e75d85.tmp
16:09:04.6127570 (lldb) command source -s 0 '/var/folders/q7/mkzwrzcn7bzf3g2v38f3c1cw0000gn/T/tmp58e75d85.tmp'
16:09:04.6130020 Executing commands in '/var/folders/q7/mkzwrzcn7bzf3g2v38f3c1cw0000gn/T/tmp58e75d85.tmp'.
16:09:04.6130200 (lldb) process attach --pid 25276
16:09:05.6458750 error: attach failed: Error 1
16:09:05.7529100 25276 Execution timed out after 1200 seconds and the process was killed.
16:09:05.7588770 Execution timed out after 1200 seconds.
If any of those commands fail, the subsequent commands aren't executed. This
includes the final "quit" command, which means we end up waiting forever for
lldb to do its thing, when lldb doesn't think it needs to do anything at all.
The fix: pass two different scripts. It turns out subsequent scripts are
executed even if previous scripts fail, so we do the equivalent of:
lldb --source <attach script> --source <quit script>
And now lldb will quit no matter what the attach script does (it still works
even if the attach script succeeds, in which case we'll ask lldb to quit
twice).
This avoids:
* Getting the resource stream for every template expansion
* Reading the resource stream
* A lot of async code.
Instead just read the resource stream once, into a string, and return that.
This speeds up the startup by ~10% (from ~0.96s to ~0.83s on my machine).
Grouping simulators means we have to wait until we know which simulators are
available, because we group by the simulator's UDID.
When running in server mode, we don't need to group simulator test runs,
because we run the as the user chooses, not in any particular order.
This way the test list loads faster at startup, since we don't have to wait
until we've loaded the simulators before we can show the list.
This is slightly faster - ~0.95s vs ~1.4s - (probably because reflection tries
to load a lot of other referenced assemblies, which may or may not exist,
causing exceptions (if they don't exist) or spend time loading them (which
Cecil won't)).
It also avoids a lot of exception details showing up when tracing xharness
execution.
* [xharness] Add support for generating a tvOS version of .NET iOS projects.
And use it to run the tvOS version of introspection for .NET.
* [xharness] Change according to reviews.
They end up with the same make targets as non-.NET targets, which prints
warnings in the terminal when running make.
We're not using the makefile targets much anymore, so postpone implementing
them for .NET until we need them for some reason.
.NET projects are vastly simplified, which means that the OutputPath can't be
determined by reading the project file itself, it has to be calculated by
MSBuild.
So that's exactly what we do: we run MSBuild on the project file and get it to
print the property we're interested in.
.NET projects will include files from the current directory by default, which
means that if we clone the project file and write the cloned project file in a
different directory, we'll have to add an automatically included files into
the cloned file manually.
Add crash reports to a logs collection we care about, instead of to a logs
collection that's promptly forgotten.
This makes sure crash reports actually show up in the html report.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
This fixes this:
Unexpected exception: System.AggregateException: One or more errors occurred. (One or more errors occurred. (ApplicationName='[...]/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/Current/bin/mlaunch', CommandLine='--listsim=/var/folders/43/h027tm1n101cdrq2_b6n9n2m0000gn/T/tmp12c3d415.tmp --output-format=XML', CurrentDirectory='', Native error= Cannot find the specified file)) ---> System.AggregateException: One or more errors occurred. (ApplicationName='[...]/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/Current/bin/mlaunch', CommandLine='--listsim=/var/folders/43/h027tm1n101cdrq2_b6n9n2m0000gn/T/tmp12c3d415.tmp --output-format=XML', CurrentDirectory='', Native error= Cannot find the specified file) ---> System.ComponentModel.Win32Exception: ApplicationName='[...]/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/Current/bin/mlaunch', CommandLine='--listsim=/var/folders/43/h027tm1n101cdrq2_b6n9n2m0000gn/T/tmp12c3d415.tmp --output-format=XML', CurrentDirectory='', Native error= Cannot find the specified file
at System.Diagnostics.Process.StartWithCreateProcess (System.Diagnostics.ProcessStartInfo startInfo) [0x0029f] in /Users/builder/jenkins/workspace/build-package-osx-mono/2020-02/external/bockbuild/builds/mono-x64/mcs/class/System/System.Diagnostics/Process.cs:778
at System.Diagnostics.Process.Start () [0x0003a] in /Users/builder/jenkins/workspace/build-package-osx-mono/2020-02/external/bockbuild/builds/mono-x64/mcs/class/referencesource/System/services/monitoring/system/diagnosticts/Process.cs:2006
at (wrapper remoting-invoke-with-check) System.Diagnostics.Process.Start()
at Microsoft.DotNet.XHarness.iOS.Shared.Execution.ProcessManager.RunAsyncInternal (System.Diagnostics.Process process, Microsoft.DotNet.XHarness.iOS.Shared.Logging.ILog log, Microsoft.DotNet.XHarness.iOS.Shared.Logging.ILog stdout, Microsoft.DotNet.XHarness.iOS.Shared.Logging.ILog stderr, System.Nullable`1[T] timeout, System.Collections.Generic.Dictionary`2[TKey,TValue] environmentVariables, System.Nullable`1[T] cancellationToken, System.Nullable`1[T] diagnostics) [0x00469] in [...]/xamarin-macios/tests/xharness/Microsoft.DotNet.XHarness.iOS.Shared/Execution/ProcessManager.cs:185
at Microsoft.DotNet.XHarness.iOS.Shared.Hardware.SimulatorLoader+<>c__DisplayClass16_0.<LoadDevices>b__0 () [0x00107] in [...]/xamarin-macios/tests/xharness/Microsoft.DotNet.XHarness.iOS.Shared/Hardware/SimulatorLoader.cs:65
at Microsoft.DotNet.XHarness.iOS.Shared.Hardware.SimulatorLoader.LoadDevices (Microsoft.DotNet.XHarness.iOS.Shared.Logging.ILog log, System.Boolean includeLocked, System.Boolean forceRefresh, System.Boolean listExtraData) [0x00155] in [...]/xamarin-macios/tests/xharness/Microsoft.DotNet.XHarness.iOS.Shared/Hardware/SimulatorLoader.cs:54
at Microsoft.DotNet.XHarness.iOS.Shared.Hardware.SimulatorLoader.FindOrCreateDevicesAsync (Microsoft.DotNet.XHarness.iOS.Shared.Logging.ILog log, System.String runtime, System.String devicetype, System.Boolean force) [0x0023c] in [...]/xamarin-macios/tests/xharness/Microsoft.DotNet.XHarness.iOS.Shared/Hardware/SimulatorLoader.cs:156
at Microsoft.DotNet.XHarness.iOS.Shared.Hardware.SimulatorLoader.FindSimulators (Microsoft.DotNet.XHarness.iOS.Shared.TestTarget target, Microsoft.DotNet.XHarness.iOS.Shared.Logging.ILog log, System.Boolean create_if_needed, System.Boolean min_version) [0x00267] in [...]/xamarin-macios/tests/xharness/Microsoft.DotNet.XHarness.iOS.Shared/Hardware/SimulatorLoader.cs:277
at Xharness.Jenkins.TestTasks.RunSimulatorTask.FindSimulatorAsync () [0x0005a] in [...]/xamarin-macios/tests/xharness/Jenkins/TestTasks/RunSimulatorTask.cs:53
at Xharness.Jenkins.Jenkins.CreateRunSimulatorTasksAsync () [0x004ee] in [...]/xamarin-macios/tests/xharness/Jenkins/Jenkins.cs:558
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional (System.Boolean includeTaskCanceledExceptions) [0x00011] in /Users/builder/jenkins/workspace/build-package-osx-mono/2020-02/external/bockbuild/builds/mono-x64/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2027
at System.Threading.Tasks.Task`1[TResult].GetResultCore (System.Boolean waitCompletionNotification) [0x0002b] in /Users/builder/jenkins/workspace/build-package-osx-mono/2020-02/external/bockbuild/builds/mono-x64/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs:496
at System.Threading.Tasks.Task`1[TResult].get_Result () [0x00000] in /Users/builder/jenkins/workspace/build-package-osx-mono/2020-02/external/bockbuild/builds/mono-x64/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs:466
at Xharness.Jenkins.Jenkins+<>c__DisplayClass72_0.<PopulateTasksAsync>b__0 (System.Threading.Tasks.Task`1[TResult] v) [0x0001e] in [...]/xamarin-macios/tests/xharness/Jenkins/Jenkins.cs:941
at System.Threading.Tasks.ContinuationTaskFromResultTask`1[TAntecedentResult].InnerInvoke () [0x00024] in /Users/builder/jenkins/workspace/build-package-osx-mono/2020-02/external/bockbuild/builds/mono-x64/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskContinuation.cs:155
at System.Threading.Tasks.Task.Execute () [0x00000] in /Users/builder/jenkins/workspace/build-package-osx-mono/2020-02/external/bockbuild/builds/mono-x64/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2319
--- End of stack trace from previous location where exception was thrown ---
Automatically detect which .NET version to use based on the global.json reachable from the project directory.
Due to how things are designed, some parts of the code wants the dotnet
executable before the code in question knows the project directory, which
isn't possible anymore. So there's a minor change to pass around a lambda that
can calculate the path to the executable instead of the executable itself.
Use both in those labs that support xamarin-storage so that we have a
backup. To do so, move the parameter to be a bool but keep the rest of
the logic the same.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
This is particularly important when we have global.json and NuGet.config in
the project directory, because those are looked up starting with the current
directory, so those two need to match.
Vsdrops does not support serving a static html. Therefore we need to use
full uris that will be used to download the logs. To make things less
dangerous, we leave the xamarin-storage report as it was and create a
new one for vsdrops.
This means that:
1. xamarin-storage index.html is left as is.
2. vsdrops_index.html contains full uris to download (the env var will
have to be set in the step) and js and css are in the header.
3. because we use and env var, jenkins won't generate the
vsdrops_index.html only device pipelines will.
For this to take effect needs updates in the device pipelines. The
solution is not yet optimal since we need to add some workaround to
rather than make the monitoring person open a text file, we should
display it in the browser.
Co-authored-by: Chris Hamons <chris.hamons@xamarin.com>
In order to be able to test the TestSelectro in PR
https://github.com/xamarin/xamarin-macios/pull/8768 we need a more
general interface to mock and ensure that the correct properties are set
by the TestSelector.
* Fix links that point to master to point to main instead.
* Implement support in the sample tester for specifying the default branch for
each sample repo.
* Fix various text / documentation to say 'main' instead of 'master.'
* Push to 'main' instead of 'master' in xamarin-macios-data.
* Fix xharness to make 'main' the special branch with regards to documentation tests as opposed to 'master'.
* Fix various CI to use 'main' instead of 'master'.
* Bump maccore
New commits in xamarin/maccore:
* xamarin/maccore@ed6d146822 Rename 'master' to 'main'. (#2233)
Diff: 424fa26148..ed6d146822
Note: affected tools are not included in the code shipping inside the SDK
Usage of deprecated cryptographic algorithms is not approved anymore,
even if not used for cryptographic purpose.
MD5 was used to create immutable GUID since it's output is 128bits which
is just what a GUID wants as it's input (16 bytes). The same can be
achieved (even if a bit slower) with a newer/longer hash function
https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1128148
Note: affected tools are not included in the code shipping inside the SDK
Usage of deprecated cryptographic algorithms is not approved anymore,
even if not used for cryptographic purpose.
MD5 was used to create immutable GUID since it's output is 128bits which
is just what a GUID wants as it's input (16 bytes). The same can be
achieved (even if a bit slower) with a newer/longer hash function
https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1128148
* [tests] Add a unit test project to test our net5 support.
* [tests] Fix clearing environment variables when launching processes.
* [tests] Add net5 macOS test app.
* [tests] Add net5 tvOS test app.
* [tests] Add net5 watchOS test app.
* [msbuild] Exclude CreateAppBundleDependsOn from net5 builds as well.
* [msbuild] We're not required to know the signing identity to figure out the app extension bundle name.
Move all the logic of the server to its own class to later be able to
add tests in Jenkins (make sure that we do run it) and in the server
itself (check that files are server, correct actions are executed etc..)
At this point Jenkins just creates the tests tasks via other objects and
executes in server mode or not depending on the env.
One step closer to make the Jenkins class just know how to spin the
tasks and what tests are selected. This new class, once we can have a
clean Jenkins class will be testeable.
The grouping of the tasks was moved to the html report. This meant that
the markdown was not getting all the correct tasks. Move the grouping of
the tasks out of the hml report and use it with the markdown so that
both reports have the same data.
Also fixes an issue when tests failed and did not appear correctly in
the comment such as in comment: https://github.com/xamarin/xamarin-macios/pull/8698#issuecomment-635006243
The failures are aggregated tasks, we need to get the, use the key as
the name of the test and use the rest of the data from the ITestTask.
The reason is that the ITestTask.Name was modified by Jenkins.cs to
group them.
fixes: https://github.com/xamarin/xamarin-macios/issues/8695
This avoids confusion when tests are run locally using 'make runner' (and
would print the simulator's stdout/stderr to the terminal) versus when run on
bots (and would write the simulator's stdout/stderr to separate files that
would show up in the html report).
Add crash reports to a logs collection we care about, instead of to a logs
collection that's promptly forgotten.
This makes sure crash reports actually show up in the html report.
This is a step towards trying to make the modification of the html
easier for other team members, the are just to small logical changes:
1. Removed the GenerateReportImpl to a HtmlReportWriter.
2. Do not make the HtmlReportWriter call the Marckdown one, the are
independent.
There is room for improvement, since there are some collections
re-calculated. That change will come once we have this out of the way.
At some point, we ought to be able to make changes in the html just be
as hard as they should be (html + css, we are not experts on that).
The knowledgebase was just added in the simulator case, not in the
device. That meant that the tcp error was not reported.
Also added another possible case of tcp errors to be picked up and be
shown as a known error (device throws a diff error when on airplane
mode).
fixes: https://github.com/xamarin/xamarin-macios/issues/8659
Following other PRs, try to reduce the work of the Jenkins class to
orchestrate the different classes and move any other logic to classes
with a single concern. Add tests for the new class.
Also switch xharness to build the csproj instead of running the makefile to
build the tests, because that way xharness is able to automatically use the
correct NUnit runner depending on the NUnit version the tests are using.
* [xharness] Don't use the MSBuild xml namespace when processing sdk-style projects.
The sdk-style projects don't use MSBuild's xml namespace, which means that it
will be added to every node we create if we use it.
* Fix merge conflict resolution failure.
Move the generation of the markdown out and add tests. Make sure that
the report writes the expected results and specially that it shows the
known issues.
Create some enumerables that will state the tests to run, that way, if
new tests need to be added, there will not be a need to edit the
jenkins class.
Add tests to ensure that all the tests are added. That away we will make
sure that tests are not removed by accident.
Try to make the Jenkins class do a single task. Move the generation of
the variations logic out (which is hard to test atm, but will be doable
in a second round).
Moved some useful methos also out and add tests.
This makes it possible to select what should be done on CI when building a
commit on internal Jenkins (as opposed to when building a pull request, in
which case labels can be set on the pull request).
The GetReader from the install log is broken and requires a re-eng.
Rollback to using the fullpath, not as nice, but that way we do not have
exceptions.
We are using issue #8545 to track the actual bug in the logs when we
request a Reader, if the reader cannot be created we should not be
throwing an exception, whats more, that reader should not be null since
otherwise we would not get the install logs.
The change in xamarin/xamarin-macios/@617777ae73ccefdddd5711fe9628243b399492be
forgot to ignore the use of the tunnel in some cases. Is not the root of
the exception, but is the root of the tests failing after that commit.
fixes: https://github.com/xamarin/maccore/issues/2217
Move the test selection logic out. Tests will land after we refactor the
Github class, which we depend on, to not be static, so that we can test
the different paths.
* [Harness] Small refactor to get the knowledge base out of jenkins.
Some small changes to make things a little easier to understand:
* Add a new method in case we find known install issues.
* enable nullable, lets go step by step.
* Move logic outside Jenkins.cs
* Add tests! \o/
* [Harness] Add support to create tunnels.
Add support to create tunnels in case the devices cannot connect to
the host. This option is false by default, which means that unless told
otherwise xharness will try to se a tcp connection over the WiFi.
We are not setting it as default because the devices in DDFun will have
access to an unrestricted network. Nevertheless after this PR we will
create a new one with the following logic:
1. Try to use the tcp connection using the network.
2. If devices cannot connect to the host via the network, fall back to
the tcp tunnel.
This change executes a tunnel process per test application, most of the
cases out of the 150 test application we execute, most of them (maybe 98%
but most % are made up) will pass, and just a few of them will fail. The
reason is that there is a daemon in the OS that gets underwater.
Rather than tryingt o find a hacky way to re-use the tunnel, lets KISS
and if we need to hack that, do it as an enhancement.
The netstandard2.0 version has been the default for a few weeks now, and no
problems have been found, so just delete the net461 version.
This speeds up testing a bit, since we won't be testing the net461 version
anymore.
Moved all the code that can be shared with the CLI to the common
library. We neede some small changes, but mainly due to namespaces and a
forgotten interface implemenation (was already implemented, was missing
in the class).
This was fun :)
Refactor TestProject so that we can move all the tasks to the common
assembly. We had to remove all the references from Harness, that
included the MonoNativeInfo.
Move all the logic outside and use it as a Composition pattern, later
this class can be used in the CLI so that we share the logic of building
and tested.
In order to de-couple the RunTestTask from Jenkins, create an interface
to be implemented, which is pass to use as a member (Composition
pattern). In order to do that, do not expose Jenkins as a property of
the interface because it is required just by the base constructor.
Moving the the use an interface meant a lot of small changes that
should have no real effect (the compiler should have caught any possible
issues).
PR https://github.com/xamarin/xamarin-macios/pull/8184 removed the
inheritance with TextWriter, therefore the `as` will return null and we
will not generate the Html report. In this particular case, we do not
need an ILog, we just use it to get a path to the correct location,
therefore, we can create the file using the full path and pass it to the
xslt.
Fixes: https://github.com/xamarin/xamarin-macios/issues/8364
The initial idea of the refactoring looked nice but as soon as we want
to get the RunTestTask out of jenkins, we have a number of naming
issues. Move the tools to not use the *Task postfix so that it is
cleaner and we can later extra the RunTestTask better.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>
PR https://github.com/xamarin/xamarin-macios/pull/8184 removed the
inheritance with TextWriter, therefore the `as` will return null and we
will not generate the Html report. In this particular case, we do not
need an ILog, we just use it to get a path to the correct location,
therefore, we can create the file using the full path and pass it to the
xslt.
Fixes: https://github.com/xamarin/xamarin-macios/issues/8364
Move all the logic outside of the Jenkins namespace. Rework a little the
inheritance to make it nicer in the constructors.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>
Move the task and use composition so that we can reuse the code. This
will later allow other projects to use the class without the need of
Jenkins or Harness and just implement the base class.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>
dotnet doesn't like projects within projects in the file system, because of
default inclusion behavior, where the top project will want to include the
source files for the nested projects.
There are ways around that, but the easiest is to just make sure there aren't
projects within other project directories.
There is a difference between mono and .netcore with regards to environment
variables when launching processes.
For the following example:
process.StartInfo.EnvironmentVariables ["FOO"] = null;
.netcore will launch the process with an empty FOO variable, while mono will
launch the process with no FOO variable set.
So unstead remove the variable from the collection of environment variables
(this works fine even if the collection doesn't contain the variable).
Ref: https://github.com/dotnet/runtime/issues/34446
Some of the Jenkins test tasks are very useful and do A LOT of stuff. So
we try to generalize the base class, to later be able to share the most
usebul ones in the shared lib.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Threading is hard, part 2. We could have a situation in which two tasks
that are instantiated async use the same id. The id is later used to
build the logs directory etc.. When they share the id we end up in a
situation in which the logs overlap.
fixes: https://github.com/xamarin/xamarin-macios/issues/8146
The testing framework dlls come from mono and their paths are harcoded
in the templates. Move the hardcoded paths out to the assembly locator
which know were to find the paths.
This is needed to later allow the cmd to pass the location of the paths.
The assets were being used from the bcl-test directory. Move them as
part of the template. Renamed a file to make things a little easier when
adding the assets (a '.' in the middle of the name made this
complicated).
The template should be selfcotained.
- Code that will later be moved to the `dotnet/xharness` repo is first moved to an isolated project before it will be extracted to the new repo and NuGetified
- New project for the shared code and new test project for tests accompanying the moved code are created
- Only Factories are left behind that are used only by XHarness
The Generator and the Factory classes are a xamarin-macios thing.
Initially, they were separated because the code that generated the bcl
tests was not inside xharness. That is not longer the case. We can merge
both classes, generalize the namespace and be more prepared to move out
of the xamarin-macios repo.
Threading is hard, even with async. We reached a situation in which the
src code of the test applications from the template was being generated
more than once. Ideally, we could have a nice solution with
AsyncLazy<bool> and use the lazy async to copy the code, but it was
moved to netcore 5. The fix is a little uglier but valid, lock and
enure we only generate it once. Please note that you CANNOT use a lock
statement inside a async method.
Fixes: https://github.com/xamarin/xamarin-macios/issues/8240
Remove two classes that are not really needed and a method. We moved to
named tuples to make things simpler. Unfortunally C# does not support:
```csharp
using MyNamedTuple = (string Name, double value, bool answer);
```
But we have to live with this. Makes the namespace simpler and removes
confusion with the already present project classes that are more widely
used.
* [Harness] Remove the BCLTestProjectInfo class.
The class is not really needed. Move to a tuple, make the definition of
the group test project nicer since the dictionary initiallization makes
it cleaner.
Looks like we have cases in which we get empty nodes when listing
devices, which makes the parsing of the nodes fail and crash.
Move the code to a diff method, try to parse the data, if there is an
exception (missing data) return null. If null is returned, do not add
it.
Fixes: https://github.com/xamarin/xamarin-macios/issues/8160
Looks like mono is not investing much time at the moment fixing
https://github.com/mono/mono/issues/18560
The above error happens always on 32b device tests. Added a new known
failure for the monitoring to use in their task so that they do not have
to goole the issue and they know that is knonw already.
Added a test in the xml parser that verifies that the parsing of the xml
is correct and that the correct number of failures is reported in the
human readable form.
fixes: https://github.com/xamarin/xamarin-macios/issues/8214
Remove two classes that are not really needed and a method. We moved to
named tuples to make things simpler. Unfortunally C# does not support:
```csharp
using MyNamedTuple = (string Name, double value, bool answer);
```
But we have to live with this. Makes the namespace simpler and removes
confusion with the already present project classes that are more widely
used.
Refactor result parsing and test it. Make sure that all the possible
corner cases are taken into account. The changes are as follow:
1. Move all the logic to a new class, which will take care of the reustl
parsing.
2. Simplify the parsing method by splitting the noise code to other
functions. There is no need to parse several files in the same method.
3. Add tests that make sure that the correct result is returned when:
a. The app cannot launch.
b. The app timesout.
c. The app crashes.
First step to try and clean up the psring of results and ensure that we
are doing the correct thing. We have several issue there. We want to
move all the complicated logic out of AppRunner and test the different
outputs and the full result matrix.
This commit first allows to use an instance class for the parsing, later
we will move things out. Step by step
XHarness TestAssemblyDefinition is very particular on the way we deal
with the mono test assemblies, in the general case, it only contains the
assembly name, but in the xamarin-macios it calculates the name of the
assembly using the platform.
Create an interface so that the command line was its own implementaiton
and move as much as possible to use the AssemblyLocator class. Later we
can add two simpler implementations for the more general cases.
Move the AssemblyLocator out of the Templates namespace since it makes
more sense.
Initially the location of the test dlls of the downloaded artefact and the compiled mono were different. That was no longer the case and therefore there was a lot of code that could be removed since the wasDownloaded variable was set to be always true.
We remove the not needed code that will calculate a diff path if mono was compiled and use the AssemblyLocator class to point to the correct root path were test dlls will be found.
Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>
Rolf Bjarne Kvinge
Manuel de la Pena
Sebastien Pouliot
monojenkins
Přemek Vysoký
Alex Soto