Also switch xharness to build the csproj instead of running the makefile to
build the tests, because that way xharness is able to automatically use the
correct NUnit runner depending on the NUnit version the tests are using.
* [xharness] Don't use the MSBuild xml namespace when processing sdk-style projects.
The sdk-style projects don't use MSBuild's xml namespace, which means that it
will be added to every node we create if we use it.
* Fix merge conflict resolution failure.
Move the generation of the markdown out and add tests. Make sure that
the report writes the expected results and specially that it shows the
known issues.
Create some enumerables that will state the tests to run, that way, if
new tests need to be added, there will not be a need to edit the
jenkins class.
Add tests to ensure that all the tests are added. That away we will make
sure that tests are not removed by accident.
Try to make the Jenkins class do a single task. Move the generation of
the variations logic out (which is hard to test atm, but will be doable
in a second round).
Moved some useful methos also out and add tests.
This makes it possible to select what should be done on CI when building a
commit on internal Jenkins (as opposed to when building a pull request, in
which case labels can be set on the pull request).
The GetReader from the install log is broken and requires a re-eng.
Rollback to using the fullpath, not as nice, but that way we do not have
exceptions.
We are using issue #8545 to track the actual bug in the logs when we
request a Reader, if the reader cannot be created we should not be
throwing an exception, whats more, that reader should not be null since
otherwise we would not get the install logs.
The change in xamarin/xamarin-macios/@617777ae73ccefdddd5711fe9628243b399492be
forgot to ignore the use of the tunnel in some cases. Is not the root of
the exception, but is the root of the tests failing after that commit.
fixes: https://github.com/xamarin/maccore/issues/2217
Move the test selection logic out. Tests will land after we refactor the
Github class, which we depend on, to not be static, so that we can test
the different paths.
* [Harness] Small refactor to get the knowledge base out of jenkins.
Some small changes to make things a little easier to understand:
* Add a new method in case we find known install issues.
* enable nullable, lets go step by step.
* Move logic outside Jenkins.cs
* Add tests! \o/
* [Harness] Add support to create tunnels.
Add support to create tunnels in case the devices cannot connect to
the host. This option is false by default, which means that unless told
otherwise xharness will try to se a tcp connection over the WiFi.
We are not setting it as default because the devices in DDFun will have
access to an unrestricted network. Nevertheless after this PR we will
create a new one with the following logic:
1. Try to use the tcp connection using the network.
2. If devices cannot connect to the host via the network, fall back to
the tcp tunnel.
This change executes a tunnel process per test application, most of the
cases out of the 150 test application we execute, most of them (maybe 98%
but most % are made up) will pass, and just a few of them will fail. The
reason is that there is a daemon in the OS that gets underwater.
Rather than tryingt o find a hacky way to re-use the tunnel, lets KISS
and if we need to hack that, do it as an enhancement.
The netstandard2.0 version has been the default for a few weeks now, and no
problems have been found, so just delete the net461 version.
This speeds up testing a bit, since we won't be testing the net461 version
anymore.
Moved all the code that can be shared with the CLI to the common
library. We neede some small changes, but mainly due to namespaces and a
forgotten interface implemenation (was already implemented, was missing
in the class).
This was fun :)
Refactor TestProject so that we can move all the tasks to the common
assembly. We had to remove all the references from Harness, that
included the MonoNativeInfo.
Move all the logic outside and use it as a Composition pattern, later
this class can be used in the CLI so that we share the logic of building
and tested.
In order to de-couple the RunTestTask from Jenkins, create an interface
to be implemented, which is pass to use as a member (Composition
pattern). In order to do that, do not expose Jenkins as a property of
the interface because it is required just by the base constructor.
Moving the the use an interface meant a lot of small changes that
should have no real effect (the compiler should have caught any possible
issues).
The initial idea of the refactoring looked nice but as soon as we want
to get the RunTestTask out of jenkins, we have a number of naming
issues. Move the tools to not use the *Task postfix so that it is
cleaner and we can later extra the RunTestTask better.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>
PR https://github.com/xamarin/xamarin-macios/pull/8184 removed the
inheritance with TextWriter, therefore the `as` will return null and we
will not generate the Html report. In this particular case, we do not
need an ILog, we just use it to get a path to the correct location,
therefore, we can create the file using the full path and pass it to the
xslt.
Fixes: https://github.com/xamarin/xamarin-macios/issues/8364
Move all the logic outside of the Jenkins namespace. Rework a little the
inheritance to make it nicer in the constructors.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>
Move the task and use composition so that we can reuse the code. This
will later allow other projects to use the class without the need of
Jenkins or Harness and just implement the base class.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>
dotnet doesn't like projects within projects in the file system, because of
default inclusion behavior, where the top project will want to include the
source files for the nested projects.
There are ways around that, but the easiest is to just make sure there aren't
projects within other project directories.
There is a difference between mono and .netcore with regards to environment
variables when launching processes.
For the following example:
process.StartInfo.EnvironmentVariables ["FOO"] = null;
.netcore will launch the process with an empty FOO variable, while mono will
launch the process with no FOO variable set.
So unstead remove the variable from the collection of environment variables
(this works fine even if the collection doesn't contain the variable).
Ref: https://github.com/dotnet/runtime/issues/34446
Some of the Jenkins test tasks are very useful and do A LOT of stuff. So
we try to generalize the base class, to later be able to share the most
usebul ones in the shared lib.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Threading is hard, part 2. We could have a situation in which two tasks
that are instantiated async use the same id. The id is later used to
build the logs directory etc.. When they share the id we end up in a
situation in which the logs overlap.
fixes: https://github.com/xamarin/xamarin-macios/issues/8146
Manuel de la Pena
Rolf Bjarne Kvinge
Přemek Vysoký