Граф коммитов

3596 Коммитов

Автор SHA1 Сообщение Дата
Manuel de la Pena 39629ab93b
[Harness] Remove old files that are not longer used. (#8424)
The bcltest dir should only have the .ignore files, the code generator
creates all the different projects and those files were left behind when
they should have been removed.

* The info.plist is needed. This was added to issue: https://github.com/xamarin/xamarin-macios/issues/8426
2020-04-18 12:33:34 -04:00
Manuel de la Pena 7a2573a99b
[Harness] Move all common code to the shared lib. (#8389)
Moved all the code that can be shared with the CLI to the common
library. We neede some small changes, but mainly due to namespaces and a
forgotten interface implemenation (was already implemented, was missing
in the class).

This was fun :)
2020-04-17 18:14:55 -04:00
Manuel de la Pena cdee7bbeb6
[Tests] Remove flacky tests from AppKit. (#8422)
Remove the tests, they give problems in headless servers and in old
versions of Mac OS X.
2020-04-17 11:51:44 -04:00
Rolf Bjarne Kvinge da5c5a0ce6 [tests] Remove the mcs variation of tests that run with both mcs and csc.
There's no need to test mcs code anymore.
2020-04-17 13:52:28 +02:00
Manuel de la Pena e0b6ff480c
[Harness] Refactor TestProject to be moved to the common lib. (#8388)
Refactor TestProject so that we can move all the tasks to the common
assembly. We had to remove all the references from Harness, that
included the MonoNativeInfo.
2020-04-16 20:20:53 -04:00
Manuel de la Pena 430ce42e44
[Tests] Ignore failing tests in older versions of Mac OS X. (#8409)
Something must be different between the virtual screen created in a
headless system in older versions of Mac OS X and 10.15. Everything
works on 10.15 older versions throw an exception.

fixes: https://github.com/xamarin/xamarin-macios/issues/8408
2020-04-16 15:03:07 -04:00
Manuel de la Pena a6dcd40c8b
[Harness] Refactor RunDeviceTask. (#8386)
Move most/all of the logic to a helper class so that it later can be
used in the CLI.
2020-04-16 10:25:24 -04:00
Rolf Bjarne Kvinge e4f8460277
[mmp/mtouch] Share more code. (#8394)
* Rearrange files in Xamarin.Mac a bit to ease code sharing between mmp and
  mtouch, by putting mono's static and dynamic libraries in
  /Library/Frameworks/Xamarin.Mac.framework/Versions/Current/Sdks/Xamarin.macOS.sdk
  to match how Xamarin.iOS does it.

* Don't use 'usr' as an intermediate directory. This removes another special
  case.

* Share many of the functions and properties that return specific directories,
  and document (as comments) what each function/property is supposed to
  return.
2020-04-16 14:19:45 +02:00
Rolf Bjarne Kvinge 8feb3c29e6
[mtouch] Handle a failure to launch the native linker better by showing better error messages. (#8391)
* [mtouch] Handle a failure to launch the native linker better by showing better error messages.

dotnet will throw a Win32Exception if the command line is too long, so handle
that scenario. Also handle any other Win32Exceptions and show a better error
message.

* Make MT5217 an error to avoid multiple potentially confusing errors.
2020-04-16 14:18:43 +02:00
Manuel de la Pena a036e6e998
[AppKit] Ignore failing test on older OS X versions. (#8397)
The API should be present, yet we have an exception thrown when using
the selector. On 10.15 we do not have issues, so we skip in older
versions. Might be related to the fact that we use headless mac minis.

fixes: https://github.com/xamarin/xamarin-macios/issues/8395
2020-04-15 18:25:48 -04:00
Rolf Bjarne Kvinge e6f64d7b38
[tests] Add script to launch manual sample test run for the current commit. (#8381) 2020-04-15 08:01:59 +02:00
Manuel de la Pena 969d5d92ed
[Harness] Generalize the RunTestTask to allow it to be used in the CLI. (#8380)
Move all the logic outside and use it as a Composition pattern, later
this class can be used in the CLI so that we share the logic of building
and tested.
2020-04-14 18:39:58 -04:00
Manuel de la Pena bbffb308e8
[Harness] Donet doest not need to restore. (#8383)
Removed line due to a bad merge, re-add it.
2020-04-14 15:03:12 -04:00
Manuel de la Pena dbc89f01a4
[Harness] Create IBuildToolTask and use it in the RunTestTask. (#8375)
In order to de-couple the RunTestTask from Jenkins, create an interface
to be implemented, which is pass to use as a member (Composition
pattern). In order to do that, do not expose Jenkins as a property of
the interface because it is required just by the base constructor.

Moving the the use an interface meant a lot of small changes that
should have no real effect (the compiler should have caught any possible
issues).
2020-04-14 13:11:55 -04:00
Rolf Bjarne Kvinge 511124f4b1
[mmp] Explicitly resolve assemblies from the GAC / system mono. (#8377)
Cecil has a fall-back mode where it looks in the GAC / system mono for
assemblies when failing to find them elsewhere. This is not the expected
behavior when using Xamarin.Mac in the Full/XM mode, because then we should
only resolve to assemblies shipped with Xamarin.Mac.

Unfortunately doing so will break apps (our own tests break), so instead
change our resolution to be explicit about where we find assemblies, and if we
find assemblies in the GAC / system mono when we're not supposed to, then show
a warning.

Also add a fall-back mechanism, where we use the old logic instead, in case
the new logic is not 100% compatible with the old one.

This showed up when I tried to port mmp to dotnet, because then Cecil stopped
looking in the GAC / system mono for assemblies (Cecil has a special case when
running on Mono to look in Mono's GAC), and tests started failing.
2020-04-14 16:32:42 +02:00
Manuel de la Pena c150679630
[Harness] Refactor certain class names to simplify the extraction of the RunTestTask. (#8372)
The initial idea of the refactoring looked nice but as soon as we want
to get the RunTestTask out of jenkins, we have a number of naming
issues. Move the tools to not use the *Task postfix so that it is
cleaner and we can later extra the RunTestTask better.


Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>
2020-04-14 08:04:35 -04:00
Manuel de la Pena 98d21d742c
[Harness] Ensure that the generator tests are set when we modify the sources. (#8371)
fixes: https://github.com/xamarin/xamarin-macios/issues/8363

Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
2020-04-14 07:58:24 -04:00
Manuel de la Pena 5e0131db27
[Harness] Fix HML report for NUnitTestTasks. (#8367)
PR https://github.com/xamarin/xamarin-macios/pull/8184 removed the
inheritance with TextWriter, therefore the `as` will return null and we
will not generate the Html report. In this particular case, we do not
need an ILog, we just use it to get a path to the correct location,
therefore, we can create the file using the full path and pass it to the
xslt.

Fixes: https://github.com/xamarin/xamarin-macios/issues/8364
2020-04-13 13:51:48 -04:00
Manuel de la Pena 01d2b62097
[AppKit] Ignore test in earlier versions of Mac OS X. (#8357)
The api is present since Mac OS X 10.11 but we get an exception when
executing it in earlier versions of Mac OS X, it just works on 10.15.

Fixes: https://github.com/xamarin/xamarin-macios/issues/8356
2020-04-13 12:21:06 -04:00
Rolf Bjarne Kvinge 00d39b9b6d
[tests] NugetRestore should only restore, not build. (#8360)
"msbuild /restore" will run nuget restore, then build. "msbuild /t:Restore"
will just run the Restore target, which should just restore.

This becomes significant when we later try to do "msbuild build", and expect
warnings to show up. If we previously built the project unintentionally, those
warnings won't show up because nothing will actually be built.
2020-04-13 17:44:17 +02:00
Rolf Bjarne Kvinge 665b20a44c
[tests] Fix location of aot.cs in mmptest.csproj (#8361) 2020-04-13 16:08:15 +02:00
Rolf Bjarne Kvinge f3ee011797
[xharness] Automatically generate xharness make dependencies. (#8324) 2020-04-13 09:26:55 +02:00
Manuel de la Pena 7ac67e6e33
[AppKit] NSScreen properties can be executed in non-UI threads. (#8345)
Allow all the properties to be executed outside the UI thread. This has
been tests with Xcode using MTC.
2020-04-10 19:25:56 -04:00
Manuel de la Pena 382f2ad997
[Tests] Bump language used in the mac binding tests. (#8348)
* [Tests] Bump language used in the mac binding tests.

They broke with nullability, we need to use the latests lang version.

Fixes: https://github.com/xamarin/xamarin-macios/issues/8346
2020-04-10 19:25:30 -04:00
Manuel de la Pena a2f57b38d6
[Harness] Generalize MSBuildTask and DotnetBuildTask. (#8306)
Move all the logic outside of the Jenkins namespace. Rework a little the
inheritance to make it nicer in the constructors.

Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>
2020-04-10 16:50:50 -04:00
Sebastien Pouliot 8d3987b84a
[xtro][bindings] Add xtro support for nullability and fix some bindings (#8335)
Enabling this will ensure that future bindings (and Xcode updates that
change nullability information) are spotted right away.

The binding fixes are **not** complete, i.e. what was done was mostly
to debug the xtro rule and find corner cases. The backlog will be
_ignored_ so the builds won't fail.
2020-04-10 11:00:47 -04:00
Manuel de la Pena 4621b467df
[AppKit] Allow static properties to execute on non-UI threads. (#8330)
VSMac has failing tests when they query the NSScrees.Screens property
which the following swift code shows that it can be executed in a diff
thread:

```swift
import Cocoa
import AppKit

DispatchQueue.global(qos: .background).async {
    print("This is run on the background queue")

    print(Thread.current)
    var screens = NSScreen.screens
    print (screens.count)
}
```

Fixes: https://github.com/xamarin/xamarin-macios/issues/8329
2020-04-09 10:24:16 -04:00
Sebastien Pouliot a5ac0ea102
[generator] Add C#8 nullability into generated code (#7570)
Goals
* Reflect Apple nullability annotations in our bindings using C#8
* No warnings when building bindings

Non-Goals
* Update (add or fix) `[NullAllowed]` to match Apple headers (next phase)
* Make the generator or internal code fully nullable aware (`nowarn` is used)

Notes
* Apple's own annotations are not 100% accurate :(
* Where known issue exists we have _fixed_ our attributes to match reality :)
* We also do additional null-checks internally that might seems not required (better safe than sorry).
2020-04-09 09:29:28 -04:00
Manuel de la Pena e668545555
[Harness] Generalize the BuildProjectTask (#8299)
Use composition to generalize the BuildProjectTask to later reuse it in
the CLI.

Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>
2020-04-08 20:09:10 -04:00
Manuel de la Pena 5b99859321
[Harness] Move BuildToolTask to a more general namespace. (#8287)
Move the task and use composition so that we can reuse the code. This
will later allow other projects to use the class without the need of
Jenkins or Harness and just implement the base class.



Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>
2020-04-08 15:03:33 -04:00
Přemek Vysoký db921689a9
[Harness] Remove launchTimeout parameter from TestReporter, move factories (#8322) 2020-04-08 11:32:47 -04:00
Rolf Bjarne Kvinge 4b917ad2de
Create a general rule to create makefile fragments for project files. (#8320)
The generator project file still needs some custom logic, because we can't put
the generated makefile fragment next to the csproj (that would break the
API/generator diff).
2020-04-08 16:59:17 +02:00
Rolf Bjarne Kvinge 58f0a0a2c0
[tests] Fix building 'test.config' by passing 'test.config' as an argument to make and not as the working directory to the process. (#8315) 2020-04-08 16:20:21 +02:00
Rolf Bjarne Kvinge 28d97d6b6b
[xharness] Don't overwrite variations. (#8312)
This means different variations of tests won't show up as identical in the
html report.
2020-04-08 16:19:58 +02:00
Rolf Bjarne Kvinge 82f3bacc57
[xharness] .NET projects don't need 'nuget restore', they're always restored automatically. (#8313) 2020-04-08 16:19:52 +02:00
Rolf Bjarne Kvinge da2090c460
[tests] Move mmptest/regression to mmp-regression. (#8314)
dotnet doesn't like projects within projects in the file system, because of
default inclusion behavior, where the top project will want to include the
source files for the nested projects.

There are ways around that, but the easiest is to just make sure there aren't
projects within other project directories.
2020-04-08 16:19:47 +02:00
Rolf Bjarne Kvinge d3674f7efc
[mmp] Move tools/mmp/tests to tests/mmp-aot-tests. (#8317)
This is to avoid having projects within other projects' directories, which
ends up being cumbersome in .NET 5 due to default inclusion behavior.
2020-04-08 16:19:39 +02:00
Rolf Bjarne Kvinge 3d7a38e464
[mtouch/mmp] Use a single SdkVersions.cs in tools/common. (#8318)
No need to have two identical files around.
2020-04-08 16:19:34 +02:00
Rolf Bjarne Kvinge f69ef4f696
Setting an environment variable to null doesn't work in .netcore. (#8316)
There is a difference between mono and .netcore with regards to environment
variables when launching processes.

For the following example:

    process.StartInfo.EnvironmentVariables ["FOO"] = null;

.netcore will launch the process with an empty FOO variable, while mono will
launch the process with no FOO variable set.

So unstead remove the variable from the collection of environment variables
(this works fine even if the collection doesn't contain the variable).

Ref: https://github.com/dotnet/runtime/issues/34446
2020-04-08 16:19:08 +02:00
Manuel de la Pena f891076bd8
[Harness] Move to create a more general TestTasks class to be reused. (#8286)
Some of the Jenkins test tasks are very useful and do A LOT of stuff. So
we try to generalize the base class, to later be able to share the most
usebul ones in the shared lib.

Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
2020-04-07 20:20:38 -04:00
Přemek Vysoký 530f2a2bfa
[Harness] Rename device management classes (Devices, Simulators, ILoadAsync) (#8295) 2020-04-06 14:33:27 -04:00
Manuel de la Pena 6c9f62f56e
[Introspection] Fix a false typo in mac modern typo tests. (#8294)
Add Sha as an allowed word in the typo tests, it is a well known
acronym.
2020-04-06 14:31:46 -04:00
Přemek Vysoký b4a262eb42
Fix unit tests, make Devices fields readonly (#8293) 2020-04-06 10:57:31 -04:00
Manuel de la Pena ab476d7fa2
[Harness] Fix the generation of the sources. (#8289)
The name of the dll changed, ergo the prefix changed.
2020-04-06 09:52:18 -04:00
Přemek Vysoký ce9f9278b6
[Harness] Move TestImporter into the shared library (#8283) 2020-04-03 18:28:49 -04:00
Manuel de la Pena 10333b0d0f
[Harness] Make sure TestsTasks do not share an id. (#8273)
Threading is hard, part 2. We could have a situation in which two tasks
that are instantiated async use the same id. The id is later used to
build the logs directory etc.. When they share the id we end up in a
situation in which the logs overlap.

fixes: https://github.com/xamarin/xamarin-macios/issues/8146
2020-04-03 14:43:04 -04:00
Manuel de la Pena 3004e7d34b
[Harness] Ensure we do not throw an exception if we do not have tasks. (#8272)
The First extension will throw an exception if the list is empty. Use
FirstOrDefault, check if the devices are null and return the test as
Ignored.

fixes: https://github.com/xamarin/xamarin-macios/issues/8271
2020-04-03 13:23:35 -04:00
Manuel de la Pena d7160b3dcf
[Harness] Completely remove the BCL prefix from the test importer. (#8224)
Remove the last BCLPrefix and move the InfoPlistGenerator to the managed
namespace since it is part of the managed template.
2020-04-03 11:01:56 -04:00
Přemek Vysoký d4cf8301cf
[Harness] Allow to create a process manager with default tool paths (#8278) 2020-04-03 11:00:45 -04:00
Manuel de la Pena feec8d4ef9
[Harness] Remove hardcoded paths to the testing framework dlls. (#8244)
The testing framework dlls come from mono and their paths are harcoded
in the templates. Move the hardcoded paths out to the assembly locator
which know were to find the paths.

This is needed to later allow the cmd to pass the location of the paths.
2020-04-01 15:10:13 -04:00
Manuel de la Pena 1518894394
[Harness] Add the assets as part of the managed template. (#8251)
The assets were being used from the bcl-test directory. Move them as
part of the template. Renamed a file to make things a little easier when
adding the assets (a '.' in the middle of the name made this
complicated).

The template should be selfcotained.
2020-04-01 15:09:08 -04:00
Manuel de la Pena f16561b49f
[Harness] If we cannot connect over tcp use Console.Out as the writer. (#8235)
If we could not connect to the host via tcp, set the writer to be
Console.Out which will fix the NRE and will report the test run in the
runlogs.

fixes: https://github.com/xamarin/xamarin-macios/issues/8130
2020-04-01 14:03:40 -04:00
Přemek Vysoký b14a055265
[Harness] Move code to a shared library (#8234)
- Code that will later be moved to the `dotnet/xharness` repo is first moved to an isolated project before it will be extracted to the new repo and NuGetified
- New project for the shared code and new test project for tests accompanying the moved code are created
- Only Factories are left behind that are used only by XHarness
2020-04-01 13:32:21 -04:00
Sebastien Pouliot 86ab2bc1d1
[tests][link all] Remove test workaround (#8252)
iOS fixed after Xcode 11 release

Fix https://github.com/xamarin/xamarin-macios/issues/6254
2020-04-01 09:00:26 -04:00
Sebastien Pouliot ac01b522f0
[generator] Add missing `EnsureUIThread` for generated `init` and `initWithCoder` (#8250)
Fix https://github.com/xamarin/xamarin-macios/issues/8249
2020-03-31 17:20:44 -04:00
Manuel de la Pena f383593924
[Harness] Merge the Generator and the Factory in a same object and rename namespace. (#8222)
The Generator and the Factory classes are a xamarin-macios thing.
Initially, they were separated because the code that generated the bcl
tests was not inside xharness. That is not longer the case. We can merge
both classes, generalize the namespace and be more prepared to move out
of the xamarin-macios repo.
2020-03-30 23:23:33 -04:00
Manuel de la Pena 63fcc22bad
[Harness] Make sure we do not copy/generate the app sources more than once. (#8243)
Threading is hard, even with async. We reached a situation in which the
src code of the test applications from the template was being generated
more than once. Ideally, we could have a nice solution with
AsyncLazy<bool> and use the lazy async to copy the code, but it was
moved to netcore 5. The fix is a little uglier but valid, lock and
enure we only generate it once. Please note that you CANNOT use a lock
statement inside a async method.

Fixes: https://github.com/xamarin/xamarin-macios/issues/8240
2020-03-30 21:20:37 -04:00
Manuel de la Pena bdce02622f
[Harness] Remove the BCLTestProjectInfo class. (#8221)
Remove two classes that are not really needed and a method. We moved to
named tuples to make things simpler. Unfortunally C# does not support:

```csharp
using MyNamedTuple = (string Name, double value, bool answer);
```

But we have to live with this. Makes the namespace simpler and removes
confusion with the already present project classes that are more widely
used.

* [Harness] Remove the BCLTestProjectInfo class.

The class is not really needed. Move to a tuple, make the definition of
the group test project nicer since the dictionary initiallization makes
it cleaner.
2020-03-30 15:20:50 -04:00
Manuel de la Pena 728442e01f
[Harness] If empty nodes are found, do not crash. (#8201)
Looks like we have cases in which we get empty nodes when listing
devices, which makes the parsing of the nodes fail and crash.

Move the code to a diff method, try to parse the data, if there is an
exception (missing data) return null. If null is returned, do not add
it.

Fixes: https://github.com/xamarin/xamarin-macios/issues/8160
2020-03-30 10:41:38 -04:00
Sebastien Pouliot f1508bff64
[tests][bcl] Ignore some known failures on arm/32 bits (#8216) 2020-03-30 08:26:36 -04:00
Manuel de la Pena 0ffd4faa64
[Harness] Add a new known failure to help the mionitoring person. (#8227)
Looks like mono is not investing much time at the moment fixing
https://github.com/mono/mono/issues/18560

The above error happens always on 32b device tests. Added a new known
failure for the monitoring to use in their task so that they do not have
to goole the issue and they know that is knonw already.
2020-03-30 07:29:52 -04:00
Manuel de la Pena b371133216
[Harness] Verify that issue #8214 is fixed. (#8229)
Added a test in the xml parser that verifies that the parsing of the xml
is correct and that the correct number of failures is reported in the
human readable form.

fixes: https://github.com/xamarin/xamarin-macios/issues/8214
2020-03-30 07:29:22 -04:00
Manuel de la Pena 3623d7a72f
[Harness] Remove not needed classes from the BCLTestImporter. (#8218)
Remove two classes that are not really needed and a method. We moved to
named tuples to make things simpler. Unfortunally C# does not support:

```csharp
using MyNamedTuple = (string Name, double value, bool answer);
```

But we have to live with this. Makes the namespace simpler and removes
confusion with the already present project classes that are more widely
used.
2020-03-30 07:26:15 -04:00
Přemek Vysoký 7d47b073cf
[Harness] Remove AppRunner references from TestReporter (#8212) 2020-03-29 12:02:03 -04:00
Přemek Vysoký 6209719f51
[Harness] Remove Harness as a dependency (#8203) 2020-03-27 12:41:31 -04:00
Alex Soto dddcba9dc4
Merge pull request #8207 from dalexsoto/master-xcode11.4
[xcode11.4] Merge d16-5-xcode11.4 into master
2020-03-27 10:58:32 -04:00
Přemek Vysoký 46a2d96c55
[Harness] Fix XHarness unit tests (#8208)
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
2020-03-27 09:46:54 -04:00
Alex Soto 3ff3e07ebe Merge remote-tracking branch 'xamarin/d16-5-xcode11.4' 2020-03-26 19:59:14 -04:00
Chris Hamons 53964c0af0
[mac] Add NSScrollView.WillStartLiveScrollNotification (#8205)
- https://github.com/xamarin/xamarin-macios/issues/7712
2020-03-26 16:15:50 -05:00
Přemek Vysoký a48667b761
[Harness] Remove TextWriter dependency, refactor Logging (#8184) 2020-03-26 09:59:43 -04:00
Přemek Vysoký 3f3c9af198
Fix BUILD_REVISION env argument (#8197)
Fixes #8185 and some other minor things problems in AppRunner
2020-03-25 12:38:10 -04:00
Manuel de la Pena a59d519775
[BCL] Ignore a failing test on devices. (#8195)
Ignore a failing test on devices. Related mono issue: https://github.com/mono/mono/issues/19308

Mono has to fix the issue, so we ignore in the meantime.



Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
2020-03-25 11:24:07 -04:00
Přemek Vysoký d98c515a4c
[Harness] Add strongly typed mlaunch args and escape them (#8167)
Co-authored-by: Premek Vysoky <prvysoky@microsoft.com>
2020-03-25 10:59:29 -04:00
Alex Soto aa73e41254
[xcode11.4][UIKit] Update bindings to Xcode 11.4 Final (#8192)
* [UIKit] Update bindings to Xcode 11.4 Final

* Apply feedback
2020-03-25 10:52:11 -04:00
Manuel de la Pena 0324fbab34
[Harness] Refactor, simplify and fix issues in the result parsing. (#8183)
Refactor result parsing and test it. Make sure that all the possible
corner cases are taken into account. The changes are as follow:

1. Move all the logic to a new class, which will take care of the reustl
parsing.
2. Simplify the parsing method by splitting the noise code to other
functions. There is no need to parse several files in the same method.
3. Add tests that make sure that the correct result is returned when:
  a. The app cannot launch.
  b. The app timesout.
  c. The app crashes.
2020-03-25 02:44:47 -04:00
Alex Soto 96e806c6e2
[FileProvider] Update bindings to Xcode 11.4 Beta 3 (#8075) (#8194) 2020-03-25 00:32:30 -04:00
Sebastien Pouliot aa5e17dfc5
[xcode11.4][intentsui] Add missing [DesignatedInitializer] (#8191) 2020-03-25 00:31:15 -04:00
Sebastien Pouliot 064886a09f
[xcode11.4][arkit] Add final bindings (ARKit 3.5) (#8190) 2020-03-24 22:29:32 -04:00
Alex Soto e8e78ff86e
[d16-5-xcode11.4] Initial bump to Xcode 11.4 Final (#8189) 2020-03-24 20:46:37 -04:00
Sebastien Pouliot e5e93f6c53
Revert "[tests] Adjust expected result of looking for 'mono_native_initialize'. Fixes xamarin/maccore#1809. (#6806)" (#8182)
This reverts commit e4a926e8f2.
2020-03-24 17:36:29 -04:00
Přemek Vysoký 6a58674960
Extract AppBundleInformationParser (#8174) 2020-03-24 10:02:22 +01:00
Sebastien Pouliot e9ff1f854d
It seems 'initWithRequest:cachedResponse:client:' is not called on 10.9 (#8173) 2020-03-23 20:55:51 -04:00
Sebastien Pouliot 9f1ecfa85f
[tests][monotouch-test] Update UrlProtocolTest (#8168)
The _nice_ thing about `NSURLProtocol` is that it does not force the use
of the network - but we inflicted this upon us on tests.

So this updates the test to avoid the network, which should make it
more reliable and not so much influenced by the URL used or the OS
versions where it's executed.

This does not cover as much callback as before. However this it does
test that the NSUrlProtocol is registered correctly (what we care) and
the rest is largely only related the the OS (which is not the target
of our unit tests).

Hopefully fix https://github.com/xamarin/maccore/issues/1956
2020-03-20 21:52:43 -04:00
Rolf Bjarne Kvinge d0b170bf8f
[d16-5-xcode11.4][mtouch/mmp] Only use stdout from xcrun as the path of the file that was found. Fixes #8147. (#8150) (#8164)
xcrun can print stuff to stderr even if it succeeds and prints what it found
to stdout, which means we have to separate stdout and stderr when capturing
xcrun's output.

This also means that we need a few changes to our RunCommand API:

* Add support for capturing stdout and stderr separately by providing
  different callbacks/StringBuilder instances.
* Use overloads instead of default arguments, because with default arguments
  we end up with ambiguous matches in a lot of places with the new overloads.

Fixes https://github.com/xamarin/xamarin-macios/issues/8147.

Backport of #8150.
2020-03-20 19:44:58 +01:00
Rolf Bjarne Kvinge dccf99a0ca
[tests] Work around xamarin/maccore#2177 by building test-libraries as part of the build. (#8163)
This is not a complete solution, because it doesn't work when running device
tests, because then we don't build locally. On the other hand, that bug has
never been a problem for device bots, so hopefully that won't change in the
future.

Ref: https://github.com/xamarin/maccore/issues/2177
2020-03-20 19:39:31 +01:00
Přemek Vysoký 29e978e0be
[Harness] Add AppRunner.RunAsync unit tests (#8159)
Co-authored-by: Premek Vysoky <prvysoky@microsoft.com>
2020-03-20 14:18:28 -04:00
Rolf Bjarne Kvinge c6e9932be9
[mtouch/mmp] Only use stdout from xcrun as the path of the file that was found. Fixes #8147. (#8150)
* [mtouch/mmp] Only use stdout from xcrun as the path of the file that was found. Fixes #8147.

xcrun can print stuff to stderr even if it succeeds and prints what it found
to stdout, which means we have to separate stdout and stderr when capturing
xcrun's output.

This also means that we need a few changes to our RunCommand API:

* Add support for capturing stdout and stderr separately by providing
  different callbacks/StringBuilder instances.
* Use overloads instead of default arguments, because with default arguments
  we end up with ambiguous matches in a lot of places with the new overloads.

Fixes https://github.com/xamarin/xamarin-macios/issues/8147.

* [tests] Fix mmptest build.

* [mtouch/mmp] Fix process log output.
2020-03-20 17:46:43 +01:00
Přemek Vysoký ec420e6740
[Harness] Make CrashSnapshotReporter injectable and add unit tests (#8129)
* Move AppInstallMonitorLog

* Move Helpers.GenerateStableGuid

* Move DirectoryUtilities

* Move DirectoryUtilities.RootDirectory

* Add CrashSnapshot ctor

* refactor CrashReportSnapshot

* Add WrenchLog

* Move stuff out of Harness

* Remove RootDirectory

* Revert RootDirectory

* Remove Initialize() calls

* Make CrashReportSnapshot injectable

* Remove Harness dependency from CrashReportSnapshot

* Add device snapshot tests

* Trim down IHarness and make properties of Harness readonly

* Memoize xcoderoot

* Use MlaunchArguments

* Add MlaunchArguments docs

* Add CrashReportSnapshot docs

* Rename method

* Rename class

* Refactor CrashSnapshotReporter

* Inject AppRunner

* Indent parameters

* Indent parameters

* Fix unit tests

* Fix gathering crash logs

* Add crash logs to output

* Fix getting xcode root

* Fix alignment

Co-authored-by: Premek Vysoky <prvysoky@microsoft.com>
2020-03-20 11:42:43 +01:00
Rolf Bjarne Kvinge fcbb90982f
[tests] Remove test that used Xamarin.Mac/Classic. (#8151)
This test need substantial updates to work with Xamarin.Mac/Unified, and
there's very little gain in doing so, so just remove it.
2020-03-20 11:26:58 +01:00
Manuel de la Pena 79a089cd0d
[Harness] Generalize XmlResultParser. (#8156)
First step to try and clean up the psring of results and ensure that we
are doing the correct thing. We have several issue there. We want to
move all the complicated logic out of AppRunner and test the different
outputs and the full result matrix.

This commit first allows to use an instance class for the parsing, later
we will move things out. Step by step
2020-03-19 17:42:51 -04:00
Manuel de la Pena a4b71841df
[Harness] Create a more general TestAssemblyDefinition. (#8149)
XHarness TestAssemblyDefinition is very particular on the way we deal
with the mono test assemblies, in the general case, it only contains the
assembly name, but in the xamarin-macios it calculates the name of the
assembly using the platform.

Create an interface so that the command line was its own implementaiton
and move as much as possible to use the AssemblyLocator class. Later we
can add two simpler implementations for the more general cases.

Move the AssemblyLocator out of the Templates namespace since it makes
more sense.
2020-03-19 13:40:13 -04:00
Přemek Vysoký f8d744cf54
Create DeviceLogCapturer factory and inject it (#8154)
Co-authored-by: Premek Vysoky <prvysoky@microsoft.com>
2020-03-19 13:38:35 -04:00
Přemek Vysoký 5422d4f9b8
Inject ProcessManager into SimulatorDevice (#8145)
Co-authored-by: Premek Vysoky <prvysoky@microsoft.com>
2020-03-19 13:37:21 -04:00
Přemek Vysoký a5fbe513a3
Increase wait time for crash logs (#8152)
Co-authored-by: Premek Vysoky <prvysoky@microsoft.com>
2020-03-19 11:13:06 -04:00
Rolf Bjarne Kvinge 4d7186028e
[mtouch/mmp] Improve target framework code. (#8137)
* Unify target framework code between mtouch and mmp.
* Simplify the code in mmp: have three possible valid target frameworks for
  most of code, and add special code to handle setting any other valid target
  frameworks to redirect to one of those three valid target frameworks (and
  warn if given any of those valid, but not "main", target frameworks). Any
  other code can then depend on the target framework having exactly one of
  those specific values, which means we can make IsUnified* variables
  convenience properties instead.
* Unify a bit more of the argument parsing code between mtouch and mmp, since
  that made a few other things easier.
* Add TargetFramework.IsValidFramework to have one validation implementation.
* Move the implementation of TargetFramework.MonoFrameworkDirectory to mmp
  itself, it's not really related to the target framework.
* Remove Driver.IsUnified and IsClassic from mmp, they're not used anymore.
* Formally deprecate --xamarin-[full|system]-framework in mmp, they've really been deprecated for many years.
* Remove LinkerOptions.TargetFramework, it's not used anymore.
* Get rid of mmp's userTargetFramework fried, it's duplicated with the
  targetFramework field.
* Add a few tests, and tweak others a bit.

Breaking changes:

* Both mtouch and mmp require --target-framework now. The only direct
  consumers should be the MSBuild tasks, which already pass --target-framework
  all the time. This simplifies code, and removes assumptions.
2020-03-19 09:28:09 +01:00
Přemek Vysoký 16cbf4a025
Create CaptureLogFactory and make CaptureLog injectable (#8141)
Co-authored-by: Premek Vysoky <prvysoky@microsoft.com>
2020-03-18 18:39:10 -04:00
Manuel de la Pena 4449b0c4c1
[Harness] Remove the code that check if the test dlls were downloaded. (#8131)
Initially the location of the test dlls of the downloaded artefact and the compiled mono were different. That was no longer the case and therefore there was a lot of code that could be removed since the wasDownloaded variable was set to be always true.

We remove the not needed code that will calculate a diff path if mono was compiled and use the AssemblyLocator class to point to the correct root path were test dlls will be found.

Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>
2020-03-18 16:54:47 -04:00
Rolf Bjarne Kvinge bacd958be0 [mtouch/mmp] Rework how we find developer tools. Partial fix for #4634 and fixes #8005. (#8121) (#8128)
Partial fix for https://github.com/xamarin/xamarin-macios/issues/4634.
Fixes https://github.com/xamarin/xamarin-macios/issues/8005.
2020-03-18 13:07:08 -04:00
Přemek Vysoký fbbd5ceede
[Harness] Move Timestamp and TTY utilities to Helpers (#8138)
Co-authored-by: Premek Vysoky <prvysoky@microsoft.com>
2020-03-18 12:10:25 -04:00
Rolf Bjarne Kvinge dae93e0396
[tests] Adjust MT5107 to work when external verbosity is higher, and make it faster. (#8136)
It runs in 36s instead of 47s on my machine now.
2020-03-18 14:56:47 +01:00
Přemek Vysoký 5f2905c5a4
[Harness] Move utilities out of Harness.cs (#8125)
Co-authored-by: Premek Vysoky <prvysoky@microsoft.com>
2020-03-17 14:42:21 -04:00
Alex Soto 3e3cd33f0c
[NetworkExtension] Update bindings to Xcode 11.4 Beta 3 (#8126)
* [NetworkExtension] Update bindings to Xcode 11.4 Beta 3

* Update src/networkextension.cs

* Update src/networkextension.cs

Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>

* Update src/networkextension.cs

Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>

* Apply feedback

* Update src/networkextension.cs

Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
2020-03-17 13:23:36 -04:00
Rolf Bjarne Kvinge 13a56ffb95
[mtouch/mmp] Rework how we find developer tools. Partial fix for #4634 and fixes #8005. (#8121)
Partial fix for https://github.com/xamarin/xamarin-macios/issues/4634.
Fixes https://github.com/xamarin/xamarin-macios/issues/8005.
2020-03-17 15:49:39 +01:00
Přemek Vysoký 723ec1857c
[Harness] Refactor simple listeners (#8120)
* Add RunMode, DeviceClass

* Inject dependencies into AppRunner

* Add all readonly fields to ctor

* Add basic AppRunner tests

* Add first proper Install test

* Add Uninstall tests

* Refactor listener ctors

* Update tests/xharness/Harness.cs

Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>

* Use InvalidOperationException

* Hide fields

* Add TODO

Co-authored-by: Premek Vysoky <prvysoky@microsoft.com>
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
2020-03-17 10:16:19 +01:00
Manuel de la Pena 5209b19ef4
[Harness] Move all test project generation to the template object. (#8122)
At this point the BCLTestProjectGenerator class just represents a configuration class that knows about the project that are particular to xamarin-macios. All the logic that creates the projects has been moved to the XamariniOSTemplate which creates the project using the information passed by the config class.

At this stage, we have the ability to generate test projects by simply passing the info classes, which can be built either by a class (like the generator one) or a command line. We have fully decoupled the project generation from Xharness.

Tests have been added or moved to the correct location.
2020-03-16 18:52:41 -04:00
Whitney Schmidt e085125ab7
[StoreKit] Update bindings for Xcode 11.4 beta 3 (#8104)
* initial commit of StoreKit changes

* frameworks.cs to 6,2

* respond to round 1 feedback

* move StoreKit into COMMON_FRAMEWORKS

* fix manuel formatting suggestions

* add availability attributes to everything, add comments with questions

* remove extra [NoWatch], fix copyright
2020-03-16 16:54:16 -04:00
Přemek Vysoký d1666d524d
Add MLaunchArgument classes (#8123)
Co-authored-by: Premek Vysoky <prvysoky@microsoft.com>
2020-03-16 16:09:27 -04:00
Přemek Vysoký 26903778fb
[Harness] Refactor and unit-test AppRunner (#8117)
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
2020-03-16 09:26:46 -04:00
Manuel de la Pena 82bd730ee3
[Harness] Add several fixes to the crash reports. (#8107)
The name of the tests is picked up in a more inner node than
expected. Fwd the test name to make sure that noe all
crashes/timeouts/launches have the same name.
Add extra information when we have a timeout. We want to know the
name and the bot used to ensure that it is easier to identify patterns.
Timeout tests now have:

test title as: App Timeout {AppName} {Variation} on bot {device_name}
message: AppName} {Variation} Test run timed out after {timeout.TotalMinutes} minute(s) on bot {device_name}.
Build issues now have:

test title: App Build {projectTask.TestName} {projectTask.Variation}
That way we know the exact app that failed to build and the exact app that timeout and the device used to run it.
2020-03-13 18:15:02 -04:00
Přemek Vysoký ef9f7e4e9b
[Harness] Make Harness properties immutable (#8093)
Co-authored-by: Premek Vysoky <prvysoky@microsoft.com>
2020-03-13 10:41:02 -04:00
Manuel de la Pena e96c56bf3c
[Harness] Fix bug in TCCDb editing that is missing using the watchapp id. (#8098)
The wrong var name was used and the table was not correctly updated.
Updated tests to take into account the new \n in the sql.
2020-03-12 13:46:29 -04:00
Manuel de la Pena 7332c92263
[Harness] Move the templates src to be embedded resources. (#8090)
Move the project templates to be a resource in the dll. That way we can
move all the code outside of xharness and used it outside. This is not a
fill refactor but a first step to decouple the bcl test generation from
the xamarin-macios project.

XHarness should not see a difference, everythign works and simply makes
the dlls larger due to the new resources.

The code is going to take into account that the mono team will want to
use and unmanaged template with no dependencies on Xamarin.iOS and
Xamarin.Mac.



Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>

Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
2020-03-12 13:45:47 -04:00
Rolf Bjarne Kvinge 0b0f6ab043
[xharness] Improve TCC database editing. (#8094)
* Delete any previous entries before adding new ones in all simulators.
* Separate statements by newlines, so logs are easier to parse by humans.
2020-03-12 15:19:32 +01:00
Rolf Bjarne Kvinge 8be4333f43
[xharness] Rework how we build test-libraries a bit. (#8095)
* It's built in parallel with populating the tests.
* A failure isn't silently ignored, it's now shown in the html report.
2020-03-12 15:18:05 +01:00
Rolf Bjarne Kvinge 800d63bb4d Merge remote-tracking branch 'origin/master' into dotnet-bgen 2020-03-12 10:37:12 +01:00
Rolf Bjarne Kvinge 5dda97a0ef [xharness] Unset a few environment variables when running .NET tests. 2020-03-12 10:36:59 +01:00
Manuel de la Pena 16a84bd7f7
[Harness] Move the project templates out of the harness dir. (#8088)
Move the project templates to be a resource in the dll. That way we can
move all the code outside of xharness and used it outside. This is not a
fill refactor but a first step to decouple the bcl test generation from
the xamarin-macios project.

XHarness should not see a difference, everythign works and simply makes
the dlls larger due to the new resources.

The code is going to take into account that the mono team will want to
use and unmanaged template with no dependencies on Xamarin.iOS and
Xamarin.Mac.
2020-03-11 20:10:07 -04:00
Manuel de la Pena ed77f3bfd1
[Harness] Add extra tests for the SimDevice and Simulator classes. (#8079)
Add tests for the SimDevice and Simulators classes that ensure that the
process is correctly called and the xml with the simulators parsed and
returns the correct number of sims.
2020-03-11 20:09:38 -04:00
Manuel de la Pena afb11d65cc
[Harness] Add more context to crashes and timeouts. (#8078)
Update the generation of the xml to include a more descriptive title
than "AppCrash". After this commit the name of the testcase and the test
follow this pattern:

- "App Crash {ApplicationName} {Variation}"

This will make it easier to filter since the an example failure will be:

"App Crash Monotouch (Debug)"

Note, the '()' in the example are part of the variation and that is
build by xharness and passed to the runner.

Fixes: https://github.com/xamarin/xamarin-macios/issues/8077
2020-03-11 17:09:21 -04:00
Manuel de la Pena 4b3d2d1cb4
[Harness] Refactor simulators and devices listing. (#8086)
Moved the implementations under Hardare. Add the following to simplify
the testing of the AppRunner:

1. Add interfaces for the listing/loading of devices and simulators.
2. Add a special class for mlaunch args. That way we are sure that
arguments are correctly passed and we do not have typos.
3. Added tests for the new classes and for the device and devices
classes.
4. Added tests for the management of the TCC db.

Tests for the SimDevice and Simulators will come in a following PR. The
only reason for it is that I realize that the commit was getting to
large.

Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>
2020-03-11 17:06:40 -04:00
Rolf Bjarne Kvinge 067c88e2f4 [xharness] Fix merge problem. 2020-03-11 12:36:09 +01:00
Rolf Bjarne Kvinge dac68f33b4 Merge remote-tracking branch 'origin/master' into dotnet-bgen 2020-03-11 12:01:31 +01:00
Rolf Bjarne Kvinge f2063efefb
Revert "[Harness] Move utilities out from Harness.cs, refactor AppRunner fields (#8074)" (#8082)
* Revert "[Harness] Refactor simulators and devices listing. (#8064)"

This reverts commit e53b686ef6.

* Revert "[Harness] Move utilities out from Harness.cs, refactor AppRunner fields (#8074)"

This reverts commit 78dd3b861c.
2020-03-11 10:38:03 +01:00
Manuel de la Pena e53b686ef6
[Harness] Refactor simulators and devices listing. (#8064)
Moved the implementations under Hardware. Add the following to simplify
the testing of the AppRunner:

1. Add interfaces for the listing/loading of devices and simulators.
2. Add a special class for mlaunch args. That way we are sure that
arguments are correctly passed and we do not have typos.
3. Added tests for the new classes and for the device and devices
classes.
4. Added tests for the management of the TCC db.

Tests for the SimDevice and Simulators will come in a following PR. The
only reason for it is that I realize that the commit was getting to
large.


Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>
Co-Authored-By: Přemek Vysoký <premek.vysoky@microsoft.com>
2020-03-10 15:10:37 -04:00
Rolf Bjarne Kvinge 7108f59f8b Merge remote-tracking branch 'origin/master' into dotnet-bgen 2020-03-10 19:27:46 +01:00
Přemek Vysoký 78dd3b861c
[Harness] Move utilities out from Harness.cs, refactor AppRunner fields (#8074)
Move all the extension methods to a class. After this refactor, we will
be able to DI the manager in the other classes and assert that the
processes are called with the correct parameters without the need of
launching them.

Also added tests for the manager. We create a dummy console app that
will be executed by the tests. The console app has a number of
parameters that will be used to ensure that the new process behaves as
we want:

- Use the passed exit code.
- Create child proecesses if needed.
- Sleep to force a timeout.
- Writer messages to stdout and stderr.

Our tests call the dummy app and ensures that the results match the
behaviour expected by the dummy app.

Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
Co-authored-by: Premek Vysoky <prvysoky@microsoft.com>
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
2020-03-10 12:44:18 -04:00
Rolf Bjarne Kvinge 5c323cebb4 Merge remote-tracking branch 'origin/master' into dotnet-bgen 2020-03-10 13:50:11 +01:00
Rolf Bjarne Kvinge d7d9baef2d Use full path to the 'dotnet' executable.
This way we don't have to change PATH on bots. It always makes it easier to
use a different location for the executable in the future if we wished to.
2020-03-10 13:41:16 +01:00
Přemek Vysoký fa7ffbcabc
[Harness] Split out Targets, Utils and remove external links (#8061)
* [Harness] Refactor process management to be testable.

Move all the extension methods to a class. After this refactor, we will
be able to DI the manager in the other classes and assert that the
processes are called with the correct parameters without the need of
launching them.

Also added tests for the manager. We create a dummy console app that
will be executed by the tests. The console app has a number of
parameters that will be used to ensure that the new process behaves as
we want:

- Use the passed exit code.
- Create child proecesses if needed.
- Sleep to force a timeout.
- Writer messages to stdout and stderr.

Our tests call the dummy app and ensures that the results match the
behaviour expected by the dummy app.

* Apply suggestions from code review

Co-Authored-By: Přemek Vysoký <premek.vysoky@microsoft.com>

* Move out utils into a separate namespace

* Move Cache to the test project

* Fix namespaces after merge

* Remove unneeded code

* Move Target files

* Sort csproj

* Refactor Targets

* Rename Cache to TempDirectory

* Fix using

* Move ProjectFileExtensions

* Remove dead code

* Move Extensions

* Add empty StringUtilsTests

* Add StringUtils tests

* Revert refactorings

* Update tests/xharness/Utilities/StringUtils.cs

Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>

* Update tests/xharness/Utilities/StringUtils.cs

Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>

* Add better formatarguments test

* Update tests/xharness/Utilities/StringUtils.cs

Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>

Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
Co-authored-by: Premek Vysoky <prvysoky@microsoft.com>
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
2020-03-10 13:12:33 +01:00
Rolf Bjarne Kvinge 1fc2f8e3c2 [xharness] Add support for running the generator tests in .NET mode.
This also meant adding support for running .NET tests in xharness. Some
refactoring was done to extract common code to shared members, in order to
avoid duplicating a lot of code.
2020-03-09 19:49:26 +01:00
Rolf Bjarne Kvinge 7a1a8bbf0b [bgen] Add netcore version of the generator tests. 2020-03-09 19:49:26 +01:00
Rolf Bjarne Kvinge bcefeb37b6
[xharness] Handle UTF8 text better. (#8065)
* Set UTF8 as the encoding for process output.
* Serve plain text as UTF-8 plain text from our test server.

This makes emojiis render as emojiis 😁😁😁
2020-03-09 18:40:46 +01:00
Rolf Bjarne Kvinge bd29f99aaa
[xharness] Rename XBuildTask to MSBuildTask. (#8067)
We're not using xbuild anymore, and haven't for quite a while.
2020-03-09 16:44:49 +01:00
Rolf Bjarne Kvinge 57db46af17
[tests] Don't assume test assemblies are at a specific distance from the root directory. (#8066)
The RootPath finds the top '.git' directory, which should be good enough
always, so use that implementation instead.
2020-03-09 16:44:29 +01:00
Alex Soto f5b3a6f1c4
Merge remote-tracking branch 'xamarin/xcode11.4' into d16-5-xcode11.4 2020-03-08 13:16:04 -04:00
Alex Soto 3e4ba1baae
[Metal] Update bindings to Xcode 11.4 Beta 3 (#8055) 2020-03-06 18:10:24 -05:00
Whitney Schmidt f4d0ee251c
remove non-bindings, add wkwebview to .ignore (#8054) 2020-03-06 10:26:59 -05:00
Sebastien Pouliot 2be35fb1c8
[mtouch][mmp] Share error.cs and make it safer against FormatException (#8047)
* [mtouch][mmp] Share error.cs and make it safer against FormatException

Beside the obvious code sharing [1] this makes the error reporting code
safer against `FormatException`.

Even with code review this happened a few times in the past - we get
a string that can't format correctly in some error handling code. This
means the `FormatException` must be fixed before we can see the
underlying root issue (delaying bug fixes and support).

Recent changes made for localization makes it harder to review the
original strings (and even harder for translated strings) for
correctness. This means we can end up with something like

```
String.Format ("{0} {1}", "foo");
``

and easily miss it (since they split between different files) if there
are several changes inside a PR.

Finally sprinkle some unit tests :)

[1] we should move everything to use `ProductException` but that will be
for another PR...
2020-03-06 09:34:16 -05:00
Manuel de la Pena 597b7d7b0b
[Harness] Refactor process management to be testable. (#8053)
Move all the extension methods to a class. After this refactor, we will
be able to DI the manager in the other classes and assert that the
processes are called with the correct parameters without the need of
launching them.

Also added tests for the manager. We create a dummy console app that
will be executed by the tests. The console app has a number of
parameters that will be used to ensure that the new process behaves as
we want:

- Use the passed exit code.
- Create child proecesses if needed.
- Sleep to force a timeout.
- Writer messages to stdout and stderr.

Our tests call the dummy app and ensures that the results match the
behaviour expected by the dummy app.


Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>
2020-03-06 09:11:33 -05:00
Přemek Vysoký 65a7168c66
[Harness] Split out classes from Jenkins.cs to separate files (#8046)
Co-authored-by: Premek Vysoky <prvysoky@microsoft.com>
2020-03-06 06:00:34 -05:00
Alex Soto a8f38e3696
[AutomaticAssessmentConfiguration] Update bindings to Xcode 11.4 Beta 3 (#8049)
* [AutomaticAssessmentConfiguration] Update bindings to Xcode 11.4 Beta 3
2020-03-05 19:48:28 -05:00
Rolf Bjarne Kvinge 7744ad82db
[linker] No need to support monotouch.dll anymore. (#8050) 2020-03-05 22:05:08 +01:00
Manuel de la Pena cd35f26d6f
[Harness] Add listeners tests.
Provide tests that will ensure that the Tcp listener and File listener
will work correctly. Tests ensure that both listeners receive the data
and write it in the ILog

The tcp test is an interesting one because it redirects the call of the
method to a diff callback that will write the data in a stream to check
the final result.
2020-03-05 06:41:52 -05:00
Sebastien Pouliot 0129618de8
[mtouch] Add `force-rejected-types-removal` optimization (#8009) (#8042)
This optimization can be enabled when it's not possible to use the
managed linker (e.g. **Don't link**) or when the managed linker cannot
remove references to deprecated types that would cause an application
to be rejected by Apple.

References to the existing types will be renamed, e.g. `UIWebView` to
`DeprecatedWebView`, in every assemblies.

The type definition is also renamed (for validity) and all custom
attributes on the types and their members will be removed.
Code inside the members will be replaced with a
`throw new NotSupportedException ();`.

The msbuild test app `MyReleaseBuild` has been updated to test that the
optimization is working as expected (device builds are slow so reusing
this test has little impact in test time).

Basically the test ensure that `UIWebView` is used and cannot be removed
by the compiler (optimization) or the managed linker (since it's
referenced). Since the optimization is enabled then we can `grep` then
final `.app` directory to ensure there's no mention of `UIWebView` inside
any of the files that would be submitted.

The application can be run, by itself, and will turn green if OK, red if
`DeprecatedWebView` can't be found (skeleton replacement for `UIWebView`)
or orange if a `NotSupportedException` is thrown.

Finally introspection tests have been updated to skip over the deprecated
(and renamed) types. It should not be an issue right now, since this
optimization is not enabled by default, but it made testing easier.
2020-03-04 21:21:41 -05:00
Alex Soto af4ed93b65
[PassKit] Update bindings to Xcode 11.4 Beta 3 (#8041)
* [PassKit] Update bindings to Xcode 11.4 Beta 3

* Implement feedback

* Update src/PassKit/PKEnums.cs

* Update src/PassKit/PKEnums.cs

* Fix attributes
2020-03-04 20:23:42 -05:00
Sebastien Pouliot bdd39b529e
[mtouch][mmp] Add a `--warn-on-type-ref=X` option (#8038)
Using this option it's possible to test for the presence of a type
reference in both pre-linked and post-linked assemblies.

This makes it possible to detect if
* a 3rd party assemblies are using some specific type you would like to avoid;
* a type reference has been removed during the build (e.g. linker)

Notes:
* Custom attributes are encoded differently and not included in the assembly type references metadata.
* Assembly that define a type `X` do not have a reference (but the definition) of the type (and won't be reported).

If either the pre or post-linked warnings are not useful then it's possible
 to add `-nowarn:150x` to exclude the results.

E.g.
* `-nowarn:1502` would not report references in pre-linked assemblies;
* `-nowarn:1503` would not report references in post-linked assemblies;

Finally `-warnaserror:150x` can be used to stop a build that would not
satisfy either the pre or post-linked condition.

* `-warnaserror:1502` would not report references in pre-linked assemblies;
* `-warnaserror:1503` would not report references in post-linked assemblies;

_side note_ this was first done on `d16-6` to ease backports as `master`
has multiple changes for localization. A similar PR will be done for
`master` once merged.
2020-03-04 17:28:39 -05:00
Manuel de la Pena 0c085cd719
[XHarness] Add tests for all the logging classes. (#8032)
Move to use interfaces, that will let us later add tests that will
verify that all the correct logging is performed. As an example, added a
test for XmlResultParser that ensures that the failures are correctly
generated. The test uses Moq to pass the different paths to be used and
later be able to verify the wirtten xml.


Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>
2020-03-04 15:27:52 -05:00
Alex Soto ac8fb30f5d
[xcode11.4] Initial commit for Xcode 11.4 Beta 3 support (#8034)
* [xcode11.4] Initial commit for Xcode 11.4 Beta 3 support

* [monotouch-test] Adjust `TextFieldTest.EmptySelection` to 13.4 changes
2020-03-04 11:24:02 -05:00
Sebastien Pouliot 004b50f590
[tests][monotouch-test] Tweak UIWebView tests to work in every case (#8033)
so if we optimize (to remove references) the tests will be ignored.
Doing this based on define was wrong since we have different (and more)
configurations executed on devices,
2020-03-03 17:07:10 -05:00
Sebastien Pouliot c2ca09dc69
[corefoundation] Fix TypeInitializationException in OSLog (#7962)
`Default` property was using a nil-handle which is incorrect since
* we don't allow that (this is generally a bad sign)
* it does not map to `OS_LOG_DEFAULT`

Since `Default` was assigned in the type (static) constructor then
the whole type became unusable :(

Header `log.h` shows that the right definition requires us to load a
field and use it.

```
define OS_LOG_DEFAULT OS_OBJECT_GLOBAL_OBJECT(os_log_t, _os_log_default)
```

While `NULL` can actually be used for disabled (not exposed) by this
contradicting (nullability-wise) macro

```
define OS_LOG_DISABLED ((os_log_t _Nonnull)NULL)
```

Also adds unit tests. A more general tests for `.cctor` will be added
to introspection tests in a separate PR.

Fixes https://github.com/xamarin/xamarin-macios/issues/7959
2020-03-03 16:38:43 -05:00