* [Harness] Refactor process management to be testable.
Move all the extension methods to a class. After this refactor, we will
be able to DI the manager in the other classes and assert that the
processes are called with the correct parameters without the need of
launching them.
Also added tests for the manager. We create a dummy console app that
will be executed by the tests. The console app has a number of
parameters that will be used to ensure that the new process behaves as
we want:
- Use the passed exit code.
- Create child proecesses if needed.
- Sleep to force a timeout.
- Writer messages to stdout and stderr.
Our tests call the dummy app and ensures that the results match the
behaviour expected by the dummy app.
* Apply suggestions from code review
Co-Authored-By: Přemek Vysoký <premek.vysoky@microsoft.com>
* Move out utils into a separate namespace
* Move Cache to the test project
* Fix namespaces after merge
* Remove unneeded code
* Move Target files
* Sort csproj
* Refactor Targets
* Rename Cache to TempDirectory
* Fix using
* Move ProjectFileExtensions
* Remove dead code
* Move Extensions
* Add empty StringUtilsTests
* Add StringUtils tests
* Revert refactorings
* Update tests/xharness/Utilities/StringUtils.cs
Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>
* Update tests/xharness/Utilities/StringUtils.cs
Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>
* Add better formatarguments test
* Update tests/xharness/Utilities/StringUtils.cs
Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
Co-authored-by: Premek Vysoky <prvysoky@microsoft.com>
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
This also meant adding support for running .NET tests in xharness. Some
refactoring was done to extract common code to shared members, in order to
avoid duplicating a lot of code.
IKVM has a bug where it doesn't correctly compare assemblies, which means it
can end up loading the same assembly (in particular any System.Runtime whose
version > 4.0, but likely others as well) more than once. This is bad, because
we compare types based on reference equality, which breaks down when there are
multiple instances of the same type.
So just don't ask IKVM to load assemblies that have already been loaded.
I couldn't figure out how to make nuget install these into the system, so I
decided to just download the package locally instead.
This is just temporary until we get real .NET 5 reference assemblies.
* Bump Xamarin.MacDev.
New commits in xamarin/Xamarin.MacDev:
* xamarin/Xamarin.MacDev@a0a11af Adjust SDK validation to allow tools/bin and not allow usr/bin. (#71)
* xamarin/Xamarin.MacDev@e21e1aa Accept 'tools/buildinfo' as an alternative path to 'buildinfo' due to an nuget bug. (#70)
* xamarin/Xamarin.MacDev@ce24236 Don't look in /Developer/MonoTouch anymore, there's nothing there. (#69)
Diff: 210c664e56..a0a11aff27
* [msbuild] Teach tests that mock MonoTouchSdk to mock better.
MonoTouchSdk ends up a bit confused when giving it an Sdk directory that isn't
a real Sdk directory, and then assuming that properties that return Sdk paths
won't throw exceptions.
Fix this by giving MonoTouchSdk a better fake Sdk directory (which
MonoTouchSdk detects as a real Sdk directory), and thus those previously
mentioned properties will return true fake values instead of throwing
exceptions.
It seems we only want to provision a few specific things, so modify the
selection logic to first ignore everything, then enable what we want.
This way we don't have to add new ignore flags here every time we add
something new to the provisioning script.
We also don't need to check if 7z and python3 provisioning is possible, since
we already know that (by manually checking the system-dependencies.sh script
for the current hash).
When .NET 5 comes, the TargetFrameworkMoniker will change, and we need the
entire moniker to distinguish between various platforms.
So change our msbuild code to consume the entire TargetFrameworkMoniker, so
that we have all the information we need when we need it.
Also redirect everything through an intermediate
_ComputedTargetFrameworkMoniker property, so that the target framework can be
overridden without affecting any other code. This becomes necessary during the
initial implementation phase, because we don't have a .NET version to test
with yet that can give us the new target frameworks. Eventually it should be
possible to remove this intermediate variable
This is restricted to debug builds.
Also tell make to not delete delegates.h and delegates.inc. That makes it much
easier to inspect them.
Example before function:
void
xamarin_throw_ns_exception (NSException * exc)
{
delegates.throw_ns_exception (exc);
}
The after version:
void
xamarin_throw_ns_exception (NSException * exc)
{
#if DEBUG
if (delegates.throw_ns_exception == NULL) {
NSLog (@PRODUCT ": The managed function throw_ns_exception could not be loaded.");
xamarin_assertion_message ("The managed function throw_ns_exception could not be loaded.");
}
#endif
delegates.throw_ns_exception (exc);
}
On old macOS bots we only want to provision Mono, so change the logic so that
we first ignore everything, then re-enable Mono.
This way we don't have to add new ignore flags here every time we add
something new to the provisioning script.
This fixes an issue where the script detects that dotnet isn't installed, and
shows an error, when trying to run tests on older macOS bots in CI.
* [mtouch][mmp] Share error.cs and make it safer against FormatException
Beside the obvious code sharing [1] this makes the error reporting code
safer against `FormatException`.
Even with code review this happened a few times in the past - we get
a string that can't format correctly in some error handling code. This
means the `FormatException` must be fixed before we can see the
underlying root issue (delaying bug fixes and support).
Recent changes made for localization makes it harder to review the
original strings (and even harder for translated strings) for
correctness. This means we can end up with something like
```
String.Format ("{0} {1}", "foo");
``
and easily miss it (since they split between different files) if there
are several changes inside a PR.
Finally sprinkle some unit tests :)
[1] we should move everything to use `ProductException` but that will be
for another PR...
Move all the extension methods to a class. After this refactor, we will
be able to DI the manager in the other classes and assert that the
processes are called with the correct parameters without the need of
launching them.
Also added tests for the manager. We create a dummy console app that
will be executed by the tests. The console app has a number of
parameters that will be used to ensure that the new process behaves as
we want:
- Use the passed exit code.
- Create child proecesses if needed.
- Sleep to force a timeout.
- Writer messages to stdout and stderr.
Our tests call the dummy app and ensures that the results match the
behaviour expected by the dummy app.
Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>
* [configure] Add --[enable|disable]-dotnet.
And make it enabled by default on CI and disabled by default elsewhere for now
(because it makes the build significantly slower).
* [system-dependencies] Add support for provisioning .NET.
Also write a global.json in the root directory which is how we select which
.NET version to use.
It took me a while to figure out why my newly added references didn't quite
work as expected... so let's make sure future me doesn't run into the same
problem again.
This allows us to replace Xamarin.MacDev.Tasks.PlatformFramework with
Xamarin.Utils.ApplePlatform.
It also allow for more complex target framework handling/parsing in the
future, using the TargetFramework class.
Provide tests that will ensure that the Tcp listener and File listener
will work correctly. Tests ensure that both listeners receive the data
and write it in the ILog
The tcp test is an interesting one because it redirects the call of the
method to a diff callback that will write the data in a stream to check
the final result.
Move to use interfaces, that will let us later add tests that will
verify that all the correct logging is performed. As an example, added a
test for XmlResultParser that ensures that the failures are correctly
generated. The test uses Moq to pass the different paths to be used and
later be able to verify the wirtten xml.
Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>
so if we optimize (to remove references) the tests will be ignored.
Doing this based on define was wrong since we have different (and more)
configurations executed on devices,
Move to use interfaces, that will let us later add tests that will
verify that all the correct logging is performed. As an example, added a
test for XmlResultParser that ensures that the failures are correctly
generated. The test uses Moq to pass the different paths to be used and
later be able to verify the wirtten xml.
* Merge IOS_native_DEFINES and IOS_GENERATOR_native_FLAGS into IOS_DEFINES and
IOS_GENERATOR_FLAGS, respectively.
* Remove MAC_*_GENERATOR and MAC_*_GENERATE, use MAC_GENERATOR and
MAC_GENERATE instead.
* Simplify the MAC_TARGETS_template by removing empty and useless arguments.
Don't use assembly identity when converting between reflected attributes and
instantiated attributes, because types will move to different assemblies in
.NET 5, and the code to keep track of everything ends up being complicated if
we were to verify assembly identity for each attribute type.
So don't verify assembly identity anymore.
When converting from IKVM.Reflection.Type to System.Type we can just create a
big switch statement with typeof expressions as the result, and let the C#
compiler figure out where those typeof types come from.
When converting from System.Type to IKVM.Reflection.Type we now look in all
the loaded assemblies in IKVM for the type name. To try to prevent unpleasant
surprises, we verify that any particular typename is only findable once (this
seems obvious, but this check actually found a bug in IKVM when loading .NET 5
assemblies - which will be fixed in a different PR).
Also this allowed us to remove some code to manually load assemblies we needed
to know about (since now we don't need to know about those assemblies
anymore).
An additional bonus is that this seems to be slightly faster too (a clean
build in src/ takes 1m33s instead of 1m39s).
This optimization can be enabled when it's not possible to use the
managed linker (e.g. **Don't link**) or when the managed linker cannot
remove references to deprecated types that would cause an application
to be rejected by Apple.
References to the existing types will be renamed, e.g. `UIWebView` to
`DeprecatedWebView`, in every assemblies.
The type definition is also renamed (for validity) and all custom
attributes on the types and their members will be removed.
Code inside the members will be replaced with a
`throw new NotSupportedException ();`.
The msbuild test app `MyReleaseBuild` has been updated to test that the
optimization is working as expected (device builds are slow so reusing
this test has little impact in test time).
Basically the test ensure that `UIWebView` is used and cannot be removed
by the compiler (optimization) or the managed linker (since it's
referenced). Since the optimization is enabled then we can `grep` then
final `.app` directory to ensure there's no mention of `UIWebView` inside
any of the files that would be submitted.
The application can be run, by itself, and will turn green if OK, red if
`DeprecatedWebView` can't be found (skeleton replacement for `UIWebView`)
or orange if a `NotSupportedException` is thrown.
Finally introspection tests have been updated to skip over the deprecated
(and renamed) types. It should not be an issue right now, since this
optimization is not enabled by default, but it made testing easier.
Instead verify that it's not a nested type, which should be more than good
enough. If someone else defines a System.Nullable`1, they'll probably have
much bigger problems.
This solves an issue with .NET 5 where System.Nullable`1 doesn't actually come
from mscorlib.
More future-proof, because new and different target frameworks will be
introduced in the future with .NET 5.
Also: fewer string comparisons is always good.
Some crashes are reported as tcp connection issues because they happen
before the app had the chance to write anything in the log. In that
case, we checked for the presence of the file, and if not present we
decided it was a tcp issue when it is not the case.
In this commit, we check for the tcp erorr message in the main_log so
that we are certain that the issue was with the connection and not
anyother.
Testingis simple, ran tests without the phone being part of the same
network and test with a branch that has a crash. For example the one in
https://github.com/xamarin/xamarin-macios/pull/8009 hash 83240612e8
Rolf Bjarne Kvinge
Přemek Vysoký
Sebastien Pouliot
Manuel de la Pena
Waleed Chaudhry