We initially added a cmd to do the bcl test import generation that has
never been used.
We are moving the bcl test importer code to xharness and creating a
NUnit test project to run the tests that we already added. Unit tests
pass.
* [mmp] Set 'disable_omit_fp' by default to provide better native stack traces in certain cases.
This may produce better stack traces when crashes occur, which could be
helpful for tracking down issues like
https://github.com/xamarin/maccore/issues/643.
Also add an mmp option to override the default behavior.
This is unlikely to cause any problems, because Visual Studio for Mac sets
MONO_DEBUG=disable_omit_fp at launch, which means that all processes launched
by VSMac will also have this variable set already.
* Fix lldb attach disabling.
* [mmp] Don't ignore failures to execute pkg-config and simplify/improve logic.
* Show proper errors if pkg-config fails to execute.
* Extract logic to verify system mono version into its own function.
* Extract logic to execute pkg-config into its own function.
* Simplify code slightly.
* [mmp] Remove the MM5301 error message.
It's never used in mmp, and the same error code is already used in mtouch.
* [mmp] Use relative path in mmp script.
Makes relocating mmp easier.
* [mmp] Resolve the right directory.
mmp might be in two locations (the second one works because the Commands
directory is a symlink to the actual bin directory):
* /Library/Frameworks/Xamarin.Mac.framework/Versions/Current/bin/mmp
* /Library/Frameworks/Xamarin.Mac.framework/Commands/mmp
The mmp script didn't work properly in the latter case, because it would
calculate where to find the mmp.exe assembly based on real location the parent
directory (which would be /Library/Frameworks/Xamarin.Mac.framework), instead
of the real location of the directory where mmp is
(/Library/Frameworks/Xamarin.Mac.framework/Commands, which would resolve to
/Library/Frameworks/Xamarin.Mac.framework/Versions/Current/bin).
The fix is to just get the real path of the Commands directory, and compute
the location of the mmp.exe from there.
bmac-mobile-mono is an old name (when we needed a specific executable to
execute bmac) that has nothing to do with its current usage (AOT code for
Xamarin.Mac apps).
So rename the executable to reflect current usage.
This makes looking at the installed files a bit less confusing.
This makes the `--time` options more useful to see inside the linker's
pipeline steps.
This was used only in `mtouch` before (and missed some steps [1]).
Now it's shared between both tools.
[1] not sure why... merge issue/backport maybe ?
Using this option it's possible to test for the presence of a type
reference in both pre-linked and post-linked assemblies.
This makes it possible to detect if
* a 3rd party assemblies are using some specific type you would like to avoid;
* a type reference has been removed during the build (e.g. linker)
Notes:
* Custom attributes are encoded differently and not included in the assembly type references metadata.
* Assembly that define a type `X` do not have a reference (but the definition) of the type (and won't be reported).
If either the pre or post-linked warnings are not useful then it's possible
to add `-nowarn:150x` to exclude the results.
E.g.
* `-nowarn:1502` would not report references in pre-linked assemblies;
* `-nowarn:1503` would not report references in post-linked assemblies;
Finally `-warnaserror:150x` can be used to stop a build that would not
satisfy either the pre or post-linked condition.
* `-warnaserror:1502` would not report references in pre-linked assemblies;
* `-warnaserror:1503` would not report references in post-linked assemblies;
_side note_ same as https://github.com/xamarin/xamarin-macios/pull/7925
except that this one uses the localized mtouch/mmp errors only in master (so far)
We can end up with a format that has `{x}` inside it and no argument. If
this is logged (which depends on verbosity level) then we end up with an
uncaught exceptions and either `MT0000` or `MM0000` errors.
This happens in #7904 due to other, unrelated, issues (a variable is not
expanded) but this can happen in other circumstances, e.g. a file could
be named `{x}.cs`.
The easy fix is to use the right `Console.WriteLine` overload if there
are no arguments provided. This is always good since it avoid an non
required call to `String.Format`.
Fixes https://github.com/xamarin/xamarin-macios/issues/7904
If the pipeline is set on debug, to be able to see any issues, propagate
that to the make calls so that we can also get the information there,
else debugging issues with the pipelines + make is really hard.
Some of the bots fail to do the checkout (miss configuration). The clean
up step is always executed and assumes the pressence of a script, which
will fail since the script is not there.
The script is small, there is no need to add the rm in an extra file
that needs to be checkout.
This removes an extra warning that is set in the pipeline which is noise
when monitoring.
If the bot could no get the provisioning profiles installed, there is no
reason for certain tasks to run since they are all going to fail. This
adds A LOT of noise in the pipeline for the monitoring person to check
when there is no reason.
Move to use a template for the DDFun pipeline. Copy the current pipeline
to a template, set some parameters and recreate the pipeline importing
the template.
* [master] Bump mono to pickup needed nunitlite changes.
Commits are:
* [interp] context can be uninitialized for get_resume_state callback (#18535) mono/mono@455cf7d
* [2019-12] [debugger] Native thread not executing managed code considered as terminated (#18504) mono/mono@a979811
* [jit] Compute has_references correctly for gshared types whose constraint is a generic valuetype. Also emit write barriers correctly for these types. (#18562) mono/mono@f9e5a6d
* [2019-12] Bump msbuild to track mono-2019-10 (#18577) mono/mono@01be275
* [runtime] Disable lldb backtrace display on osx, it hangs on attaching in lldb. (#18591) mono/mono@ef8188a
* configure.ac: remove AC_SEARCH_LIBS for libintl (#18595) mono/mono@e55302c
* Bump bockbuild for Pango patch 1e2d68b
* [corlib] Split corlib xunit tests even more for iOS (#18620) mono/mono@8b72dbb
* [2019-12] [merp] MONO_DEBUG=no-gdb-stacktrace shouldn't disable MERP (#18611) mono/mono@4c93e38
* [aot] Avoid inflating gparams with byreflike types during generic sharing. (#18682) mono/mono@0011444
* Update deprecated query parameter to header (#18705) mono/mono@e65846b
* [NUnitLite] Bump nunitlite submodule. (#18733) mono/mono@36073a0
Diff: 2edccc52a7...36073a0c74
Other commits need for this to land"
* Add a MONO define so that we get the extensions written for mono.
* Update the sources on iphone to remove the TextUI which is not used.
* BONUS: Remove an annoying warning when compiling NUnitLite
* Update the templates too which are use by xharness.
```
warning CS0659: 'ResultState' overrides Object.Equals(object o) but does not override Object.GetHashCode()
```
When building extensions, we first store all the mtouch arguments in a file
when msbuild builds the extension, and then when msbuild builds the main
project, we load those arguments again and actually build the extension at the
same time as we build the main app.
As such, it's important to make sure that when we reload the extension
arguments we end up with the exact same build configuration as the first time.
Unfortunately that was not the case regarding the interpreter: we
automatically set the 'EnableRepl' value in Main according to whether the
interpreter was enabled or not, but Main is not called after re-loading the
arguments when building extensions.
Fix this by moving the logic that automatically sets 'EnableRepl' to somewhere
that is executed when re-loading arguments when building extensions.
Fixes https://github.com/xamarin/xamarin-macios/issues/7780.
* [registrar] Fixes NSString trampoline code generation in static registrar
Fixesxamarin/xamarin-macios#7733
This was introduced as a side effect of commit 8425129, we
used to generate `id foo` instead of the full block signature
in the trampoline code used by the static registrar, this is
the reason we never caught this condition before.
Added registrar test.
* Move tests to the appropiate test file
We need to make sure that the iOS SDK and the iOS version do match the
ones present in Xcode.
* Add new variables to track the target version.
* Add method to get the target version.
* Modify mtouch to check agains the target framework rather than the
SDK.
This will allow to keep track of three independent things:
1. The SDK max version.
2. The Simulator max version.
3. The target version of the device.
This had to be added becuase 13.2 has targets to 13.2 but simulators for
13.3
Fixes: https://github.com/xamarin/xamarin-macios/issues/7705
- https://github.com/xamarin/xamarin-macios/issues/5738
- There are a number of managed exceptions Apple can throw at you during
debugging, such as expanding a NSColor in the wrong colorspace
- Throwing a managed exception is a nicer debugging experience, and
during debug we don't care about any performance penality.
TL&DR
* re-apply the fix to cache.cs from https://github.com/xamarin/xamarin-macios/pull/7544
* which was reverted in https://github.com/xamarin/xamarin-macios/pull/7589
* since it regressed mscorlib/sim testing in xharness (for other reasons)
* Final part to fix https://github.com/xamarin/xamarin-macios/issues/7514
This was the ~night~ day before christmas... amd a tough nut to crack!
Thanksfully we had a good test case (inside #7514) and then xharness
regressed one test in consistent, reproducible manner.
xharness builds mscorlib tests twice (32 and 64 bits) even if it's a
fat application (could be reused). That should not be a huge problem
since the 2nd build should be identical and the cache should be (re)used.
An earlier attempt fixed this (comparison was true for the wrong
reasons [1]) but the fix did not end up with the same arguments !?! and
was reverted.
This is the diff between the first and second builds:
```diff
--- /Users/poupou/a.txt 2019-12-23 09:55:01.000000000 -0500
+++ /Users/poupou/b.txt 2019-12-23 09:55:01.000000000 -0500
@@ -182,5 +182,5 @@
-r=/Users/poupou/git/xamarin/xamarin-macios/builds/downloads/ios-release-Darwin-8f396bbb408b5758fccb8602030b9fa5293ce718/ios-bcl/monotouch/tests/Xunit.NetCore.Extensions.dll \
' --target-framework=Xamarin.iOS,v1.0' \
--root-assembly=/Users/poupou/git/xamarin/xamarin-macios/tests/xharness/tmp-test-dir/mscorlib/bin/mscorlib/iPhoneSimulator/Debug-unified/com.xamarin.bcltests.mscorlib.exe \
- ' -v -v -v -v' \
+ ' -v -v' \
@/Users/poupou/git/xamarin/xamarin-macios/tests/xharness/tmp-test-dir/mscorlib/obj/iPhoneSimulator/Debug/response-file.rsp \
```
Since they are not identical the cache is invalidated (which is normal,
cache-wise) and produce an output app that is incorrect (and crash
32bits).
Now there is code to ignore verbosity options (both `-v` and -q`) since
they will not affect what `mtouch` generates. However this was broken
because mtouch's response-file parser is quite basic and stricter the the
specification
spec: https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/compiler-options/response-file-compiler-option
issue: https://github.com/xamarin/xamarin-macios/issues/7644
That failed in two different ways
1. note the extra space before the first `-v` in the diff (before the `'`
quote). That skipped the line.
2. there are multiple `-v` in the same line, again that make the
filtering skip the line.
*Unknowns*
It's too close to xmas/vacation so I might not find the reasons/issues
for the following, unanswered questions...
1. Why is the re-build app bundle failing at runtime when p/invoking ?
Something is not regenerated (symbol maps?) ?
2. Why xharness 2nd build has more verbosity than the first one (likely
harmless) ?
[1] the original cache.cs issue (prequel)
issue w/test case: https://github.com/xamarin/xamarin-macios/issues/7514
first attempt: https://github.com/xamarin/xamarin-macios/pull/7544
While incorrect the first attempt to fix `cache.cs` was a logical, if
not entirely complete, fix. Without it this is what we _currently_ cache:
```
/Users/poupou/git/xamarin/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/lib/mtouch/mtouch.exe \
```
and that does not include any of mtouch's arguments, that can change
between executions and (should) invalidate the cache.
In this case it means the cache is used (no difference) but this does
**not** parse the content of the **response file** which is obviously
wrong (and we do have code to process it).
On the original issue's test case this is what makes the difference
between using the same *old* nuget assembly after an update (and fail)
by itself and also because the updated framework was not copied (due
to the 2nd part of the bug report wrt `copyfile`).
OTOH re-using (incorrectly) the cache is what makes xharness's mscorlib
unit tests works right now :(
Inspiration was taken from other yaml pipelines but those multiple jobs are not behaving the way I expected.
1. Each job will clone `xamarin-macios` in a dedicated folder (for 3 jobs you'll have `./1/s`; `./2/s`; `./3/s`)
2. The goal of the last job was to upload the `TestSummary.md` but because of the cloning strategy it couldn't find it. There might be ways to pass files (artifacts?) between jobs but there's no need to be that fancy right now.
3. Cloning takes time so I'm assuming having everything in one job can actually save us some time.
- We had `./jenkins/vsts-device-tests-set-status.sh: line 88: HTML_REPORT: unbound variable` in the `Report results to GitHub as comment / status` step. Now that the HTML_REPORT is set this shouldn't happen anymore.
- The `Add summaries` task is reporting errors but it's working regardless. I believe this might be due to -x being set instead of the original +x (I changed that by mistake).
Bump mono to get the new splited test dlls and add them to be ran in
xharness. Special logic is used for mscorlib so we make sure that all
the 'parts' of the test dll do have the same configurations.
Co-Authored-By: Waleed Chaudhry <54864665+wachaudh@users.noreply.github.com>
- Needed for the DDFun migration of our tests
- Don't use SSH because it's restricted on the new bots
- Use brand new DevOps feature to clone multiple repos via HTTPS
- Update names of all tasks (clearer)
- Simplify and re-use code
- Update all paths because our cloning strategy is different compared to old pipeline
- Update xharness to print "experimental" titles in GitHub comments
- Update xharness to not add the Html link for tests from the DDFun pipeline
- Fix all timeouts to be relevant to the tests length
- Split in multiple jobs (for GitHub status)
- Move longer scripts to independant files
- Make inline bash prettier
- Use a variable for the pool name
- Fix single space alignment
- Remove unused variable in `vsts-device-tests-set-status.sh` that broke it
- Bump maccore to use the right certificates and profiles
- Bump maccore to target the right pipeline
- Bump maccore to not use ssh in scripts
- Bump maccore and use persistCredentials
Notes:
- Right now the xharness test output can only be seen when downloading the zip file locally (better output soon)
- https://devdiv.visualstudio.com/DevDiv/_workitems/edit/947932
- It turns out the linker in some cases can wrap expected exceptions in a outer exception. By drilling in we can produce better errors, again.
- Refactor pipeline exception handling to be shared between mmp/mtouch
Copying the framework could fail (error 260) and the failure was never
reported so the build succeeded - but without updating (completely) the
framework.
The exact reason it fails is unknown :( but we can recover from it by
deleting the target and copying (everything) back to the expected
(target) location.
Build logs will now indicate when this fails and will try to recover
before reporting a build error. Best case it works :) worse case we'll
be aware something is wrong (which is better than ignoring)
This is half the fix from https://github.com/xamarin/xamarin-macios/pull/7544
which was reverted due to the second half causing a regression (under
investigation).
Reference: https://github.com/xamarin/xamarin-macios/issues/7514 (partial fix)
* [sample-tester] Collect app size and build duration statistics.
* Always publish performance data.
* [mmp] Add support for MMP_ENV_OPTIONS to mirror mtouch's MTOUCH_ENV_OPTIONS.
* [sample-tester] Make mmp/mtouch show timing information, and get the diagnostic msbuild log.
* Collect task and target info.
* Add more perf data and a baseline test.
* Redo the base test a bit.
* More tweaks.
* More tweaks.
* Huh?
* Debug stuff.
* diagnostics.
* Better diagnostics.
* cleanup
* tmp
* Use a separate stage to push data.
* Next attempt.
* Unique artifact names.
* Make the right name unique.
* More progress
* Improvements
* Don't need new maccore.
* cleanup
* Remove debug spew.
* Realign stuff.
* duh
* More path fixes.
* zippity zip.
* Show publishing errors.
* dependencies
* Not fake results anymore.
* Tweaks
* Merge xml files
* [sampletester] Bump nuget timeout to 5 minutes.
The TodoREST.iOS test seems to use a lot of packages, and it can apparently
take a while to restore them, so give a few more minutes.
* Improvements.
* Fix glob.
* [sampletester] Fix fetching of sample repository and hash.
* [sampletester] Centralize verbosity for mtouch and mmp.
Some of the test assemblies are too large and will be splitted by mono so
that they can be compiled for iOS 32b. In that case, we are using the
following pattern
`
test_assembly_name.dll
`
becomes
`
test_assembly_name.part1.dll
test_assembly_name.part2.dll
`
Perse the only change we need to add to make our life easier is to be
able to mantain a single .ignore file since mantaining more .ignore
files is hard and more error prone. This change simply checks if we are
working with a splitted dll and ensures that the correct .ignore files
are added to the BCL test application.
Co-Authored-By: Pramit Mallick <prmallic@microsoft.com>
After this commit we will have more application but they will be the
appropiate size so that they can be built with the linker for iOS 32b.
It is important to note that the following apps WILL CONTINUE to fail
since the dlls need to be splitted in mono:
* mscorlib tests
* Mono BCL tests group 5 - Which is monotouch_System.Core_xunit-test.dll
and is too large.
Updating the msbuild tasks to use netstandard2.0 requires us to bump NUnit to 3+.
This requires:
* A few code changes due to breaking API changes in NUnit.
* Changes in xharness and a makefile to cope with the new location for the
NUnit console runner (I added a helper script to make things slightly
easier).
It's possible for a nuget to have dependencies that will bring satellite
assemblies into a project.
When doing so the satellite assemblies are copied to the right output
(e.g. `Debug`) directory, so they work fine at runtime. However it's
not 100% fine at build time, e.g. msbuild won't detected them as
satellite assemblies.
Something similar happens with `mtouch` (and `mmp`) since the satellite
assemblies won't be found inside a (culture-named) subdirectory from the
original assembly location. In our case this means the assemblies won't
be copied into the .app bundle (since we don't run from `Debug`) and
localization won't work properly.
The solution is to check for both the (culture-named) subdirectories
from the assembly location (like before) and, if nothing is found, also
try to locate them in the _build_ (like `Debug`) directory - so if
something else (nuget or custom build scripts) tried to outsmart the
build logic then we'll still bring those satellite assemblies in the
app bundle we produce.
https://github.com/xamarin/xamarin-macios/issues/7113
Sebastien Pouliot
Manuel de la Pena
Rolf Bjarne Kvinge
Alex Soto
Waleed Chaudhry
Vincent Dondain
Chris Hamons