The mmp regression tests are already executed in parallel (using `make -j8` in
the Makefile), and they're quite CPU intensitive, which means that if xharness
runs other tests in addition to the these tests, then those other tests may
fail randomly.
commit xamarin/maccore@14c3eca3aa
Author: Rolf Bjarne Kvinge <rolf@xamarin.com>
Date: Fri Nov 24 09:53:13 2017 +0100
Bump mac-api-docs.
commit xamarin/mac-api-docs@057bb83848
Author: Timothy Risi <timothy.risi@xamarin.com>
Date: Thu Nov 23 23:52:21 2017 -0900
Docs should also be generated for Xamarin.Mac (#2)
* Docs should also be generated for Xamarin.Mac
* Remove the push target
commit xamarin/maccore@bb3ebb634f
Author: Rolf Bjarne Kvinge <rolf@xamarin.com>
Date: Thu Nov 23 19:06:51 2017 +0100
[tests/external/device-builds] Use BUILD_WORK_HOST instead of BUILD_HOST so that we calculate the right wrench url.
Dispose logs (so that the corresponding files are closed) when we're done with
them, and also don't open a file log by default (usually we just want a
filename to pass to somebody else), but instead open the file if needed.
This should decrease the number of open file descriptors in xharness, which
sometimes become a problem when running many tests.
* [tests] Add support for passing arguments to XM unit tests from the command line.
* [xharness] Get xml results for mac unit tests and parse it to show failures inline in the html report.
The parsing done by `System.Version` does not accept a major-only string,
e.g. providing "11" would throw an exception.
Since people generally refer version as iOS 11 (and not iOS 11.0) this
is, at best, a nuisance. Xcode toolchain accept "11" as a valid string.
The first part of message was updated to show both the option name and
the (user provided) value.
The 2nd part remain the text of the .net exception message, i.e. what
`Version.Parse` tells you when it validates the string. Seeing the input
value should make it more obvious for other, incorrect version strings.
reference:
https://bugzilla.xamarin.com/show_bug.cgi?id=60280
Usually an exit code of neither 0 nor 1 means a test run crashed (in
particular if the exit code is 130+), but this may not always be true, so
report the exit code as well so that a human can evaluate properly.
Change [Intermediate]OutputPath of Xamarin.Mac test projects to include the
project flavor, so that they don't build into (and run from) the same location
(which leads to random and incorrect behavior for at least one of the
flavors).
Fixes https://github.com/xamarin/maccore/issues/584
* [xharness] Add support for executing a command periodically.
This will be used to run 'rsync' on bots to upload the html report somewhere
while the tests are running.
* [tests] Disable all the wrench test targets, and instead run the jenkins target (only) on wrench.
* [xharness] Write xharness log to stdout as well on wrench.
Wrench has a 30-min stdout timeout: if nothing is printed in 30 minutes, then
the step fails. Printing the harness log to stdout makes us not hit this
timeout.
* [xharness] Timestamp a few more logs.
* [xharness] Disable the @MonkeyWrench calls, since we're not uploading directly to wrench anymore.
* [tests][generator] Port bindas1048error to NUnit.
* [tests][generator] Port bindas1049error to NUnit.
* [tests][generator] Port bindas1050modelerror to NUnit.
* [tests][generator] Port bindas1050protocolerror to NUnit.
* [tests][generator] Port bug42855 to NUnit.
* [tests][generator] Port bug57070 to NUnit.
* [tests][generator] Port bug52570classinternal to NUnit.
* [tests][generator] Port bug52570methodinternal to NUnit.
* [tests][generator] Port bug52570allowstaticmembers to NUnit.
* [tests][generator] Port protocol-duplicate-abstract-error to NUnit.
* [tests][generator] Port protocol-duplicate-method-diff-length to NUnit.
* [tests][generator] Port protocol-duplicate-method-diff-out to NUnit.
* [tests][generator] Port protocol-duplicate-method-diff-type to NUnit.
* [tests][generator] Port protocol-duplicate-method-diff-return to NUnit.
* [tests][generator] Port warnaserror to NUnit.
* [tests][generator] Port nowarn to NUnit.
* [tests][generator] Add support for inspecting/asserting the generated content.
* [tests][generator] Port some Xamarin.Mac tests to NUnit.
Ported:
* bmac_smoke
* bmac-with-hyphen-in-name
* property-redefination-mac
* NSApplicationPublicEnsureMethods
* protocol-duplicate-abstract
* [tests][generator] Point bgen to our local installation.
* [tests][generator] Port the bug31788 test to a unit test.
* [generator] Make the 'bgen' helper target more complete.
* [tests][generator] Port non-custom iOS tests to unit tests.
* [tests][generator] Add new test.
* [tests][generator] Port the forum54078 test to a unit test.
* [tests][generator] Port the desk63279 test to a unit test.
* [tests][generator] Port the desk79124 test to a unit test.
* [tests][generator] Port the multiple-api-definitions tests to unit tests.
* [generator] Use mono code style.
* [tests][generator] Port the bug29493 test to a unit test.
* [tests][generator] Port the classNameCollision test to a unit test.
* [tests][generator] Port the bi1036 test to a unit test.
* [tests][generator] Port the bug37527 test to a unit test.
Also fix BI1112 and BI1113 to show up as errors in the console output (since
they're exceptions they're already treated as errors and would cause bgen to
fail).
* [tests][generator] Port the bug27986 test to a unit test.
* [tests][generator] Port the bug35176 test to a unit test.
* [tests][generator] Port the bi1046 test to a unit test.
* [tests][generator] Port the virtualwrap test to a unit test.
* [tests][generator] Port the bug42742 test to a unit test.
* [tests][generator] Port the noasyncinternalwrapper test to a unit test.
* [tests][generator] Port the noasyncwarningcs0219 test to a unit test.
* [tests][generator] Port the bug53076 test to a unit test.
* [tests][generator] Port the bug53076withmodel test to a unit test.
* [tests][generator] Port the fieldenumtests test to a unit test.
* [tests][generator] Port the smartenumwithframework test to a unit test.
* [tests][generator] Port the forcedtype test to a unit test.
* [tests][generator] Port the bug46292 test to a unit test.
* [tests][generator] Build tests with MSBuild.
There's no need to use xbuild for these tests.
* [tests][generator] Remove dead code.
* [xharness] Don't run the makefile-based generator tests anymore.
Since there aren't any makefile-based generator tests anymore, they've all
been ported to NUnit tests.
* [tests][generator] Make the bug39614 test do what it was supposed to do: make sure a namespace isn't required (but recommended).
Hopefully third time's the charm...
Don't do date math (adding hours) to a local datetime, since DST *will* muddy
the waters and prove that 1=2.
Instead convert the date we want to calculate on to UTC, which should be DST-agnostic.
I've tested this by running the test for every hour during the next 10 years,
so that should cover mostly everything (although I'm still waiting for the
Delorean I ordered to be able to test both in the future and the past).
Previous attempts:
0442cdf9c05caddb3571
Should fix https://github.com/xamarin/maccore/issues/573.
* [mtouch/mmp/bgen] Add support for response files.
This is the first part of the fix for #56501.
https://bugzilla.xamarin.com/show_bug.cgi?id=56501
* [tests] Make sure no single argument starting with a '@' is passed to mtouch unless it's a response file.
--assembly-build-target takes arguments starting with '@', for instance:
--assembly-build-target @all=framework
which does not work anymore, because that's interpreted as a response file
(mtouch tries to read the file '@all=framework', which obviously doesn't
exist).
The fix is simple, don't put a space between the two arguments:
--assembly-build-target=@all=framework
* Add --root-assembly to mtouch/mmp and make the MSBuild tasks use this new option.
This makes it possible to pass root assemblies starting with `@` to mtouch/mmp
without getting mistaken for response files.
* [msbuild] Always use the command-line option that takes an equals or colon.
Always use the command-line option that takes an equals or colon instead of a
space.
Do either of these:
--foo=something
--foo:something
instead of this:
--foo something
so that `something` can start with an at (`@`) sign without being mistaken for
a response file.
* [msbuild] Fix tests according to recent task changes.
* Update the function name used to initialize libmono-profiler-log, its called mono_profiler_init_log () now.
* [builds] Pass --with-cross-offsets= to crosstv's configure.
* Bump mono to 2017-08.
* Bump mono to 2017-08.
* Force disable 'futimens' and 'utimensat' so that we build with Xcode 9.
This is also needed to build with Xcode 8.3 on High Sierra.
* Remove old AppleTls implementation.
* Bump mono.
* Bump mono to 2017-08.
* Bump mono to 2017-08
* Reenable link-keep-resources-2 test
- This reverts commit 76b759ef22.
- 2017-08 has linker fix
* Bump mono to 2017-10
* Revert "Bump mono to 2017-10"
This reverts commit bb7832724e.
* Bump system mono to 2017-10
* Bump embedded mono to 2017-10
* [runtime] reflect eglib move
9be68f8952
* bump mono
* [btouch] remove Security.Tls usage from test
* [mtouch tests] update the function name used to initialize libmono-profiler-log, its called mono_profiler_init_log () now.
see
ea4e4a9ef6
fixes:
```
1) Failed : Xamarin.MTouch.Profiling(tvOS)
_mono_profiler_startup_log
Expected: collection containing "_mono_profiler_startup_log"
But was: < "_mono_profiler_init_log" >
at Xamarin.MTouch.Profiling (Xamarin.Profile profile) [0x00106] in <511889694a624cc9a50e0e9b259b05c5>:0
2) Failed : Xamarin.MTouch.Profiling(watchOS)
_mono_profiler_startup_log
Expected: collection containing "_mono_profiler_startup_log"
But was: < "_xamarin_get_block_descriptor", "_mono_profiler_init_log" >
at Xamarin.MTouch.Profiling (Xamarin.Profile profile) [0x00106] in <511889694a624cc9a50e0e9b259b05c5>:0
```
* [mmptest] update log profiler options.
826558a4af
deprecated the dash prefix for the mlpd path.
`noallocs` or `nocalls` are not needed, neither of them are default anymore.
* [mmptest] fix link-keep-resources-2 test to cope with more corlib resources.
another corlib resource (mscorlib.xml) was added:
https://github.com/mono/mono/commit/11e95169e787#diff-2d1c64decd91d9a6e8842ab0f0e9438d
* Revert "[mmptest] fix link-keep-resources-2 test to cope with more corlib resources."
This reverts commit 350eb3c174.
* [XHarness] Add the Mono.Data.Tds tests.
* Address comments from rolf in the review.
* [mmp regresssion tests] bump mono linker, so mscorlib.xml gets stripped
the test was failing in that way:
> Executing link-keep-resources-2...
> [FAIL] i18n 4/2 data files present: charinfo.nlp, collation.core.bin, collation.tailoring.bin, mscorlib.xml
also update the output, because it's actually expected at least three
elements.
fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=59277
* bump mono
fixes crash in tvOS: https://github.com/mono/mono/pull/5812
* bump mono for updated BCL tests
see https://github.com/mono/mono/pull/5820
* [mono] set 2017-10 branch in .gitmodules
* [macos] Fix guiunit error on clean builds by depending on correct copy (#2912)
* [macos] Fix guiunit error on clean builds by depending on correct copy
- From a clean build making a BCL test would error due to the non-mobile guiunit not being built
- This was because the Makefile-mac.inc target was incorrect
- This was because xharness assumed that non variation based targets were always Modern
- However, BCL tests are Full, not Modern
* Code review change
* Swap to var to reduce diff
* Revert changes in the paths for GuiUnit.
* [XHarness] Add the System.IO.Compression bcl tests. (#2918)
* [XHarness] Add the System.IO.Compression bcl tests.
* [XHarness] Add bcl tests for System.IO.Compression.FileSystem. (#2924)
* [XHarness] Add the System.IO.Compression bcl tests.
* Ensure that resources are correctly copied in the bundles.
* [XHarness] Add bcl tests for System.IO.Compression.FileSystem.
* As per review, make the Mac test app name match the tests that are ran.
* [XHarness] Add Mono.CSharp tests on ios. (#2927)
* [XHarness] Add Mono.CSharp tests on ios.
* Bump mono to bring changes in the mono.csharp tests.
* [xtro-sharpie] fix TypeDefinition access due to Cecil change
* Bump mono
* bump mono
fixes
- https://bugzilla.xamarin.com/show_bug.cgi?id=60480
- https://bugzilla.xamarin.com/show_bug.cgi?id=60482
* bump mono
more fixes around conflicting paths when tests are run in parallel.
* Bump for mono/mono@2017-10
* [generator] Improve BI1014 - include name of unsupported field and update valid types on docs, fixes bug 57094.
https://bugzilla.xamarin.com/show_bug.cgi?id=57094
* Implement feedback
* fix error message
* More feedback
* [Generator] BindAs attribute for smart enums of an array of nullable values generates code that doesn't compile, Fixes bug 57797
https://bugzilla.xamarin.com/show_bug.cgi?id=57797
We now correctly compiles the array nullabe types
* Disable Nullable array types in BindAs until we add registrar support
In 0442cdf9c0 the `now` variable was changed to
be a UTC date, but unfortunately the code that compares it to a local date
(NSDate.Now) wasn't updated.
Only year/month/day values were compared, which meant the test would fail if
run when UTC and local time didn't represent the same date (and conversely
would pass if the UTC and local date was the same date, which is why the
changed did not fail the PR test run: the PR was tested during the 19 hours of
the day when EST and UTC represent the sam date).
Fix this by converting the UTC `now` to NSDate instead of using NSDate.Now.
This has the additional benefit of also fixing a (much smaller) race
condition: if midnight occurred just between calculating `now` and NSDate.Now,
the test would also fail.
Rolf Bjarne Kvinge
Manuel de la Pena
Alex Soto
Sebastien Pouliot
Chris Hamons
Bernhard Urban
Alex Soto
yaliashkevich
Timothy Risi