Commits are:
* [domain] Don't add NULL runtime to runtimes list (#18265) dc5902c243
* [eglib] Don't define _FORTIFY_SOURCE on Cygwin builds (#18312) 5f0704ce37
* Socket.BeginMConnect() should not attempt connections on unsupported address families. 99e3aed6ec
* [2019-10] [metadata] Size 0 Blob heap is ok when resolving assembly refs (#18338) 2faea92a92
* [FileSystemWatcher] Fix wrong behaviour when move multiple files out of watched folder on mac (#41963) df42020fe6
Complete diff is: bfcac9bd30...df42020fe6
The order of the check matters, we only skip with a sim && not on
xcode11.
The other whay around will do the wrong short circuit. Since if we are
not on Xcode11 we will always skip, including device, which is not
correct.
Fixes: https://github.com/xamarin/xamarin-macios/issues/6243.
The 32bits **debug** binaries are now too big for Apple's native linker
to process, which gives us (non useful) build errors on the bots.
This will still run the release builds configuration of the tests since
they are smaller and still within the limits of the tooling.
TL&DR
* re-apply the fix to cache.cs from https://github.com/xamarin/xamarin-macios/pull/7544
* which was reverted in https://github.com/xamarin/xamarin-macios/pull/7589
* since it regressed mscorlib/sim testing in xharness (for other reasons)
* Final part to fix https://github.com/xamarin/xamarin-macios/issues/7514
This was the ~night~ day before christmas... amd a tough nut to crack!
Thanksfully we had a good test case (inside #7514) and then xharness
regressed one test in consistent, reproducible manner.
xharness builds mscorlib tests twice (32 and 64 bits) even if it's a
fat application (could be reused). That should not be a huge problem
since the 2nd build should be identical and the cache should be (re)used.
An earlier attempt fixed this (comparison was true for the wrong
reasons [1]) but the fix did not end up with the same arguments !?! and
was reverted.
This is the diff between the first and second builds:
```diff
--- /Users/poupou/a.txt 2019-12-23 09:55:01.000000000 -0500
+++ /Users/poupou/b.txt 2019-12-23 09:55:01.000000000 -0500
@@ -182,5 +182,5 @@
-r=/Users/poupou/git/xamarin/xamarin-macios/builds/downloads/ios-release-Darwin-8f396bbb408b5758fccb8602030b9fa5293ce718/ios-bcl/monotouch/tests/Xunit.NetCore.Extensions.dll \
' --target-framework=Xamarin.iOS,v1.0' \
--root-assembly=/Users/poupou/git/xamarin/xamarin-macios/tests/xharness/tmp-test-dir/mscorlib/bin/mscorlib/iPhoneSimulator/Debug-unified/com.xamarin.bcltests.mscorlib.exe \
- ' -v -v -v -v' \
+ ' -v -v' \
@/Users/poupou/git/xamarin/xamarin-macios/tests/xharness/tmp-test-dir/mscorlib/obj/iPhoneSimulator/Debug/response-file.rsp \
```
Since they are not identical the cache is invalidated (which is normal,
cache-wise) and produce an output app that is incorrect (and crash
32bits).
Now there is code to ignore verbosity options (both `-v` and -q`) since
they will not affect what `mtouch` generates. However this was broken
because mtouch's response-file parser is quite basic and stricter the the
specification
spec: https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/compiler-options/response-file-compiler-option
issue: https://github.com/xamarin/xamarin-macios/issues/7644
That failed in two different ways
1. note the extra space before the first `-v` in the diff (before the `'`
quote). That skipped the line.
2. there are multiple `-v` in the same line, again that make the
filtering skip the line.
*Unknowns*
It's too close to xmas/vacation so I might not find the reasons/issues
for the following, unanswered questions...
1. Why is the re-build app bundle failing at runtime when p/invoking ?
Something is not regenerated (symbol maps?) ?
2. Why xharness 2nd build has more verbosity than the first one (likely
harmless) ?
[1] the original cache.cs issue (prequel)
issue w/test case: https://github.com/xamarin/xamarin-macios/issues/7514
first attempt: https://github.com/xamarin/xamarin-macios/pull/7544
While incorrect the first attempt to fix `cache.cs` was a logical, if
not entirely complete, fix. Without it this is what we _currently_ cache:
```
/Users/poupou/git/xamarin/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/lib/mtouch/mtouch.exe \
```
and that does not include any of mtouch's arguments, that can change
between executions and (should) invalidate the cache.
In this case it means the cache is used (no difference) but this does
**not** parse the content of the **response file** which is obviously
wrong (and we do have code to process it).
On the original issue's test case this is what makes the difference
between using the same *old* nuget assembly after an update (and fail)
by itself and also because the updated framework was not copied (due
to the 2nd part of the bug report wrt `copyfile`).
OTOH re-using (incorrectly) the cache is what makes xharness's mscorlib
unit tests works right now :(
Some types are not available in any 32bits iOS versions. We are _mostly_
(but not always) throwing a `PlatformNotSupportedException` when they are
used. However we're missing cases where we can reduce the generated code.
This is one (of a few) occurrences.
New generated code looks like:
```csharp
[BindingImpl (BindingImplOptions.GeneratedCode | BindingImplOptions.Optimizable)]
static readonly IntPtr class_ptr;
static readonly IntPtr class_ptr = Class.GetHandle ("ARFaceGeometry");
```
Beside the obvious `Class.GetHandle` call removal, this has the nice,
side benefit of removing several `.cctor` from the assembly (that were
only needed to set this field).
This reduce the code size for the 32bits version of Xamarin.iOS.dll [1]
```
Before: 16,338,944
After: 16,255,488
Diff: 83,456
```
[1] While this might not sound very useful, since it's going away, its
becoming a problem (because it's not going away fast enough) since we're
not able to build some (non-linked) test suites anymore due to size
constraints (and Apple's `ld` branching limit).
* [Makefile] Make csc strict and fix some small errors.
Make csc to bee more strict when compiling the projects and mix some
small errors we had in the bindings.
Fixes: https://github.com/xamarin/xamarin-macios/issues/5398
There are two important things to look at this:
1. That we do set the cookies that are present in the CookieContainer in
the request. That is, we need to set Cookie headers for all of them.
2. That if we receive a Set-Cookie from the server, the CookieContainer
gets correctly updated else we will have issues since we do not respect
the data sent from the server side.
Tests show both that we are setting the Cookie header and that we honour
the Set-Cookie header.
Fixes: https://github.com/xamarin/xamarin-macios/issues/5665
The tests in xunit are ran in parallel, the default number of threads is
to large which is making the iOS watchdog kill the process before we are
done with the tests.
The default value is either 0 (use as many as possible) of the
Enviroment.ProcessorsCount value, in our case we divide that by two to
make sure we do not use too much CPU.
Fixes: https://github.com/xamarin/maccore/issues/2083
Since the "quote" refactoring there's no real difference between `Add*`
and `Add*Line` methods, everything is now line-oriented.
However `Add` and `AddQuoted` behaved differently, which meant that `Add`
would add a whitespace before a new argument in the `StringBuilder` was
not empty (as a separator needed _when_ different lines were not used).
That resulted in response files that had some lines starting with a white
space. That was not an issue (spec [1] wise) but it broke some (not so
correct) assumptions in `mtouch` and could also break anything that added
comments (which must start with `#`) in the future.
This brings consistency in the output whether quoted/non-quoted arguments
are used.
Reference:
[0] https://github.com/xamarin/xamarin-macios/issues/7645
[1] https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/compiler-options/response-file-compiler-option
Inspiration was taken from other yaml pipelines but those multiple jobs are not behaving the way I expected.
1. Each job will clone `xamarin-macios` in a dedicated folder (for 3 jobs you'll have `./1/s`; `./2/s`; `./3/s`)
2. The goal of the last job was to upload the `TestSummary.md` but because of the cloning strategy it couldn't find it. There might be ways to pass files (artifacts?) between jobs but there's no need to be that fancy right now.
3. Cloning takes time so I'm assuming having everything in one job can actually save us some time.
- We had `./jenkins/vsts-device-tests-set-status.sh: line 88: HTML_REPORT: unbound variable` in the `Report results to GitHub as comment / status` step. Now that the HTML_REPORT is set this shouldn't happen anymore.
- The `Add summaries` task is reporting errors but it's working regardless. I believe this might be due to -x being set instead of the original +x (I changed that by mistake).
Bump mono to get the new splited test dlls and add them to be ran in
xharness. Special logic is used for mscorlib so we make sure that all
the 'parts' of the test dll do have the same configurations.
Co-Authored-By: Waleed Chaudhry <54864665+wachaudh@users.noreply.github.com>
- Needed for the DDFun migration of our tests
- Don't use SSH because it's restricted on the new bots
- Use brand new DevOps feature to clone multiple repos via HTTPS
- Update names of all tasks (clearer)
- Simplify and re-use code
- Update all paths because our cloning strategy is different compared to old pipeline
- Update xharness to print "experimental" titles in GitHub comments
- Update xharness to not add the Html link for tests from the DDFun pipeline
- Fix all timeouts to be relevant to the tests length
- Split in multiple jobs (for GitHub status)
- Move longer scripts to independant files
- Make inline bash prettier
- Use a variable for the pool name
- Fix single space alignment
- Remove unused variable in `vsts-device-tests-set-status.sh` that broke it
- Bump maccore to use the right certificates and profiles
- Bump maccore to target the right pipeline
- Bump maccore to not use ssh in scripts
- Bump maccore and use persistCredentials
Notes:
- Right now the xharness test output can only be seen when downloading the zip file locally (better output soon)
Fixes these warnings:
IPhoneSdks.cs(22,4): warning CS0618: 'AppleSdkSettings.Changed' is obsolete: 'This event is never raised' [/work/maccore/onedotnet/xamarin-macios/msbuild/Xamarin.iOS.Tasks.Core/Xamarin.iOS.Tasks.Core.csproj]
IPhoneSdks.cs(31,4): warning CS0618: 'AppleSdkSettings.CheckChanged()' is obsolete: 'This method does nothing' [/work/maccore/onedotnet/xamarin-macios/msbuild/Xamarin.iOS.Tasks.Core/Xamarin.iOS.Tasks.Core.csproj]
MacOSXSdks.cs(18,4): warning CS0618: 'AppleSdkSettings.Changed' is obsolete: 'This event is never raised' [/work/maccore/onedotnet/xamarin-macios/msbuild/Xamarin.Mac.Tasks.Core/Xamarin.Mac.Tasks.Core.csproj]
MacOSXSdks.cs(25,4): warning CS0618: 'AppleSdkSettings.CheckChanged()' is obsolete: 'This method does nothing' [/work/maccore/onedotnet/xamarin-macios/msbuild/Xamarin.Mac.Tasks.Core/Xamarin.Mac.Tasks.Core.csproj]
Also bump Xamarin.MacDev to pick up updated Xamarin.MacDev:
* xamarin/Xamarin.MacDev@0f578f5 [tests] Upgrade to NUnit 3.12 and use package references. (#65)
* xamarin/Xamarin.MacDev@55a30e2 [tests] Adjust tests to not expect the provisioning profiles in the index in any particular order. (#63)
* xamarin/Xamarin.MacDev@dc270f6 Make methods that do nothing obsolete and fix a compiler warning. (#67)
* xamarin/Xamarin.MacDev@681aef6 Remove unused csproj. (#66)
* xamarin/Xamarin.MacDev@d78a92f Update gitignore (#55)
Diff: ca221c8fd6..0f578f51e6
- https://devdiv.visualstudio.com/DevDiv/_workitems/edit/947932
- It turns out the linker in some cases can wrap expected exceptions in a outer exception. By drilling in we can produce better errors, again.
- Refactor pipeline exception handling to be shared between mmp/mtouch
Copying the framework could fail (error 260) and the failure was never
reported so the build succeeded - but without updating (completely) the
framework.
The exact reason it fails is unknown :( but we can recover from it by
deleting the target and copying (everything) back to the expected
(target) location.
Build logs will now indicate when this fails and will try to recover
before reporting a build error. Best case it works :) worse case we'll
be aware something is wrong (which is better than ignoring)
This is half the fix from https://github.com/xamarin/xamarin-macios/pull/7544
which was reverted due to the second half causing a regression (under
investigation).
Reference: https://github.com/xamarin/xamarin-macios/issues/7514 (partial fix)
Although we provide the correct property to set the proxy configuration,
we do not add a strong dict that will make it nicer for users. This
fixes that API.
Fixes: https://github.com/xamarin/xamarin-macios/issues/5811
* [sample-tester] Collect app size and build duration statistics.
* Always publish performance data.
* [mmp] Add support for MMP_ENV_OPTIONS to mirror mtouch's MTOUCH_ENV_OPTIONS.
* [sample-tester] Make mmp/mtouch show timing information, and get the diagnostic msbuild log.
* Collect task and target info.
* Add more perf data and a baseline test.
* Redo the base test a bit.
* More tweaks.
* More tweaks.
* Huh?
* Debug stuff.
* diagnostics.
* Better diagnostics.
* cleanup
* tmp
* Use a separate stage to push data.
* Next attempt.
* Unique artifact names.
* Make the right name unique.
* More progress
* Improvements
* Don't need new maccore.
* cleanup
* Remove debug spew.
* Realign stuff.
* duh
* More path fixes.
* zippity zip.
* Show publishing errors.
* dependencies
* Not fake results anymore.
* Tweaks
* Merge xml files
* [sampletester] Bump nuget timeout to 5 minutes.
The TodoREST.iOS test seems to use a lot of packages, and it can apparently
take a while to restore them, so give a few more minutes.
* Improvements.
* Fix glob.
* [sampletester] Fix fetching of sample repository and hash.
* [sampletester] Centralize verbosity for mtouch and mmp.
Some of the test assemblies are too large and will be splitted by mono so
that they can be compiled for iOS 32b. In that case, we are using the
following pattern
`
test_assembly_name.dll
`
becomes
`
test_assembly_name.part1.dll
test_assembly_name.part2.dll
`
Perse the only change we need to add to make our life easier is to be
able to mantain a single .ignore file since mantaining more .ignore
files is hard and more error prone. This change simply checks if we are
working with a splitted dll and ensures that the correct .ignore files
are added to the BCL test application.
Co-Authored-By: Pramit Mallick <prmallic@microsoft.com>
* [msbuild] Don't use gdk-sharp, use native functions instead.
There's no netstandard version of gdk-sharp.
* Don't require the source checkout to be named 'xamarin-macios'.
After this commit we will have more application but they will be the
appropiate size so that they can be built with the linker for iOS 32b.
It is important to note that the following apps WILL CONTINUE to fail
since the dlls need to be splitted in mono:
* mscorlib tests
* Mono BCL tests group 5 - Which is monotouch_System.Core_xunit-test.dll
and is too large.
Manuel de la Pena
Pramit Mallick
Sebastien Pouliot
Marius Ungureanu
Vincent Dondain
Rolf Bjarne Kvinge
Chris Hamons
Alex Soto
monojenkins