* Add a server mode, which launches a web server (and a web page) that can be
used to interactively run tests and view their results.
* Add support for running test assemblies in a today extension (generating a
new set of projects, similar to how we generate tvOS/watchOS projects based
on the iOS project, we now generate a today extension project in addition to
the tvOS and watchOS projects).
* Load all the different tests (and show them in the html report, although
they show up as 'ignored'), even for disabled/ignored tests. This makes
disabled/ignored tests more visible, and also makes it possible to actually
run them using the embedded web server.
* Add support for running tests on device. Tests will be executed on multiple
devices simulatenously (any connected devices will be used).
* [jenkins] Make jenkins non-parellel to see if it fixes#46097.
* [jenkins] Limit serialization to watchos builds.
* [jenkins] Only allow Mac builds to be paralellized.
* [mtouch] Always require a SDK version when building.
Technically it was required before too, but the error messages were non-optimal:
it could for instance complain that the user is using an iOS framework that
was introduced in iOS 2.0.
* [mtouch tests] Rewrite MT0060 and MT0061 tests to use MTouchTool.
This makes sure we pass --sdk to mtouch (which MTouchTool does by default), so
that we don't run into MT0025 before the errors we're testing for.
* [xharness] Add Jenkins support for mac binding project tests.
* [jenkins] Rework how mac binding projects are tested.
Facts:
* The wrench-mac-binding-project target will clean itself when done: [1]
* The mac-msbuild tests depend on mac-binding-project build artifacts (but
will build those artifacts automatically if they don't exist when the mac-
msbuild tests are built).
* Jenkins runs tests in parallel.
Result:
* The mac-msbuild tests may fail if the mac-binding-project tests are run
concurrently and delete its build artifacts.
Fix:
* Don't call the wrench-mac-binding-project target (which will clean itself),
instead just run the tests (without cleaning).
[1]: d99212a712/tests/Makefile (L280-L282)
The BundleId property is used by the code that generates the mSYM directory,
but its value was always the default value 'com.yourcompany.sample' instead of
looked up in the app's Info.plist.
So fix the BundleId property to do the expected.
Also fix the mSYM test (SymbolicationData) to actually test mSYM stuff (it was
partially disabled when we disabled automatic mSYM generation for C8, and
never re-enabled), and port it to the new and better test syntax, and add a
few more asserts to check the manifest.xml generation.
-lsqlite3 is a linker flag, not a file to be linked with, so when
automatically determining that we need to pass -lsqlite3 we need to put it in
the right list of linker information.
Otherwise we may end up passing `-force_load -lsqlite3` to the linker (if the
assembly's ForceLoad flag is set), which won't compile.
https://bugzilla.xamarin.com/show_bug.cgi?id=49220
dontlink/64-bit release times out on our Sierra bots, so try to bump the
timeout to see if this is working on other bots because those other bots are
faster.
* [tests] Fix framework-test to actually work.
* [xharness] Properly replace 'ios' with corresponding platform for paths to our test frameworks as well.
* [framework-test] Fix watchOS build.
* Bump maccore to get new mlaunch.
A new mlaunch that:
* Should have fewer random failures when launching watchOS apps.
* Supports launching extensions on device.
* Supports uninstalling apps from devices.
* [jenkins] Automatically detect mono bumps and enable device build.
And do this before fetching labels, so that we can skip fetching labels if we
know we're already enabling the device build.
* [tests] Bumping LLVM merits enabling device build and running mtouch + BCL tests.
e.g. running twice
> make run-ios-sim32-introspection
results in
Unhandled Exception:
System.AggregateException: One or more errors occurred. ---> System.IO.IOException: /Users/poupou/git/master/xamarin-macios/tests/logs/exec-ios-sim32-introspection/iPhone 5.log already exists
at System.IO.File.Copy (System.String sourceFileName, System.String destFileName, System.Boolean overwrite) [0x001bd] in /private/tmp/source-mono-4.8.0/bockbuild-mono-4.8.0-branch/profiles/mono-mac-xamarin/build-root/mono-x86/mcs/class/corlib/System.IO/File.cs:109
at System.IO.File.Copy (System.String sourceFileName, System.String destFileName) [0x00000] in /private/tmp/source-mono-4.8.0/bockbuild-mono-4.8.0-branch/profiles/mono-mac-xamarin/build-root/mono-x86/mcs/class/corlib/System.IO/File.cs:69
at xharness.CaptureLog.StopCapture () [0x00019] in /Users/poupou/git/master/xamarin-macios/tests/xharness/Log.cs:252
Also:
* some refactoring to reduce the reflection usage for each field-based
introspection tests;
* some fixes for existing bindings, mostly missing [Notification]
* removal of `unsafe` in the notification binding generation (not needed)
* Ignore the new test on XM until the API have been fixed
This reduce the metadata size and this information, even if part of the
header files, is not required (as some types are just not refcounted)
E.g.
public bool MicrophoneEnabled {
[Export ("isMicrophoneEnabled", ArgumentSemantic.UnsafeUnretained)]
should be
public bool MicrophoneEnabled {
[Export ("isMicrophoneEnabled")]
This could have been done in different places but not generating them has
the smallest impact versus:
1. Check bindings input and report them as errors
- con: break existing binding code;
- con: sharpie outputs them;
2. Removed by the linker
- con: linking not always enabled, e.g. 3rd party bindings
- con: extra logic == extra time for each build
Generator diff
https://gist.github.com/spouliot/cc36e68bf7bd6097064ed6ba0bb3275a
Commit ba37aa44 workaround around a signature clash incorrectly and
turned the selector to static ones (and incorrectly set the handle)
Also fix a typo in the [Advice] attribute of the old API
https://bugzilla.xamarin.com/show_bug.cgi?id=48382
* [XM] Add 32-bit variants of apitest and introspection and fix bindings
- Previous to this we were only running 64-bit tests, which caused us
to make mistakes
* Fix cherry-pick to build
* Correctly skip jenkins XM45 when asked
* Fix logic
* Run 32-bit tests on Jenkins
* Fix Jenkins grouping by respecting -32 suffix
* More fixing of jenkins. This time make enum for 32-bit types
* Enable default constructor on NEPacketTunnelProvider
* NEPacketTunnelNetworkSettings needs the same constructor as parent class
* skip NEPacketTunnelProvider in ctor init test
From @VincentDondain
`NEPacketTunnelProvider *a = [[NEPacketTunnelProvider alloc] init];` return `nil` without these entitlements:
```
<key>com.apple.developer.networking.networkextension</key>
<array>
<string>packet-tunnel-provider</string>
<string>app-proxy-provider</string>
<string>content-filter-provider</string>
</array>
```
Note: you're also supposed to call it from its "extension process" otherwise you get: `NEProvider objects cannot be instantiated from non-extension processes.`
Hence from an iOS App Extension that has the `com.apple.networkextension.packet-tunnel` extension point.
* [XM] Change mmp defaults to static registrar and disable lldb attach in release mode
- Static registrar is now proven and can be flipped on for default on release
- LLDB attach isn't very helpful in release mode and pops up the install xcode dialog
- Relax Unified_HelloWorld_ShouldHaveNoWarnings to exclude static registrar warnings due to https://bugzilla.xamarin.com/show_bug.cgi?id=48311
Trello: https://trello.com/c/mKsUDti8
This adds a new overload to our Notifications class
Current:
public static NSObject ObserveDidStart (EventHandler<NSNotificationEventArgs> handler)
New overload:
public static NSObject ObserveDidStart (NSObject objectToObserve, EventHandler<NSNotificationEventArgs> handler)
This allows our users to have single object subscription to our
easy to find notifications.
Also this commit adds an Advice attribute encouraging to use the
Notifications class instead of the NSString based notifications
The previous method of getting the base and head sha from the pull request
doesn't work if the pull request has been rebased.
Example: https://github.com/xamarin/xamarin-macios/pull/1238
Resulted in this [1]:
Fetching modified files for commit range cf07825667aa444c988c82b7e29cefc5f8ba7bcd..6aa2b9517ac35374dfa4ded41d1e2ff52778da07
git diff-tree --no-commit-id --name-only -r cf07825667aa444c988c82b7e29cefc5f8ba7bcd..6aa2b9517ac35374dfa4ded41d1e2ff52778da07
Found 20 modified file(s) in the pull request #1238.
Makefile
Versions-ios.plist.in
Versions-mac.plist.in
external/llvm
external/mono
msbuild/Xamarin.Mac.Tasks.Core/Tasks/MmpTaskBase.cs
msbuild/Xamarin.Mac.Tasks/Xamarin.Mac.Common.props
msbuild/Xamarin.Mac.Tasks/Xamarin.Mac.Common.targets
msbuild/Xamarin.iOS.Tasks.Core/Tasks/MTouchTaskBase.cs
msbuild/Xamarin.iOS.Tasks.Core/Xamarin.iOS.Common.props
msbuild/Xamarin.iOS.Tasks.Core/Xamarin.iOS.Common.targets
src/Makefile
src/UIKit/UIScreen.cs
src/appkit.cs
src/generator-diff.mk
tests/introspection/ApiTypoTest.cs
tests/introspection/Mac/MacApiTypoTest.cs
tests/introspection/Mac/introspection-mac.csproj
tests/xharness/Harness.cs
versions-check.csharp
which is listing way too many files.
[1]: https://jenkins.mono-project.com/job/xamarin-macios-pr-builder/2118/Test_Report/Harness.log
* [jenkins] Add support for enabling device builds using labels.
* [xharness] Give the iOS MSBuild tests 30 minutes to finish.
* [mtouch tests] Give the BuildTestProject 10 minutes to compile each test case.
Wrench bots build the dontlink test in ~3m40, but that's apparently not enough
for the Jenkins bots (slower bots?), which time out the test after 5 minutes.
So double the timeout to 10 minutes, which will hopefully give the Jenkins
bots enough time to run the test to completion.
* [xharness] Don't crash if we can't find a simulator.
* [xharness] Create a device pair if none applicable is found.
* [xharness] Use an enum instead of string values for the target.
* [xharness] Unify the simulator selection code between Jenkins and Wrench.
* [Jenkins] Make test to write output as an xml file so that it can be parsed by the jenkins bot.
* Point to the correct Touvh.Unit repo.
* Use the available property to determine if we are being ran in Jenkins.
* Log where are test results stored.
* Add @MonkeyWrench: prefix.
* Ensure that we do set the build env in jenkins/run-tests.sh
* Do not mix Wrench with Jenkins. The reports in jenkins can be Xml, in Wrench we prefer the old style.
* Ensure that the main node of the unit tests does contain the target, that will improve the tests results reporting.
* Revert "Fix binding project LinkWithAttributes generation to prevent rebuilds" (#1018)
* Added xslt to be used to keep the old Test Reports until we move to only Jenkins reports.
* Add an extra log for the xslt transformation.
* Point to the correcto dir in jenkins.
* Deal with the xslt once we have finished rather than in a batch.
* Remove noise.
* Readd case removed in rebase.
* Fix indentation.
* Skip lock keychain.
- https://bugzilla.xamarin.com/show_bug.cgi?id=46508
Since we were previously looking for the .exe instead of the launcher, mmp
failures would come back as good and we wouldn't rebuild. What we want
to do is look for the native launcher, which we perviously were doing wrong.
Example log output when bumping mono:
> Found 1 modified file(s) in the pull request #1161.
> external/mono
> Enabled 'mtouch' tests because the modified file 'external/mono' matches prefix 'external/mono'
> Enabled 'mmp' tests because the modified file 'external/mono' matches prefix 'external/mono'
> Enabled 'bcl' tests because the modified file 'external/mono' matches prefix 'external/mono'
> Found 1 label(s) in the pull request #1161: cla-already-signed
Or when changing the static registrar:
> Found 1 modified file(s) in the pull request #1164.
> tools/common/StaticRegistrar.cs
> Enabled 'mtouch' tests because the modified file 'tools/common/StaticRegistrar.cs' matches prefix 'tools/common'
> Enabled 'mmp' tests because the modified file 'tools/common/StaticRegistrar.cs' matches prefix 'tools/common'
> Found 2 label(s) in the pull request #1164: cla-already-signed, run-mmp-tests
> Enabled 'mmp' tests because the label 'run-mmp-tests' is set.
This reverts commit abb0449b6c.
_Seriously_ this time. Here's the story:
- TouchBar support was breaking tests in master, even after my static
registrar fix in 6422000c27
- However, the fix wasn't broad enough, because the TouchBar APIs were
in Xcode 8.1 as well.
- Confusion on my end (You need latest macOS but not latest Xcode) and
confusion on others (since there were changes in Xcode 8.2 to touchbar,
but they were subtractive, caused us to think this needed to go in
Xcode 8.2
Convert MT1016 and MT1017 to newer test syntax, and at the same time change
them to not disable the managed linker.
For these tests it doesn't matter if they're linked or not, but linking is
much faster (20s vs 82s for both tests).
In 7c6d04f1 the code to create the NOTICE file was simplified, and the new
implementation is writing to a temporary file and then replacing the existing
file.
This makes the scenario that MT1017 was testing (failure if a readonly NOTICE
file already exists) go away, since we don't write to the existing file
anymore (so the build succeeds).
- https://bugzilla.xamarin.com/show_bug.cgi?id=46508
Since we were previously looking for the .exe instead of the launcher, mmp
failures would come back as good and we wouldn't rebuild. What we want
to do is look for the native launcher, which we perviously were doing wrong.
Store strings in the exception data when converting CFError to CFException, to
make sure the data stored is serializable (which is apparently a requirement
with the reference sources).
https://bugzilla.xamarin.com/show_bug.cgi?id=46626
The old `legacy` option will now be reported as a warning.
That's by design an warning would require manually editing the .csproj
file (when the UI gets removed, as planned, from the IDE).
This is part of
https://trello.com/c/SrgU38DN/647-only-ship-support-appletls
Note: The BCL changes will happen in later stages.
Earlier versions of Xamarin Studio stored an invalid http message handler in
watchOS project files, which would cause a build error. In addition Xamarin
Studio removed the UI to set the http message handler (since only one value is
valid), which meant that the user had to edit the project file by hand to get
around this build error.
So make it a warning instead (and document what the user has to do to fix the
warning).
https://bugzilla.xamarin.com/show_bug.cgi?id=46552
This makes us only put packages in one directory (saves disk space and time),
and it also makes project files in multiple solutions work properly
(mtouch.csproj is in tests/tests.sln and tests/mtouch/mtouch.sln).
Rolf Bjarne Kvinge
Vincent Dondain
Chris Hamons
Sebastien Pouliot
Alex Soto
Alex Soto
batmaninpink
Timothy Risi
Manuel de la Pena