Fixes this NUnit warning:
> labels=All is deprecated and will be removed in a future release. Please use labels=Before instead.
We don't follow the suggestion from the warning, because the advantage of
writing the label after each test is that the test result will also be
printed, which means it's possible to see if any tests failed during the test
run, as opposed to having to wait until the entire test run is completed
(which can take a while) to realize that pretty much every test failed with
some silly mistake which could have been quickly fixed before re-running the
tests.
* [tests][sampletester] Add nuget.org single feed to sampletester
Since we no longer upstream to nuget.org our sampletester is not
able to find packages so we create a nuget config file with a single
source to nuget.org to fix it and still be compliant.
* [tests][sampletester] Remove ConferenceVision sample, it no longer exists
* [tests][sampletester] Double TimeOut for a couple of tests
This makes it so that we use the configured Xcode, and not the default (/Applications/Xcode.app), which may be a symlink, which doesn't work with Xcode 12:
Tool /Applications/Xcode.app/Contents/Developer/usr/bin/ibtool execution started with arguments: --errors --warnings --notices --output-format xml1 --minimum-deployment-target 8.0 --target-device iphone --target-device ipad --auto-activate-custom-fonts --sdk /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS14.2.sdk --compilation-directory /Users/runner/work/1/s/xamarin-macios/tests/sampletester/BaselineTest/obj/iPhone/Debug/ibtool /Users/runner/work/1/s/xamarin-macios/tests/sampletester/BaselineTest/LaunchScreen.storyboard
(TaskId:13)
Tool /Applications/Xcode.app/Contents/Developer/usr/bin/ibtool execution finished (exit code = 134).
(TaskId:13)
LaunchScreen.storyboard : error : 2020-10-22 17:47:39.854 ibtoold[5120:92020] [MT] DVTAssertions: Warning in /Library/Caches/com.apple.xbs/Sources/DVTFrameworks/DVTFrameworks-17515.1/DVTFoundation/PlugInArchitecture/DataModel/DVTPlugIn.m:232 [/Users/runner/work/1/s/xamarin-macios/tests/sampletester/BaselineTest/BaselineTest.csproj]
LaunchScreen.storyboard : error : Details: Requested but did not find required plug-in with identifier com.apple.rc.RCIDESupportCore [/Users/runner/work/1/s/xamarin-macios/tests/sampletester/BaselineTest/BaselineTest.csproj]
LaunchScreen.storyboard : error : Object: <DVTPlugIn: 0x7fe30d129560> [/Users/runner/work/1/s/xamarin-macios/tests/sampletester/BaselineTest/BaselineTest.csproj]
LaunchScreen.storyboard : error : Method: -awakeWithPropertyList: [/Users/runner/work/1/s/xamarin-macios/tests/sampletester/BaselineTest/BaselineTest.csproj]
LaunchScreen.storyboard : error : Thread: <NSThread: 0x7fe30780acc0>{number = 1, name = main} [/Users/runner/work/1/s/xamarin-macios/tests/sampletester/BaselineTest/BaselineTest.csproj]
LaunchScreen.storyboard : error : Please file a bug at https://feedbackassistant.apple.com with this warning message and any useful information you can provide. [/Users/runner/work/1/s/xamarin-macios/tests/sampletester/BaselineTest/BaselineTest.csproj]
LaunchScreen.storyboard : error : 2020-10-22 17:47:39.863 ibtoold[5120:92020] Requested but did not find extension point with identifier Xcode.InterfaceBuilderBuildSupport.PlatformDefinition for extension Xcode.IBCocoaTouchBuildSupport.PlatformDefinition.MacCatalyst of plug-in com.apple.dt.IDE.IBCocoaBuildSupport [/Users/runner/work/1/s/xamarin-macios/tests/sampletester/BaselineTest/BaselineTest.csproj]
LaunchScreen.storyboard : error : 2020-10-22 17:47:39.863 ibtoold[5120:92020] Requested but did not find extension point with identifier Xcode.InterfaceBuilderBuildSupport.PlatformDefinition for extension Xcode.IBCocoaBuildSupport.PlatformDefinition.Cocoa of plug-in com.apple.dt.IDE.IBCocoaBuildSupport [/Users/runner/work/1/s/xamarin-macios/tests/sampletester/BaselineTest/BaselineTest.csproj]
LaunchScreen.storyboard : error : 2020-10-22 17:47:39.880 ibtoold[5120:92020] Requested but did not find extension point with identifier Xcode.InterfaceBuilderBuildSupport.PlatformDefinition for extension Xcode.IBCocoaTouchBuildSupport.PlatformDefinition.CocoaTouch of plug-in com.apple.dt.IDE.IBCocoaTouchBuildSupport [/Users/runner/work/1/s/xamarin-macios/tests/sampletester/BaselineTest/BaselineTest.csproj]
LaunchScreen.storyboard : error : 2020-10-22 17:47:39.880 ibtoold[5120:92020] Requested but did not find extension point with identifier Xcode.InterfaceBuilderBuildSupport.PlatformDefinition for extension Xcode.IBAppleTVBuildSupport.PlatformDefinition.AppleTV of plug-in com.apple.dt.IDE.IBAppleTVBuildSupport [/Users/runner/work/1/s/xamarin-macios/tests/sampletester/BaselineTest/BaselineTest.csproj]
LaunchScreen.storyboard : error : 2020-10-22 17:47:39.883 ibtoold[5120:92020] Requested but did not find extension point with identifier Xcode.InterfaceBuilderBuildSupport.PlatformDefinition for extension Xcode.IDEInterfaceBuilder.PlatformDefinition.WatchOS of plug-in com.apple.dt.IDE.IDEInterfaceBuilderWatchKitBuildSupport [/Users/runner/work/1/s/xamarin-macios/tests/sampletester/BaselineTest/BaselineTest.csproj]
LaunchScreen.storyboard : error : 2020-10-22 17:47:39.979 ibtoold[5120:92020] [MT] DVTAssertions: ASSERTION FAILURE in /Library/Caches/com.apple.xbs/Sources/IDEInterfaceBuilder/IDEInterfaceBuilder-17506/InterfaceBuilderKit/Utilities/IBScopedSingletonRegistry.m:45 [/Users/runner/work/1/s/xamarin-macios/tests/sampletester/BaselineTest/BaselineTest.csproj]
Make the bgen tests pass in the path to the attribute library, platform
assembly and all the .NET reference assemblies to bgen. This way we execute
these tests using the .NET version of everything.
* Fix links that point to master to point to main instead.
* Implement support in the sample tester for specifying the default branch for
each sample repo.
* Fix various text / documentation to say 'main' instead of 'master.'
* Push to 'main' instead of 'master' in xamarin-macios-data.
* Fix xharness to make 'main' the special branch with regards to documentation tests as opposed to 'master'.
* Fix various CI to use 'main' instead of 'master'.
This is a backport of PR #9561
* Create a simple Xamarin.Utils.Execution class that can handle all our
process execution needs:
* Captures or streams stdout/stderr (in UTF8).
* Supports async
* Supports a timeout
* Does not depend on any other source file we have, only uses BCL API.
* Have the execution helper classes from mtouch/mmp
(Xamarin.BundlerDriver.RunCommand) and the tests
(Xamarin.Tests.ExecutionHelper) use this new class.
* Some simplifications were made:
* All API that took a string array for the environment now takes a
Dictionary<string, string>.
* The Driver.RunCommand methods were split out to a separate file. This
file also contains a Verbosity field, which is conditioned on not being
in mtouch nor mmp, which makes including this file from other projects
simpler (such as bgen - in particular bgen was modified to use this
Verbosity field instead of its own).
* Fix links that point to master to point to main instead.
* Implement support in the sample tester for specifying the default branch for
each sample repo.
* Fix various text / documentation to say 'main' instead of 'master.'
* Push to 'main' instead of 'master' in xamarin-macios-data.
* Fix xharness to make 'main' the special branch with regards to documentation tests as opposed to 'master'.
* Fix various CI to use 'main' instead of 'master'.
* Bump maccore
New commits in xamarin/maccore:
* xamarin/maccore@ed6d146822 Rename 'master' to 'main'. (#2233)
Diff: 424fa26148..ed6d146822
Goals
* Reflect Apple nullability annotations in our bindings using C#8
* No warnings when building bindings
Non-Goals
* Update (add or fix) `[NullAllowed]` to match Apple headers (next phase)
* Make the generator or internal code fully nullable aware (`nowarn` is used)
Notes
* Apple's own annotations are not 100% accurate :(
* Where known issue exists we have _fixed_ our attributes to match reality :)
* We also do additional null-checks internally that might seems not required (better safe than sorry).
Goals
* Reflect Apple nullability annotations in our bindings using C#8
* No warnings when building bindings
Non-Goals
* Update (add or fix) `[NullAllowed]` to match Apple headers (next phase)
* Make the generator or internal code fully nullable aware (`nowarn` is used)
Notes
* Apple's own annotations are not 100% accurate :(
* Where known issue exists we have _fixed_ our attributes to match reality :)
* We also do additional null-checks internally that might seems not required (better safe than sorry).
Most false positives are (so far) because `nuget restore` fails - both
due to network issues and a mono issue [1].
This PR allows to filter/remove some projects from a solution before
nuget is invoked. This can reduce the number of packages to download
which, beside reducing false positives, is good performance wise.
It also allows some project (e.g. Xappy.iOS) to build even if some other
projects (e.g. Xappy.UWP) in the solution can't be built.
[1] https://github.com/mono/mono/issues/15418
* [sample-tester] Collect app size and build duration statistics.
* Always publish performance data.
* [mmp] Add support for MMP_ENV_OPTIONS to mirror mtouch's MTOUCH_ENV_OPTIONS.
* [sample-tester] Make mmp/mtouch show timing information, and get the diagnostic msbuild log.
* Collect task and target info.
* Add more perf data and a baseline test.
* Redo the base test a bit.
* More tweaks.
* More tweaks.
* Huh?
* Debug stuff.
* diagnostics.
* Better diagnostics.
* cleanup
* tmp
* Use a separate stage to push data.
* Next attempt.
* Unique artifact names.
* Make the right name unique.
* More progress
* Improvements
* Don't need new maccore.
* cleanup
* Remove debug spew.
* Realign stuff.
* duh
* More path fixes.
* zippity zip.
* Show publishing errors.
* dependencies
* Not fake results anymore.
* Tweaks
* Merge xml files
* [sampletester] Bump nuget timeout to 5 minutes.
The TodoREST.iOS test seems to use a lot of packages, and it can apparently
take a while to restore them, so give a few more minutes.
* Improvements.
* Fix glob.
* [sampletester] Fix fetching of sample repository and hash.
* [sampletester] Centralize verbosity for mtouch and mmp.
* Implement a different escaping/quoting algorithm for arguments to System.Diagnostics.Process.
mono changed how quotes should be escaped when passed to
System.Diagnostic.Process, so we need to change accordingly.
The main difference is that single quotes don't have to be escaped anymore.
This solves problems like this:
System.ComponentModel.Win32Exception : ApplicationName='nuget', CommandLine='restore '/Users/vsts/agent/2.158.0/work/1/s/tests/sampletester/bin/Debug/repositories/ios-samples/WorkingWithTables/Part 3 - Customizing a Table\'s appearance/3 - CellCustomTable/CellCustomTable.sln' -Verbosity detailed -SolutionDir '/Users/vsts/agent/2.158.0/work/1/s/tests/sampletester/bin/Debug/repositories/ios-samples/WorkingWithTables/Part 3 - Customizing a Table\'s appearance/3 - CellCustomTable'', CurrentDirectory='/Users/vsts/agent/2.158.0/work/1/s/tests/sampletester/bin/Debug/repositories', Native error= Cannot find the specified file
at System.Diagnostics.Process.StartWithCreateProcess (System.Diagnostics.ProcessStartInfo startInfo) [0x0029f] in /Users/builder/jenkins/workspace/build-package-osx-mono/2019-08/external/bockbuild/builds/mono-x64/mcs/class/System/System.Diagnostics/Process.cs:778
ref: https://github.com/mono/mono/pull/15047
* Rework process arguments to pass arrays/lists around instead of quoted strings.
And then only convert to a string at the very end when we create the Process
instance.
In the future there will be a ProcessStartInfo.ArgumentList property we can
use to give the original array/list of arguments directly to the BCL so that
we can avoid quoting at all. These changes gets us almost all the way there
already (except that the ArgumentList property isn't available quite yet).
We also have to bump to target framework version v4.7.2 from v4.5 in several
places because of 'Array.Empty<T> ()' which is now used in more places.
* Parse linker flags from LinkWith attributes.
* [sampletester] Bump to v4.7.2 for Array.Empty<T> ().
* Fix typo.
* Rename GetVerbosity -> AddVerbosity.
* Remove unnecessary string interpolation.
* Remove unused variable.
* [mtouch] Simplify code a bit.
* Use implicitly typed arrays.
Use simply expanded variable as test result filename, so that it's only
calculated once. This avoids ending up with different filenames when the
timestamp in the filename changes between calculations, such as:
Results (nunit3) saved as /Users/vsts/agent/2.150.3/work/1/s/tests/sampletester/TestResult-20190511-100657.xml
warning: failed to load external entity "TestResult-20190511-100658.xml"
unable to parse TestResult-20190511-100658.xml
make: *** [run-tests] Error 6
* [tests] Add sample tester.
Add a unit project that looks for iOS/macOS/tvOS sample projects in several
repositories, and builds them all.
* [tests][sampletester] Remove known issue which has now been fixed.
* [tests] Only run sample tests on CI in Azure Devops.
* Remove the possibility of automatically running the sample tests with
xharness (so the sample tests won't run on PR bots or internal bots when the
'run-all-tests' label is added). It's still possible to run the sample tests
manually from the xharness web UI.
* Automatically trigger the sample test run in Azure Devops if the
'run-sample-tests' label is applied to a PR (and that PR is executed on
internal Jenkins).
* Fix typo.
* Fix path.
* Verbose output to track down scheduling failure.
* Bump maccore to get improved debug spew.
Diff: f527c9c526..f89d74b165
* [tests][sampletester] Fix build for TodoWCF.
Rolf Bjarne Kvinge
Alex Soto
Manuel de la Pena
Sebastien Pouliot
monojenkins
Waleed Chaudhry