Stop building the test dependencies on each test run, and instead use the archived test dependencies we have from the main build:
* Stop running 'make all' in tests/ on every separate test run.
* Add a lot more stuff in the package-test-libraries.zip archive.
* Extract all the new stuff on every test run. We add stuff from outside the tests/
directory, so adjust archive creation and extraction to use the root directory
of the repository as the root of the zip archive as well.
* Also add the introspection dependencies to the same archive to simplify the logic.
* Fix xharness to not store absolute paths in generated projects.
* Fix test project to not automatically run make in tests/test-libraries when running
on the bots.
Building the test dependencies takes ~10 minutes for each test run, so this saves
about that time for each test run.
Implement a launch timeout for macOS and Mac Catalyst apps where if a certain
environment variable (LAUNCH_SENTINEL_FILE) is set, the app will create that
file at launch. The code launching the test app will wait 10 seconds and check
if the file is there: if it's not, something went wrong, in which case the app
should be terminated and launched again.
This necessitated re-implementing the launch script in C#, since it got quite
complicated to implement in bash.
This fixes an issue with Mac Catalyst apps where something would go wrong
during the app launch and nothing would happen (but the app wouldn't be
deadlocked, it would just sit there, doing nothing).
The TestRuntime.cs and ApplePlatform.cs had to be added to a few test projects
to make this compile, which required a few fixes in these files for building
with legacy Xamarin.Mac.
Fixes https://github.com/xamarin/maccore/issues/2414.
Ask ditto to thin native libraries and frameworks when copying them to the app
bundle to remove slices for architectures we're not building for.
Also add tests.
Fixes https://github.com/xamarin/xamarin-macios/issues/13081.
Co-authored-by: TJ Lambert <50846373+tj-devel709@users.noreply.github.com>
I recently deleted the generated makefile support for building and running our
test suites. It turned out that it was used for building the packaged
Xamarin.Mac tests, so it wasn't as unused as I thought.
So fix the building and packaging of Xamarin.Mac tests to not use the
(non-existent) makefile targets, but instead replicate it with manual make
code.
Also take the opportunity to add packaging and execution of the .NET versions
of these test suites we execute on other macOS versions (both for macOS and
the Mac Catalyst).
* [devops] Use stricter matching when finding the Xamarin.Mac pkg link.
Otherwise the branch name in any package could end up matching the pattern we
were looking for:
XM_PACKAGE=https://bosstoragemirror.blob.core.windows.net/wrench/tests-package-xamarin-mac-tests/15759261d425ae08494b0a26862a0b1356c5f8ec/5268864/package/Microsoft.iOS.Bundle.15.0.101-ci.tests-package-xamarin-mac-tests.68.pkg
is just clearly wrong.
* [apitest] Sanitize files by adding missing eols.
* [apitest] Add #if __MACOS__ to all test files.
In preparation for the move into monotouchtest.
* [apitest] Move test files into monotouchtest.
* [tests] Remove the apitest project.
* [monotouch-test] Remove MessagingMac.cs, it's not needed.
* [xammac-tests] Add file PlatformInfo.cs to the build.
* [xammac-tests] Move files into monotouch-test.
* [monotouch-test] Rename test class to not clash with another test class of the same (Objective-C) name.
* [tests] How did this ever work?
Answer: it never did.
* [monotouch-test] Remove duplicated test code.
* [xammac-tests] Define DYNAMIC_REGISTRAR when we're using the dynamic registrar.
* [monotouch-test] Adjust the BundleTest.TestGetBundleId test to cope with having multiple apps for the same bundle id.
* [monotouch-test] Ignore a test that doesn't work with the static registrar (due to a bug in the static registrar).
Prebuild the Touch.Client projects for macOS when packaging the Xamarin.Mac
tests, so that when we try to build all the Xamarin.Mac test projects in
parallel, we don't end up trying to build the Touch.Client projects from
multiple build processes at the same time, causing numerous random failures
because all the processes are stomping on eachother.
* Use the commonly used casing for `MSBuildSDKsPath` property
Handle "incorrectly" cased msbuild property names
msbuild property names are case insensitive. While generating the custom
app.config, in `SetToolsetProperty(..)` we try to update the property if
it already exists. But the name lookup was case sensitive, thus causing
the lookup to fail, resulting in two entries for the same property name
differing only in case. Eg. `MSBuildSDKsPath` vs `MSBuildSdksPath`.
* [mtouch] Whitelist new Brotli native symbols in Xamarin.Tests.Misc.PublicSymbols test
* [mtouch] Better assert in NoLLVMFailuresInWatchOS() test
We'd list the "LLVM failed" messages before even though the AOT might've crashed and the list is meaningless. Assert the exit code before that.
* [mtouch] Use new LLVM even for 32bit targets
See https://github.com/mono/mono/issues/14841 and https://github.com/mono/mono/issues/9621
* [mtouch] Work around slow LLVM in "don't link" test
See https://github.com/mono/mono/issues/14843
* Remove useless conditional
* Remove LLVM36 from Makefile
* [watch4] set right min version for arm64_32 based watch devices (#6307)
Fixes the confusion around `libmono-native*` (see for example ce5ba1e41d (commitcomment-33834491) ) when building with `MONO_BUILD_FROM_SOURCE=1`.
* reflect watchos64_32_version_min change from mono sdk
* Move mono hash info to mk/mono.mk so that existing scripts work.
* Add Makefile dependency on mono.mk where necessary
With 3e7bc29ade the Mono hash was moved from Make.config to mono.mk.
We need to add a Makefile dependency on this file wherever Make.config was used to track a Mono dependency.
* [tests] Copy mk/mono.mk to the XM test package.
* [tests] Update minOS version test after consolidating min watchOS versions everywhere.
Fixes this mtouch and mmptest failure:
1) Failed : Xamarin.Tests.ProductTests.MinOSVersion(watchOS,MinwatchOS,WatchOSSimulator,False)
Failures
Expected: <empty>
But was: < "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (mono-runtime-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (bindings-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (bindings-generated-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (shared-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (runtime-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (trampolines-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (trampolines-invoke-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (xamarin-support-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (nsstring-localization-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (trampolines-varargs-debug.arm64_32.o)."... >
* [mmp] Fix make clean target
It needs an -r to remove directories:
```
rm: bin: is a directory
rm: obj: is a directory
```
* Add new xamarin_timezone_get_local_name() to a few more places
* [tests] Point MSBuild to the right Xamarin.Mac location when building packaged Xamarin.Mac tests. Fixes maccore#1115.
Fixes https://github.com/xamarin/maccore/issues/1115.
* [xharness] Ensure all makefile targets set the proper environment variables.
* [tests] Move linker tests to match introspection directory layout.
Move linker tests to match introspectio directory layout: tests/linker/ios and
tests/linker/mac instead of tests/linker-ios and tests/linker-mac.
This creates a logical place for shared linker files (tests/linker).
* [tests] Fix path to GuiUnit_NET_4_5.csproj in sln as well.
This is the path used to find referenced projects when building Classic.
Fixes this build problem:
warning: Referenced project 'GuiUnit_NET_4_5' not found in the solution.
/Users/builder/jenkins/workspace/xamarin-macios-pr-builder/tests/common/mac/MacTestMain.cs(11,7) : error CS0246: The type or namespace name `GuiUnit' could not be found. Are you missing an assembly reference?
/Users/builder/jenkins/workspace/xamarin-macios-pr-builder/tests/common/mac/MacTestMain.cs(64,42) : error CS0246: The type or namespace name `IMainLoopIntegration' could not be found. Are you missing an assembly reference?
/Users/builder/jenkins/workspace/xamarin-macios-pr-builder/tests/common/mac/MacTestMain.cs(75,34) : error CS0246: The type or namespace name `InvokerHelper' could not be found. Are you missing an assembly reference?
Make sure to package the linker tests when packaging XM tests (this broke
recently because the dontlink tests were moved).
Fixes this problem when running the packaged tests:
make[2]: linker-mac/dont link/bin/x86/Debug/dont link.app/Contents/MacOS/dont link: No such file or directory
make[2]: *** [exec-mac-classic-dont link] Error 1
make[2]: linker-mac/dont link/bin/x86/Debug-unified/dont link.app/Contents/MacOS/dont link: No such file or directory
make[2]: *** [exec-mac-unified-dont link] Error 1
make[2]: linker-mac/dont link/bin/x86/Debug-unifiedXM45/dont link.app/Contents/MacOS/dont link: No such file or directory
make[2]: *** [exec-mac-unifiedXM45-dont link] Error 1
make[2]: linker-mac/dont link/bin/x86/Debug-unified-32/dont link.app/Contents/MacOS/dont link: No such file or directory
make[2]: *** [exec-mac-unified32-dont link] Error 1
make[2]: linker-mac/dont link/bin/x86/Debug-unifiedXM45-32/dont link.app/Contents/MacOS/dont link: No such file or directory
make[2]: *** [exec-mac-unifiedXM4532-dont link] Error 1
exec-mac-classic-dont\ link failed
exec-mac-unified-dont\ link failed
exec-mac-unifiedXM45-dont\ link failed
exec-mac-unified32-dont\ link failed
exec-mac-unifiedXM4532-dont\ link failed
make[1]: *** [exec-mac-dontlink] Error 1
Rolf Bjarne Kvinge
Manuel de la Pena
Jo Shields
Chris Hamons