xamarin-macios/tests/Makefile

422 строки
17 KiB
Makefile
Исходник Обычный вид История

TOP = ..
SUBDIRS=test-libraries dotnet
2016-05-26 16:06:52 +03:00
# disabled for now: mac-test
include $(TOP)/Make.config
2016-05-26 16:06:52 +03:00
include $(TOP)/mk/rules.mk
MTOUCH=$(IOS_DESTDIR)/$(MONOTOUCH_PREFIX)/bin/mtouch
UNIT_SERVER_DIR=$(TOUCH_UNIT_PATH)/Touch.Server
UNIT_SERVER=$(UNIT_SERVER_DIR)/bin/Debug/Touch.Server.exe
2018-11-14 21:41:36 +03:00
TEST_SUITES=monotouch-test link\ sdk link\ all dont\ link framework-test mini mono-native
[runtime] integrate mono 2017-10 (#2905) * Update the function name used to initialize libmono-profiler-log, its called mono_profiler_init_log () now. * [builds] Pass --with-cross-offsets= to crosstv's configure. * Bump mono to 2017-08. * Bump mono to 2017-08. * Force disable 'futimens' and 'utimensat' so that we build with Xcode 9. This is also needed to build with Xcode 8.3 on High Sierra. * Remove old AppleTls implementation. * Bump mono. * Bump mono to 2017-08. * Bump mono to 2017-08 * Reenable link-keep-resources-2 test - This reverts commit 76b759ef22c06cda3ba30aba1ac0d45634e4fbf4. - 2017-08 has linker fix * Bump mono to 2017-10 * Revert "Bump mono to 2017-10" This reverts commit bb7832724e18f8578449e46426382a537f3f4823. * Bump system mono to 2017-10 * Bump embedded mono to 2017-10 * [runtime] reflect eglib move https://github.com/mono/mono/commit/9be68f8952ea0e1aad582bfe2f47bad71aee2cc7 * bump mono * [btouch] remove Security.Tls usage from test * [mtouch tests] update the function name used to initialize libmono-profiler-log, its called mono_profiler_init_log () now. see https://github.com/mono/mono/commit/ea4e4a9ef6fc42570a23026adbe826cf7248290e fixes: ``` 1) Failed : Xamarin.MTouch.Profiling(tvOS) _mono_profiler_startup_log Expected: collection containing "_mono_profiler_startup_log" But was: < "_mono_profiler_init_log" > at Xamarin.MTouch.Profiling (Xamarin.Profile profile) [0x00106] in <511889694a624cc9a50e0e9b259b05c5>:0 2) Failed : Xamarin.MTouch.Profiling(watchOS) _mono_profiler_startup_log Expected: collection containing "_mono_profiler_startup_log" But was: < "_xamarin_get_block_descriptor", "_mono_profiler_init_log" > at Xamarin.MTouch.Profiling (Xamarin.Profile profile) [0x00106] in <511889694a624cc9a50e0e9b259b05c5>:0 ``` * [mmptest] update log profiler options. https://github.com/mono/mono/commit/826558a4af624bc0acaea98ec39784e65d278091 deprecated the dash prefix for the mlpd path. `noallocs` or `nocalls` are not needed, neither of them are default anymore. * [mmptest] fix link-keep-resources-2 test to cope with more corlib resources. another corlib resource (mscorlib.xml) was added: https://github.com/mono/mono/commit/11e95169e787#diff-2d1c64decd91d9a6e8842ab0f0e9438d * Revert "[mmptest] fix link-keep-resources-2 test to cope with more corlib resources." This reverts commit 350eb3c174288bbffcc3b7acb15cadb913af25b7. * [XHarness] Add the Mono.Data.Tds tests. * Address comments from rolf in the review. * [mmp regresssion tests] bump mono linker, so mscorlib.xml gets stripped the test was failing in that way: > Executing link-keep-resources-2... > [FAIL] i18n 4/2 data files present: charinfo.nlp, collation.core.bin, collation.tailoring.bin, mscorlib.xml also update the output, because it's actually expected at least three elements. fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=59277 * bump mono fixes crash in tvOS: https://github.com/mono/mono/pull/5812 * bump mono for updated BCL tests see https://github.com/mono/mono/pull/5820 * [mono] set 2017-10 branch in .gitmodules * [macos] Fix guiunit error on clean builds by depending on correct copy (#2912) * [macos] Fix guiunit error on clean builds by depending on correct copy - From a clean build making a BCL test would error due to the non-mobile guiunit not being built - This was because the Makefile-mac.inc target was incorrect - This was because xharness assumed that non variation based targets were always Modern - However, BCL tests are Full, not Modern * Code review change * Swap to var to reduce diff * Revert changes in the paths for GuiUnit. * [XHarness] Add the System.IO.Compression bcl tests. (#2918) * [XHarness] Add the System.IO.Compression bcl tests. * [XHarness] Add bcl tests for System.IO.Compression.FileSystem. (#2924) * [XHarness] Add the System.IO.Compression bcl tests. * Ensure that resources are correctly copied in the bundles. * [XHarness] Add bcl tests for System.IO.Compression.FileSystem. * As per review, make the Mac test app name match the tests that are ran. * [XHarness] Add Mono.CSharp tests on ios. (#2927) * [XHarness] Add Mono.CSharp tests on ios. * Bump mono to bring changes in the mono.csharp tests. * [xtro-sharpie] fix TypeDefinition access due to Cecil change * Bump mono * bump mono fixes - https://bugzilla.xamarin.com/show_bug.cgi?id=60480 - https://bugzilla.xamarin.com/show_bug.cgi?id=60482 * bump mono more fixes around conflicting paths when tests are run in parallel. * Bump for mono/mono@2017-10
2017-11-14 23:30:08 +03:00
BCL_TEST_SUITES=mscorlib \
System \
System.Core \
System.Data \
System.Net.Http \
System.Numerics \
System.Runtime.Serialization \
System.Transactions \
System.Web.Services \
System.Xml \
System.Xml.Linq \
Mono.Security \
System.ComponentModel.DataAnnotations \
System.Json \
System.ServiceModel.Web \
Mono.Data.Sqlite \
Mono.Data.Tds \
System.IO.Compression \
System.IO.Compression.FileSystem \
Mono.CSharp \
System.Security \
System.ServiceModel \
System.IdentityModel \
2016-05-26 16:06:52 +03:00
ALL_TEST_SUITES=$(TEST_SUITES) $(BCL_TEST_SUITES)
EXEC_UNIT_SERVER=XCODE_DEVELOPER_ROOT=$(XCODE_DEVELOPER_ROOT) MONOTOUCH_ROOT=$(IOS_DESTDIR)/$(MONOTOUCH_PREFIX) $(SYSTEM_MONO) --debug $(UNIT_SERVER)
2016-05-26 16:06:52 +03:00
export MD_APPLE_SDK_ROOT=$(abspath $(XCODE_DEVELOPER_ROOT)/../..)
export MD_MTOUCH_SDK_ROOT=$(IOS_DESTDIR)/$(MONOTOUCH_PREFIX)
export MSBUILD_EXE_PATH=$(MONO_PREFIX)/lib/mono/msbuild/15.0/bin/MSBuild.dll
export TargetFrameworkFallbackSearchPaths=$(IOS_DESTDIR)/Library/Frameworks/Mono.framework/External/xbuild-frameworks
export MSBuildExtensionsPathFallbackPathsOverride=$(IOS_DESTDIR)/Library/Frameworks/Mono.framework/External/xbuild
2016-05-26 16:06:52 +03:00
export XAMMAC_FRAMEWORK_PATH=$(MAC_DESTDIR)/Library/Frameworks/Xamarin.Mac.framework/Versions/Current
export XamarinMacFrameworkRoot=$(MAC_DESTDIR)/Library/Frameworks/Xamarin.Mac.framework/Versions/Current
2020-10-15 18:42:44 +03:00
# This is a temporary variable to enable the .NET workload resolver, because it's opt-in for now.
# Ref: https://github.com/dotnet/sdk/issues/13849
export MSBuildEnableWorkloadResolver=true
2016-05-26 16:06:52 +03:00
ifneq ($(RELEASE),)
ifneq ($(BITCODE),)
CONFIG=Release-bitcode
else
2016-05-26 16:06:52 +03:00
CONFIG=Release
endif
2016-05-26 16:06:52 +03:00
else
CONFIG=Debug
endif
ifeq ($(V),)
ifeq ($(BUILD_REVISION),)
# non-verbose local build
XHARNESS_VERBOSITY=
else
# wrench build
XHARNESS_VERBOSITY=--verbose
endif
else
# verbose build
XHARNESS_VERBOSITY=--verbose
endif
#
# To run all the tests, just do:
#
# make run-tests
#
# and go take a long tea break.
#
# For faster turnaround there are specific targets that can be used
# to run specific test suites. For more information read the README.md.
#
package-tests:
$(MAKE) qa-test-dependencies.zip mac-test-package.zip
2016-05-26 16:06:52 +03:00
Bump to mono:2019-06 (#6196) * Use the commonly used casing for `MSBuildSDKsPath` property Handle "incorrectly" cased msbuild property names msbuild property names are case insensitive. While generating the custom app.config, in `SetToolsetProperty(..)` we try to update the property if it already exists. But the name lookup was case sensitive, thus causing the lookup to fail, resulting in two entries for the same property name differing only in case. Eg. `MSBuildSDKsPath` vs `MSBuildSdksPath`. * [mtouch] Whitelist new Brotli native symbols in Xamarin.Tests.Misc.PublicSymbols test * [mtouch] Better assert in NoLLVMFailuresInWatchOS() test We'd list the "LLVM failed" messages before even though the AOT might've crashed and the list is meaningless. Assert the exit code before that. * [mtouch] Use new LLVM even for 32bit targets See https://github.com/mono/mono/issues/14841 and https://github.com/mono/mono/issues/9621 * [mtouch] Work around slow LLVM in "don't link" test See https://github.com/mono/mono/issues/14843 * Remove useless conditional * Remove LLVM36 from Makefile * [watch4] set right min version for arm64_32 based watch devices (#6307) Fixes the confusion around `libmono-native*` (see for example https://github.com/xamarin/xamarin-macios/commit/ce5ba1e41d526eddd9b61f68fae22a747831e9f2#commitcomment-33834491 ) when building with `MONO_BUILD_FROM_SOURCE=1`. * reflect watchos64_32_version_min change from mono sdk * Move mono hash info to mk/mono.mk so that existing scripts work. * Add Makefile dependency on mono.mk where necessary With https://github.com/xamarin/xamarin-macios/commit/3e7bc29adec8dff3558c284a3050f2db36ea3cf2 the Mono hash was moved from Make.config to mono.mk. We need to add a Makefile dependency on this file wherever Make.config was used to track a Mono dependency. * [tests] Copy mk/mono.mk to the XM test package. * [tests] Update minOS version test after consolidating min watchOS versions everywhere. Fixes this mtouch and mmptest failure: 1) Failed : Xamarin.Tests.ProductTests.MinOSVersion(watchOS,MinwatchOS,WatchOSSimulator,False) Failures Expected: <empty> But was: < "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (mono-runtime-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (bindings-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (bindings-generated-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (shared-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (runtime-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (trampolines-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (trampolines-invoke-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (xamarin-support-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (nsstring-localization-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (trampolines-varargs-debug.arm64_32.o)."... > * [mmp] Fix make clean target It needs an -r to remove directories: ``` rm: bin: is a directory rm: obj: is a directory ``` * Add new xamarin_timezone_get_local_name() to a few more places
2019-07-16 18:24:00 +03:00
test.config: Makefile $(TOP)/Make.config $(TOP)/mk/mono.mk
2016-05-26 16:06:52 +03:00
@rm -f $@
@echo "MONOTOUCH_PREFIX=$(abspath $(MONOTOUCH_PREFIX))" >> $@
@echo "IOS_DESTDIR=$(abspath $(IOS_DESTDIR))" >> $@
@echo "MAC_DESTDIR=$(abspath $(MAC_DESTDIR))" >> $@
@echo "JENKINS_RESULTS_DIRECTORY=$(abspath $(JENKINS_RESULTS_DIRECTORY))" >> $@
@echo "INCLUDE_DEVICE=$(INCLUDE_DEVICE)" >> $@
@echo "XCODE_DEVELOPER_ROOT=$(XCODE_DEVELOPER_ROOT)" >> $@
@echo "MONO_IOS_SDK_DESTDIR=$(MONO_IOS_SDK_DESTDIR)" >> $@
@echo "MONO_MAC_SDK_DESTDIR=$(MONO_MAC_SDK_DESTDIR)" >> $@
@echo "ENABLE_XAMARIN=$(ENABLE_XAMARIN)" >> $@
@echo "DOTNET=$(DOTNET)" >> $@
@echo "DOTNET5=$(DOTNET5)" >> $@
@echo "IOS_SDK_VERSION=$(IOS_SDK_VERSION)" >> $@
@echo "TVOS_SDK_VERSION=$(TVOS_SDK_VERSION)" >> $@
@echo "WATCH_SDK_VERSION=$(WATCH_SDK_VERSION)" >> $@
@echo "OSX_SDK_VERSION=$(OSX_SDK_VERSION)" >> $@
@echo "DOTNET5_BCL_DIR=$(DOTNET5_BCL_DIR)" >> $@
2016-05-26 16:06:52 +03:00
Bump to mono:2019-06 (#6196) * Use the commonly used casing for `MSBuildSDKsPath` property Handle "incorrectly" cased msbuild property names msbuild property names are case insensitive. While generating the custom app.config, in `SetToolsetProperty(..)` we try to update the property if it already exists. But the name lookup was case sensitive, thus causing the lookup to fail, resulting in two entries for the same property name differing only in case. Eg. `MSBuildSDKsPath` vs `MSBuildSdksPath`. * [mtouch] Whitelist new Brotli native symbols in Xamarin.Tests.Misc.PublicSymbols test * [mtouch] Better assert in NoLLVMFailuresInWatchOS() test We'd list the "LLVM failed" messages before even though the AOT might've crashed and the list is meaningless. Assert the exit code before that. * [mtouch] Use new LLVM even for 32bit targets See https://github.com/mono/mono/issues/14841 and https://github.com/mono/mono/issues/9621 * [mtouch] Work around slow LLVM in "don't link" test See https://github.com/mono/mono/issues/14843 * Remove useless conditional * Remove LLVM36 from Makefile * [watch4] set right min version for arm64_32 based watch devices (#6307) Fixes the confusion around `libmono-native*` (see for example https://github.com/xamarin/xamarin-macios/commit/ce5ba1e41d526eddd9b61f68fae22a747831e9f2#commitcomment-33834491 ) when building with `MONO_BUILD_FROM_SOURCE=1`. * reflect watchos64_32_version_min change from mono sdk * Move mono hash info to mk/mono.mk so that existing scripts work. * Add Makefile dependency on mono.mk where necessary With https://github.com/xamarin/xamarin-macios/commit/3e7bc29adec8dff3558c284a3050f2db36ea3cf2 the Mono hash was moved from Make.config to mono.mk. We need to add a Makefile dependency on this file wherever Make.config was used to track a Mono dependency. * [tests] Copy mk/mono.mk to the XM test package. * [tests] Update minOS version test after consolidating min watchOS versions everywhere. Fixes this mtouch and mmptest failure: 1) Failed : Xamarin.Tests.ProductTests.MinOSVersion(watchOS,MinwatchOS,WatchOSSimulator,False) Failures Expected: <empty> But was: < "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (mono-runtime-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (bindings-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (bindings-generated-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (shared-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (runtime-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (trampolines-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (trampolines-invoke-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (xamarin-support-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (nsstring-localization-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (trampolines-varargs-debug.arm64_32.o)."... > * [mmp] Fix make clean target It needs an -r to remove directories: ``` rm: bin: is a directory rm: obj: is a directory ``` * Add new xamarin_timezone_get_local_name() to a few more places
2019-07-16 18:24:00 +03:00
test-system.config: Makefile $(TOP)/Make.config $(TOP)/mk/mono.mk
@rm -f $@
@echo "MONOTOUCH_PREFIX=$(IOS_FRAMEWORK_DIR)/Versions/Current" >> $@
@echo "IOS_DESTDIR=/" >> $@
@echo "MAC_DESTDIR=/" >> $@
@echo "JENKINS_RESULTS_DIRECTORY=$(abspath $(JENKINS_RESULTS_DIRECTORY))" >> $@
@echo "INCLUDE_DEVICE=$(INCLUDE_DEVICE)" >> $@
@echo "MONO_IOS_SDK_DESTDIR=$(MONO_IOS_SDK_DESTDIR)" >> $@
@echo "MONO_MAC_SDK_DESTDIR=$(MONO_MAC_SDK_DESTDIR)" >> $@
@echo "DOTNET=$(DOTNET)" >> $@
@echo "DOTNET5=$(DOTNET5)" >> $@
@echo "IOS_SDK_VERSION=$(IOS_SDK_VERSION)" >> $@
@echo "TVOS_SDK_VERSION=$(TVOS_SDK_VERSION)" >> $@
@echo "WATCH_SDK_VERSION=$(WATCH_SDK_VERSION)" >> $@
@echo "OSX_SDK_VERSION=$(OSX_SDK_VERSION)" >> $@
2016-05-26 16:06:52 +03:00
clean-local::
$(Q) $(SYSTEM_XBUILD) /t:Clean /p:Platform=iPhoneSimulator /p:Configuration=$(CONFIG) $(XBUILD_VERBOSITY) tests.sln
$(Q) $(SYSTEM_XBUILD) /t:Clean /p:Platform=iPhone /p:Configuration=$(CONFIG) $(XBUILD_VERBOSITY) tests.sln
$(Q) $(SYSTEM_XBUILD) /t:Clean /p:Platform=iPhoneSimulator /p:Configuration=$(CONFIG) $(XBUILD_VERBOSITY) tests-unified.sln
$(Q) $(SYSTEM_XBUILD) /t:Clean /p:Platform=iPhone /p:Configuration=$(CONFIG) $(XBUILD_VERBOSITY) tests-unified.sln
$(Q) $(SYSTEM_XBUILD) /t:Clean /p:Platform=iPhoneSimulator /p:Configuration=$(CONFIG) $(XBUILD_VERBOSITY) tests-tvos.sln
$(Q) $(SYSTEM_XBUILD) /t:Clean /p:Platform=iPhoneSimulator /p:Configuration=$(CONFIG) $(XBUILD_VERBOSITY) tests-tvos.sln
$(Q) $(SYSTEM_XBUILD) /t:Clean /p:Platform=iPhone /p:Configuration=$(CONFIG) $(XBUILD_VERBOSITY) tests-watchos.sln
$(Q) $(SYSTEM_XBUILD) /t:Clean /p:Platform=iPhone /p:Configuration=$(CONFIG) $(XBUILD_VERBOSITY) tests-watchos.sln
## run targets = build + [install] + exec
run run-all run-tests run-test:
$(Q) for subdir in $(SUBDIRS); do \
$(MAKE) -C $$subdir run || exit 1; \
done
$(Q) $(MAKE) run-local
# separate build-dev-* entries because some of them are build with debug other (llvm) with release
build-dev-bots:
@echo "These tests are now in the mtouch test suite"
2016-05-26 16:06:52 +03:00
build-% run-% exec-% install-%:
@echo ""
@echo "\033[0;31mUnknown target: $@\033[0m"
@echo ""
@echo "Syntax is: <action>-<platform>-<where>-<what>"
@echo " Action: build, install (only for device), exec, run (= build + install + exec)."
@echo " Platform: ios, tvos, watchos (can be skipped, in which case all platforms are executed)"
@echo " Where: "
@echo " ios: sim = [simclassic + simunified = [sim32, sim64]]; dev = [devclassic + devunified]"
@echo " tvos/watchos: sim, dev"
@echo " What: The test project to run (BCL tests do not need to be treated specially anymore, just use test project's name)"
@echo ""
@echo "Example:"
@echo ""
@echo " # this will run monotouch-test on an Apple TV device"
@echo " $ make run-tvos-dev-monotouch-test"
@echo ""
@echo "For more info read README.md."
@echo ""
@exit 1
check-result:
@test -z "$(BUILD_REVISION)" || echo "@MonkeyWrench: AddFile: $(abspath $(FILE))"
@if grep "Tests run" "$(FILE)"; then \
if grep FAIL "$(FILE)"; then \
test "x" == "x$(BUILD_REVISION)" -o "x" == "x$(TESTNAME)" || echo "@MonkeyWrench: AddSummary: <b>$(TESTNAME) failed: `grep "Tests run:" "$(FILE)" | sed 's/Tests run: //'`</b><br/>"; \
echo Test run failed; \
exit 1; \
else \
test "x" == "x$(BUILD_REVISION)" -o "x" == "x$(TESTNAME)" || echo "@MonkeyWrench: AddSummary: $(TESTNAME) succeeded: `grep "Tests run:" "$(FILE)" | sed 's/Tests run: //'`<br/>"; \
echo Test run succeeded; \
fi; \
else \
test "x" == "x$(BUILD_REVISION)" -o "x" == "x$(TESTNAME)" || echo "@MonkeyWrench: AddSummary: <b><i>$(TESTNAME) crashed</i></b><br/>"; \
echo Test run crashed; \
exit 1; \
fi
logdev:
$(MTOUCH) --logdev
$(UNIT_SERVER): $(wildcard $(UNIT_SERVER_DIR)/*.cs)
(cd $(UNIT_SERVER_DIR) && $(SYSTEM_MSBUILD))
2016-05-26 16:06:52 +03:00
build-test-libraries:
@$(MAKE) -C $(TOP)/tests/test-libraries
Makefile.inc: xharness/xharness.exe
$(Q_GEN) $(SYSTEM_MONO) --debug $(CURDIR)/$< $(XHARNESS_VERBOSITY) --configure --autoconf --rootdir $(CURDIR)
2016-05-26 16:06:52 +03:00
-include Makefile.inc
Makefile-mac.inc: xharness/xharness.exe
$(Q_GEN) $(SYSTEM_MONO) --debug $(CURDIR)/$< $(XHARNESS_VERBOSITY) --configure --autoconf --mac --rootdir $(CURDIR)
2016-05-26 16:06:52 +03:00
-include Makefile-mac.inc
$(TOP)/tools/common/SdkVersions.cs: $(TOP)/tools/common/SdkVersions.cs.in
@$(MAKE) -C $(TOP)/tools/mtouch ../common/SdkVersions.cs
2016-05-26 16:06:52 +03:00
.stamp-src-project-files:
@$(MAKE) -C $(TOP)/src project-files
@touch $@
xharness/xharness.exe: $(xharness_dependencies) test.config test-system.config .stamp-src-project-files $(TOP)/tools/common/SdkVersions.cs
$(Q_GEN) $(SYSTEM_MSBUILD) /restore $(MSBUILD_VERBOSITY_QUIET) xharness/xharness.csproj
xharness/xharness.csproj.inc: export ABSOLUTE_PATHS=1
-include xharness/xharness.csproj.inc
2016-05-26 16:06:52 +03:00
killall:
@killall "iPhone Simulator" >/dev/null 2>&1 || true
@killall "iOS Simulator" >/dev/null 2>&1 || true
@killall Touch.Server >/dev/null 2>&1 || true
NUNIT_MSBUILD_DIR=$(TOP)/packages/NUnit.Runners.2.6.4/tools/lib
test-ios-tasks: test-macdev-tests test-macdev-tasks
test-macdev-tests: verify-system-vsmac-xcode-match
$(SYSTEM_MSBUILD) $(TOP)/tests/msbuild/Xamarin.MacDev.Tests/Xamarin.MacDev.Tests.csproj /p:Configuration=Debug /r
cd $(TOP)/tests/msbuild/Xamarin.MacDev.Tests && $(SYSTEM_XIBUILD) -t -- $(abspath $(TOP)/tools/nunit3-console-3.11.1) $(abspath $(TOP)/tests/msbuild/Xamarin.MacDev.Tests/bin/Debug/net461/Xamarin.MacDev.Tests.dll) "--result=$(abspath $(CURDIR)/TestResults_Xamarin.MacDev.Tests.xml);format=nunit2" -labels=Before $(TEST_FIXTURE)
test-macdev-tasks: verify-system-vsmac-xcode-match
$(SYSTEM_MSBUILD) $(TOP)/tests/msbuild/Xamarin.MacDev.Tasks.Tests/Xamarin.MacDev.Tasks.Tests.csproj /p:Configuration=Debug /r
cd $(TOP)/tests/msbuild/Xamarin.MacDev.Tasks.Tests && $(SYSTEM_XIBUILD) -t -- $(abspath $(TOP)/tools/nunit3-console-3.11.1) $(abspath $(TOP)/tests/msbuild/Xamarin.MacDev.Tasks.Tests/bin/Debug/net472/Xamarin.MacDev.Tasks.Tests.dll) "--result=$(abspath $(CURDIR)/TestResults_Xamarin.MacDev.Tasks.Tests.xml)" -labels=Before $(TEST_FIXTURE)
2016-05-26 16:06:52 +03:00
test-install-sources:
Move tests to use new tool xibuild and thus fix msbuild tests (#5128) * xibuild: New wrapper tool to run msbuild or managed executables MSBuild supports fallback paths for projects imported using `$(MSBuildExtensionsPath)`, but these must be specified explicitly in the app.config of the main executable. There was a PR to allow use of properties for this in the app.config, but that was not accepted upstream. This is required for being able to: 1. build projects with msbuild against the in-tree XI/XM build output 2. and to run nunit tests against the same. For this we introduce a new tool, `xibuild`, based on XA's `xabuild`. This supports the fallback paths to be specified via the environment variable `MSBuildExtensionsPathFallbackPathsOverride`[1]. It essentially operates in 3 modes: 1. `xibuild -c /path/to/foo.exe` Generates /path/to/foo.exe.config with the fallback paths inserted into that. 2. `xibuild -- /v:diag /path/to/project.csproj` Runs msbuild with the arguments after `--` with a custom app.config based on `MSBuild.dll.config`, with the fallback paths correctly inserted. This is in a temporary file and the original config file is not touched. 3. `xibuild -t -- /path/to/managed_tool.exe args` Generates `/path/to/managed_tool.exe.config` based on `MSBuild.dll.config` with the fallback paths inserted, and runs `managed_tool.exe` with the arguments. The default is to overwrite the config file. But there is also a switch to merge it with an existing config file. -- 1. Value of the environment variable $MSBuildExtensionsPathFallbackPathsOverride is prepended to any existing list of search paths in `MSBuild.dll.config`, IOW, it takes precedence. So, the order of lookup becomes: - Value of the property `$(MSBuildExtensionsPath)` - Value of the environment variable `$MSBuildExtensionsPathFallbackPathsOverride` - /Library/Frameworks/Mono.framework/External/xbuild on macOS * Integrate use of `xibuild` with the tests Update all uses of `msbuild` and invocations of tools like nunit that might depend on using the in-tree builds to use `xibuild`. * xibuild: Move help descriptions to OptionSet itself.
2018-11-16 22:24:35 +03:00
$(SYSTEM_XIBUILD) -- $(TOP)/tools/install-source/InstallSourcesTests/InstallSourcesTests.csproj
cd $(NUNIT_MSBUILD_DIR) && $(SYSTEM_XIBUILD) -t -- ../nunit-console.exe ../../../../tools/install-source/InstallSourcesTests/bin/Release/InstallSourcesTests.dll -xml=TestResults_InstallSourcesTests.xml -labels $(TEST_FIXTURE) || touch .failed-stamp
@[[ -z "$$BUILD_REPOSITORY" ]] || ( xsltproc $(TOP)/tests/HtmlTransform.xslt $(NUNIT_MSBUILD_DIR)/TestResults_InstallSourcesTests.xml > $(TOP)/tests/index.html && echo "@MonkeyWrench: AddFile: $$PWD/index.html" )
@if test -e $(NUNIT_MSBUILD_DIR)/.failed-stamp; then rm $(NUNIT_MSBUILD_DIR)/.failed-stamp; exit 1; fi
ifdef ENABLE_XAMARIN
2016-05-26 16:06:52 +03:00
ifdef INCLUDE_IOS
qa-test-dependencies.zip:
@$(MAKE) build-test-libraries
@# Make sure we start from a clean slate
$(Q) rm -rf $@ $@.tmpdir
ifdef INCLUDE_TVOS
@# TVOS
$(Q) mkdir -p $@.tmpdir/tvos $@.tmpdir/tvsimulator
$(Q) $(CP) $(IOS_DESTDIR)$(MONOTOUCH_PREFIX)/lib/mono/Xamarin.TVOS/MonoTouch.Dialog-1.dll* $@.tmpdir/tvos
$(Q) $(CP) $(IOS_DESTDIR)$(MONOTOUCH_PREFIX)/lib/mono/Xamarin.TVOS/MonoTouch.NUnitLite.dll* $@.tmpdir/tvos
$(Q) $(CP) $(TOP)/tests/test-libraries/.libs/tvos/libtest.a $@.tmpdir/tvos
$(Q) $(CP) $(TOP)/tests/test-libraries/.libs/tvos/libtest.dylib $@.tmpdir/tvos
$(Q) $(CP) -a $(TOP)/tests/test-libraries/.libs/tvos/XTest.framework $@.tmpdir/tvos
$(Q) $(CP) $(IOS_DESTDIR)$(MONOTOUCH_PREFIX)/lib/mono/Xamarin.TVOS/MonoTouch.Dialog-1.dll* $@.tmpdir/tvsimulator
$(Q) $(CP) $(IOS_DESTDIR)$(MONOTOUCH_PREFIX)/lib/mono/Xamarin.TVOS/MonoTouch.NUnitLite.dll* $@.tmpdir/tvsimulator
$(Q) $(CP) $(TOP)/tests/test-libraries/.libs/tvsimulator/libtest.a $@.tmpdir/tvsimulator
$(Q) $(CP) $(TOP)/tests/test-libraries/.libs/tvsimulator/libtest.dylib $@.tmpdir/tvsimulator
$(Q) $(CP) -a $(TOP)/tests/test-libraries/.libs/tvsimulator/XTest.framework $@.tmpdir/tvsimulator
2016-05-26 16:06:52 +03:00
endif
@# iOS
$(Q) mkdir -p $@.tmpdir/iphoneos $@.tmpdir/iphonesimulator
$(Q) $(CP) $(TOP)/tests/test-libraries/.libs/iphoneos/libtest.a $@.tmpdir/iphoneos
$(Q) $(CP) $(TOP)/tests/test-libraries/.libs/iphoneos/libtest.dylib $@.tmpdir/iphoneos
$(Q) $(CP) -a $(TOP)/tests/test-libraries/.libs/iphoneos/XTest.framework $@.tmpdir/iphoneos
$(Q) $(CP) $(TOP)/tests/test-libraries/.libs/iphonesimulator/libtest.a $@.tmpdir/iphonesimulator
$(Q) $(CP) $(TOP)/tests/test-libraries/.libs/iphonesimulator/libtest.dylib $@.tmpdir/iphonesimulator
$(Q) $(CP) -a $(TOP)/tests/test-libraries/.libs/iphonesimulator/XTest.framework $@.tmpdir/iphonesimulator
$(Q) printf '#!/bin/bash -e\n\n/Library/Frameworks/Xamarin.iOS.framework/Versions/Current/bin/mlaunch "$$@"\n' > $@.tmpdir/mlaunch
$(Q) chmod +x $@.tmpdir/mlaunch
2016-05-26 16:06:52 +03:00
@# Generate zip
$(Q_GEN) cd $@.tmpdir && zip -9r $(abspath $@) .
@# Cleanup
$(Q) rm -rf $@.tmpdir
else
qa-test-dependencies.zip:
@echo Not enabled
endif
else
qa-test-dependencies.zip:
@echo Xamarin build not enabled
endif
2016-05-26 16:06:52 +03:00
ifdef INCLUDE_MAC
mac-test-package.zip:
./package-mac-tests.sh
else
mac-test-package.zip:
@echo Not enabled
endif
#XI
ifdef INCLUDE_IOS
wrench-mtouch:
@echo Not here anymore
2016-05-26 16:06:52 +03:00
wrench-btouch:
@echo Not here anymore
2016-05-26 16:06:52 +03:00
wrench-build-unit-test-dev:
@echo Not here anymore
2016-05-26 16:06:52 +03:00
wrench-sim-linksdk:
@echo Not here anymore
2016-05-26 16:06:52 +03:00
wrench-sim-linkall:
@echo Not here anymore
2016-05-26 16:06:52 +03:00
wrench-sim-dontlink:
@echo Not here anymore
2016-05-26 16:06:52 +03:00
else
wrench-mtouch wrench-btouch wrench-build-unit-test-dev:
@echo "iOS tests have been disabled [$@]"
endif
## XM
ifdef INCLUDE_MAC
wrench-mac-dontlink:
@echo Not here anymore
2016-05-26 16:06:52 +03:00
wrench-mac-mmp:
@echo Now part of wrench-mac-mmptest
wrench-mac-misc:
@echo Tests mac-misc tests removed
wrench-mac-apidiff:
@echo This is done in the normal api-diff target now.
wrench-mac-drm:
@echo "The DRM tests have all been removed."
wrench-mac-api:
@echo Not here anymore
2016-05-26 16:06:52 +03:00
wrench-mac-mmptest:
@echo Not here anymore
2016-05-26 16:06:52 +03:00
wrench-mac-msbuild:
@echo Not here anymore
2016-05-26 16:06:52 +03:00
wrench-mac-binding-project:
@echo Not here anymore
2016-05-26 16:06:52 +03:00
wrench-mac-introspection:
@echo Not here anymore
wrench-mac-xammac_tests:
@echo Not here anymore
2016-05-26 16:06:52 +03:00
else
wrench-mac-%:
@echo "Mac tests have been disabled [$@]"
endif
wrench-msbuild:
@echo Not here anymore
2016-05-26 16:06:52 +03:00
wrench-docs:
2016-05-27 10:56:48 +03:00
ifdef ENABLE_XAMARIN
2016-05-26 16:06:52 +03:00
ifdef INCLUDE_IOS
$(MAKE) -C $(MACCORE_PATH) update-docs
2016-05-26 16:06:52 +03:00
else
@echo "iOS tests have been disabled [$@]"
endif
2016-05-27 10:56:48 +03:00
else
@echo "Docs have been disabled [$@]"
endif
2016-05-26 16:06:52 +03:00
# the wrenchhelper-* targets are generated.
wrench-sim-%:
@echo Not here anymore
2016-05-26 16:06:52 +03:00
wrench-bcl-sim-%:
@echo Not here anymore
2016-05-26 16:06:52 +03:00
wrench-%:
@echo Not here anymore
wrench-jenkins: xharness/xharness.exe
$(Q) rm -f $@-failed.stamp
$(Q) ulimit -n 4096 && $(SYSTEM_MONO) --trace=E:all --debug $(CURDIR)/$< $(XHARNESS_VERBOSITY) --jenkins --autoconf --rootdir $(CURDIR) --sdkroot $(XCODE_DEVELOPER_ROOT) --label run-all-tests,skip-ios-32-tests,skip-device-tests --markdown-summary=$(abspath $(CURDIR))/TestSummary.md $(TESTS_PERIODIC_COMMAND) || echo "$$?" > $@-failed.stamp
@echo "@MonkeyWrench: SetSummary: <br/>`cat $(abspath $(CURDIR))/TestSummary.md | awk 1 ORS='<br/>'`"
@echo "@MonkeyWrench: AddFile: $(abspath $(CURDIR))/TestSummary.md"
$(Q) if test -e $@-failed.stamp; then EC=`cat $@-failed.stamp`; rm -f $@-failed.stamp; exit $$EC; fi
wrench-xtro:
@echo Not here anymore
jenkins: xharness/xharness.exe
$(Q) $(SYSTEM_MONO) --trace=E:all,-E:System.Reflection.ReflectionTypeLoadException --debug $(CURDIR)/$< $(XHARNESS_VERBOSITY) --jenkins --autoconf --rootdir $(CURDIR) --sdkroot $(XCODE_DEVELOPER_ROOT) --markdown-summary=$(abspath $(CURDIR))/TestSummary.md
# This will launch xharness' interactive test runner in the system's default browser
runner: xharness/xharness.exe
@echo "Running xharness in server mode. Press Ctrl-C to exit (or click Quit / press Q in the browser page)"
$(Q) $(SYSTEM_MONO) $(CURDIR)/xharness/xharness.exe $(XHARNESS_VERBOSITY) --jenkins:server --autoconf --rootdir $(CURDIR) --sdkroot $(XCODE_DEVELOPER_ROOT)
# This makefile target will run the device tests using the Xamarin.iOS version
# installed on the system.
vsts-device-tests: xharness/xharness.exe
$(MAKE) -C $(TOP)/builds .stamp-mono-ios-sdk-destdir download -j
$(Q) ulimit -n 4096 && $(SYSTEM_MONO) --debug $(CURDIR)/$< $(XHARNESS_VERBOSITY) --jenkins --autoconf --rootdir $(CURDIR) --sdkroot $(XCODE_DEVELOPER_ROOT) --use-system:true --label=skip-all-tests,run-device-tests,run-bcl-tests --markdown-summary=$(CURDIR)/TestSummary.md $(TESTS_EXTRA_ARGUMENTS) $(TESTS_PERIODIC_COMMAND)
ifdef ENABLE_XAMARIN
wrench-launch-external wrench-report-external:
$(MAKE) -C $(MACCORE_PATH)/tests/external $@
# This will schedule a manual device test run for the current commit. The current commit must already have packages.
launch-manual-device-tests:
./launch-manual-device-tests.sh
# This will schedule a manual sample test run for the current commit. The current commit must already have packages.
launch-manual-sample-tests:
./launch-manual-sample-tests.sh
else
wrench-launch-external wrench-report-external:
@echo "External tests have been disabled"
endif
verify-system-vsmac-xcode-match:
@SYSTEM_XCODE=$$(dirname $$(dirname $$(xcode-select -p))); \
VSMAC_XCODE=$$(/usr/libexec/PlistBuddy -c 'print :AppleSdkRoot' ~/Library/Preferences/Xamarin/Settings.plist); \
if [[ "$$SYSTEM_XCODE" != "$$VSMAC_XCODE" ]]; then echo "Error: the system's Xcode ($$SYSTEM_XCODE) does not match the Xcode selected in Visual Studio for Mac ($$VSMAC_XCODE). This will make tests fail in non-trivial ways, so please make sure both match the expected Xcode ($(shell dirname $(shell dirname $(XCODE_DEVELOPER_ROOT))))."; exit 1; fi