Problem:
1. interdependent-binding-project references binding-test and bindings-test2.
2. bindings-test2 references bindings-test.
This means bindings-test is reached through 2 projects: both
interdependent-binding-projects and bindings-test2, and previously we'd
process each of these references separately, effectively making two separate
clones of the bindings-test project.
Instead keep track of all referenced projects from the leaf project, and if we
encounter a project we've already cloned, use that directly.
We have noticed the following message from Apple when performing
submissions with Xamarin.iOS:
> ITMS-90338: Non-public API usage - The app references non-public
> selectors in WcBc.iOS: behaviorTypes, convolutionState,
> discoverAllContactUserInfosWithCompletionHandler:,
> discoverAllContactsCompletionBlock,
> discoverUserInfoWithEmailAddress:completionHandler:,
> discoverUserInfoWithUserRecordID:completionHandler:,
> discoverUserInfosCompletionBlock, displayContact, drawableResizesAsynchronously,
> encodeToCommandBuffer:sourceImage:convolutionState:,
> encodeToCommandBuffer:sourceImage:destinationImage:state:,
> getProperty:onChannel:responseHandler:, hasProperty:onChannel:responseHandler:,
> initWithEmailAddresses:userRecordIDs:, initWithMIDIEntity:dataReadyHandler:,
> initWithZoneID:options:, initWithZoneID:subscriptionID:options:,
> isPublicDatabase, mouseUpAction, newDrawable, propertyChangedCallback,
> removeAllAppearanceStreams, replaceTextStorage:, retrieveConnectedPeripherals,
> retrievePeripherals:, setDiscoverAllContactsCompletionBlock:,
> setDiscoverUserInfosCompletionBlock:, setDrawableResizesAsynchronously:,
> setEditedMask:, setMouseUpAction:, setMovieControlMode:,
> setProperty:onChannel:responseHandler:, setPropertyChangedCallback:,
> setSocketFamily:, setTemporaryAttributes:forCharacterRange:, setUserRecordIDs:,
> sourceOffset, subscriptionOptions, takeBackgroundColorFrom:, takePasswordFrom:,
> temporalAntialiasingEnabled, userRecordIDs. If method names in your source code
> match the private Apple APIs listed above, altering your method names will help
> prevent this app from being flagged in future submissions. In addition, note
> that one or more of the above APIs may be located in a static library that was
> included with your app. If so, they must be removed. For further information,
> visit the Technical Support Information at http://developer.apple.com/support/technical/
All of them have been removed but without a break in the API excep
"initWithMIDIEntity:dataReadyHandler:" wich does look like an error on
Apples side.
Empty stubs are used as much as possible except on those cases in which
a handler is called or an output variable should be modified (buffer,
out param) to minimize the users surprise at runtime.
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
beta 3 included a ton of bad update of availability. While it correctly
changed `13` to `14`, is also added `API_UNAVAILABLE(ios` to everything
```diff
-SR_EXTERN API_AVAILABLE(ios(13.0)) API_UNAVAILABLE(watchos) API_UNAVAILABLE(tvos, macos)
+SR_EXTERN API_AVAILABLE(ios(14.0)) API_UNAVAILABLE(ios, watchos) API_UNAVAILABLE(tvos, macos)
```
Those were corrected (lot of noise) if beta 4, along with few API
changes.
xharness needs to inspect project files, but can't do so through imported
projects, so move some of the logic from the imported shared.csproj to the
main csproj.
We have noticed the following message from Apple when performing
submissions with Xamarin.iOS:
> ITMS-90338: Non-public API usage - The app references non-public
> selectors in WcBc.iOS: behaviorTypes, convolutionState,
> discoverAllContactUserInfosWithCompletionHandler:,
> discoverAllContactsCompletionBlock,
> discoverUserInfoWithEmailAddress:completionHandler:,
> discoverUserInfoWithUserRecordID:completionHandler:,
> discoverUserInfosCompletionBlock, displayContact, drawableResizesAsynchronously,
> encodeToCommandBuffer:sourceImage:convolutionState:,
> encodeToCommandBuffer:sourceImage:destinationImage:state:,
> getProperty:onChannel:responseHandler:, hasProperty:onChannel:responseHandler:,
> initWithEmailAddresses:userRecordIDs:, initWithMIDIEntity:dataReadyHandler:,
> initWithZoneID:options:, initWithZoneID:subscriptionID:options:,
> isPublicDatabase, mouseUpAction, newDrawable, propertyChangedCallback,
> removeAllAppearanceStreams, replaceTextStorage:, retrieveConnectedPeripherals,
> retrievePeripherals:, setDiscoverAllContactsCompletionBlock:,
> setDiscoverUserInfosCompletionBlock:, setDrawableResizesAsynchronously:,
> setEditedMask:, setMouseUpAction:, setMovieControlMode:,
> setProperty:onChannel:responseHandler:, setPropertyChangedCallback:,
> setSocketFamily:, setTemporaryAttributes:forCharacterRange:, setUserRecordIDs:,
> sourceOffset, subscriptionOptions, takeBackgroundColorFrom:, takePasswordFrom:,
> temporalAntialiasingEnabled, userRecordIDs. If method names in your source code
> match the private Apple APIs listed above, altering your method names will help
> prevent this app from being flagged in future submissions. In addition, note
> that one or more of the above APIs may be located in a static library that was
> included with your app. If so, they must be removed. For further information,
> visit the Technical Support Information at http://developer.apple.com/support/technical/
All of them have been removed but without a break in the API excep
"initWithMIDIEntity:dataReadyHandler:" wich does look like an error on
Apples side.
Empty stubs are used as much as possible except on those cases in which
a handler is called or an output variable should be modified (buffer,
out param) to minimize the users surprise at runtime.
This requires passing the root directory around in multiple places, since ResolveAllPaths
doesn't have access to the static class where we define the root directory.
Also call ResolveAllPaths in a few more places to ensure paths everywhere are resolved.
In .NET projects there's a default value for most properties, which means that
there won't necessarily be an AssemblyName property in a csproj. We need to know the
AssemblyName, so calculate it from the csproj filename (which is how .NET does it).
This turned out slighly complicated, because we're pass an XmlDocument around,
and the XmlDocument doesn't know the file from where it was loaded, so we need
to keep that information separately.
This works around a build problem that occurs because NUnit ships with a
P/Invoke to a function that doesn't exist on Apple platforms:
MTOUCH : error MT5210: Native linking failed, undefined symbol: _GetVersionEx. Please verify that all the necessary frameworks have been referenced and native libraries are properly linked in. [/Users/xamarinqa/myagent/_work/8/s/xamarin-macios/tests/xharness/tmp-test-dir/monotouch-test58/monotouch-test-tvos.csproj]
MTOUCH : error MT5201: Native linking failed. Please review the build log and the user flags provided to gcc: -fembed-bitcode-marker [/Users/xamarinqa/myagent/_work/8/s/xamarin-macios/tests/xharness/tmp-test-dir/monotouch-test58/monotouch-test-tvos.csproj]
clang : error : linker command failed with exit code 1 (use -v to see invocation) [/Users/xamarinqa/myagent/_work/8/s/xamarin-macios/tests/xharness/tmp-test-dir/monotouch-test58/monotouch-test-tvos.csproj]
Also fix an issue in mtouch where we would overwrite any previous --dlsym
values; they're now accumulative (`--dlsym:foo.dll --dlsym:bar.dll` works
as expected)
Ref: https://github.com/nunit/nunit/issues/3618
* [MapKit] Update the framework to Xcode 12 beta 4.
* Apply suggestions from code review
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
* Address reviews.
* Add ignore to init test because we should use the Create method.
* Fix failing tests.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Use two separate output variables (EntitlementsInExecutable/EntitlementsInSignature)
instead of using the same output variable for two different purposes. This makes
the code more self-explanatory.
Also move the simulator check to the C# code, that way it's easier to re-use elsewhere.
* [msbuild] Share the _CompileEntitlements task.
* [msbuild] Create the same class hierarchy for Xamarin.Mac and Xamarin.iOS for the CompileEntitlements task.
CompileEntitlementsTaskBase
└─── iOS/CompileEntitlementsTaskCore
│ └─── iOS/CompileEntitlements
└─── Mac/CompileEntitlementsTaskCore
└─── Mac/CompileEntitlements
This also means we can remove a known failure in the list of MSBuild tasks that don't
conform to our 'no code in final task implementation' requirements.
This fixes numerous P/Invoke test failures like this:
Could not find the symbol 'GlobalizationNative_GetCalendars' in QCall
Could not find the symbol 'GlobalizationNative_GetCalendarInfo' in QCall
Could not find the symbol 'GlobalizationNative_EnumCalendarInfo' in QCall
Could not find the symbol 'GlobalizationNative_GetLatestJapaneseEra' in QCall
[...]
Ref: a56d6a8034
Ref: fae477f34b
This also means that setting 'run-all-tests' enables the .NET tests, which
means they'll now start running on internal Jenkins (as was the intention from
the beginning).
With Mono, Environment.OSVersion returns the macOS version when running in the
simulator. On Catalina, Mono returns 19.* as the Environment.OSVersion, which
means that the check for major version = 15 in the simulator turns out to be a
constant value now (since we don't support running in the simulator on a
4-year-old macOS version).
With .NET, Environment.OSVersion returns the iOS version instead, which means
that the check would have to be reworked.
However, since these version checks have effectively been constant for a few
years now, we can just remove the checks and the resulting unreachable code.
* [Tests] Fix TryGetSupportedInterfaces when we have values. Fixes#2273
Make the test check against the value returned by the native API rather
than expect the result to always be null.
Fixes https://github.com/xamarin/maccore/issues/2273
* Address reviews.
* [Harness] Add support to create tunnels.
Add support to create tunnels in case the devices cannot connect to
the host. This option is false by default, which means that unless told
otherwise xharness will try to se a tcp connection over the WiFi.
We are not setting it as default because the devices in DDFun will have
access to an unrestricted network. Nevertheless after this PR we will
create a new one with the following logic:
1. Try to use the tcp connection using the network.
2. If devices cannot connect to the host via the network, fall back to
the tcp tunnel.
This change executes a tunnel process per test application, most of the
cases out of the 150 test application we execute, most of them (maybe 98%
but most % are made up) will pass, and just a few of them will fail. The
reason is that there is a daemon in the OS that gets underwater.
Rather than tryingt o find a hacky way to re-use the tunnel, lets KISS
and if we need to hack that, do it as an enhancement.
The bcltest dir should only have the .ignore files, the code generator
creates all the different projects and those files were left behind when
they should have been removed.
* The info.plist is needed. This was added to issue: https://github.com/xamarin/xamarin-macios/issues/8426
Moved all the code that can be shared with the CLI to the common
library. We neede some small changes, but mainly due to namespaces and a
forgotten interface implemenation (was already implemented, was missing
in the class).
This was fun :)
Refactor TestProject so that we can move all the tasks to the common
assembly. We had to remove all the references from Harness, that
included the MonoNativeInfo.
Move all the logic outside and use it as a Composition pattern, later
this class can be used in the CLI so that we share the logic of building
and tested.
In order to de-couple the RunTestTask from Jenkins, create an interface
to be implemented, which is pass to use as a member (Composition
pattern). In order to do that, do not expose Jenkins as a property of
the interface because it is required just by the base constructor.
Moving the the use an interface meant a lot of small changes that
should have no real effect (the compiler should have caught any possible
issues).
The initial idea of the refactoring looked nice but as soon as we want
to get the RunTestTask out of jenkins, we have a number of naming
issues. Move the tools to not use the *Task postfix so that it is
cleaner and we can later extra the RunTestTask better.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>
Move all the logic outside of the Jenkins namespace. Rework a little the
inheritance to make it nicer in the constructors.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>
Move the task and use composition so that we can reuse the code. This
will later allow other projects to use the class without the need of
Jenkins or Harness and just implement the base class.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>
Some of the Jenkins test tasks are very useful and do A LOT of stuff. So
we try to generalize the base class, to later be able to share the most
usebul ones in the shared lib.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
* [xcode12] Bump to Xcode 12 Beta 4
* [Tests] Disable Trust_FullChain and Trust2_FullChain tests
These tests started showing a different error than the one we expect
I'll let our security expert chime in
Update the framework to include Xcode beta3 and add support for tvOS.
As of Xcode12 beta 3 it cannot yet be compiled on macOS X, please refer
to https://github.com/xamarin/maccore/issues/2261
* Touch.Client is a tiny mobile UI/runner for the official NUnitLite. This
means we'll stop using a very old fork of NUnitLite to run our tests, and
instead use the most recent upstream version instead.
* Some test changes were required to use the official NUnitLite, because there
have been breaking changes there.
* NUnitLite is now referenced using a NuGet, which means there are numerous
changes to support this.
* We're not using GuiUnit anymore, so that dependency has been removed.
* Reduces code duplication.
* Makes the macOS versions thread-safe (the iOS versions have been thread-safe
for years: 2c6a5303a7).
* A few parameters names were different in the definitions; I chose to keep the ones in Xamarin.iOS, since they looked better.
* Xamarin.Mac had two methods, SetTextBlocks and SetTextLists, in place of an actual property override (for the mutable setter), this was fixed to be an actual property overload, and compat methods were implemented.
* xtro needed an update to cope with multiple static methods for the same selector.
This solves a rebuild problem if an assembly has an invalid or unsupported symbol
file, where we'd detect that the symbol file exists, and expect it to be copied,
but then the linker would drop it, causing us to always rebuild the app (this is
not the same as when a symbol file is out of date).
This happens for NUnitLite 3.12.0's nunit.framework.dll, which ships with an old-style
pdb.
Also add a warning that is shown when we detect that there's a symbol file, but it
couldn't be loaded for some reason.
We weren't properly reporting all failures when there were multiple failures
in parameterized tests, because we'd incorrectly skip too many xml nodes when
we were looking for the next test failure.
I would have liked to automatically get all the references required by bindings-test.dll,
but that turned out to be not as simple as I first thought (since bindings-test is
a library project, the exact dependencies actually depend on the type of the executable
project that references it, which means it's quite difficult to calculate the final
references given just the binding project).
Instead just hardcode the fact that there's a reference to nunit.framework.dll, and
the corresponding on-disk path (we still look in the actual csproj to get the version
of nunit.framework.dll).
In particular NUnit uses reflection to get a private method, and the linker removes
the corresponding private method:
1c680b4dc8/src/NUnitFramework/framework/Internal/TestExecutionContext.cs (L552)
So add an xml definition to keep this private method, and modify project files to
pass the xml definition to mtouch and mmp.
Some care needs to be taken to make sure xharness is still able to clone these project
files.
These tests now end up testing the NUnitLite and MonoTouch.Dialog assemblies from
NuGet, not the ones we ship, which means they're out of our control to fix.
* Touch.Client references the official NUnitLite package, which means we're using
a non-forked version of NUnit.
* This makes it easier for our .NET 5 effort, since we won't have to port an ancient
version of NUnitLite to .NET 5 (nor will we have to keep using it in our existing
code, we can use more modern NUnit patterns).
* Reference MonoTouch.Dialog from the NuGet package. This also eases the .NET 5 effort,
since we won't have to port MonoTouch.Dialog to .NET 5 (we'll probably still do it
though at some point, but it doesn't have to be done right away), nor build it
ourselves / ship it.
* [xharness] Mark the .NET tests project as a .NET project so that we don't try to call nuget restore on it.
* [xharness] Mark the .NET generator project as a .NET project as well.
Packages are listed in the csproj when we're using package references.
Also only list files in git, otherwise we pick up all the generated project files as well.
- This commit adds a hook, "AdditionalAppExtensions", to the msbuild to allow
extensions written in other languages, such as Swift, to be embedded and signed in an
Xamarin App bundle easily.
- Example:
<AdditionalAppExtensions Include="$(MSBuildProjectDirectory)/../../native">
<Name>NativeTodayExtension</Name>
<BuildOutput Condition="'$(Platform)' == 'iPhone'">build/Debug-iphoneos</BuildOutput>
<BuildOutput Condition="'$(Platform)' == 'iPhoneSimulator'">build/Debug-iphonesimulator</BuildOutput>
</AdditionalAppExtensions>
* [MetalPerformanceShaders] Neural Networks Update to Xcode 11
This includes updates from PRs xamarin/xamarin-macios#6932, xamarin/xamarin-macios#6935 and xamarin/xamarin-macios#7461
It adds new functionality to the neural network components.
This is still not the complete API for 11.3
* Update src/metalperformanceshaders.cs
Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>
* Make changes requested by @rolfbjarne
* Fix binding of default random distribution creation
The parameterless function creates new
default distributions and is not a property.
It is a counterpart of CreateUniform.
* Expose public APIs for MPSCnnConvolutionTransposeNode
These APIs are public and documented at: https://developer.apple.com/documentation/metalperformanceshaders/mpscnnconvolutiontransposenode/2942641-initwithsource?language=objc
I have also tested that they work.
* Reintroduce compat API.
* Fix acronym casing.
* Fix introspection tests.
* Fix xtro.
* One last xtro issue.
* Fix more xtro.
* Another introspection fix.
Co-authored-By: Frank A. Krueger <fak@praeclarum.org>
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
* Apply suggestions from code review
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
* Apply feedback
* Please the typo guardians
Co-authored-by: Frank A. Krueger <fak@praeclarum.org>
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
* [dotnet] Only pass a single custom step to the linker.
The linker will load the assemblies with the custom steps once per custom step
argument, which means that each step is effectively in a different assembly,
making it impossible to share state between steps.
This behavior is filed as a linker bug: https://github.com/mono/linker/issues/1314
Until this is fixed, we can just have a single step that injects all the other
steps programmatically.
* [tests] Adjust .NET tests according to new behavior.
It turns out some Xamarin.Mac projects are library projects (for unit tests),
and we still need to generate makefile targets for those (because we use them
to build Xamarin.Mac tests to run on older macOS bots).
* [MetalPerformanceShaders] Neural Networks Update to Xcode 11
This includes updates from PRs xamarin/xamarin-macios#6932, xamarin/xamarin-macios#6935 and xamarin/xamarin-macios#7461
It adds new functionality to the neural network components.
This is still not the complete API for 11.3
* Update src/metalperformanceshaders.cs
Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>
* Make changes requested by @rolfbjarne
* Fix binding of default random distribution creation
The parameterless function creates new
default distributions and is not a property.
It is a counterpart of CreateUniform.
* Expose public APIs for MPSCnnConvolutionTransposeNode
These APIs are public and documented at: https://developer.apple.com/documentation/metalperformanceshaders/mpscnnconvolutiontransposenode/2942641-initwithsource?language=objc
I have also tested that they work.
* Reintroduce compat API.
* Fix acronym casing.
* Fix introspection tests.
* Fix xtro.
* One last xtro issue.
* Fix more xtro.
* Another introspection fix.
Co-authored-By: Frank A. Krueger <fak@praeclarum.org>
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
* Apply suggestions from code review
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
* Apply feedback
* Please the typo guardians
Co-authored-by: Frank A. Krueger <fak@praeclarum.org>
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
It's a valid scenario to set a null environment variable.
This happens when changing InCI to always return true (to test locally what's
done on the bots), in which case we try to forward BUILD_REVISION to mlaunch,
but BUILD_REVISION isn't necessarily set. Not forwarding it to mlaunch should
not cause problems in most tests, so just allow this scenario.
According to the existing code, we're already not supposed to create makefile
targets for BCL tests. However, the code to detect BCL tests wasn't quite
reliable, so we ended up creating makefile targets for some BCL tests.
So always use the same logic to detect BCL tests.
Also remove a lot of dead code to generate makefile code for BCL tests.
Split the BCL project generation in two: one part that figures out which
projects to generate, and which is done synchronously (because we need it to
compute the list of tests), then split out the actual generation and run it on
a background thread (since that doesn't have to happen until we want to
execute those tests).
This speeds up launching xharness in server mode significantly (from ~2s to
~0.2s).
It's running into different behavior between OS versions, and it's getting
time-consuming and annoying to keep up with the changes.
So instead just remove the test, because as far as I can see it's not testing
something important anyway (we don't care exactly which library provides the
'mono_native_initialize' function, the important part is that it's callable,
which we already test elsewhere).
* [xharness] Don't generate into the same directory/files for macOS Modern and macOS Full.
This meant that we were overwriting some generated files, which meant that we
were executing the Modern set of tests instead of the Full set of tests for at
least some configurations.
* Add a few ignored tests to the System.Configuration test now that we actually run it.
Manuel de la Pena
Whitney Schmidt
Dorothy Tam
Sebastien Pouliot
Rolf Bjarne Kvinge
monojenkins
Přemek Vysoký
Alex Soto
Chris Hamons