I started fixing nullability in one place, and then it snowballed a bit
and I had to fix nullability in a lot of places.
Most are trivial, except for the `generate-frameworks-constants`
project: I had to create a .NET version of the project in order to
compile a .NET version of the tool.
Replaced the existing type map in StaticRegistrar.cs with a
`CSToObjCMap`.
Added code to write it out to a specified path as XML.
Currently the path is a parameter that defaults to null and is not (yet)
used.
* Update `SdkVersions.cs` after the latest Xcode 14.2 bump.
* Rename `[SdkVersions|ProductConstants].cs.in` to `[SdkVersions|ProductConstants].in.cs`.
This way the autoformatter makes sure it's formatted correctly.
We need parts of tools/common/SdkVersions.cs when building tests on Windows.
In order to simplify our Windows-life, we're going to check in the generated
SdkVersions.cs file, that way it won't have to be re-generated on Windows (the
logic is very make-based, and not easily executed on Windows).
However, parts of SdkVersions.cs would change every commit, which would make
the above solution rather annoying. So split out those parts into a new file
(ProductConstants.cs), which is still generated during the build (and not
checked in).
The Xamarin.MacDev.Tasks.sln solution is built with dotnet, while other projects
are still built with msbuild. This becomes a problem when generating Errors.designer.cs,
because depending on the runtime the output is different.
This means that the Errors.designer.cs will sometimes randomly change (depending
on which project re-generated the file), leaving the file modified in git. This is
quite annoying, but it also breaks the api comparison, which depends on the build
not leaving modified files behind. So for now, we generate Errors.designer.cs separately
for Xamarin.MacDev.Tasks.sln to not conflict with the mtouch version.
Also fix capitalization in numerous places to be consistent (it's Errors.designer.cs,
not Errors.Designer.cs).
We already have a few attributes that can specify the native name for a type, whenever the native name doesn't match the managed name:
* [Register ("DifferentClassName"): specifies the Objective-C class name
* [Native ("DifferentEnumName")]: specifies the Objective-C enum name (and also that it's a native-sized enum)
* [Protocol ("DifferentProtocolName")]: specifies the Objective-C protocol name
* [Category ("DifferentCategoryName")]: specifies the Objective-C category name
Unfortunately this leaves (at least) two cracks:
* Objective-C structs.
* Objective-C enums which aren't native-sized.
So I'm adding a [NativeName] attribute for this purpose, and updating numerous
types to specify the native name (either using an existing [Native] attribute
for enums that already have one, or by adding a new [NativeName] attribute).
The static registrar needs to know the native name for such types, in case
they appear as parameter types in function signatures.
This also allows us to simplify xtro a bit, to not have a separate map of
managed name given a native name, because we can now build that map
dynamically.
* [tools] Extract the logic to parse OSPlatformAttribute platform names to a separate file/class.
* [introspection] Migrate .NET code to use the new .NET-style availability attributes.
This also means using the 'ApplePlatform' enum instead of the 'PlatformName'
enum, because the latter will be removed in .NET.
* [FileProvider] Exclude some deprecated API from .NET.
* [AVFoundation] Adjust availability attribute for AVCaptureStillImageOutput.HighResolutionStillImageOutputEnabled.
* Update tests.
* making sure new strings get added to designer and resources plus the test
* Next wave of changes to csproj to incorporate Rolf's changes
* fixing path
* Update tests/msbuild/Xamarin.MacDev.Tasks.Tests/TaskTests/LocalizationStringTest.cs
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
* Update tests/mtouch/LocalizationTests.cs
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
* forgot the include
Co-authored-by: tj_devel709 <antlambe@microsoft.com>
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
* Update to new linker custom steps API
* PR feedback
- Fix indentation
- Add Initialize(LinkContext) to ExceptionalMarkHandler
- Remove unnecessary ifdef
- Use IsSetter/IsGetter
- Use [0] instead of Single()
- Avoid allocating empty collections
* Note override issue
* Clean up comments
* Move `DynamicRegistrationSupported` change earlier, along with the
detection code.
This solve the issue that `ILLink` does a similar job _before_ we have
the chance to disable the dynamic registrar.
* ILLink does not support considering other attributes as `[Preserve]`
when it is itself preserved at the assembly-level.
This ignored test is checking that feature so it cannot be enabled
for `NET`
Added to known breaking changes https://github.com/xamarin/xamarin-macios/issues/8900
* Fix removal of the dynamic registrar on legacy
* Fix IntPtr size inlining
Co-authored-by: Sebastien Pouliot <sebastien.pouliot@microsoft.com>
Use a private property (prefixed with underscore) for now, until we can decide
on a better/general name.
Also add a variation to xharness to build monotouch-test with CoreCLR
(building works fine, but it crashes at startup, which is expected at this
point).
* Bump Xamarin.MacDev.
New commits in xamarin/Xamarin.MacDev:
* xamarin/Xamarin.MacDev@1e738e9 [Xamarin.MacDev] Extract the code to convert between Mac Catalyst versions to a separate file. (#89)
* xamarin/Xamarin.MacDev@a3bb12c [Xamarin.MacDev] Add methods to map between iOS and macOS versions for Mac Catalyst. (#88)
Diff: 02d6d05be3..1e738e9f7f
* [msbuild] Use the macOS SDK to build Mac Catalyst apps instead of the iOS SDK
From a native compilation perspective, compilating a Mac Catalyst is the macOS SDK
+ a dash of iOS, so use the native macOS SDK to compile, and then do the corresponding
adjustments elsewhere.
At the same time document which version we want for the sdk version and the deployment
target in mtouch, and adjust the code accordingly (sdk version: macOS version; deployment
target: iOS version).
* Update resource files
* Add new entry to canary test for string localization.
This means:
* Move the parts of the ApplyPreserveAttribute step that we don't need for.NET
into a new MobileApplyPreserveAttribute step, and have mtouch and mmp use
that step instead of the ApplyPreserveAttribute step.
* Copy ApplyPreserveAttributeBase into dotnet-linker from the upstream tuner
source (with minor modifications) so that our ApplyPreserveAttribute step
compiles.
* Create a DotNetSubStepDispatcher class that we're going to use as our
substep dispatcher, create an instance of it and insert it into the list of
linker steps.
* Also a workaround for the lack of LinkContext.GetAssemblies (): add a step
that collects all the assemblies and stores them in a list, so that we can
have our own GetAssemblies implementation.
I filed a linker issue to see if we can get LinkContext.GetAssemblies ()
exposed to us: https://github.com/mono/linker/issues/1455.
Fixes this startup crash with the linkall test:
2020-08-26 19:47:10.219697+0200 link all[32709:6065783] Xamarin.iOS: Fatal error: failed to load the method 'ObjCRuntime.Runtime.Initialize'.
=================================================================
Native Crash Reporting
=================================================================
Got a abrt while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.
=================================================================
=================================================================
Native stacktrace:
=================================================================
0x104007b0e - /Users/rolf/Library/Developer/CoreSimulator/Devices/289E372A-501C-4499-A1A6-59C5B3B6A9AE/data/Containers/Bundle/Application/4A5F3968-6980-4E90-88A2-2E77AE039C40/link all.app/libmonosgen-2.0.dylib : mono_dump_native_crash_info
0x103fb4437 - /Users/rolf/Library/Developer/CoreSimulator/Devices/289E372A-501C-4499-A1A6-59C5B3B6A9AE/data/Containers/Bundle/Application/4A5F3968-6980-4E90-88A2-2E77AE039C40/link all.app/libmonosgen-2.0.dylib : mono_handle_native_crash
0x104007365 - /Users/rolf/Library/Developer/CoreSimulator/Devices/289E372A-501C-4499-A1A6-59C5B3B6A9AE/data/Containers/Bundle/Application/4A5F3968-6980-4E90-88A2-2E77AE039C40/link all.app/libmonosgen-2.0.dylib : sigabrt_signal_handler
0x7fff51c005fd - /Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 13.5.simruntime/Contents/Resources/RuntimeRoot/usr/lib/system/libsystem_platform.dylib : _sigtramp
0x0 - Unknown
0x7fff51af0b7c - /Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 13.5.simruntime/Contents/Resources/RuntimeRoot/usr/lib/system/libsystem_c.dylib : abort
0x103daad98 - /Users/rolf/Library/Developer/CoreSimulator/Devices/289E372A-501C-4499-A1A6-59C5B3B6A9AE/data/Containers/Bundle/Application/4A5F3968-6980-4E90-88A2-2E77AE039C40/link all.app/libxamarin-debug.dylib : xamarin_assertion_message
0x103dade77 - /Users/rolf/Library/Developer/CoreSimulator/Devices/289E372A-501C-4499-A1A6-59C5B3B6A9AE/data/Containers/Bundle/Application/4A5F3968-6980-4E90-88A2-2E77AE039C40/link all.app/libxamarin-debug.dylib : xamarin_initialize
0x103dbf80b - /Users/rolf/Library/Developer/CoreSimulator/Devices/289E372A-501C-4499-A1A6-59C5B3B6A9AE/data/Containers/Bundle/Application/4A5F3968-6980-4E90-88A2-2E77AE039C40/link all.app/libxamarin-debug.dylib : xamarin_main
0x103cd2f0d - /Users/rolf/Library/Developer/CoreSimulator/Devices/289E372A-501C-4499-A1A6-59C5B3B6A9AE/data/Containers/Bundle/Application/4A5F3968-6980-4E90-88A2-2E77AE039C40/link all.app/link all : main
0x7fff51a231fd - /Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 13.5.simruntime/Contents/Resources/RuntimeRoot/usr/lib/system/libdyld.dylib : start
This is a partial/modified port of the initial linker support (bc88790201)
Co-authored-by: Sebastien Pouliot <sebastien@xamarin.com>
The shared version isn't used by mmp yet as far as I can tell (mmp has its own logic
to copy assemblies), but sharing this code is the first step towards having the same
implementation as well.
* Create a simple Xamarin.Utils.Execution class that can handle all our
process execution needs:
* Captures or streams stdout/stderr (in UTF8).
* Supports async
* Supports a timeout
* Does not depend on any other source file we have, only uses BCL API.
* Have the execution helper classes from mtouch/mmp
(Xamarin.BundlerDriver.RunCommand) and the tests
(Xamarin.Tests.ExecutionHelper) use this new class.
* Some simplifications were made:
* All API that took a string array for the environment now takes a
Dictionary<string, string>.
* The Driver.RunCommand methods were split out to a separate file. This
file also contains a Verbosity field, which is conditioned on not being
in mtouch nor mmp, which makes including this file from other projects
simpler (such as bgen - in particular bgen was modified to use this
Verbosity field instead of its own).
Solves build errors like this:
xamarin-macios/tools/dotnet-linker/packages/xlifftasks/1.0.0-beta.20154.1/build/XliffTasks.targets(91,5): error : 'Errors.cs.xlf' is out-of-date with 'Errors.resx'. Run `msbuild /t:UpdateXlf` to update .xlf files or set UpdateXlfOnBuild=true to update them on every build, but note that it is strongly discouraged to set UpdateXlfOnBuild=true in official/CI build environments as they should not modify source code during the build.
See also: https://github.com/xamarin/xamarin-macios/pull/8157
* Move much of ErrorHandler.cs into a partial class in ErrorHandler.tools.cs,
which is referenced by mtouch and mmp (but not our runtime).
* Add ErrorHandler.runtime.cs for runtime-specific bits, including a simpler
version of ErrorHandler.Show. In particular this gets rid of the call to
Environment.Exit, which should never happen at runtime.
* Rename MonoTouchException and MonoMacException to ProductException, which
allows us to remove a lot of ifdefs.
* This required moving Application.LoadSymbols and Target.LoadSymbols to
shared mtouch/mmp code.
* Files outside the project directory now show up with a much more logical
name in the IDE. External files have a link target which is relative to the
root xamarin-macios directory, and files included from the mono archive show
up as such as well.
* Rename a few mtouch/mmp-specific files to contain 'mtouch' or 'mmp' in the
filename, to avoid having multiple files in the projects with the same name
(it's confusing every time you search for a filename in the IDE and get
multiple filenames where only the directory is different).
* Add a tools.sln that contains only the mtouch and mmp projects. This makes
it easier to work with both mtouch and mmp and the same time, while not
making VSfM unbearably slow by loading many projects in the same solution.
We're using the Release configuration to build the mtouch and mmp binaries
that we ship, which means that we can use the Debug configuration for
debugging from an IDE, and use the standard conditional compilation symbols to
identify that case.
Bump mono to 2020-02 head to be in sync with Android.
Co-authored-by: Imran Hameed <imhameed@microsoft.com>
Co-authored-by: Sebastien Pouliot <sebastien.pouliot@microsoft.com>
Co-authored-by: Ankit Jain <radical@gmail.com>
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
Co-authored-by: Sebastien Pouliot <sebastien.pouliot@gmail.com>
* [mtouch][mmp] Share error.cs and make it safer against FormatException
Beside the obvious code sharing [1] this makes the error reporting code
safer against `FormatException`.
Even with code review this happened a few times in the past - we get
a string that can't format correctly in some error handling code. This
means the `FormatException` must be fixed before we can see the
underlying root issue (delaying bug fixes and support).
Recent changes made for localization makes it harder to review the
original strings (and even harder for translated strings) for
correctness. This means we can end up with something like
```
String.Format ("{0} {1}", "foo");
``
and easily miss it (since they split between different files) if there
are several changes inside a PR.
Finally sprinkle some unit tests :)
[1] we should move everything to use `ProductException` but that will be
for another PR...
Using this option it's possible to test for the presence of a type
reference in both pre-linked and post-linked assemblies.
This makes it possible to detect if
* a 3rd party assemblies are using some specific type you would like to avoid;
* a type reference has been removed during the build (e.g. linker)
Notes:
* Custom attributes are encoded differently and not included in the assembly type references metadata.
* Assembly that define a type `X` do not have a reference (but the definition) of the type (and won't be reported).
If either the pre or post-linked warnings are not useful then it's possible
to add `-nowarn:150x` to exclude the results.
E.g.
* `-nowarn:1502` would not report references in pre-linked assemblies;
* `-nowarn:1503` would not report references in post-linked assemblies;
Finally `-warnaserror:150x` can be used to stop a build that would not
satisfy either the pre or post-linked condition.
* `-warnaserror:1502` would not report references in pre-linked assemblies;
* `-warnaserror:1503` would not report references in post-linked assemblies;
_side note_ same as https://github.com/xamarin/xamarin-macios/pull/7925
except that this one uses the localized mtouch/mmp errors only in master (so far)
- https://devdiv.visualstudio.com/DevDiv/_workitems/edit/947932
- It turns out the linker in some cases can wrap expected exceptions in a outer exception. By drilling in we can produce better errors, again.
- Refactor pipeline exception handling to be shared between mmp/mtouch
## Miscellaneous fixes
* Fixed
`/Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/builds/mono-ios-sdk-destdir/ios-sources/external/linker/src/linker/Linker.Steps/OutputStep.cs(110,15): error CS0246: The type or namespace name ‘OutputException’ could not be found (are you missing a using directive or an assembly reference?) [/Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/tools/mmp/mmp.csproj]`
* Changed the name of the method that is used from linker. Because of this commit 6be26771b9
* Added `OutputException.cs` file on `mtouch.csproj`.
* Removing enter_gc_safe and exit_gc_safe because now it's already gc_safe in this part of code, after a mono change.
* Added known exceptions to LLVM exception list.
* Needs `ifdef` because of this https://github.com/mono/mono/pull/17260.
* Bump MIN_MONO_VERSION to 6.8.0.41 and point MIN_MONO_URL to the PR.
* Add ENABLE_IOS=1 and ENABLE_MAC=1.
* Added switch to disable packaged mono build
* [Tests] Ignore tests that fail on 32b.
Ignore the test on 32b, and filled issue: https://github.com/mono/mono/issues/17752
* [Tests] Ignore a couple of tests causing OOM.
Hopefully fixes https://github.com/xamarin/maccore/issues/1659 for good.
* Ignore `MM0135` test on Catalina+ because it needs Xcode 9.4.
* [monotouch-test] Add null checks for teardown when test didn't run because of a too early OS version.
* [CFNetwork]: Http 2.0 requires OS X 10.11 or later.
Check whether `_HTTPVersion2_0` is available and fallback to HTTP 1.1 otherwise.
## Bring HttpClient from CoreFX
* #7346
* This bumps Mono to use https://github.com/mono/mono/pull/17645 (which is the 2019-10 backport
of https://github.com/mono/mono/pull/17628).
* The big user-visible change is in regards to certificate validation, everything below are just
some minor adjustments to tests.
### SocketsHttpHandler
CoreFX uses a completely new `HttpClientHandler` implementation called `SocketsHttpHandler`,
which you can find at https://github.com/dotnet/corefx/tree/release/3.0/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler.
Since this is not based on the web stack anymore, it does not use any of the related APIs such
as `ServicePointManager` or `WebException`.
### Certificate Validation Changes
There is a new API called `HttpClientHandler.ServerCertificateCustomValidationCallback`.
- https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpclienthandler.servercertificatecustomvalidationcallback?view=netframework-4.8
- c1778515a3/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Unix.cs (L154)
- c1778515a3/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Windows.cs (L383)
The `ServicePointManager.ServerCertificateValidationCallback` is no longer invoked and on
certificate validation failure, `AuthenticationException` (from `System.Security.Authentication`)
is thrown instead of `WebException`.
At the moment, the `NSUrlSessionHandler` still uses it's own validation callback and also still
throws `WebException` on failure; we should probably look into making this consistent with the
other handlers.
### Minor adjustments related to internal Mono APIs
* `HttpContent.SerializeToStreamAsync()` is now `protected` (changed from `protected internal`).
- src/Foundation/NSUrlSessionHandler.cs: changed overload accordingly.
- src/System.Net.Http/CFContentStream.cs: likewise.
* `HttpHeaders.GetKnownHeaderKind()` is an internal Mono API.
There is a new internal API called `System.Net.Http.PlatformHelper.IsContentHeader(key)`
which exists in both the old as well as the new implementation.
The correct way of doing it with the CoreFX handler is
`HeaderDescriptor.TryGet (key, out var descriptor) && descriptor.HeaderType == HttpHeaderType.Content`
### Minor adjustments to tests.
* `HttpClientHandler.MaxRequestContentBufferSize` is now longer supported, you can set it to
any non-negative value, the getter will always return 0.
See c1778515a3/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Core.cs (L18).
- tests/linker/ios/link sdk/HttpClientHandlerTest.cs: removed assertion from test.
* `HttpMessageInvoker.handler` is a `protected private` field - in the CoreFX handler, it is
called `_handler` and `private`. This is accessed via reflection by some of the tests, which are
now using the new name.
- tests/mmptest/src/MMPTest.cs: here
- tests/mtouch/MTouch.cs: here
* tests/monotouch-test/System.Net.Http/MessageHandlers.cs:
Adjust `RejectSslCertificatesServicePointManager` to reflect the certificate validation
changes described above.
- FIXME: There was an `Assert.Ignore()` related to `NSUrlSessionHandler` and macOS 10.10;
I removed that to reenable the test because the description linked to an old issue in
the private repo that was referenced by several "Merged" PR's, so it looked to me that
this might have already been fixed - and I also didn't see why it would fail there.
Rolf Bjarne Kvinge
Steve Hawley
Filip Navara
Sebastien Pouliot
TJ Lambert
Sven Boemer
Jo Shields
Waleed Chaudhry
Chris Hamons
Alexander Köplinger