Also rename DOTNET_VERSION to SYSTEM_DOTNET_VERSION to make it clear what it's
referring to (and to not clash with DOTNET6_VERSION which has now been renamed
to DOTNET_VERSION).
.NET 7 is right around the corner.
The main theme here is that code signing will be done in the outermost
executable project, not in any app extension projects or watch projects, nor
during the RID-specific build of a .NET universal app. This makes codesigning
easier to reason about and other affected logic (such as strip/dsymutil)
easier to handle, in particular for .NET universal apps. Another benefit is
that the differences between the iOS and macOS code bases have been
eliminated.
The first step is to collect all the information we need from the targets
files. Every app bundle (be it app extension, watch app or main app) will add
its own output app bundle (.app/.appex) to the _CodesignBundle item group.
Then every app bundle will load this informarion from referenced app bundles,
and finally store this information on disk (in the 'codesign-bundle.items'
file). This means that in the end the main app bundle will have a list of all
contained app bundles in the app (recursively), in the _CodesignBundle item
group.
Separately we keep a list of other items that need signing, in the
_CodesignItems item group, and we do the same store/load logic for every
contained/contained app bundle (in the 'codesign.items' file, so a the end the
main app bundle will have a list of all the _CodesignItems for all contained
app bundles (recursively).
The previous steps occur in the _CollectCodesigningData and
_StoreCodesigningData targets.
The next step is to use the new ComputeCodesignItems task to compute
everything we need to know for code signing. This task takes over the
responsibility for listing all the *.dylib and *.metallib files, and the
*.framework directories in the app bundles, that need signing (which was
previously done in the targets file). This logic is significantly easier to
write, debug and test in C# than MSBuild.
In addition the ComputeCodesignItems also figures out a stamp file path we use
to determine if something needs (re-)signing. Previously .framework
directories did not have a stamp location, so they'd always end up resigned in
a rebuild, while now we'll automatically skip signing *.framework directories
unless something changed in them.
I've also tried to comment everything thorougly, for the next poor soul having
to deal with any bugs.
Behavioral differences:
* We were always signing *.dylib files for macOS. We're now doing the same
thing for all platforms.
* We're now always signing *.framework directories for all platforms (like we
do for *.dylib files), since frameworks are pretty much like dylibs anyways.
I've verified that this works both by running the submission tests and running
and launching a sample project on device from Windows.
* Remove existing attributes
* Attribute Conversion
* First cut of code for review.
* Clean up csproj
* nullable enable and cleanup nullability. Compiles with no errors or warnings.
* typo
* Remove nullables with cleaner code, reworked things with Try...pattern.
* last couple '== null' changes
The main theme here is that code signing will be done in the outermost executable
project, not in any app extension projects or watch projects, nor during the RID-specific
build of a .NET universal app. This makes codesigning easier to reason about and
other affected logic (such as strip/dsymutil) easier to handle, in particular for
.NET universal apps. Another benefit is that the differences between the iOS and
macOS code bases have been eliminated.
The first step is to collect all the information we need from the targets files.
Every app bundle (be it app extension, watch app or main app) will add its own output
app bundle (.app/.appex) to the _CodesignBundle item group. Then every app bundle
will load this informarion from referenced app bundles, and finally store this information
on disk (in the 'codesign-bundle.items' file). This means that in the end the main
app bundle will have a list of all contained app bundles in the app (recursively),
in the _CodesignBundle item group.
Separately we keep a list of other items that need signing, in the _CodesignItems
item group, and we do the same store/load logic for every contained/contained app
bundle (in the 'codesign.items' file, so a the end the main app bundle will have
a list of all the _CodesignItems for all contained app bundles (recursively).
The previous steps occur in the _CollectCodesigningData and _StoreCodesigningData
targets.
The next step is to use the new ComputeCodesignItems task to compute everything we
need to know for code signing. This task takes over the responsibility for listing
all the *.dylib and *.metallib files, and the *.framework directories in the app
bundles, that need signing (which was previously done in the targets file). This
logic is significantly easier to write, debug and test in C# than MSBuild.
In addition the ComputeCodesignItems also figures out a stamp file path we use to
determine if something needs (re-)signing. Previously .framework directories did
not have a stamp location, so they'd always end up resigned in a rebuild, while now
we'll automatically skip signing *.framework directories unless something changed
in them.
I've also tried to comment everything thorougly, for the next poor soul having to
deal with any bugs, as well has adding a comprehensive test for the new task.
Behavioral differences:
* We were always signing *.dylib files for macOS. We're now doing the same thing
for all platforms.
* We're now always signing *.framework directories for all platforms (like we do
for *.dylib files), since frameworks are pretty much like dylibs anyways.
Or make files canculate the version of the nuget based on the hash of
the source, because VSTS does a merge of the branhc rather than checking
out, the version calculated in the tests bot via de makefiles is
different to the one that was used in the build bots.
We solve this by creating the rollback file at build time so that it is
consumed by the test bot.
The static registrar usually stores a compressed version of metadata tokens in
the generated code. However, when there are many assemblies in the app (>127),
we can't use the compressed version anymore, and fall back to a full version.
In this case, we weren't comparing type metadata tokens correctly when looking
for a type in our table of types, and thus we weren't finding the type we were
looking for.
The result is an exception like this:
> Can’t register the class MyClass when the dynamic registrar has been linked away.
In the generated table of types we're storing the full metadata token, which
includes a few bits indicating which type of token it is (in this particular
case a TypeDef token). When going through the table looking for a type, we
need to compare with those few bits set on the input type token as well to
find what we're looking for.
Also make it possible to use the remove-dynamic-registrar optimization on
macOS (which is useful by itself, but it also makes adding a test case
easier).
Fixes https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1476585.
Fixes https://github.com/xamarin/xamarin-macios/issues/11641.
If we have an issue unlocking the UI prompt we continue on error. This
is considered a warning in the pipeline, but doing so teaches users to
ignore warnings and will result in us ignoring real actual issues.
Setting the step to not have a warning will ensure that we only see
warnings that we care about.
This fixes an issue where we'd do logic with Windows-style paths on macOS, and that's
never the right thing to do.
For the LinkNativeCode task, this would manifest as this error when building from windows:
> ld: file too small (length=0) file 'obj/Debug/net6.0-ios/iossimulator-x64/nativelibraries/libSystem.Native.dylib' for architecture x86_64
because the 'ShouldCopyToBuildServer' method would return incorrect results.
For the Codesign task, it would manifest as an exception trying to create a
directory with an empty string (because the directory name of a windows-style
path is an empty string on macOS).
Since this exception was quite useless (just getting the exception message
didn't tell me much about what caused the exception, because it had no stack
trace information), I've also improved error reporting in both of these tasks.
* [CI] Ensure we have dotnet 3.x to be able to run ESRP.
* do runtime instead
* [CI] Force dotnet installation path
* [CI] Move task after the build created the path dotnet is looking for
* [CI] Remove extra /
* [CI] Try getting 5.x and 3x installed
* Avoid blank space
Co-authored-by: Alex Soto <alex@alexsoto.me>
CHIP framework seems to not be stable yet from Apple's side
each xcode update it brings breaking changes and it is also
not documented anywhere so let's disable it for now and
we can re-enable it in the future once it is stable.
* Better text.
* Better number ("7000" is more like "this is kind of normal" - while "6999"
was more like "something quite expected happened").
* Use resources to make it localizable.
* Add to Mac Catalyst. Fixes#13931.
* Manually include CoreTelephony headers in xtro. There's no umbrella header
in CoreTelephony 😡😞
* Fix availability attributes
* Only CTCall and CTCallCenter are deprecated in the CoreTelephony API.
* None of these APIs are obsolete, just deprecated.
* Add Mac Catalyst attributes.
One curious fact is that the PCSC framework interferes with compiling CTCarrer.h:
In file included from /private/var/folders/43/h027tm1n101cdrq2_b6n9n2m0000gn/T/n0b0byrt.h:163:
/Applications/Xcode_13.2.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/System/Library/Frameworks/CoreTelephony.framework/Headers/CTCarrier.h:62:41: error: reference to 'BOOL' is ambiguous
@property (nonatomic, readonly, assign) BOOL allowsVOIP __OSX_AVAILABLE_STARTING(__MAC_NA,__IPHONE_4_0);
^
/Applications/Xcode_13.2.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/System/Library/Frameworks/PCSC.framework/Headers/wintypes.h:59:18: note: candidate found by name lookup is 'BOOL'
typedef int16_t BOOL;
^
/Applications/Xcode_13.2.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/usr/include/objc/./objc.h:78:18: note: candidate found by name lookup is 'BOOL'
typedef bool BOOL;
^
1 error generated.
but since we don't bind the PCSC framework, we can just ask ObjectiveSharpie
to exclude it.
Fixes https://github.com/xamarin/xamarin-macios/issues/13931.
* Remove ObjCRuntime.nfloat (in favor of System.Runtime.InteropServices.NFloat).
* Automatically add a reference to the System.Runtime.InteropServices.Internal
package, so that developers get the new NFloat API (with operators) we've
added post .NET 6 (but don't do this for .NET 7).
* Automatically add a global using alias for
System.Runtime.InteropServices.NFloat -> nfloat. This is not behind the
usual `ImplicitUsings` condition our other implicit usings are, because
they're off by default for existing projects, and the main target for the
global using alias for nfloat is upgraded projects.
* Automatically generate a global using alias (like above) in the generator
for all code the generator compiles.
* Update xtro entries to reference System.Runtime.InteropServices.NFloat
instead of ObjCRuntime.nfloat.
* Add a workaround for a hopefully temporary issue with .NET/CoreCLR where the
wrong runtime pack is selected otherwise (without the new NFloat API, so
nothing works at runtime).
Ref: https://github.com/xamarin/xamarin-macios/issues/13087
There was a large change to rename a lot of our Xamarin assemblies to Microsoft
ie) Xamarin.iOS -> Microsoft.iOS
There is a mismatch with some of the prerequisites in our tools/apidiff/Makefile where dependencies
are looking for ...Microsoft.iOS... but they are still named ...Xamarin.iOS...
This PR takes any remaining "Xamarin" names and changes them to "Microsoft" for all dotnet related rules.
We will also change other dotnet rules to use the new naming convention of "macOS" and "tvOS"
The only exception is to the Xamarin.PLATFORM.dll's coming from the zip - those remain as Xamarin.iOS.dll
We should expect to see the gists showing up in ApiDiffs from this PR!
Co-authored-by: TJ Lambert <tjlambert@microsoft.com>
Rename our product assemblies to:
* Microsoft.iOS.dll
* Microsoft.tvOS.dll
* Microsoft.macOS.dll
* Microsoft.MacCatalyst.dll
This makes it easy to distinguish between legacy Xamarin and .NET whenever the
product assembly is mentioned, and I've also chosen the platform part of the
name to match how the platforms are named elsewhere (this also makes it
possible to simplify our build logic, since we can remove a lot of special
casing).
Fixes https://github.com/xamarin/xamarin-macios/issues/13748.
As a part of the breaking changes in .NET, we introduced a new type,
`ObjCRuntime.NativeHandle`, to represent native handles.
This meant that constructors taking taking `IntPtr handle`:
```cs
public class MyUIViewController : UIViewController {
protected MyUIViewController (IntPtr handle)
: base (handle)
{
}
}
```
would have to be ported to take `NativeHandle handle`:
```cs
public class MyUIViewController : UIViewController {
protected MyUIViewController (NativeHandle handle)
: base (handle)
{
}
}
```
The unfortunate part is that there will be no compiler warnings or errors
flagging this, so users won't know to do it unless they either read the
documentation (🤣) or run into the problem, googles for a while, runs into
someone else who had the same problem, and applies their (probably broken)
fix.
So we change our logic to:
1. Look for and use an `(IntPtr)` (or `(IntPtr, bool)`) constructor in .NET if
the `NativeHandle` version isn't found.
2. Show a warning.
3. Some time in the future maybe remove this hack/workaround.
Fixes https://github.com/xamarin/xamarin-macios/issues/14046.
* Propagate the IsAppExtension variable correctly.
* Don't try to call mono_domain_set_config for app extensions in .NET.
It doesn't look like it's needed, and it also immediately aborts anyway, so
if it turns out to be needed, another solution would have to be implemented.
Fixes https://github.com/xamarin/xamarin-macios/issues/13742.
Related work item: VS #1458467
Per Executive Order (EO) produce a Software Bill of Materials (SBOM) capturing the produced package files using a dedicated task
https://eng.ms/docs/cloud-ai-platform/devdiv/one-engineering-system-1es/1es-docs/secure-supply-chain/ado-sbom-generator
SBOM manifest generation only runs as part of a CI build since the steps are integrated with the Upload packages to Azure job
As a result of this change you will find an artifact named sbom attached to each build. Within that artifact is a manifest.json file under a _manifest directory capturing all of the files that constitute the Software Bill of Materials
The sbom steps capture the *.nupkg, *.pkg and *.msi package files published by the build
Test build having the Upload packages to Azure step enabled
https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=5720455&view=results
Thinks are working fine as-is (with this XAMCORE_4_0 variable set to false),
and I see no particular reason in the code to change it, nor does the original
implementation explain much (b2bcad7a94).
So just remove this XAMCORE_4_0 condition as if it had never existed.
When exception marshalling was originally implemented, for backwards
compatibility concerns it was only turned on by default for platforms that
really needed it (watchOS).
However, exception marshalling is by far the safest option, so in .NET we're
enabling it by default for all platforms (it's still possible to disable it
for those who wants to).
Ref: https://docs.microsoft.com/en-us/xamarin/ios/platform/exception-marshaling
* Make Runtime.Arch a readonly field.
* Tell the AOT compiler Runtime.Arch is a constant value.
* Tell the linker to stub out the method we use to fetch the current
architecture from native code (it turned out a bit complicated to set the
Arch field when it's readonly - the solution I came up with was to call a
P/Invoke).
Test case (size of the main executable): link all (debug)
* Before: 33.522.704 bytes
* After: 33.506.112 bytes
* Difference: -16.592 bytes (-0.05 %)
There were no size differences in release mode, nor were there any size
differences in the "don't link" test, neither for debug nor release mode.
Fixes https://github.com/xamarin/xamarin-macios/issues/5518.
Allow to skip it for several reasons:
1. We might want to seepd up a build during a release.
2. We want to make the build.yml template for pflexible, later we will
have to instances, one for the diff one to run tests and build.
Running those in parallel should buy us an hour in the total build.
Make the code more readable, this is the first step to clean the scripts
and to be able to add a switch so that we can diff between comments for
device tests and general comments.
* Change dotnet-linker to only care about whether we're actually trimming anything or not.
* Allow LinkMde/MtouchLink to not be set if TrimMode is set.
* Detect if any assemblies are linked or not by checking the global TrimMode
property + any TrimMode properties on assemblies.
Fixes https://github.com/xamarin/xamarin-macios/issues/13518.
This change allows to have a parameter (false by default) that allows to
get a build to be able to do an insertion even when it is comming from a
not predefined branch.
Uses cases:
1. Trigger a buiild with no tests from a special branch to insert.
2. Work with the CI to test the deployment.
Rolf Bjarne Kvinge
Manuel de la Pena
Steve Hawley
Bekir Ozturk
Alex Soto
TJ Lambert
Sebastien Pouliot
Mike Bond