Try hard to always add a GitHub comment after (trying to) running macOS tests on other bots.
We now:
* Create a file with a generic error message at the very beginning.
* Update the error message if something we detect goes wrong (with a better error message).
* Delete the file in case of success.
* Check for the file at the very end, reporting any contents as a failure, and only if it doesn't exist, then we report success.
This way we'll pretty much always report _something_.
This is what Azure Devops defines:
* Build.SourceBranchName: the last path component of the branch.
* Build.SourceBranch: the complete ref spec for the branch.
We want the branch name without the 'refs/heads/' part. Unfortunately we can't
use Build.SourceBranchName, because for a branch name like
'release/6.0.3xx-rc1' the last path component is '6.0.3xx-rc1', not
'release/6.0.3xx-rc' - in other words it doesn't have all the information we
need.
That means we need to use the Build.SourceBranch value instead, and remove the
'refs/heads/' part. We already compute BRANCH_NAME like this everywhere else,
so just copy that implementation.
Backport of #14568
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
* Remove existing attributes
* Attribute Conversion
* let's add some options and put in feedback
* don't need this check anymore
* Don't copy paste code without reading, Steve
* fallout from no more nulls
* catch exception from options parsing, check for file existence, fix white space
* whitespace
* add --force-overwrite
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Some tests will build xibuild automatically when needed, some won't. This
means that the ones that don't won't succeed if executed before the ones that
do.
To avoid this scenario, just manually build xibuild before running the tests.
The internal bots are giving issues running xtro, public ones are ok.
There is no real reason to use the internal pool to run tests and we can
debug the bots in that pool better.
We move to run tests in the public bots unless told otherwise (when
using devices for example).
This reverts commit 073165fef6.
Fixes this build error:
/Library/Frameworks/Mono.framework/Versions/6.12.0/lib/mono/msbuild/Current/bin/Microsoft.Common.CurrentVersion.targets(1232,5): error MSB3971: The reference assemblies for ".NETFramework,Version=v5.0" were not found. You might be using an older .NET SDK to target .NET 5.0 or higher. Update Visual Studio and/or your .NET SDK. [/Users/rolf/work/maccore/dotnet/xamarin-macios/tools/nnyeah/nnyeah/nnyeah.csproj]
gmake[2]: *** [Makefile:9: bin/Debug/net5.0/nnyeah.dll] Error 1
It can happen that we use the same agent for more than one bot, if that
is the case, we need to make sure that we do not have the same name for
the logs. We use the job id + job name for that.
Adding the needed Makefile and the dir to the parent one so that the
tool is also built in the CI.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
The problem is:
1. On Mac Catalyst we AOT-compile managed code by default (since we can't JIT).
2. We recently added a binding for the `ASAuthorizationAllSupportedPublicKeyCredentialDescriptorTransports` native function, which is only available on Mac Catalyst 15+. The containing framework (`AuthenticationServices`) is available in Mac Catalyst 13+ (which means we won't link weakly with the framework).
3. The AOT compiler emits a non-weak reference to the `ASAuthorizationAllSupportedPublicKeyCredentialDescriptorTransports` symbol.
4. macOS verifies that there aren't any undefined non-weak symbol references at launch.
5. The `ASAuthorizationAllSupportedPublicKeyCredentialDescriptorTransports` symbol doesn't exist on macOS 11, and thus the app crashes at launch on macOS 11.
I tried the same thing on iOS, and the problem doesn't occur because iOS doesn't to step 4 - the symbol reference are resolved at first use, which means that everything works just fine until the first time the P/Invoke is called (and then the app aborts).
The workaround is to tell the AOT compiler to not emit a direct reference to the P/Invoke (and use dlsym instead) - this fixes case 3) above. Drawbacks: dlsym is slower than a direct reference.
Fixes https://github.com/xamarin/xamarin-macios/issues/14437.
Also rename DOTNET_VERSION to SYSTEM_DOTNET_VERSION to make it clear what it's
referring to (and to not clash with DOTNET6_VERSION which has now been renamed
to DOTNET_VERSION).
.NET 7 is right around the corner.
The main theme here is that code signing will be done in the outermost
executable project, not in any app extension projects or watch projects, nor
during the RID-specific build of a .NET universal app. This makes codesigning
easier to reason about and other affected logic (such as strip/dsymutil)
easier to handle, in particular for .NET universal apps. Another benefit is
that the differences between the iOS and macOS code bases have been
eliminated.
The first step is to collect all the information we need from the targets
files. Every app bundle (be it app extension, watch app or main app) will add
its own output app bundle (.app/.appex) to the _CodesignBundle item group.
Then every app bundle will load this informarion from referenced app bundles,
and finally store this information on disk (in the 'codesign-bundle.items'
file). This means that in the end the main app bundle will have a list of all
contained app bundles in the app (recursively), in the _CodesignBundle item
group.
Separately we keep a list of other items that need signing, in the
_CodesignItems item group, and we do the same store/load logic for every
contained/contained app bundle (in the 'codesign.items' file, so a the end the
main app bundle will have a list of all the _CodesignItems for all contained
app bundles (recursively).
The previous steps occur in the _CollectCodesigningData and
_StoreCodesigningData targets.
The next step is to use the new ComputeCodesignItems task to compute
everything we need to know for code signing. This task takes over the
responsibility for listing all the *.dylib and *.metallib files, and the
*.framework directories in the app bundles, that need signing (which was
previously done in the targets file). This logic is significantly easier to
write, debug and test in C# than MSBuild.
In addition the ComputeCodesignItems also figures out a stamp file path we use
to determine if something needs (re-)signing. Previously .framework
directories did not have a stamp location, so they'd always end up resigned in
a rebuild, while now we'll automatically skip signing *.framework directories
unless something changed in them.
I've also tried to comment everything thorougly, for the next poor soul having
to deal with any bugs.
Behavioral differences:
* We were always signing *.dylib files for macOS. We're now doing the same
thing for all platforms.
* We're now always signing *.framework directories for all platforms (like we
do for *.dylib files), since frameworks are pretty much like dylibs anyways.
I've verified that this works both by running the submission tests and running
and launching a sample project on device from Windows.
* Remove existing attributes
* Attribute Conversion
* First cut of code for review.
* Clean up csproj
* nullable enable and cleanup nullability. Compiles with no errors or warnings.
* typo
* Remove nullables with cleaner code, reworked things with Try...pattern.
* last couple '== null' changes
The main theme here is that code signing will be done in the outermost executable
project, not in any app extension projects or watch projects, nor during the RID-specific
build of a .NET universal app. This makes codesigning easier to reason about and
other affected logic (such as strip/dsymutil) easier to handle, in particular for
.NET universal apps. Another benefit is that the differences between the iOS and
macOS code bases have been eliminated.
The first step is to collect all the information we need from the targets files.
Every app bundle (be it app extension, watch app or main app) will add its own output
app bundle (.app/.appex) to the _CodesignBundle item group. Then every app bundle
will load this informarion from referenced app bundles, and finally store this information
on disk (in the 'codesign-bundle.items' file). This means that in the end the main
app bundle will have a list of all contained app bundles in the app (recursively),
in the _CodesignBundle item group.
Separately we keep a list of other items that need signing, in the _CodesignItems
item group, and we do the same store/load logic for every contained/contained app
bundle (in the 'codesign.items' file, so a the end the main app bundle will have
a list of all the _CodesignItems for all contained app bundles (recursively).
The previous steps occur in the _CollectCodesigningData and _StoreCodesigningData
targets.
The next step is to use the new ComputeCodesignItems task to compute everything we
need to know for code signing. This task takes over the responsibility for listing
all the *.dylib and *.metallib files, and the *.framework directories in the app
bundles, that need signing (which was previously done in the targets file). This
logic is significantly easier to write, debug and test in C# than MSBuild.
In addition the ComputeCodesignItems also figures out a stamp file path we use to
determine if something needs (re-)signing. Previously .framework directories did
not have a stamp location, so they'd always end up resigned in a rebuild, while now
we'll automatically skip signing *.framework directories unless something changed
in them.
I've also tried to comment everything thorougly, for the next poor soul having to
deal with any bugs, as well has adding a comprehensive test for the new task.
Behavioral differences:
* We were always signing *.dylib files for macOS. We're now doing the same thing
for all platforms.
* We're now always signing *.framework directories for all platforms (like we do
for *.dylib files), since frameworks are pretty much like dylibs anyways.
Or make files canculate the version of the nuget based on the hash of
the source, because VSTS does a merge of the branhc rather than checking
out, the version calculated in the tests bot via de makefiles is
different to the one that was used in the build bots.
We solve this by creating the rollback file at build time so that it is
consumed by the test bot.
The static registrar usually stores a compressed version of metadata tokens in
the generated code. However, when there are many assemblies in the app (>127),
we can't use the compressed version anymore, and fall back to a full version.
In this case, we weren't comparing type metadata tokens correctly when looking
for a type in our table of types, and thus we weren't finding the type we were
looking for.
The result is an exception like this:
> Can’t register the class MyClass when the dynamic registrar has been linked away.
In the generated table of types we're storing the full metadata token, which
includes a few bits indicating which type of token it is (in this particular
case a TypeDef token). When going through the table looking for a type, we
need to compare with those few bits set on the input type token as well to
find what we're looking for.
Also make it possible to use the remove-dynamic-registrar optimization on
macOS (which is useful by itself, but it also makes adding a test case
easier).
Fixes https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1476585.
Fixes https://github.com/xamarin/xamarin-macios/issues/11641.
If we have an issue unlocking the UI prompt we continue on error. This
is considered a warning in the pipeline, but doing so teaches users to
ignore warnings and will result in us ignoring real actual issues.
Setting the step to not have a warning will ensure that we only see
warnings that we care about.
This fixes an issue where we'd do logic with Windows-style paths on macOS, and that's
never the right thing to do.
For the LinkNativeCode task, this would manifest as this error when building from windows:
> ld: file too small (length=0) file 'obj/Debug/net6.0-ios/iossimulator-x64/nativelibraries/libSystem.Native.dylib' for architecture x86_64
because the 'ShouldCopyToBuildServer' method would return incorrect results.
For the Codesign task, it would manifest as an exception trying to create a
directory with an empty string (because the directory name of a windows-style
path is an empty string on macOS).
Since this exception was quite useless (just getting the exception message
didn't tell me much about what caused the exception, because it had no stack
trace information), I've also improved error reporting in both of these tasks.
Rolf Bjarne Kvinge
Steve Hawley
VS MobileTools Engineering Service 2
Manuel de la Pena
Bekir Ozturk