The Xamarin.MacDev.Tasks.sln solution is built with dotnet, while other projects
are still built with msbuild. This becomes a problem when generating Errors.designer.cs,
because depending on the runtime the output is different.
This means that the Errors.designer.cs will sometimes randomly change (depending
on which project re-generated the file), leaving the file modified in git. This is
quite annoying, but it also breaks the api comparison, which depends on the build
not leaving modified files behind. So for now, we generate Errors.designer.cs separately
for Xamarin.MacDev.Tasks.sln to not conflict with the mtouch version.
Also fix capitalization in numerous places to be consistent (it's Errors.designer.cs,
not Errors.Designer.cs).
Fix crash report collection to not try to zip up the
~/Library/Logs/DiagnosticReports directory if there's nothing in it, because
it creates a warning in Azure DevOps:
+ zip -9rj /Users/builder/azdo/_work/2/s/crash-reports.zip /Users/builder/Library/Logs/DiagnosticReports
zip error: Nothing to do! (try: zip -9rj /Users/builder/azdo/_work/2/s/crash-reports.zip . -i /Users/builder/Library/Logs/DiagnosticReports)
##[error]Bash exited with code '12'.
Also remove some debug spew, and make it possible to create a crash report
collection locally by writing the zip file to the current directory if
SYSTEM_DEFAULTWORKINGDIRECTORY isn't set.
Remove support/logic for the following labels:
* run-internal-tests
* skip-external-tests
* trigger-device-tests
because they didn't actually do anything.
Also use underscores instead of dashes in output variables, because these
variables are also exported as environment variables, and environment variable
names can't contain dashes (bash can't access them).
Remove the argument to specify the GitHub comments file, because it's overwritten with a computed value (i.e. completely ignored), and thus confusing what's supposed to happen in the calling script.
Also compute the default earlier, so that it's set to the expected value by the time we have error reporting that will try to write to the comments file.
This way the api diff job can be re-run and not fail because the artifact
already exists:
> ##[error]Artifact 'DropMetadata-ChangeDetection' already exists. If the build is producing multiple Drops, the Drop Metadata Container Name should be unique for each Drop.
We can't process a common-fx.ignore file for a given framework if that
framework isn't included in any of the platforms we're building for.
Example: we can't process common-AppKit.ignore when only iOS is enabled,
because none of the errors listed in common-AppKit.ignore will be reported for
an iOS build.
This will hopefully make it easier to correctly subscribe to our maestro feeds
and only pick certain platforms.
It also fixes a problem where publishing wouldn't work unless we were building
for iOS, because the code was assuming that iOS was always enabled.
This PR adds support for Mac and MacCatalyst. Apple says that these
platforms are now supported; framework's headers does not specify them
but do not deny them either, so let's see what intro says about this.
I'm just creating this PR to test it against intro once we can build the
branch.
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
Also make the build go into the tools/devops directory, which will run
shellcheck on the bash files. This also required fixing a couple of issues in
one of the bash scripts.
We currently don't use yamllint, but that's coming very soon.
* Add net8.0, because that's coming soon.
* Add release-test/\*, because we want to run some automated tests on various
release configurations, and all the release/\* branches are branch-protected,
which means CI can't commit any such branch without going through a pull
request (which needs to be approved, etc.), and that's not very automated at
all. So add a branch pattern to the post-build pipeline that isn't
branch-protected.
This change splits the signing of the pkgs so that we can have botnet
pkgs as early as possible without needing to wait for the legacy ones to
be completed, this will allow to do VS insertions earlier.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
The SensorKit framework isn't available on all devices (for instance iPads),
and as such we can't link with it strongly.
This seems to be a bug in Apple's toolchain, because Xcode runs into the same
problem if you try to use an app referencing SensorKit on an iPad.
Fixes https://github.com/xamarin/xamarin-macios/issues/9938.
We have some problems when autoformatting PRs and the PR branch isn't fully
up-to-date with regards to the target branch.
I believe this is what happens:
1. When a PR is created (or modified), GitHub Actions will merge the PR branch
with the target branch, and parse/load the merged *.yml files.
2. Then when we run the autoformatter, we're working on the tip of the PR
branch (and not the merged result).
3. This means that we were using the list of projects to autoformat from the
merged branch, but exeuting on the PR branch. This resulted in spurious
autoformatting, because the autoformatted would autoformat more code than
expected.
The fix I'm implementing is to move the list of projects to autoformat to a
separate script in source code. That way we'll work upon the list of projects
as they show up in the PR branch, and not the merged results.
Now that it's possible to disable legacy xamarin tests, we must also
enable
them after using 'skip-all-tests' if we want to run legacy tests.
Otherwise we end up only executing .NET tests:
f592de721d (commitcomment-84832124)
Context: dotnet/runtime#68610
Context: https://github.com/xamarin/xamarin-android-tools/commit/0be567a9
In Mono and .NET prior to .NET 8, the
[`System.Environment.SpecialFolder`][0]`.Personal` enum value would refer to
`$HOME` on Unix platforms.
This will be changing in .NET 8, such that
`Environment.SpecialFolder.Personal` will instead refer to
`$XDG_DOCUMENTS_DIR` (if set) or `$HOME/Documents`. This is for "semantic
compatibility" with .NET on Windows.
Replace usage of `Environment.SpecialFolder.Personal` with
`Environment.SpecialFolder.UserProfile`, so that our code continues to work as
expected under .NET 8.
[0]: https://docs.microsoft.com/en-us/dotnet/api/system.environment.specialfolder?view=net-6.0
Also fix a merge conflict that made autoformatting not work, and make sure to not add any temporary files to the diff.
Co-authored-by: GitHub Actions Autoformatter <github-actions-autoformatter@xamarin.com>
Apple has deprecated bitcode, and will apparently reject app submissions
containing bitcode starting with Xcode 14. So automatically disable bitcode if
building using Xcode 14+ (and show a warning so that app developers can remove
the 'MtouchEnableBitcode' property from their project files).
Fixes https://github.com/xamarin/xamarin-macios/issues/15210.
Backport of #15804
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Change package signing so that we only try to sign packages for platforms that aren't disabled.
This makes the 'Prepare packages' job green in Azure Devops.
The 'Prepare Release' job still fails in the 'Convert NuGet to MSI' step (for
the branch I've been working on), because the package names are too long for
MSIs. This should be resolved for a proper release branch (i.e. when this PR
is merged), because package names are much shorter in a release branch.
Example results: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=6711641&view=results
Backport of #16028.
Don't try to compare legacy vs .NET for platforms that aren't included in the build, because this happens:
> make: *** No rule to make target 'output/diff/dotnet/legacy-diff/Microsoft.macOS.Ref/ref/net6.0/Microsoft.macOS.html', needed by 'output/api-diff.html'. Stop.
We do this by not hardcoding the list of legacy platforms, but instead starting with DOTNET_PLATFORMS variable (which won't contain platforms that aren't included in the build), and then removing any .NET-only platforms (i.e. Mac Catalyst).
Also fix the `update-refs` target to not try to update refs for platforms that aren't enabled.
Fixes https://github.com/xamarin/xamarin-macios/issues/16011.
Backport of #16029
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Add support for two new MSBuild item groups:
* CodesignBundle: lists additional app bundles inside the main bundle which should
be signed (typically manually copied into the app bundle by the developer).
* SkipCodesignItems: lists files we'd sign by default, but which shouldn't be signed.
Fixes https://github.com/xamarin/xamarin-macios/issues/15594.
Because we use mtouch and mmp to build the partial static registrar code for .NET.
Eventually we'll look into generating the partial static registrar some other
way, but that's for another time.
The 'Expand tests.' step sometimes fails with:
> ##[error]Bash exited with code '1'.
Which is very unhelpful. Make bash more verbose to see if we can figure out what's going wrong.
Otherwise this happens if the directory is empty:
+ test -d /Users/runner/Library/Logs/DiagnosticReports
+ zip -9rj /Users/runner/work/1/s/crash-reports.zip /Users/runner/Library/Logs/DiagnosticReports
zip error: Nothing to do! (try: zip -9rj /Users/runner/work/1/s/crash-reports.zip . -i /Users/runner/Library/Logs/DiagnosticReports)
Don't try to publish test results unless there are any tests results.
Fixes this [horribly/amusingly incorrect error][1] in the publish task:
##[error]Error: Failed find: ENOENT: no such file or directory, lstat '/System/Library/Frameworks/iTunesLibrary.framework/Versions/Versions'
##[section]Finishing: Publish NUnit Device Test Results
Also stop failing the task on failing tests, because we already have another task that fail if there are failing tests (the task that runs the tests).
[1]: https://github.com/microsoft/azure-pipelines-tasks/issues/16786
* Move the bash in the yml file to a separate script file to ease reading, writing & debugging.
* Don't install any symlinks if legacy Xamarin isn't enabled.
* Only install the iOS / macOS symlink if the corresponding build is enabled.
Only install the XI and/or XM package if the corresponding part of the build is enabled.
Also don't install either if the legacy Xamarin build is disabled.
This PR has the AVKit updates and introduces the AVRouting bindings that
are interconnected with AVKit
Co-authored-by: TJ Lambert <tjlambert@microsoft.com>
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
Co-authored-by: tj_devel709 <antlambe@microsoft.com>
* Special characters in powershell are rather, hrm, _uncommon_, in that
they're prefixed with a backtick instead of backslash. Fix code accordingly.
* Use 'Write-Debug' instead of 'Write-Host' in a few places.
* Simplified/improved a few debug statements to make them clearer/less redundant.
* Added tabs in a few places to make debug statements indent properly.
* Fixed a typo.
This will increase app size a little bit: the space for the MVID + 4 bytes for each
assembly, but we'll be able to validate and show a helpful error message if the generated
static registrar code does not match the assembly loaded at runtime.
It's also a step toward per-assembly static registration (ref: #12067).
This makes it possible to re-run tests when they fail (since Azure DevOps only
allows re-running failed jobs).
It shouldn't affect any release pipelines anymore, because the release
pipeline only depends on the job that builds the packages now.
This also involved some CI changes, to be able to figure out the last test results when a test step is executed multiple times. Also, the GitHub comment will now state the run attempt (if >1) for each test ([example](https://github.com/xamarin/xamarin-macios/pull/15764#issuecomment-1235891944))
This makes it easier to both read & write bash code (syntax highlighting in
the script file, shellscript to validate, etc.), as well as testing out the
script locally.
This avoids one case where we we embed metadata tokens to a different assembly
in the generated static registrar code.
This is required for supporting per-assembly static registration
(https://github.com/xamarin/xamarin-macios/issues/12067).
Make sure bash doesn't ignore any errors during signing. This makes it easier
to diagnose signing failures, because they don't show up in weird ways later.
Context: https://github.com/xamarin/yaml-templates/pull/180
Context: https://github.com/xamarin/yaml-templates/pull/195
Context: https://github.com/xamarin/yaml-templates/pull/199
Context: https://github.com/xamarin/xamarin-macios/pull/15761
Updates the build to use the latest MSI generation template. The v3
template uses the latest changes from arcade which include a large
refactoring, support for multi-targeting, and support for workload pack
group MSIs.
The build will now produce two different VS Drop artifacts. The MSI and
VSMAN files generated for SDK packs have been split out into a new
`vsdrop-multitarget-signed` artifact, allowing us to include multiple
versions of the SDK packs in VS.
All of the SDK packs have been renamed to include a `.net6` suffix to
match the pack aliases that will be referenced in the .NET 7 manifests.
Backport of #15776
Co-authored-by: Peter Collins <pecolli@microsoft.com>
Make the binlog artifact name unique across build attempts, so that uploading the binlog archive doesn't fail in subsequent build attempts:
> ##[error]Artifact all-binlogs-test-simulator_cecil-6594281 already exists for build 6594281.
This involved:
* Make the mono-api-info and mono-api-diff tools run with .NET 7 (instead of
requiring .NET 6).
* Make the code cope with the fact that we're comparing .NET 6 assemblies (in
a net6.0 directory) with .NET 7 assemblies (in a net7.0 directory).
Co-authored-by: TJ Lambert <50846373+tj-devel709@users.noreply.github.com>
Hopefully works around this problem:
[...]
[08:14:16 VRB] Preloading sudo access since brew installation cannot be run as root
[08:14:16 VRB] Exec[0] (flags: RedirectStdout, RedirectStderr, Default): /usr/bin/sudo -v
[08:14:16 DBG] Adding main (originally refs/heads/main) to telemetry
[08:14:16 DBG] Adding main (originally refs/heads/main) to telemetry
[08:14:16 VRB] Exec[0] exited 1
Unhandled exception. Xamarin.Provisioning.Exec+ExitException: /usr/bin/sudo terminated with exit code 1
at Xamarin.Provisioning.Exec.Run(ExecFlags flags, String command, String[] arguments) in /Users/runner/work/1/s/Provisionator/Exec.cs:line 297
at Xamarin.Provisioning.ProvisioningScript.BrewPackages(BrewOptions options, String[] packages) in /Users/runner/work/1/s/Provisionator/ProvisioningScript_Brew.cs:line 104
at Xamarin.Provisioning.ProvisioningScript.BrewPackages(String[] packages) in /Users/runner/work/1/s/Provisionator/ProvisioningScript_Brew.cs:line 23
at Submission#0.<<Initialize>>d__0.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.CodeAnalysis.Scripting.ScriptExecutionState.RunSubmissionsAsync[TResult](ImmutableArray`1 precedingExecutors, Func`2 currentExecutor, StrongBox`1 exceptionHolderOpt, Func`2 catchExceptionOpt, CancellationToken cancellationToken)
at Microsoft.CodeAnalysis.Scripting.Script`1.RunSubmissionsAsync(ScriptExecutionState executionState, ImmutableArray`1 precedingExecutors, Func`2 currentExecutor, Func`2 catchExceptionOpt, CancellationToken cancellationToken)
at Xamarin.Provisioning.ProvisioningScript.RunScriptAsync(String scriptContents, String scriptFile, CancellationToken cancellationToken) in /Users/runner/work/1/s/Provisionator/ProvisioningScript.cs:line 118
at Xamarin.Provisioning.Entry.MainAsync(String[] args) in /Users/runner/work/1/s/Provisionator/Entry.cs:line 256
at Xamarin.Provisioning.Entry.MainAsync(String[] args) in /Users/runner/work/1/s/Provisionator/Entry.cs:line 339
at Xamarin.Provisioning.Entry.Main(String[] args) in /Users/runner/work/1/s/Provisionator/Entry.cs:line 60
##[error]The process '/Users/builder/azdo/_work/_tool/provisionator/0.2.635/x64/provisionator' failed with exit code null
Fix an issue when doing API comparison for PRs with multiple commits, where we'd only do the comparison for the last commit, by not ignoring the provided base hash in the compare-commits script.
* [xcode14] Bump to latest Xcode 14 Beta 5
* [CloudKit] Fix cloudkit intro on tvOS
* Revert "[Tests] Fix an monotouch-test test that landed broken. (#15503)"
This reverts commit 161de84bcf.
* Update tools/common/StaticRegistrar.cs
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
* Fix typo
* [CHIP] Tell our drivers to not link CHIP at all
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
* [mlaunch] Fix permisisons after extracting from nuget
- A side effect of ac1fa25816 is that the permission of bin/mlaunch is no longer +x for non-root, which means it is unusable.
* Apply suggestions from code review
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
* Apply fix to app bundle mlaunch as well
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
For some reason the C# compilers crash a lot during the build in src/ when
building for the API diff (but not the normal build!). So test the theory that
we're overloading the bot in question (OOM maybe?) by slowing down a bit.
I have to say that if this works and the theory is proven, it's kind of sad
that after over a decade doing -j8 the bot situation has gotten worse...
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
For some reason the C# compilers crash a lot during the build in src/ when
building for the API diff (but not the normal build!). So test the theory that
we're overloading the bot in question (OOM maybe?) by slowing down a bit.
I have to say that if this works and the theory is proven, it's kind of sad
that after over a decade doing -j8 the bot situation has gotten worse...
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
* Update dependencies from https://github.com/dotnet/installer build 20220707.5
Microsoft.Dotnet.Sdk.Internal
From Version 7.0.100-preview.6.22306.4 -> To Version 7.0.100-preview.7.22357.5
Dependency coherency updates
Microsoft.NET.ILLink.Tasks,Microsoft.NETCore.App.Ref,Microsoft.AspNetCore.App.Ref
From Version 7.0.100-1.22273.1 -> To Version 7.0.100-1.22354.1 (parent: Microsoft.Dotnet.Sdk.Internal
* Update global.json
* Update dependencies from https://github.com/dotnet/installer build 20220708.13
Microsoft.Dotnet.Sdk.Internal
From Version 7.0.100-preview.6.22306.4 -> To Version 7.0.100-preview.7.22358.13
Dependency coherency updates
Microsoft.NET.ILLink.Tasks,Microsoft.NETCore.App.Ref,Microsoft.AspNetCore.App.Ref
From Version 7.0.100-1.22273.1 -> To Version 7.0.100-1.22354.1 (parent: Microsoft.Dotnet.Sdk.Internal
* Bump global.json
* Update dependencies from https://github.com/dotnet/installer build 20220711.1
Microsoft.Dotnet.Sdk.Internal
From Version 7.0.100-preview.6.22306.4 -> To Version 7.0.100-preview.7.22361.1
Dependency coherency updates
Microsoft.NET.ILLink.Tasks,Microsoft.NETCore.App.Ref,Microsoft.AspNetCore.App.Ref
From Version 7.0.100-1.22273.1 -> To Version 7.0.100-1.22354.1 (parent: Microsoft.Dotnet.Sdk.Internal
* Bump global.json
* Update dependencies from https://github.com/dotnet/installer build 20220712.1
Microsoft.Dotnet.Sdk.Internal
From Version 7.0.100-preview.6.22306.4 -> To Version 7.0.100-preview.7.22362.1
Dependency coherency updates
Microsoft.NET.ILLink.Tasks,Microsoft.NETCore.App.Ref,Microsoft.AspNetCore.App.Ref
From Version 7.0.100-1.22273.1 -> To Version 7.0.100-1.22354.1 (parent: Microsoft.Dotnet.Sdk.Internal
* Re-generate global.json
* [registrar] Improve naming of generated P/Invoke wrappers.
C# method names generated by the C# compiler can have all sorts of invalid
characters, so just sanitize this a bit.
* [tests] Adjust InvalidRuntimeIdentifier_Restore to expect failure for Mac Catalyst.
* Update dependencies from https://github.com/dotnet/installer build 20220713.4
Microsoft.Dotnet.Sdk.Internal
From Version 7.0.100-preview.6.22306.4 -> To Version 7.0.100-rc.1.22363.4
Dependency coherency updates
Microsoft.NET.ILLink.Tasks,Microsoft.NETCore.App.Ref,Microsoft.AspNetCore.App.Ref
From Version 7.0.100-1.22273.1 -> To Version 7.0.100-1.22362.1 (parent: Microsoft.Dotnet.Sdk.Internal
* Re-generate global.json
Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Co-authored-by: GitHub Actions <github-actions@xamarin.com>
We can't use the global.json located in the root of our repo, because makes it
required to use the exact .NET version we're referencing in our
eng/Versions.Details.xml file. So in order to not use it, we set the working
directory to the parent directory of xamarin-macios.
Otherwise this happens:
Could not execute because the application was not found or a compatible .NET SDK is not installed.
Possible reasons for this include:
* You intended to execute a .NET program:
The application 'build' does not exist.
* You intended to execute a .NET SDK command:
A compatible installed .NET SDK for global.json version [6.0.301-rtm.22280.1] from [D:\a\1\s\xamarin-macios\global.json] was not found.
6.0.201 [C:\hostedtoolcache\windows\dotnet\sdk]
Install the [6.0.301-rtm.22280.1] .NET SDK or update [D:\a\1\s\xamarin-macios\global.json] with an installed .NET SDK:
& : The term 'C:\hostedtoolcache\windows\darc\darc' is not recognized as the name of a cmdlet, function, script file,
or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and
try again.
We can't use the global.json located in the root of our repo, because makes it
required to use the exact .NET version we're referencing in our
eng/Versions.Details.xml file. So in order to not use it, we set the working
directory to the parent directory of xamarin-macios.
Otherwise this happens:
Could not execute because the application was not found or a compatible .NET SDK is not installed.
Possible reasons for this include:
* You intended to execute a .NET program:
The application 'build' does not exist.
* You intended to execute a .NET SDK command:
A compatible installed .NET SDK for global.json version [6.0.301-rtm.22280.1] from [D:\a\1\s\xamarin-macios\global.json] was not found.
6.0.201 [C:\hostedtoolcache\windows\dotnet\sdk]
Install the [6.0.301-rtm.22280.1] .NET SDK or update [D:\a\1\s\xamarin-macios\global.json] with an installed .NET SDK:
Download mlaunch from NuGet instead of building from maccore, and copy the
downloaded files into the packages we ship (both legacy Xamarin's pkg and .NET
nupkgs).
Eventually we'll want to reference the mlaunch NuGet from the .NET nupkgs, but
that's a later step.
Rolf Bjarne Kvinge
VS MobileTools Engineering Service 2
Israel Soto
Manuel de la Pena
Alex Soto
TJ Lambert
Jamie Magee
Peter Collins
Steve Hawley
Chris Hamons
dotnet-maestro[bot]