The 'Expand tests.' step sometimes fails with:
> ##[error]Bash exited with code '1'.
Which is very unhelpful. Make bash more verbose to see if we can figure out what's going wrong.
Otherwise this happens if the directory is empty:
+ test -d /Users/runner/Library/Logs/DiagnosticReports
+ zip -9rj /Users/runner/work/1/s/crash-reports.zip /Users/runner/Library/Logs/DiagnosticReports
zip error: Nothing to do! (try: zip -9rj /Users/runner/work/1/s/crash-reports.zip . -i /Users/runner/Library/Logs/DiagnosticReports)
Don't try to publish test results unless there are any tests results.
Fixes this [horribly/amusingly incorrect error][1] in the publish task:
##[error]Error: Failed find: ENOENT: no such file or directory, lstat '/System/Library/Frameworks/iTunesLibrary.framework/Versions/Versions'
##[section]Finishing: Publish NUnit Device Test Results
Also stop failing the task on failing tests, because we already have another task that fail if there are failing tests (the task that runs the tests).
[1]: https://github.com/microsoft/azure-pipelines-tasks/issues/16786
* Move the bash in the yml file to a separate script file to ease reading, writing & debugging.
* Don't install any symlinks if legacy Xamarin isn't enabled.
* Only install the iOS / macOS symlink if the corresponding build is enabled.
* Special characters in powershell are rather, hrm, _uncommon_, in that
they're prefixed with a backtick instead of backslash. Fix code accordingly.
* Use 'Write-Debug' instead of 'Write-Host' in a few places.
* Simplified/improved a few debug statements to make them clearer/less redundant.
* Added tabs in a few places to make debug statements indent properly.
* Fixed a typo.
This makes it possible to re-run tests when they fail (since Azure DevOps only
allows re-running failed jobs).
It shouldn't affect any release pipelines anymore, because the release
pipeline only depends on the job that builds the packages now.
This also involved some CI changes, to be able to figure out the last test results when a test step is executed multiple times. Also, the GitHub comment will now state the run attempt (if >1) for each test ([example](https://github.com/xamarin/xamarin-macios/pull/15764#issuecomment-1235891944))
This makes it easier to both read & write bash code (syntax highlighting in
the script file, shellscript to validate, etc.), as well as testing out the
script locally.
Make sure bash doesn't ignore any errors during signing. This makes it easier
to diagnose signing failures, because they don't show up in weird ways later.
Context: https://github.com/xamarin/yaml-templates/pull/180
Context: https://github.com/xamarin/yaml-templates/pull/195
Context: https://github.com/xamarin/yaml-templates/pull/199
Context: https://github.com/xamarin/xamarin-macios/pull/15761
Updates the build to use the latest MSI generation template. The v3
template uses the latest changes from arcade which include a large
refactoring, support for multi-targeting, and support for workload pack
group MSIs.
The build will now produce two different VS Drop artifacts. The MSI and
VSMAN files generated for SDK packs have been split out into a new
`vsdrop-multitarget-signed` artifact, allowing us to include multiple
versions of the SDK packs in VS.
All of the SDK packs have been renamed to include a `.net6` suffix to
match the pack aliases that will be referenced in the .NET 7 manifests.
Backport of #15776
Co-authored-by: Peter Collins <pecolli@microsoft.com>
Make the binlog artifact name unique across build attempts, so that uploading the binlog archive doesn't fail in subsequent build attempts:
> ##[error]Artifact all-binlogs-test-simulator_cecil-6594281 already exists for build 6594281.
Hopefully works around this problem:
[...]
[08:14:16 VRB] Preloading sudo access since brew installation cannot be run as root
[08:14:16 VRB] Exec[0] (flags: RedirectStdout, RedirectStderr, Default): /usr/bin/sudo -v
[08:14:16 DBG] Adding main (originally refs/heads/main) to telemetry
[08:14:16 DBG] Adding main (originally refs/heads/main) to telemetry
[08:14:16 VRB] Exec[0] exited 1
Unhandled exception. Xamarin.Provisioning.Exec+ExitException: /usr/bin/sudo terminated with exit code 1
at Xamarin.Provisioning.Exec.Run(ExecFlags flags, String command, String[] arguments) in /Users/runner/work/1/s/Provisionator/Exec.cs:line 297
at Xamarin.Provisioning.ProvisioningScript.BrewPackages(BrewOptions options, String[] packages) in /Users/runner/work/1/s/Provisionator/ProvisioningScript_Brew.cs:line 104
at Xamarin.Provisioning.ProvisioningScript.BrewPackages(String[] packages) in /Users/runner/work/1/s/Provisionator/ProvisioningScript_Brew.cs:line 23
at Submission#0.<<Initialize>>d__0.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.CodeAnalysis.Scripting.ScriptExecutionState.RunSubmissionsAsync[TResult](ImmutableArray`1 precedingExecutors, Func`2 currentExecutor, StrongBox`1 exceptionHolderOpt, Func`2 catchExceptionOpt, CancellationToken cancellationToken)
at Microsoft.CodeAnalysis.Scripting.Script`1.RunSubmissionsAsync(ScriptExecutionState executionState, ImmutableArray`1 precedingExecutors, Func`2 currentExecutor, Func`2 catchExceptionOpt, CancellationToken cancellationToken)
at Xamarin.Provisioning.ProvisioningScript.RunScriptAsync(String scriptContents, String scriptFile, CancellationToken cancellationToken) in /Users/runner/work/1/s/Provisionator/ProvisioningScript.cs:line 118
at Xamarin.Provisioning.Entry.MainAsync(String[] args) in /Users/runner/work/1/s/Provisionator/Entry.cs:line 256
at Xamarin.Provisioning.Entry.MainAsync(String[] args) in /Users/runner/work/1/s/Provisionator/Entry.cs:line 339
at Xamarin.Provisioning.Entry.Main(String[] args) in /Users/runner/work/1/s/Provisionator/Entry.cs:line 60
##[error]The process '/Users/builder/azdo/_work/_tool/provisionator/0.2.635/x64/provisionator' failed with exit code null
For some reason the C# compilers crash a lot during the build in src/ when
building for the API diff (but not the normal build!). So test the theory that
we're overloading the bot in question (OOM maybe?) by slowing down a bit.
I have to say that if this works and the theory is proven, it's kind of sad
that after over a decade doing -j8 the bot situation has gotten worse...
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
For some reason the C# compilers crash a lot during the build in src/ when
building for the API diff (but not the normal build!). So test the theory that
we're overloading the bot in question (OOM maybe?) by slowing down a bit.
I have to say that if this works and the theory is proven, it's kind of sad
that after over a decade doing -j8 the bot situation has gotten worse...
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
We can't use the global.json located in the root of our repo, because makes it
required to use the exact .NET version we're referencing in our
eng/Versions.Details.xml file. So in order to not use it, we set the working
directory to the parent directory of xamarin-macios.
Otherwise this happens:
Could not execute because the application was not found or a compatible .NET SDK is not installed.
Possible reasons for this include:
* You intended to execute a .NET program:
The application 'build' does not exist.
* You intended to execute a .NET SDK command:
A compatible installed .NET SDK for global.json version [6.0.301-rtm.22280.1] from [D:\a\1\s\xamarin-macios\global.json] was not found.
6.0.201 [C:\hostedtoolcache\windows\dotnet\sdk]
Install the [6.0.301-rtm.22280.1] .NET SDK or update [D:\a\1\s\xamarin-macios\global.json] with an installed .NET SDK:
& : The term 'C:\hostedtoolcache\windows\darc\darc' is not recognized as the name of a cmdlet, function, script file,
or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and
try again.
We can't use the global.json located in the root of our repo, because makes it
required to use the exact .NET version we're referencing in our
eng/Versions.Details.xml file. So in order to not use it, we set the working
directory to the parent directory of xamarin-macios.
Otherwise this happens:
Could not execute because the application was not found or a compatible .NET SDK is not installed.
Possible reasons for this include:
* You intended to execute a .NET program:
The application 'build' does not exist.
* You intended to execute a .NET SDK command:
A compatible installed .NET SDK for global.json version [6.0.301-rtm.22280.1] from [D:\a\1\s\xamarin-macios\global.json] was not found.
6.0.201 [C:\hostedtoolcache\windows\dotnet\sdk]
Install the [6.0.301-rtm.22280.1] .NET SDK or update [D:\a\1\s\xamarin-macios\global.json] with an installed .NET SDK:
Download mlaunch from NuGet instead of building from maccore, and copy the
downloaded files into the packages we ship (both legacy Xamarin's pkg and .NET
nupkgs).
Eventually we'll want to reference the mlaunch NuGet from the .NET nupkgs, but
that's a later step.
* [devops] Enable verbose build.
* Allow to set the build in verbose when the pipeline is in debug mode.
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
Xcode14 broke actool. We have created two diff bot images:
- Stable: Contains the latests stable xcode (xcode13.4)
- Beta: Contains the latests beta xcode OR will allow it to be
installed.
By doing this we mitigate the possible issues that the new Xcode might
create. Some of the bots have already been migrated.
This commit SHOULD NOT be backported to xcode14.
When the label is not present we set the value to false. We also needed
to make sure that the project is checked out when we try to write the
comment.
This should be backported to xcode14.
Allow to pass the xcode channel we are going to be using. This is to
ensure that we do not try to build the xcode14 branch in beta bots. At
the moment all bots are using the Beta channel, but once the lab has
been updated we should be able to move main to Stable.
Get the bot information to improve the debugging of build failures. The
following is an example of the output:
Software:
System Software Overview:
System Version: macOS 12.4 (21F79)
Kernel Version: Darwin 21.5.0
Boot Volume: Macintosh HD
Boot Mode: Normal
Computer Name: Mandels Home iMac Pro
User Name: Manuel de la Pena Saenz (mandel)
Secure Virtual Memory: Enabled
System Integrity Protection: Enabled
Time since boot: 3 days 23:43
Hardware:
Hardware Overview:
Model Name: iMac Pro
Model Identifier: iMacPro1,1
Processor Name: 8-Core Intel Xeon W
Processor Speed: 3,2 GHz
Number of Processors: 1
Total Number of Cores: 8
L2 Cache (per Core): 1 MB
L3 Cache: 11 MB
Hyper-Threading Technology: Enabled
Memory: 64 GB
System Firmware Version: 1731.120.10.0.0 (iBridge: 19.16.15071.0.0,0)
OS Loader Version: 540.120.3~6
Serial Number (system): C02WD0V7HX8F
Hardware UUID: 85EE3276-4E8F-592A-A47B-599DFAB6DF1C
Provisioning UDID: 85EE3276-4E8F-592A-A47B-599DFAB6DF1C
Activation Lock Status: Disabled
Developer:
Developer Tools:
Version: 13.3.1 (13E500a)
Location: /Applications/Xcode_13.3.0.app
Applications:
Xcode: 13.3.1 (20103)
Instruments: 13.3.1 (64552.71)
SDKs:
DriverKit:
21,4:
iOS:
15,4: (19E239)
iOS Simulator:
15,4: (19E239)
macOS:
12,3: (21E226)
tvOS:
15,4: (19L439)
tvOS Simulator:
15,4: (19L439)
watchOS:
8,5: (19T241)
watchOS Simulator:
8,5: (19T241)
Other teams in xamarin/microsoft do no use forks but use dev branhces.
The correct thing in our repo is to use forks, yet other developers
insist in not following our developement practices. The fact that this
branches are created results in 2 builds:
- One for CI
- On for the PR
It is harder to educate other developers than it is to ignore their
branches, therefore we have added the pattern dev/* to the exclude
list for branches in the CI build.
There are certain bots misbehaving and taking longer. We believ ethat it
might be due to some throttiling depending on their positio in the lab.
We are increasing the timeout for those cases.
Fixed the following two issues:
1. Update tests to use the correct constructor.
2. Renamed $context because it is a known pester variable name and would
get overriden.
We are getting errors with the following:
> Error creating archive at '/Users/builder/azdo/_work/2/s/diagnostic-sim-output/output.tar.gz'.
> Files are still in /Users/builder/azdo/_work/2/s/diagnostic-sim-output/output
> An error was encountered processing the command (domain=NSPOSIXErrorDomain, code=17):
> Unable to write or file already exists
> File exists
we remove the path in case it is present and continue even if there was
an error since it does not imply a test failure.
This allows the CI to run ALL the tests that the project has in
parallel. This is divided in two main changes:
1. Xharness - We move away from using boolenas to use a flag that states
the tests to run.
2. yaml - We have move the code to use a template per label. This new
jobs all run in parallel and the results are later collected by a
funel job
3. pwsh - Added a new class that understands that we have several mark
downs with the tests results. The classes parses them and them writes
a single comment (and example can be found here: https://github.com/xamarin/xamarin-macios/pull/15201#issuecomment-1162366240
The changes gives the following advantages vs how we used to run tests:
1. The CI run for all tests moves from taking 13 hours to 3/4 hours
(depending on the number of bots in the pool).
2. The download needed to verify the results on a case of failure is
smaller. Rather than downloading several GBs we now just download
that part of the html that we are interested in.
3. Better bot utlization. Bots are just used to a max of 2 hours, this
means that we can use the bots better since they are fragmented.
4. Less VMs. VSDrops has added support for macOS and Linux, we take
advanges of that here.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Collect diagnostic logs from the simulator to try to get better diagnostic information for https://github.com/xamarin/maccore/issues/2558.
Interestingly it seems like just collecting diagnostic information makes the problem much less likely to occur...
The job that hides the previous bot comments tries to find comments with
[OR\CI build] to decide if a comment should be hidden. This is added
when a title is present, so we just add a titles so that we can hide the
comments.
When we changed SCNMatrix4 to be column-major instead of row-major in .NET, there
were several other related changes we should have done but didn't do. In particular
we should have made transformation operations based on column-vectors instead of
row-vectors.
In legacy Xamarin, a vector would be transformed by a transformation matrix by doing
matrix multiplication like this:
[ x y z w] * [ 11 21 31 41 ]
| 12 22 32 42 |
| 13 23 33 43 |
[ 14 24 34 41 ]
In this case the vector is a row-vector, and it's the left operand in the multiplication.
When using column-major matrices, we want to use column-vectors, where the vector
is the right operand, like this:
[ 11 21 31 41 ] * [ x ]
| 12 22 32 42 | | y |
| 13 23 33 43 | | z |
[ 14 24 34 41 ] [ w ]
This affects numerous APIs in SCNMatrix4, SCNVector3 and SCNVector4:
* The M## fields have been changed to make the first number the column and the
second number the row, to reflect that it's a column-major matrix (this is
also how it's defined in the native SCNMatrix4 type).
* Functions that return a transformation matrix have been modified to return column-vector
transformers. Technically this means that these matrices are transposed compared
to legacy Xamarin. The functions involved are:
* CreateFromAxisAngle
* CreateRotation[X|Y|Z]
* CreateTranslation
* CreatePerspectiveFieldOfView
* CreatePerspectiveOffCenter
* Rotate
* LookAt
* Combining two column-vector transforming transformation matrices is done by multiplying
them in the reverse order, so the Mult function (and the multiplication operator)
have been modified to multiply the given matrices in the opposite order (this matches
how the SCNMatrix4Mult function does it). To make things clearer I've changed the
parameter names for XAMCORE_5_0.
* Functions that transform a vector using a transformation matrix have been modified
to do a column-vector transformation instead of a row-vector transformation. This
involves the following functions:
* SCNVector3.TransformVector
* SCNVector3.TransformNormal
* SCNVector3.TransformNormalInverse
* SCNVector3.TransformPosition
* SCNVector4.Transform
* Numerous new tests.
Fixes https://github.com/xamarin/xamarin-macios/issues/15094.
There is no need to provision the simulators on a build. Provisioning
simuatlors is giving problems with the EO pool after an upgrade to macOS
12.4.
We remove the provisioning from the build BUT not from the tests since
those tests are executed in non EO complient machines.
Deciding if we build a PR or not used to be more complicated since we
had to make the diff between a CI build and a PR build. Now, since we
added diff pipelines we do not longer need to check any variable since
we can use a parameter.
This new fact makes the decision making simpler (although forces use to
add a new parameter in a few templates). The overall result is a simple
way to decide what can be used or not in the pipeline.
* Simplify logic.
* Add missing param.
* Fix the checkout for signing in the pr build.
* There is not need to sign in PR builds.
The signature is not needed for the tests and using -s in codesign means
that it is only valid in the machine that signed it.
Make our local .NET the default .NET (in the root's global.json), and then if
a directory wants to use the system .NET, then that directory would have to
opt-in (using its own global.json).
This way we don't have to copy global.json/NuGet.config files around to run
tests with the correct .NET setup.
The files of the bundle.zip are being re-zipped inside a 'bundle' folder after signing, which is wrong since everything else is expecting the files directly inside the zip file without parent folders (as we were doing before)
Remove our dependency on Visual Studio. Use the 'dotnet-t4' tool instead of
invoking the t4 tool embedded in Visual Studio.
Fixes this build error after installing VS Mac 2022:
> Cannot open assembly '/Applications/Visual Studio.app/Contents/Resources/lib/monodevelop/AddIns/MonoDevelop.TextTemplating/TextTransform.exe': No such file or directory.
Previously we'd compare the tip of the PR branch with the commit before merge commit into the target branch created by GitHub.
This doesn't work in the following scenario:
main gh pr
| G1 | (merge commit created by GitHub)
| / \ |
|/ \|
M2 |
| P2
| /|
| / |
| / |
| / |
|/ |
M1 |
| |
| P1
| /
| /
| /
| /
|/
We'd end up comparing P2 with M2, which is not what we want, because M2 could
have API changes that would show up as missing in P2.
It's actually even worse than that, because we could be in the following
scenario:
main gh pr
| G2 | (merge commit created by GitHub when api diff started)
| / \ |
|/ \|
M3 |
| P3
| |
| G1 | (merge commit created by GitHub when build started)
| / \ |
|/ \|
M2 |
| P2
| /|
| / |
| / |
| / |
|/ |
M1 |
| |
| P1
| /
| /
| /
| /
|/
And we still want to P2 with M2, but we'd compare P2 with M3.
We want to compare P2 with M1 instead. This is done by asking git for the
merge-base between P2 and M2.
Rolf Bjarne Kvinge
VS MobileTools Engineering Service 2
Manuel de la Pena
Alex Soto
Mauro Agnoletti
TJ Lambert