On old macOS bots we only want to provision Mono, so change the logic so that
we first ignore everything, then re-enable Mono.
This way we don't have to add new ignore flags here every time we add
something new to the provisioning script.
This fixes an issue where the script detects that dotnet isn't installed, and
shows an error, when trying to run tests on older macOS bots in CI.
* [configure] Add --[enable|disable]-dotnet.
And make it enabled by default on CI and disabled by default elsewhere for now
(because it makes the build significantly slower).
* [system-dependencies] Add support for provisioning .NET.
Also write a global.json in the root directory which is how we select which
.NET version to use.
Move to use a template for the DDFun pipeline. Copy the current pipeline
to a template, set some parameters and recreate the pipeline importing
the template.
* Fix fetching author's name + email for failure reporting by running git in
the directory where the repo actually is.
* Don't fail the build if we fail to generate/upload artifacts.json, instead
show the failure everywhere and continue as if nothing happened. There seems
to be a random issue with GitHub where a network request fails, and there's
no need for that to fail the entire build.
Bump maccore so that a status is posted when the device tests cannot be
scheduled. Update the JenkinsFile to make sure that the required token
is passed to the launch.sh script.
In the DDFun pipeline case `cat: tmp.p: No such file or directory` so the `P` variable cannot be set and that's failing vsts-device-tests-set-status.sh
- We had `./jenkins/vsts-device-tests-set-status.sh: line 88: HTML_REPORT: unbound variable` in the `Report results to GitHub as comment / status` step. Now that the HTML_REPORT is set this shouldn't happen anymore.
- The `Add summaries` task is reporting errors but it's working regardless. I believe this might be due to -x being set instead of the original +x (I changed that by mistake).
- Needed for the DDFun migration of our tests
- Don't use SSH because it's restricted on the new bots
- Use brand new DevOps feature to clone multiple repos via HTTPS
- Update names of all tasks (clearer)
- Simplify and re-use code
- Update all paths because our cloning strategy is different compared to old pipeline
- Update xharness to print "experimental" titles in GitHub comments
- Update xharness to not add the Html link for tests from the DDFun pipeline
- Fix all timeouts to be relevant to the tests length
- Split in multiple jobs (for GitHub status)
- Move longer scripts to independant files
- Make inline bash prettier
- Use a variable for the pool name
- Fix single space alignment
- Remove unused variable in `vsts-device-tests-set-status.sh` that broke it
- Bump maccore to use the right certificates and profiles
- Bump maccore to target the right pipeline
- Bump maccore to not use ssh in scripts
- Bump maccore and use persistCredentials
Notes:
- Right now the xharness test output can only be seen when downloading the zip file locally (better output soon)
* [tests] Add make target to manually launch a device test run for the current commit.
* Upload a manifest that can be found using only the hash, and use it to find the packages.
* Only notarize CI builds; timeout after 90 min
* Update jenkins/Jenkinsfile
Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>
* Don't upload notarized pkgs if it fails
* Update jenkins/Jenkinsfile
Co-Authored-By: Manuel de la Pena <mandel@microsoft.com>
* Automatically build mono from source if any of the archives aren't available.
* [jenkins] Show in the PR report if mono was built from source.
* [jenkins] Make sure we got everything after running configure.
* Add a dummy x86_64 slice to all our native libraries that don't have one. (#6848)
Apple's notarization tool has a bug where they incorrectly flag Mach-O
binaries without an x86_64 slice, so make sure all our libraries have one.
* Jenkinsfile notarization (#6869)
* Add in notarization script for xamarin.mac/xamarin.iOS
* Flatten the list to get rid of the braces
* Add in keychain password
* Add login.keychain back in to access codesigning certificates
* Always sign pkgs, upload notarized copies
* Enable ios notarization and make notarized pkgs public
* Make notarization non-fatal
* Publish GH statuses for notarized PKGs
* Don't forget to declare URI variables for notarized pkgs
* report proper package links
* [jenkins] Improve package reporting.
* Use dummy function name which our tests won't complain about.
* Add in notarization script for xamarin.mac/xamarin.iOS
* Flatten the list to get rid of the braces
* Add in keychain password
* Add login.keychain back in to access codesigning certificates
* Always sign pkgs, upload notarized copies
* Enable ios notarization and make notarized pkgs public
* Make notarization non-fatal
* Publish GH statuses for notarized PKGs
* Don't forget to declare URI variables for notarized pkgs
* report proper package links
* [jenkins] Improve package reporting.
brew may fail to install 7z if 7z is already installed, so ignore any brew
failures when trying to install 7z.
If 7z really is non-functional on the machine, then it'll fail later anyway.
* macOS 10.15 starts putting up permission dialogs we can't automatically
dismiss anymore, so start honoring the 'IncludeSystemPermissionTests' option
for macOS tests.
* Improve the 'IncludeSystemPermissionTests' option to have three states: if
set (either true or false), that takes precedence, but if not set, we now
don't run any tests that require permission dialogs on macOS or on device if
we're running in CI. Tests executed locally will still put up dialogs, both
on macOS and on device.
* This needed a few changes to the html report, since the
'IncludeSystemPermissionTests' is exposed in the UI and the code didn't
handle the three different states.
* Update a few tests to check for permission to the contacts.
Fixes https://github.com/xamarin/maccore/issues/1856.
Many people are running into problems building xcode11 because they have to
"./configure --disable-packaged-mono" first.
So disable the packaged mono by default (and add a way to enable it again
manually with "./configure--disable-packaged-mono=no").
We have a 13-hour timeout for running the tests [1], but that won't work if
the timeout for the entire Jenkins pipeline is lower, so adjust the timeout
for the entire Jenkins pipeline to be more than the sum of all the nested
timeouts.
[1] 94fe39b118 (diff-68c8473bbe1d4346ecff3d74522a31f8R675)
This updates the project generation. We cannot yet fully remove the submodule because:
* We are missing the xunit dlls which should be added in the SDK.
* We have not yet remove all the old style tests. Would make the PR huge, better to deal with it in a diff PR.
* The xunit CoreLib tests have issues loading all the tests, needs some extra work and again, the PR is already large.
Fixes: xamarin/maccore#1199Fixes: xamarin/maccore#1204Fixes: xamarin/maccore#1209Fixes: xamarin/maccore#1510
We had issues in the code that adds a type found in an assembly to
ensure that it was not removed by the linker. This resulted in some
assemblies having 0 tests.
Added the needed ignore for the corlib tests and system ones.
* [builds] Fix race condition when building device runtimes.
We need to complete the corresponding cross builds first, because the cross
builds will also try to configure device builds, so we might end up with two
make processes trying to configure the same target simulatenously.
* [jenkins] Add support for building mono from source in pull requests by using labels.
* [jenkins] Add support to build.sh for timing out the build.
* Test timeout support
* [jenkins] Tell the world what happened.
* [jenkins] Minor announcement fix + disable test mode.
* [tests] Add sample tester.
Add a unit project that looks for iOS/macOS/tvOS sample projects in several
repositories, and builds them all.
* [tests][sampletester] Remove known issue which has now been fixed.
* [tests] Only run sample tests on CI in Azure Devops.
* Remove the possibility of automatically running the sample tests with
xharness (so the sample tests won't run on PR bots or internal bots when the
'run-all-tests' label is added). It's still possible to run the sample tests
manually from the xharness web UI.
* Automatically trigger the sample test run in Azure Devops if the
'run-sample-tests' label is applied to a PR (and that PR is executed on
internal Jenkins).
* Fix typo.
* Fix path.
* Verbose output to track down scheduling failure.
* Bump maccore to get improved debug spew.
Diff: f527c9c526..f89d74b165
* [tests][sampletester] Fix build for TodoWCF.
Builds take approximately 30 minutes on a bot now, so 1 hour should be plenty.
This makes sure a 32-bit dialog doesn't waste 9h of bot time just sitting
there.
Updates the jenkinsFile to include generating ci-checkout.json and dependency-info.json.
ci-checkout.json is a list of all of the repositories included in the workspace of the project and their respective hashs.
dependency-info.json includes all of the info in ci-checkout.json, as well as a scan of the containing cs/fsproj files and their containing package and project references.
Fixes this warning:
make: *** tools/siminstaller: No such file or directory. Stop.
Can't check if simulators are available, because siminstaller failed to build.
* [jenkins] Don't execute the packaged XM tests on the main macOS version.
Don't execute the packaged XM tests on the main macOS version, since it's redundant.
It also prevents a potential test deadlock if all main bots are busy.
* Rework to make it show better in the Jenkins UI.
Rolf Bjarne Kvinge
Manuel de la Pena
Sebastien Pouliot
Vincent Dondain
Connor Adsit
Alexander Köplinger
Tim Miller