* [jenkins] Add support to build.sh for timing out the build.
* Test timeout support
* [jenkins] Tell the world what happened.
* [jenkins] Minor announcement fix + disable test mode.
* [tests] Add sample tester.
Add a unit project that looks for iOS/macOS/tvOS sample projects in several
repositories, and builds them all.
* [tests][sampletester] Remove known issue which has now been fixed.
* [tests] Only run sample tests on CI in Azure Devops.
* Remove the possibility of automatically running the sample tests with
xharness (so the sample tests won't run on PR bots or internal bots when the
'run-all-tests' label is added). It's still possible to run the sample tests
manually from the xharness web UI.
* Automatically trigger the sample test run in Azure Devops if the
'run-sample-tests' label is applied to a PR (and that PR is executed on
internal Jenkins).
* Fix typo.
* Fix path.
* Verbose output to track down scheduling failure.
* Bump maccore to get improved debug spew.
Diff: f527c9c526..f89d74b165
* [tests][sampletester] Fix build for TodoWCF.
Builds take approximately 30 minutes on a bot now, so 1 hour should be plenty.
This makes sure a 32-bit dialog doesn't waste 9h of bot time just sitting
there.
Updates the jenkinsFile to include generating ci-checkout.json and dependency-info.json.
ci-checkout.json is a list of all of the repositories included in the workspace of the project and their respective hashs.
dependency-info.json includes all of the info in ci-checkout.json, as well as a scan of the containing cs/fsproj files and their containing package and project references.
Fixes this warning:
make: *** tools/siminstaller: No such file or directory. Stop.
Can't check if simulators are available, because siminstaller failed to build.
* [jenkins] Don't execute the packaged XM tests on the main macOS version.
Don't execute the packaged XM tests on the main macOS version, since it's redundant.
It also prevents a potential test deadlock if all main bots are busy.
* Rework to make it show better in the Jenkins UI.
Jenkins has a limitation where you can't mark a step a failure, it has to
*fail* to be reported as such.
This means that running multiple tests, and reporting a failure if any of
those tests fail is not possible.
We run into this with the normal test run + the docs tests; where we currently
don't show a red result in the UI if any of those fail.
So incorporate the test-docs step into xharness, so that we only have one
thing to in the Jenkinsfile, which makes it possible for us to fail the test
run step properly.
This also required a few upgrades to xharness to get more info for pull
requests, since the logic to enable the docs tests is a bit more complicated
than anything else we have (if the current branch (or the target branch for a
PR) is 'master' AND xamarin mode is enabled).
Fix retry-download-on-error-56 logic to not fail forever if the first time
failed by resetting the EC variable after each try.
Fixes https://github.com/xamarin/maccore/issues/1098 (third time's the charm!)
* [jenkins] Don't execute the packaged XM tests on the main macOS version.
Don't execute the packaged XM tests on the main macOS version, since it's redundant.
It also prevents a potential test deadlock if all main bots are busy.
* Rework to make it show better in the Jenkins UI.
* Bump maccore to get fix for test-docs.
Commit list for xamarin/maccore:
* xamarin/maccore@6e9b63e537 Merge pull request #1162 from rolfbjarne/docfixer-fixes
* xamarin/maccore@4cf1d63b7b [docfixer] Create project files and fix a few issues. Fixes#1118.
* xamarin/maccore@714bd73a55 [docfixer] Avoid declaring the same target twice.
* xamarin/maccore@34f4bfa339 [populate] Directories need to call Directory.Exists for existence checks.
* xamarin/maccore@341333db88 [docs] Remove truly ancient and outdated targets to publish updated docs.
Diff: 7ba9c5a962...6e9b63e537
* [jenkins] Make it possible to force docs testing by applying a label to pull requests.
This means we'll get an empty diff if there are no generator changes, so
update the logic that detects/reports empty generator diffs.
Also there's no need to ignore mdbs anymore, since we don't create mdbs.
* [builds] Adjust ifdefs to fix not building device architectures. Fixes maccore#1074.
Fixes https://github.com/xamarin/maccore/issues/1074.
* [jenkins] Running configure and then cleaning everything is kind of useless, so reverse the order.
* [builds] Adjust ifdefs to fix not building device architectures. Fixes maccore#1074.
Fixes https://github.com/xamarin/maccore/issues/1074.
* [jenkins] Running configure and then cleaning everything is kind of useless, so reverse the order.
Our tests can take quite a while to run sometimes, and this makes sure we
don't get random failures when the tests that access the keychain are executed
at the end of the test run.
Bump maccore to get the same fix there.
Fixes https://github.com/xamarin/maccore/issues/973.
Commit diff for maccore bump: 9937926f56...07fde84362
Something in VSTS changed, and now fake branch names don't work anymore.
So instead use real branch names (and for pull requests I've created a
'pull-request' branch we can use).
We keep track of the current stage by using the 'currentStage' variable, so that we can properly report what failed.
There were a couple of problems with this:
* The 'Package Xamarin.Mac tests' is not a fatal step, which means that the
pipeline will continue executing even if it fails. In this case the previous
code would assign any failure to package the XM tests to the last stage
('Test docs').
* Something similar seems to happen if there are failures in the Xamarin.Mac
tests executed in parallel (on other bots), where any failure would be
reported as a failure in the last stage ('Test docs').
* No support for reporting failure in multiple stages.
So do a couple of things:
* Clear out the 'currentStage' variable at the end, and handle it being empty
when reporting results. This should solve at least some of the problems
where blame as being assigned incorrectly (we'll probably run into cases
where blame isn't assigned at all, but this is still half the way of getting
it right).
* Add a 'failedStages' variables where we can keep track of (and report)
multiple failed stages.
Implement support for slack notifications.
Notifications will only be sent when something fails, and unfortunately I wasn't able to make notifications ping due to https://github.com/jenkinsci/slack-plugin/issues/353.
Rolf Bjarne Kvinge
Vincent Dondain
Manuel de la Pena
Tim Miller
Sam Schwarz
Connor Adsit
Xamarin Jenkins User
Sebastien Pouliot