xamarin-macios/tests/sampletester
Rolf Bjarne Kvinge bd457212d5
[tests] Add sample tester. (#5870)
* [tests] Add sample tester.

Add a unit project that looks for iOS/macOS/tvOS sample projects in several
repositories, and builds them all.

* [tests][sampletester] Remove known issue which has now been fixed.

* [tests] Only run sample tests on CI in Azure Devops.

* Remove the possibility of automatically running the sample tests with
  xharness (so the sample tests won't run on PR bots or internal bots when the
  'run-all-tests' label is added). It's still possible to run the sample tests
  manually from the xharness web UI.

* Automatically trigger the sample test run in Azure Devops if the
  'run-sample-tests' label is applied to a PR (and that PR is executed on
  internal Jenkins).

* Fix typo.

* Fix path.

* Verbose output to track down scheduling failure.

* Bump maccore to get improved debug spew.

Diff: f527c9c526..f89d74b165

* [tests][sampletester] Fix build for TodoWCF.
2019-04-25 17:18:43 +02:00
..
Properties [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
images [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
.gitignore [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
BaseTester.cs [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
Configuration.cs [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
GitHub.cs [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
HtmlReport.xslt [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
Makefile [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
ProcessHelper.cs [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
README.md [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
SampleTester.cs [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
Samples.cs [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
TestPlatform.cs [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
packages.config [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
sampletester.csproj [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
sampletester.sln [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00

README.md

Sample testing

These unit tests clone a series of known repositories that contain sample projects, and build all the relevant projects in those repositories.

It is executed automatically in Azure DevOps every [Saturday][2] for the following branches:

  • master
  • d16-*
  • xcode*

It can also be triggered manually, but have in mind that the commit in question must already have packages (as GitHub statuses).

It's also possible to use the sample tests from one commit, and then test with Xamarin.iOS/Xamarin.Mac a completely different commit, by setting the PROVISION_FROM_COMMIT variable to the commit that's to be tested:

screenshot

The previous point is still required: the commit to provision from must have packages.

There are two ways to run these tests locally:

  • Launching xharness in server mode and execute the "Sample tests" (they're disabled by default).
  • Executing make in this directory.