xamarin-macios/tests/sampletester
Rolf Bjarne Kvinge 277baaf521
[tests] Request labels after each test from NUnit. (#14628)
Fixes this NUnit warning:

> labels=All is deprecated and will be removed in a future release. Please use labels=Before instead.

We don't follow the suggestion from the warning, because the advantage of
writing the label after each test is that the test result will also be
printed, which means it's possible to see if any tests failed during the test
run, as opposed to having to wait until the entire test run is completed
(which can take a while) to realize that pretty much every test failed with
some silly mistake which could have been quickly fixed before re-running the
tests.
2022-04-04 19:19:45 +02:00
..
BaselineTest Fix some whitespace issues in various files. (#12399) 2021-08-11 10:06:46 +02:00
Properties [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
images [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
.gitignore [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
BaseTester.cs [xcode12] Rename master to main (#9631) 2020-09-15 07:09:29 -04:00
Configuration.cs [tests] Make sampletester greener (#10713) 2021-02-24 10:03:59 -05:00
GitHub.cs [xcode12] Rename master to main (#9631) 2020-09-15 07:09:29 -04:00
HtmlReport.xslt [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
Makefile [tests] Request labels after each test from NUnit. (#14628) 2022-04-04 19:19:45 +02:00
ProcessHelper.cs [SampleTester] Add 4 new samples (#7758) 2020-01-28 15:40:52 -05:00
README.md [xcode12] Rename master to main (#9631) 2020-09-15 07:09:29 -04:00
SampleTester.cs [tests] Make the baseline sample test pass the correct environment variables. (#9952) 2020-10-23 16:21:01 +02:00
Samples.cs [tests] Make sampletester greener (#10713) 2021-02-24 10:03:59 -05:00
TestPlatform.cs Fix some whitespace issues in various files. (#12399) 2021-08-11 10:06:46 +02:00
sampletester.csproj [tests] Make the .NET bgen tests actually reference the .NET BCL. (#9693) 2020-09-24 14:57:05 +02:00
sampletester.sln [sample-tester] Collect app size and build duration statistics. (#7602) 2019-12-20 16:27:36 +01:00

README.md

Sample testing

These unit tests clone a series of known repositories that contain sample projects, and build all the relevant projects in those repositories.

It is executed automatically in Azure DevOps every Saturday for the listed branches. This includes:

  • main
  • d16-x - i.e. one or many active release branches
  • xcodeY - i.e. the current xcode beta feature branch
  • mono-202z-zz - i.e. the mono integration branches (new one every two months)

where x, Y and z needs to be periodically updated as new branches are created (e.g. d16-x) and old ones are merged into main (e.g. xcodeY and mono-202z-zz).

It can also be triggered manually, but have in mind that the commit in question must already have packages (as GitHub statuses).

It's also possible to use the sample tests from one commit, and then test with Xamarin.iOS/Xamarin.Mac a completely different commit, by setting the PROVISION_FROM_COMMIT variable to the commit that's to be tested:

screenshot

The previous point is still required: the commit to provision from must have packages.

There are two ways to run these tests locally:

  • Launching xharness in server mode and execute the "Sample tests" (they're disabled by default).
  • Executing make in this directory.