xamarin-macios/tests/sampletester
Rolf Bjarne Kvinge c3bcfac582
Unify code for executing processes. (#8848)
* Create a simple Xamarin.Utils.Execution class that can handle all our
  process execution needs:
    * Captures or streams stdout/stderr (in UTF8).
    * Supports async
    * Supports a timeout
    * Does not depend on any other source file we have, only uses BCL API.
* Have the execution helper classes from mtouch/mmp
  (Xamarin.BundlerDriver.RunCommand) and the tests
  (Xamarin.Tests.ExecutionHelper) use this new class.
* Some simplifications were made:
    * All API that took a string array for the environment now takes a
      Dictionary<string, string>.
    * The Driver.RunCommand methods were split out to a separate file. This
      file also contains a Verbosity field, which is conditioned on not being
      in mtouch nor mmp, which makes including this file from other projects
      simpler (such as bgen - in particular bgen was modified to use this
      Verbosity field instead of its own).
2020-06-18 12:34:07 +02:00
..
BaselineTest [sample-tester] Collect app size and build duration statistics. (#7602) 2019-12-20 16:27:36 +01:00
Properties [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
images [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
.gitignore [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
BaseTester.cs Rename master to main. (#8851) 2020-06-16 15:51:44 +02:00
Configuration.cs [sample-tester] Collect app size and build duration statistics. (#7602) 2019-12-20 16:27:36 +01:00
GitHub.cs Rename master to main. (#8851) 2020-06-16 15:51:44 +02:00
HtmlReport.xslt [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
Makefile [sample tests] Fix variable calculation to calculate once. (#6061) 2019-05-17 09:21:00 -07:00
ProcessHelper.cs [SampleTester] Add 4 new samples (#7758) 2020-01-28 15:40:52 -05:00
README.md Rename master to main. (#8851) 2020-06-16 15:51:44 +02:00
SampleTester.cs Rename master to main. (#8851) 2020-06-16 15:51:44 +02:00
Samples.cs Rename master to main. (#8851) 2020-06-16 15:51:44 +02:00
TestPlatform.cs [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
packages.config [tests] Add sample tester. (#5870) 2019-04-25 17:18:43 +02:00
sampletester.csproj Unify code for executing processes. (#8848) 2020-06-18 12:34:07 +02:00
sampletester.sln [sample-tester] Collect app size and build duration statistics. (#7602) 2019-12-20 16:27:36 +01:00

README.md

Sample testing

These unit tests clone a series of known repositories that contain sample projects, and build all the relevant projects in those repositories.

It is executed automatically in Azure DevOps every Saturday for the listed branches. This includes:

  • main
  • d16-x - i.e. one or many active release branches
  • xcodeY - i.e. the current xcode beta feature branch
  • mono-202z-zz - i.e. the mono integration branches (new one every two months)

where x, Y and z needs to be periodically updated as new branches are created (e.g. d16-x) and old ones are merged into main (e.g. xcodeY and mono-202z-zz).

It can also be triggered manually, but have in mind that the commit in question must already have packages (as GitHub statuses).

It's also possible to use the sample tests from one commit, and then test with Xamarin.iOS/Xamarin.Mac a completely different commit, by setting the PROVISION_FROM_COMMIT variable to the commit that's to be tested:

screenshot

The previous point is still required: the commit to provision from must have packages.

There are two ways to run these tests locally:

  • Launching xharness in server mode and execute the "Sample tests" (they're disabled by default).
  • Executing make in this directory.