2691791ad2
The added test case has not shown any major performance enhancement, anything (if) is lost in the noise. Beside the performance the change has two small size advantages * `Create` does not call another method (less metadata) * `GetCount` is now the p/invoke `CFArrayGetCount` Before // * Summary * BenchmarkDotNet=v0.12.1.1528-nightly, OS=macOS Big Sur 11.5.2 (20G95) [Darwin 20.6.0] Intel Core i7-4790K CPU 4.00GHz (Haswell), 1 CPU, 8 logical and 4 physical cores [Host] : Mono 6.12.0 (2020-02/3cf59ad33da), X64 Job=InProcess Toolchain=InProcessEmitToolchain IterationCount=3 LaunchCount=1 WarmupCount=3 | Method | Length | Mean | Error | StdDev | |------- |------- |------------:|------------:|----------:| | Create | 0 | 255.4 ns | 53.68 ns | 2.94 ns | | Create | 1 | 343.0 ns | 8.55 ns | 0.47 ns | | Create | 16 | 443.8 ns | 30.54 ns | 1.67 ns | | Create | 256 | 1,898.4 ns | 246.80 ns | 13.53 ns | | Create | 4096 | 40,503.2 ns | 5,395.81 ns | 295.76 ns | // * Legends * Length : Value of the 'Length' parameter Mean : Arithmetic mean of all measurements Error : Half of 99.9% confidence interval StdDev : Standard deviation of all measurements 1 ns : 1 Nanosecond (0.000000001 sec) // ***** BenchmarkRunner: End ***** Global total time: 00:00:33 (33.26 sec), executed benchmarks: 5 After // * Summary * BenchmarkDotNet=v0.12.1.1528-nightly, OS=macOS Big Sur 11.5.2 (20G95) [Darwin 20.6.0] Intel Core i7-4790K CPU 4.00GHz (Haswell), 1 CPU, 8 logical and 4 physical cores [Host] : Mono 6.12.0 (2020-02/3cf59ad33da), X64 Job=InProcess Toolchain=InProcessEmitToolchain IterationCount=3 LaunchCount=1 WarmupCount=3 | Method | Length | Mean | Error | StdDev | |------- |------- |------------:|------------:|---------:| | Create | 0 | 246.3 ns | 55.49 ns | 3.04 ns | | Create | 1 | 341.1 ns | 18.13 ns | 0.99 ns | | Create | 16 | 425.4 ns | 34.97 ns | 1.92 ns | | Create | 256 | 1,741.1 ns | 298.87 ns | 16.38 ns | | Create | 4096 | 40,333.6 ns | 1,646.70 ns | 90.26 ns | // * Legends * Length : Value of the 'Length' parameter Mean : Arithmetic mean of all measurements Error : Half of 99.9% confidence interval StdDev : Standard deviation of all measurements 1 ns : 1 Nanosecond (0.000000001 sec) // ***** BenchmarkRunner: End ***** Global total time: 00:00:31 (31.68 sec), executed benchmarks: 5 |
||
|---|---|---|
| .. | ||
| Assets.xcassets | ||
| dotnet | ||
| legacy | ||
| .gitignore | ||
| AppDelegate.cs | ||
| Entitlements.plist | ||
| Info.plist | ||
| Main.cs | ||
| Main.storyboard | ||
| Makefile | ||
| ManagedRuntime.cs | ||
| MessageSend.cs | ||
| Messaging.cs | ||
| NativeArrayPerf.cs | ||
| ObjCBridge.cs | ||
| ObjectCreation.cs | ||
| README.md | ||
| TollFreeBridge.cs | ||
| ViewController.cs | ||
| ViewController.designer.cs | ||
| perftest.sln | ||
README.md
Benchmarks
This folder contains a test suite to test performance in Xamarin.iOS/Xamarin.Mac.
There are three variations of the test suite:
-
Legacy mode (Xamarin).
-
.NET Mode (Mono): runs tests in .NET, with MonoVM.
-
.NET Mode (CoreCLR): runs tests using .NET, with CoreCLR.
Running the benchmarks
Run all three variations of the test suite:
make run-perftest
Run each variation:
Legacy mode
make run-perftest-with-legacy
.NET Mode (Mono)
make run-perftest-with-mono
.NET Mode (CoreCLR)
make run-perftest-with-coreclr
Results
At the end of the test run, something like this will be printed:
Log files were stored in xamarin-macios/tests/perftest/output/2021-04-22--21:43:28
and that's where you'll find log files. Console output from the executable is stored as perflog-[timestamp].log, while BenchmarkDotNet results are exported to the results/ subfolder (in json, markdown, csv and html formats).
There's no easy way to compare results across variations or runs (yet), the exported results will have to be examined by a human and compared that way.