1394420fdc
This requires less native calls (one for all values) than the current code (one per value inside the array). However the performance enhancement is not dramatic as the largest cost is not in that code path (and the extra allocation undermine it too). Still it's I tried a few other variations (e.g. loop over a stackalloc'ed buffer) but they did not perform better and the code was more complex. Note: the NSArray* code/tests are identical between both sets (and the mean results are also closer) Actual Implementation | Method | name | value | Mean | Error | StdDev | |------------------------------ |-------------- |------------------- |------------------:|-------------------:|----------------:| | CFArray_StringArrayFromHandle | empty | () | 124.856 ns | 13.1140 ns | 0.7188 ns | | NSArray_StringArrayFromHandle | empty | () | 1,438.752 ns | 120.3745 ns | 6.5981 ns | | CFArray_ArrayFromHandle | empty | () | 124.727 ns | 21.3443 ns | 1.1700 ns | | NSArray_ArrayFromHandle | empty | () | 1,433.047 ns | 164.6988 ns | 9.0277 ns | | CFArray_StringArrayFromHandle | few | ( (...).") [108] | 1,818.681 ns | 457.2275 ns | 25.0622 ns | | NSArray_StringArrayFromHandle | few | ( (...).") [108] | 8,636.130 ns | 2,505.6452 ns | 137.3429 ns | | CFArray_ArrayFromHandle | few | ( (...).") [108] | 4,105.635 ns | 1,151.4048 ns | 63.1124 ns | | NSArray_ArrayFromHandle | few | ( (...).") [108] | 8,769.657 ns | 1,946.9970 ns | 106.7215 ns | | CFArray_StringArrayFromHandle | large_mutable | ((...)) [8419330] | 20,103,343.750 ns | 13,762,095.1905 ns | 754,346.8755 ns | | NSArray_StringArrayFromHandle | large_mutable | ((...)) [8419330] | 24,256,992.708 ns | 9,357,034.9200 ns | 512,890.6579 ns | | CFArray_ArrayFromHandle | large_mutable | ((...)) [8419330] | 3,888,197.135 ns | 395,545.6860 ns | 21,681.1938 ns | | NSArray_ArrayFromHandle | large_mutable | ((...)) [8419330] | 7,326,581.510 ns | 622,485.7999 ns | 34,120.5472 ns | | CFArray_StringArrayFromHandle | lot_mutable | ( (...)") [34427] | 194,289.884 ns | 27,658.0701 ns | 1,516.0322 ns | | NSArray_StringArrayFromHandle | lot_mutable | ( (...)") [34427] | 549,336.410 ns | 98,695.3827 ns | 5,409.8270 ns | | CFArray_ArrayFromHandle | lot_mutable | ( (...)") [34427] | 222,837.679 ns | 38,823.2619 ns | 2,128.0340 ns | | NSArray_ArrayFromHandle | lot_mutable | ( (...)") [34427] | 441,116.243 ns | 46,578.8698 ns | 2,553.1450 ns | | CFArray_StringArrayFromHandle | null | ? | 4.568 ns | 0.8750 ns | 0.0480 ns | | NSArray_StringArrayFromHandle | null | ? | 4.728 ns | 1.0713 ns | 0.0587 ns | | CFArray_ArrayFromHandle | null | ? | 4.818 ns | 1.0416 ns | 0.0571 ns | | NSArray_ArrayFromHandle | null | ? | 4.793 ns | 0.5478 ns | 0.0300 ns | | CFArray_StringArrayFromHandle | one | ( 1) | 682.173 ns | 83.0811 ns | 4.5540 ns | | NSArray_StringArrayFromHandle | one | ( 1) | 3,409.570 ns | 1,407.8715 ns | 77.1702 ns | | CFArray_ArrayFromHandle | one | ( 1) | 1,008.268 ns | 340.9156 ns | 18.6867 ns | | NSArray_ArrayFromHandle | one | ( 1) | 3,174.222 ns | 412.6530 ns | 22.6189 ns | | CFArray_StringArrayFromHandle | small_mutable | ( (...)9e") [54] | 909.008 ns | 271.9224 ns | 14.9050 ns | | NSArray_StringArrayFromHandle | small_mutable | ( (...)9e") [54] | 5,017.722 ns | 66.9051 ns | 3.6673 ns | | CFArray_ArrayFromHandle | small_mutable | ( (...)9e") [54] | 1,900.176 ns | 131.5765 ns | 7.2121 ns | | NSArray_ArrayFromHandle | small_mutable | ( (...)9e") [54] | 4,913.822 ns | 1,264.8949 ns | 69.3332 ns | New Implementation | Method | name | value | Mean | Error | StdDev | |------------------------------ |-------------- |------------------- |------------------:|------------------:|----------------:| | CFArray_StringArrayFromHandle | empty | () | 112.335 ns | 0.7103 ns | 0.0389 ns | | NSArray_StringArrayFromHandle | empty | () | 1,497.473 ns | 1,508.7464 ns | 82.6995 ns | | CFArray_ArrayFromHandle | empty | () | 114.670 ns | 3.7721 ns | 0.2068 ns | | NSArray_ArrayFromHandle | empty | () | 1,486.298 ns | 767.5367 ns | 42.0713 ns | | CFArray_StringArrayFromHandle | few | ( (...).") [108] | 1,526.398 ns | 205.6683 ns | 11.2734 ns | | NSArray_StringArrayFromHandle | few | ( (...).") [108] | 8,503.264 ns | 981.4524 ns | 53.7967 ns | | CFArray_ArrayFromHandle | few | ( (...).") [108] | 3,814.918 ns | 1,868.8035 ns | 102.4354 ns | | NSArray_ArrayFromHandle | few | ( (...).") [108] | 8,628.285 ns | 2,416.9366 ns | 132.4805 ns | | CFArray_StringArrayFromHandle | large_mutable | ((...)) [8419330] | 16,351,876.042 ns | 3,691,993.1957 ns | 202,370.6051 ns | | NSArray_StringArrayFromHandle | large_mutable | ((...)) [8419330] | 24,589,916.667 ns | 2,800,427.9107 ns | 153,500.9034 ns | | CFArray_ArrayFromHandle | large_mutable | ((...)) [8419330] | 3,390,725.260 ns | 2,950,426.1305 ns | 161,722.8120 ns | | NSArray_ArrayFromHandle | large_mutable | ((...)) [8419330] | 7,296,976.823 ns | 4,376,785.2488 ns | 239,906.3683 ns | | CFArray_StringArrayFromHandle | lot_mutable | ( (...)") [34427] | 167,120.182 ns | 14,246.9472 ns | 780.9232 ns | | NSArray_StringArrayFromHandle | lot_mutable | ( (...)") [34427] | 554,940.788 ns | 116,685.5145 ns | 6,395.9268 ns | | CFArray_ArrayFromHandle | lot_mutable | ( (...)") [34427] | 192,447.498 ns | 15,447.8562 ns | 846.7491 ns | | NSArray_ArrayFromHandle | lot_mutable | ( (...)") [34427] | 431,045.597 ns | 42,844.2650 ns | 2,348.4387 ns | | CFArray_StringArrayFromHandle | null | ? | 5.157 ns | 4.3360 ns | 0.2377 ns | | NSArray_StringArrayFromHandle | null | ? | 5.040 ns | 0.0694 ns | 0.0038 ns | | CFArray_ArrayFromHandle | null | ? | 5.514 ns | 0.3849 ns | 0.0211 ns | | NSArray_ArrayFromHandle | null | ? | 5.373 ns | 0.8358 ns | 0.0458 ns | | CFArray_StringArrayFromHandle | one | ( 1) | 730.162 ns | 65.9006 ns | 3.6122 ns | | NSArray_StringArrayFromHandle | one | ( 1) | 3,421.896 ns | 315.0152 ns | 17.2670 ns | | CFArray_ArrayFromHandle | one | ( 1) | 1,052.039 ns | 295.2081 ns | 16.1814 ns | | NSArray_ArrayFromHandle | one | ( 1) | 3,150.829 ns | 375.5461 ns | 20.5849 ns | | CFArray_StringArrayFromHandle | small_mutable | ( (...)9e") [54] | 845.227 ns | 41.1244 ns | 2.2542 ns | | NSArray_StringArrayFromHandle | small_mutable | ( (...)9e") [54] | 5,062.850 ns | 1,601.6880 ns | 87.7939 ns | | CFArray_ArrayFromHandle | small_mutable | ( (...)9e") [54] | 1,824.390 ns | 261.6341 ns | 14.3410 ns | | NSArray_ArrayFromHandle | small_mutable | ( (...)9e") [54] | 4,871.647 ns | 165.4350 ns | 9.0680 ns | Fix https://github.com/xamarin/xamarin-macios/issues/12375 |
||
|---|---|---|
| .. | ||
| Assets.xcassets | ||
| dotnet | ||
| legacy | ||
| .gitignore | ||
| AppDelegate.cs | ||
| Entitlements.plist | ||
| Info.plist | ||
| Main.cs | ||
| Main.storyboard | ||
| Makefile | ||
| MessageSend.cs | ||
| Messaging.cs | ||
| ObjCBridge.cs | ||
| ObjectCreation.cs | ||
| README.md | ||
| TollFreeBridge.cs | ||
| ViewController.cs | ||
| ViewController.designer.cs | ||
| perftest.sln | ||
README.md
Benchmarks
This folder contains a test suite to test performance in Xamarin.iOS/Xamarin.Mac.
There are three variations of the test suite:
-
Legacy mode (Xamarin).
-
.NET Mode (Mono): runs tests in .NET, with MonoVM.
-
.NET Mode (CoreCLR): runs tests using .NET, with CoreCLR.
Running the benchmarks
Run all three variations of the test suite:
make run-perftest
Run each variation:
Legacy mode
make run-perftest-with-legacy
.NET Mode (Mono)
make run-perftest-with-mono
.NET Mode (CoreCLR)
make run-perftest-with-coreclr
Results
At the end of the test run, something like this will be printed:
Log files were stored in xamarin-macios/tests/perftest/output/2021-04-22--21:43:28
and that's where you'll find log files. Console output from the executable is stored as perflog-[timestamp].log, while BenchmarkDotNet results are exported to the results/ subfolder (in json, markdown, csv and html formats).
There's no easy way to compare results across variations or runs (yet), the exported results will have to be examined by a human and compared that way.