Граф коммитов

300 Коммитов

Автор SHA1 Сообщение Дата
Cameron Aavik 88a140a223
Pass argument into scenarios to enable uploading results (#4485)
* WIP work on fixing upload condition

* Pass in upload to perf lab arg to all scenarios
2024-10-09 07:20:33 +10:00
Parker Bibus c7c287eaaf
Fix info log format to properly use lazy formatting to keep run_performance_job from throwing an error in the middle of execution. The error was not impactful, just kept the current dotnet directory from being printed to user and dirtying the run logs. (#4495) 2024-10-01 22:00:00 -07:00
Parker Bibus 3087b49821
Update the libgdiplus to point to the community for the current Alpine version we have (3.18) (#4476)
Update the libgdiplus to point to the community for the current Alpine version we have (3.18) as libgdiplus is now part of the community and not testing.
2024-09-24 09:18:15 -07:00
Adeel Mujahid d384536c59
Parse version > 9 in dotnet.py (#4470)
* Parse version > 9 in dotnet.py

* More fixes

* Print sdk dirs found in the CI

* Update scripts/dotnet.py

* Move diag to exception message

* Delete special condition (since we were not using the fork earlier)

* Reapply the workaround (needed after all=

* Allow more digits on the 1st position of version.

* return 10.0 for nativeaot9.0

* Update azure-pipelines.yml

* revert 9.0->10.0

* Update channel_map.py

* Update Data.cs

* make regex non-greedy

* Address feedback

---------

Co-authored-by: Ilona Tomkowicz <32700855+ilonatommy@users.noreply.github.com>
2024-09-24 06:28:46 +10:00
Cameron Aavik b43446a698
Fix configs for wasm aot and mono aot (#4469)
* Fix configs for wasm aot and mono aot

* Fix check in other places

* Address PR comments

* Don't pass corerun on Mono AOT
2024-09-21 03:13:37 +10:00
Cameron Aavik 6722ac3d71
Fix sanity check arg (#4464) 2024-09-18 06:32:15 +10:00
Cameron Aavik 254b737747
Supporting changes to implement python pipeline for performance runs on runtime repo (#4156)
* Make changes to python pipeline for runtime repo

* More fixes

* Clean up some parts

* Add latest changes from runtime repo

* Fix wasm bundle path and add arg support for local commit time

* Fix local commit time code

* Only set core_root for coreclr runs

* Fix ios hello world copying

* Fix wasm copying and prereq check on linux

* Fix wasm args quoting and PreparePayloadWorkItems usage

* Use new browser versions path

* Fix prerequisites script building

* Fix wasm args building

* Fix directory creation and prevent pip reinstalls

* Fix trailing space on wasm args

* More various fixes

* Fix file copy into root

* Fix FailOnTestFailure value

* Include recent changes from runtime repo

* Only build startup and SOD for runtime scenarios

* Fix infinite loop in zip creation

* Fix some directory issues

* Fix some copytree logic

* Copy ios apps into correct dir

* Fix iosSampleApp copying + add better logging

* Fix PDN download and error code

* Address PR comments

* Fix incorrect tfm condition check

* Address PR comments
2024-09-17 07:17:05 +10:00
Parker Bibus fc3a30a094
Add release/9.0 and matching 10.0 channel maps. (#4439) 2024-09-09 09:31:32 -07:00
Parker Bibus 0618d71d60
Update Maui Versions and name for runtime version (#4396)
Update Maui versions and rollback channels.
2024-08-15 10:20:18 -07:00
Parker Bibus 7ac5c58b29
Update Maui version flows (#4321)
Updated how we get the dotnet version and rollback file for Maui .Net testing. Specifically updated from using maui generated information stored as json at certain urls to generating it based on maestro/darc updated version information now stored in the dotnet/performance Version.Details.XML. This means that we will be in charge of merging the maestro PRs for these versions as well as meaning we will have the option to further gate and better track versions between runs.
2024-08-01 14:32:40 -07:00
Parker Bibus c6ad24d7f8
Remove Rlcse experiment (#4353) 2024-07-30 15:37:04 -07:00
Cameron Aavik 6151741fab
Enable log verbosity on seutp build commands (#4347) 2024-07-30 08:36:11 +10:00
Cameron Aavik 2068136cb1
Use correct python executable inside venv for perf tests on Windows (#4339)
* Use correct python executable inside venv

* Put "echo on" on windows, not linux
2024-07-26 07:26:44 +10:00
Parker Bibus 6f22a8b0bb
Add OpenTelemetry capabilities to python scripts (#4063)
* Attach basic log provider to the python logging we already have setup.

* Add more tracer spans and include them in ci_setup.py

* Add tracer.py helper file.

* Add opentelemetry api and sdk to install precommands.

* Wrap opentelemetry logging in try except so that normally logging still works without opentelemetry installed.

* Add support for the logging and tracing libraries to be imported and in code even without OpenTelemetry installed on the machine. This was accomplished by making no-op flows for when OpenTelemetry is not installed, and creating an AwareTracer that no-ops the Tracer start_as_current_span decorator when OTel is not installed.
2024-07-23 09:05:36 -07:00
Cameron Aavik 353f629ec0
Add zip file of BenchmarkDotNet.Artifacts directory to Helix Upload Root (#4325)
* Zip BDN Artifacts to helix upload root

* Fix file name
2024-07-23 20:56:11 +10:00
Cameron Aavik 9086621167
Bump urllib3 from 1.26.18 to 1.26.19 (#4319) 2024-07-20 13:18:30 +10:00
Cameron Aavik b16e62227b
Download PDN with Managed Identity (#4287)
* Use managed identity to download PDN

* fix indentation in yaml

* Specify auth mode and place PDN zip in working directory

* Fix pdn path

* Fix broken escape sequence

* Move condition to if statement, use artifact staging dir

* Use condition instead of if statement

* Fix console arg quoting
2024-07-02 05:47:19 +10:00
Parker Bibus 04a3afef57
Update azure.storage.blob/queue and azure.identity pip install packages. (#4263) 2024-06-13 09:52:25 -07:00
Drew Scoggins b459418f82
Reapply "Add DAC as a cred source before we use SAS token (#4216)" (#4252) (#4257)
* Reapply "Add DAC as a cred source before we use SAS token (#4216)" (#4252)
2024-06-10 10:31:43 -07:00
Parker Bibus a66ca3e1ca
Use full dotnet version instead of the version in file path (#4251)
Updated get_dotnet_version to get_dotnet_version_from_path which gets the latest dotnet version from the folder names in sdk path, and added get_dotnet_version_precise to get the precise version from the .version file available in net6.0 and above.
2024-06-06 09:07:33 -07:00
Parker Bibus 40a4000a4c
Revert "Add DAC as a cred source before we use SAS token (#4216)" (#4252)
This reverts commit efa42472bf.
2024-06-05 11:04:58 -07:00
Drew Scoggins efa42472bf
Add DAC as a cred source before we use SAS token (#4216)
* Add DAC as a cred source before we use SAS token

* Update creds to use federated model

* Use correct app id

* Fix scopes for federated credential

* Add client and tenant id as constants
2024-06-04 15:39:25 -07:00
Parker Bibus 1ec0cee4e6
Add a fallback to the archived installer repo for commit date checking (#4239)
* Add a fallback to the installer repo for commit patch checking for dotnet-performance runs for during the clearing of the runs referencing commits in the installer repo.

* Set fallback_url to None to start and set it to the normal url when a repository is passed.
2024-06-03 11:24:15 -07:00
Parker Bibus 735cf2216f
Update core-sdk url to point at sdk instead. (#4231)
* Update core-sdk url to point at sdk instead and update the rest of the core-sdk non-variable name instances.
2024-05-22 17:11:54 -07:00
Drew Scoggins 40b2861b27
Switch queue upload message to JSON (#4206)
* Remove SAS token from uploaded queue message

* Update to just send blob name

* Switch to using JSON object for queue payload

* Fix serialization issue
2024-05-09 14:18:46 -07:00
Andy Ayers 40e75af0b8
fix rpolayout config setting (#4208) 2024-05-09 09:55:42 -07:00
Aman Khalid 49ca9d1bf8
Remove old experiments; add layout experiment (#4200) 2024-05-06 10:03:07 -07:00
Parker Bibus de97739e8d
Fix Wasm benchmark_local setup (#4118)
Updated the paths used in wasm runs for generating wasm-data, moved dotnet-path to be defined individually in each of the single benchmark cases so wasm AOT and interp use the wasm_bundle dotnet, and added RestoreAdditionalProjectSources for WasmAOT and WasmInterpreter tests so the built packages are found.
2024-04-09 13:19:17 -07:00
Bruce Forstall a5cc38b160
Enable JitOptRepeat as a perf experiment (#4121) 2024-04-03 14:01:14 -07:00
Parker Bibus 482ffb53f1
Setup performance-ci runs to only use the 9.0 and 8.0 branch from channel map when running for main to ensure non-main runs don't get included in main runs. (#4054) 2024-03-19 11:01:24 -07:00
Cameron Aavik f54f703c84
Don't set branch on performance CI runs (#4031) 2024-03-07 13:28:20 -08:00
Parker Bibus 12a95aa594
Don't always write MAUI_VERSION and ensure version env vars have value (#3967)
* Only add entries to Additional data if they have values and don't write MAUI_VERSION env var in ci_setup if maui_version is not passed.
* Set all platforms to set version env vars in uppercase to match the functionality on Windows.
* Also have the reporter set MAUIVERSION to all uppercase.
2024-02-28 09:33:52 -08:00
Andy Ayers 06fc49ab36
Add setup for RLCSE experiment (#3975) 2024-02-22 22:08:10 -08:00
Parker Bibus 8ea9fe8eb2
Minor benchmarks_local.py cleanup (#3946)
Formatting and cleanup of benchmarks_local.py.
2024-02-20 10:23:27 -08:00
Parker Bibus e7d3ff259d
Benchmarks_local disable terminating of all dotnet processes (#3934)
* Update benchmarks_local with the default of disabling Node Reuse and switched default of killing dotnet to not killing dotnet. Added arguments to reenable each of these options, but this new default will allow the script to run completely while also using the machine for other dotnet/msbuild tasks without interruption. Updated the benchmark_local README with these new updates.

* Moved get_mono_corerun duplicated logic to its own function and updated the artifact path to include the framework version being run so you can run net9.0 and then net8.0.

* Fix bug with leftover dotnet_mono files.
2024-02-15 10:14:20 -08:00
Parker Bibus 9ededaa2cb
Fix WASM benchmarks_local generation (#3935)
* Move wasm-tools install so that it is always installed when running wasm jobs.
* Add support for pre and post wasm -> browser folder move.
* Remove the EMSDK setup as it is already automatic and update the test-main.js file find for both wasm and browser directory paths.
2024-02-15 09:55:51 -08:00
Cameron Aavik 06b0d49c84
Set _BuildConfig earlier (#3903) 2024-02-13 11:36:54 +10:00
Cameron Aavik 9a6ab436db
Consolidate yaml/script/proj logic into a single python script that is reusable across all contexts (#3806)
* CI Python Pipeline

* Fix yaml

* More bug fixes

* Ensure helix access token is propagated

* Ensure variables are always defined in yaml

* Fix env var parsing

* Prevent passing in commit sha for performance ci runs

* Ensure root payload dir is created

* Fix Nuget.config copy code

* More bug fixes for scenarios runs

* Remove pip install and ensure payload dir has ending path segment

* Also ensure dotnet dir has path segment

* Some more fix attempts

* Install python packages on osx, fix runtime flavour argument

* Try pass perf command token via env

* Fix PDN output file path being treated as a directory

* Try fix scenarios directories

* Install urllib last so that it doesn't get reinstalled by azure storage packages

* Don't install packages at the user-level as it will be in a venv

* Some more fixes

* Fix more bugs

* Delete the right directory

* Disable node reuse and fix escaping for azure pipelines variables

* Don't run apt-get on osx

* Debugging the env var

* Fix duplicate upload token

* Ensure _BuildConfig is set

* Default to not internal and fix pdn param

* Disable prerequisite check if no prerequisites

* Fix some more warnings

* Fix not_in_lab value for performance CI runs

* Ensure PerfLabGenericEventSourceLTTngProvider is build on ubuntu

* Remember to run the command

* Ensure updated global.json is copied to payload

* Support runCategories and csproj in benchmark_jobs.yml

* Fix typo in yaml

* Try surround csproj path with quotes to keep backslash

* Convert target csproj path to target OS path

* Fix performance directory issues

* Fix SYSTEM_ACCESSTOKEN not being propagated

* Add binlog output when building all scenario dotnet projects

* Update scripts/run_performance_job.py

Co-authored-by: Parker Bibus <pbibus@hotmail.com>

---------

Co-authored-by: Parker Bibus <pbibus@hotmail.com>
2024-02-07 08:02:44 +10:00
Parker Bibus 1d11344caa
Fix 8.0 runs by replacing global.json with net8.0 compatible global.json (#3853)
Setup the replacing of the global.json with the new global.net8.json file when running ci_setup and targetting net8.0 or nativeaot8.0. Also reenabled the performance-ci 8.0 runs.
2024-02-01 09:59:03 -08:00
Jiri Cincura ↹ f2e017e804
Setup for GDV with max type checks 3 (part of #3745) (#3776) 2024-01-18 20:07:06 +01:00
Parker Bibus a72a41ae54
Update urllib to 1.26.18. (#3775) 2024-01-18 09:38:18 -08:00
Parker Bibus 51d85c3726
Add nativeaot9.0 runs and some other 9.0 framework parsing support (#3587)
Added net9.0 support to moniker retrieval in ResultsComparer, added nativeaot9.0 to the channel map, and added run for nativeaot9.0.
2023-12-12 12:41:03 -08:00
Jiri Cincura ↹ cecc4ed54e
Add CrossBlockLocalAssertionProp config. (#3495) 2023-11-15 20:45:42 +01:00
Jiri Cincura ↹ 2525ba2571
Add support for specifying "experiment name". (#3489) 2023-11-15 14:58:26 +01:00
Jiri Cincura ↹ e95893374f
Cleanup and fix some R2R and Physical Promotion configs. (#3487) 2023-11-13 19:27:48 +01:00
Parker Bibus 3d5cdf00e6
Add initial README for the benchmarks_local.py script. (#3468) 2023-11-08 11:18:59 -08:00
Jiri Cincura ↹ d30e0d9984
Cleanup in ci_setup.py (#3471)
* Unify variable names.

* Use DOTNET_ instead of older COMPlus_ env vars.
2023-11-07 19:02:34 +01:00
Jiri Cincura ↹ 5f37f34b25
Add option for R2R configured runs. (#3467) 2023-11-07 09:40:35 +01:00
Parker Bibus 8779ed6e3b
Update benchmarks_local default framework from net8.0 -> net9.0. (#3463) 2023-11-02 10:41:24 -07:00
Parker Bibus 6396fdf8e0
Add clarification that bdn-arguments must be passed with an equals sign. (#3434) 2023-10-23 11:21:55 -07:00