Граф коммитов

49 Коммитов

Автор SHA1 Сообщение Дата
Eric StJohn 06d5b38031
Make it so validation builds don't trigger for docs changes. 2024-03-21 10:14:43 -07:00
Rich Lander d507a1e699
Update VMs (#7087)
* Update VMs

* Remove common files
2024-03-21 09:44:00 -07:00
Eric StJohn 50258301de
Update readmes for Tokenizers and Microsoft.ML (#7070)
* Make docs changes skip validation builds

* Apply package readme templates

* Fill in content for package readmes

* Address feedback
2024-03-15 13:32:23 -07:00
Michael Sharp 59973fc257
testing arm python brew overwite (#7058)
* testing arm python brew overwite

* more m1 testing

* Install python@3.12 before libomp

* Update official build libomp install command

---------

Co-authored-by: Eric StJohn <ericstj@microsoft.com>
2024-03-12 10:52:01 -06:00
dotnet-maestro[bot] f625080a07
[main] Update dependencies from dotnet/arcade (#6703)
* Update dependencies from https://github.com/dotnet/arcade build 20230519.2

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 8.0.0-beta.23269.2

* Add dotnet8 nuget feed

* Update dependencies from https://github.com/dotnet/arcade build 20230529.1

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 8.0.0-beta.23279.1

* Update dependencies from https://github.com/dotnet/arcade build 20230602.3

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 8.0.0-beta.23302.3

* Update dependencies from https://github.com/dotnet/arcade build 20230609.8

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 8.0.0-beta.23309.8

* Update dependencies from https://github.com/dotnet/arcade build 20230616.6

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 8.0.0-beta.23316.6

* Update dependencies from https://github.com/dotnet/arcade build 20230622.2

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 8.0.0-beta.23322.2

* Update dependencies from https://github.com/dotnet/arcade build 20230630.1

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 8.0.0-beta.23330.1

* Update dependencies from https://github.com/dotnet/arcade build 20230710.1

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 8.0.0-beta.23360.1

* Update dependencies from https://github.com/dotnet/arcade build 20230714.2

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 8.0.0-beta.23364.2

* Update dependencies from https://github.com/dotnet/arcade build 20230721.1

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 8.0.0-beta.23371.1

* Update dependencies from https://github.com/dotnet/arcade build 20230728.2

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 8.0.0-beta.23378.2

* Update dependencies from https://github.com/dotnet/arcade build 20230804.2

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 8.0.0-beta.23404.2

* Update dependencies from https://github.com/dotnet/arcade build 20230811.1

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 8.0.0-beta.23411.1

* Update dependencies from https://github.com/dotnet/arcade build 20230819.1

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 8.0.0-beta.23419.1

* Update dependencies from https://github.com/dotnet/arcade build 20230825.2

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 8.0.0-beta.23425.2

* Update dependencies from https://github.com/dotnet/arcade build 20230901.1

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 8.0.0-beta.23451.1

* Update dependencies from https://github.com/dotnet/arcade build 20230901.1

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 8.0.0-beta.23451.1

* Update dependencies from https://github.com/dotnet/arcade build 20230913.1

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 8.0.0-beta.23463.1

* Update dependencies from https://github.com/dotnet/arcade build 20230913.1

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 8.0.0-beta.23463.1

* Update dependencies from https://github.com/dotnet/arcade build 20230913.1

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 8.0.0-beta.23463.1

* Update dependencies from https://github.com/dotnet/arcade build 20231008.1

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 9.0.0-beta.23508.1

* Update dependencies from https://github.com/dotnet/arcade build 20231010.4

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 9.0.0-beta.23510.4

* Update dependencies from https://github.com/dotnet/arcade build 20231018.2

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 9.0.0-beta.23518.2

* Update dependencies from https://github.com/dotnet/arcade build 20231028.2

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 9.0.0-beta.23528.2

* Update dependencies from https://github.com/dotnet/arcade build 20231103.1

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 9.0.0-beta.23553.1

* Update dependencies from https://github.com/dotnet/arcade build 20231110.1

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 9.0.0-beta.23560.1

* Update dependencies from https://github.com/dotnet/arcade build 20231117.1

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 9.0.0-beta.23567.1

* Fixed version update breaks.

* Update dependencies from https://github.com/dotnet/arcade build 20231122.2

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 9.0.0-beta.23572.2

* Update dependencies from https://github.com/dotnet/arcade build 20231201.1

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 9.0.0-beta.23601.1

* Update dependencies from https://github.com/dotnet/arcade build 20231207.2

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 9.0.0-beta.23607.2

* Update dependencies from https://github.com/dotnet/arcade build 20231215.2

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 9.0.0-beta.23615.2

* Update XUnitVersion

* Update MicrosoftMLOnnxRuntimeVersion to 1.16.3

* Rollback OnnxRuntime and suppress warning

* Update to Xunit with fix for https://github.com/xunit/xunit/issues/2821

* Ensure we pull down 8.0 runtime.

* Update Centos docker containers

* Fix packaging step

* Try including stdint.h to fix missing uint8_t on centos

* Update Centos test queue

* Attempt to use runtime centos-stream8-helix container for tests

* Use centos-stream8-mlnet-helix container for testing

* Undo changes to test data

* Make NETFRAMEWORK ifdef versionless

* Switch back to centos7 for testing

* Revert "Switch back to centos7 for testing"

This reverts commit ab0d41e4b7.

* Update dependencies from https://github.com/dotnet/arcade build 20231221.2

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 9.0.0-beta.23621.2

* Update dependencies from https://github.com/dotnet/arcade build 20231228.1

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XliffTasks , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 9.0.0-beta.23628.1

---------

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: Eric StJohn <ericstj@microsoft.com>
Co-authored-by: Michael Sharp <misharp@microsoft.com>
2024-01-02 15:46:00 -08:00
Michael Sharp db08da61b8
added in win-arm64 (#6813)
* added in win-arm64

* fixed MKL arm64 cmake issue

* right helix queue

* added in win-arm64

* fixed MKL arm64 cmake issue

* right helix queue

* fixing arm tests

* makes x64 test detection better

* change test label

* fixed onnx files not being included

* added new win-arm baselines

* baseline changes

* fixed build issue

* fixed test

* one more basleine

* fixed pack for mkl redist

* .NET update
2023-11-20 22:05:11 -07:00
Michael Sharp d2cf997d90
Changes some of the CPU Math implemenation from our current version to use the new TensorPrimitives package. (#6875)
* using tensor primitives

* added missing files

* some with indexes changed

* Initial swap for TensorPrimitives done

* Rebased and cleaned code

* more minor cleanup

* added system.numerics.tensors version to props

* build fixes

* added net6 again

* updates from PR comments

* fixed sumabsu

* fixed baseline tests

* test fixes

* fixed test failure for kmeans

* changed decimal comparison

* updated more baselines

* Test fixes.

* template update

* Test Fixes.

* fixed performance test csproj

* added baselines for linux arm/64

* fixed linux arm baselines

* fixed arm baselines

* removed extra files

* arm32 baselines updated

* fixed arm baselines
2023-11-14 22:46:15 -07:00
Jake 4c799ab1c8
Update build templates to handle feature branches (#6744)
* Update build templates

* Update build templates to include all releases/* and feature/*

* Update releases to release

* Update triggers for PR Validation Build

* Add triggers for Code Coverage
2023-06-27 17:02:56 -07:00
Eric StJohn c8b3ca4720
Attempt to retarget tests to .NET 6.0 (#6367)
* Attempt to retarget tests to .NET 6.0

* Fix places where tests had hardcoded framework versions

* Update build naming

Ideally we should just remove the knowledge of TargetFramework from the
build, but that can happen in a follow up change.

* Update test data for rounding differences on net6.0

* Fix condition on CPUMathNative reference in tests

It should only be included when TargetFramework is not compatible with
netcoreapp3.1.

* Ensure we generate runtimeconfig.dev.json

We need this file to tell the runtime where the NuGet package cache is.
It uses that location when we disable copying of NuGet dependencies.

* Rename test baseline output directory.

Also fix a couple places where ifdefs were hardcoded to the version of .NETCore.
2022-10-18 08:29:37 -07:00
Michelle McDaniel 470593c7da
Use latest docker containers (#6365) 2022-10-12 12:34:19 -07:00
Michael Sharp d2e2f4fe1a
testing pool change for PR (#6343) 2022-10-04 22:29:58 -07:00
Michael Sharp 632c373169
mac os 12 fix (#6342) 2022-10-03 12:53:59 -07:00
Michael Sharp 791e37ed70
Multi-targeting with TargetFrameworks (#6003) 2021-11-17 13:36:29 -08:00
Michael Sharp fb2adf84a4
Ubuntu 18.04 queues (#5970) 2021-10-13 18:07:49 -07:00
Michael Sharp 34eb983687
Arm64 build/test/publish. (#5860)
* Arm helix

* yml cross fixes.

* yml cross fixes.

* yml cross fixes.

* yml cross fixes.

* yml cross testing

* Adding docker again.

* queue changes.

* queue changes.

* queue changes.

* arm pre commands

* arm pre commands

* arm pre commands

* arm pre commands

* arm pre commands

* arm pre commands

* arm pre commands

* Arm helix changes.

* Arm helix changes.

* Removing arm so PR can go.

* FIxes from PR comments. Adding arm32 back.

* Fix using statements.

* Namespace fixes.

* Added missed cmake code.

* Cmake fixes

* M1 Additions

* Changes from PR comments.

* x86x64 fact attribute.

* Fixes from PR comments.

* Fixes from PR comments.

* Fixes for Centos 7 fact and Featurizers fact.

* Code Analysis version and Not fail on M1.

* disable m1 tests in prs.

* disable m1 tests in prs.

* disable m1 tests in prs.

* disable m1 tests in prs.

* Change to full url

* Look in osx-arm64 folder for baseline files.

* Fixed -mthumb flag issue.

* Fixes from PR comments.

* Fixes from PR comments.

* Final test fixed.

* Final Fixes from PR comments.
2021-07-12 15:14:54 -07:00
Michael Sharp ce7f91a2c1
Helix Integration. (#5837)
* Helix working. Squashed local commits

* Updates from PR comments.

* Updates from PR comments.

* Update Directory.Build.targets

Co-authored-by: Santiago Fernandez Madero <safern@microsoft.com>

* Fixes from pr comments.

* Switched tabs to spaces.

* Helix changes from pr comments.

* yml changes.

* yml changes.

* yml changes.

* yml changes.

* yml changes.

* Indentation fixes.

* Indentation fixes.

Co-authored-by: Santiago Fernandez Madero <safern@microsoft.com>
2021-06-22 19:44:27 -07:00
Harish Kulkarni c2f8ed5c9c
Merge arcade to master (#5525)
* Initial commit for Arcade migration

* Added omitted files

* Changed strong name signing to use the same key for shipping and test assemblies

* arcade linux build (#5423)

* arcade linux build

* put file execution permission change into source control

* The `-test` command for windows. Nuget packages (#5464)

* working on testing

* testing updates

* tests almost working

* build changes

* all tests should be working

* changes from PR comments

* fixes for .net 3.1

* Fixed extension check. Removed <PackageId> where not needed

* Removed pkg folder and updated paths.

* Added test key. (#5475)

* Added test key.

* Update PublicKey.cs

Removed extra newline.

* Update ComponentCatalog.cs

Fixed 3 spaces to 4.

* Windows CI working (#5477)

* ci testing changes

* comments from pr

* Added Linux & Mac changes for Arcade (#5479)

* Initial Windows, Linux, Macos builds test

* Add Linux/MacOS specific CI requirements

* Run Arcade CI tests on MacOS/Linux

* Fix final package building

* Add benchmark download to benchmars .csporj file

* Print detailed status of each unit test

* Install CentOS & Ubuntu build dependencies

* Use container names to differenciate between Ubuntu & CentOS

* Remove sudo usage in CentOS

* Fix Linux build dependencies

* Add -y param to apt install

* Remove installation of Linux dependencies

* Minor additions

* Rename Benchmarks to PerformanceTests for Arcade

* Changes

* Added benchmark doc changes

* Pre-merge changes

* Fixing failing Arcade Windows Builds (#5482)

* Try Windows build single quote fix

* Remove %20

* Added variable space value

* Using variables for spacing

* Added space values as job parameters

* Try conditional variables again

* fix official builds

* Revert "fix official builds"

This reverts commit 7dbbdc7b94.

* fixing tensorflow rebase issue

* Fixes for many of the CI builds. (#5496)

* yml log changes

* Fix NetFX builds by ensuring assembly version is set correctly and not to Arcade default of 42.42.42.42 (#5503)

* Fixed official builds for Arcade SDK (#5512)

* Added fixes for official builds

* Make .sh files executable

* fix mkl nuget issue

Co-authored-by: Frank Dong <frdong@microsoft.com>

* fix code generator tests failure (#5520)

* Added fixes for official builds

* Make .sh files executable

* fix mkl nuget issue

* fix code generate test fails

* only add necessary dependency

Co-authored-by: Mustafa Bal <5262061+mstfbl@users.noreply.github.com>

* Fixed memory leaks from OnnxTransformer (#5518)

* Fixed memory leak from OnnxTransformer and related x86 build fixes

* Reverting x86 build related fixes to focus only on the memory leaks

* Updated docs

* Reverted OnnxRuntimeOutputCatcher to private class

* Addressed code review comments

* Refactored OnnxTransform back to using MapperBase based on code review comments

* Handle integration tests and nightly build testing (#5509)

* Make -integrationTests work

* Update .yml file

* Added the TargetArchitecture properties

* Try out -integrationTest

* Missed -integrationTest flag

* Renamed FunctionalTestBaseClass to IntegrationTestBaseClass

* Missed rename

* Modified tests to make them more stable

* Fixed leak in object pool (#5521)

Co-authored-by: frank-dong-ms <55860649+frank-dong-ms@users.noreply.github.com>
Co-authored-by: Michael Sharp <51342856+michaelgsharp@users.noreply.github.com>
Co-authored-by: Mustafa Bal <5262061+mstfbl@users.noreply.github.com>
Co-authored-by: Frank Dong <frdong@microsoft.com>
Co-authored-by: Michael Sharp <misharp@microsoft.com>
Co-authored-by: Antonio Velázquez <38739674+antoniovs1029@users.noreply.github.com>
2020-12-02 17:13:27 -08:00
Mustafa Bal 7d8c85b1f9
Update Ubuntu build and reenable test on Linux (#5137) 2020-05-15 15:59:55 -07:00
frank-dong-ms fd6f86cd95
test move back to host see if still hangs (#5108)
* test

* clean test data after test running

* also try move ubuntu pool back to hosted

* nit: add tale new line

* skip downloading some benchmark files that not used in CI benchmark

* use symbol link instead

* add step to show disk usage

* try use hardlink instead of symbol link

* simply code

* fix in proj
2020-05-08 23:39:09 -07:00
frank-dong-ms a190ce7745
move CI windows build to netcore public pool (#5043)
* move CI windows build to netcore public pool

* try set SecurityProtocol at static constructor of BaseTest class as this setting is per app domain
2020-04-21 19:59:34 -07:00
frank-dong-ms fbea448b7e
move ubuntu and centos to agent pool with larger disk space (#5024) 2020-04-13 17:38:07 -07:00
Mustafa Bal 94af8262f2
Updates NetCoreApp 3.0 builds to NetCoreApp 3.1 (#4888)
* Update .vsts-dotnet-ci.yml

* Changed 3.0 references to 3.1

* Update DotnetCLIVersion.txt

* Update Microsoft.ML.sln

* Update BuildToolsVersion.txt

* Update DotnetCLIVersion.txt

* Fixing small typos
2020-03-23 15:07:30 -07:00
Mustafa Bal 1d904397eb
Added hanging test mem dump feature (#4931)
* Added ability to collect memory dumps on hanging and crashing tests

Added ability to collect memory dumps on hanging and crashing tests through VSTest Tasks and Procdump. Also added an example VSTest version of the ColumnTypes XUnit test file.

* Update ColumnTypesVSTask.cs

* Improvements

* Build changes

* Removed example VSTest and added explanation in developer guide for collecting memory dumps

* Added downloading procdump.zip instead of adding to repo directly
2020-03-13 12:01:58 -07:00
frank-dong-ms ce0041e047
upgrade hosted mac agent pool to use new version of mac os (#4897) 2020-02-27 10:02:09 -08:00
frank-dong-ms 8b31f62edd
fix nightly-build, centos is building with netcore3.0 not netcore2.1, add targetFramework parameter (#4860) 2020-02-20 14:35:18 -08:00
frank-dong-ms 9f276123c0
not copy nuget package to build ouput inorder to save disk space (#4854) 2020-02-19 04:23:17 +00:00
frank-dong-ms 97c4533df0
separate build pipelines (#4688)
* remove flaky test from CI

* refine text

* remove timeout setting

* also remove benchmarktest from inner loop

* fix skip theory

* fix define constants

* skip benchmark properly

* skip tests in compile time

* skip one more tensorflow test from CI
2020-01-27 13:43:08 -08:00
Zeeshan Siddiqui 7cb9893252
Delete build files before packaging and increase build timeout (#4377)
* Revert "Move windows build machines to NetCorePublic-Pool. (#4375)"

This reverts commit c922529e66.

* Add step to cleanup test data before building packages to free up space

* Increase build timeout.
2019-10-25 11:04:01 -07:00
Zeeshan Siddiqui c922529e66
Move windows build machines to NetCorePublic-Pool. (#4375)
* Move windows build machines to NetCorePublic-Pool.

* PR feedback.

* PR feedback.
2019-10-24 11:36:28 -07:00
Eric Erhardt 3b9528373a
Update official build and CI to run on macOS 10.13. (#4108)
* Update official build and CI to run on macOS 10.13.

Also update the building instructions to reflect the new requirement to build for .NET Core 3.0.

* Update template for the new pool name.
2019-08-14 10:41:32 -05:00
Eric Erhardt 365ccf2927
Enabling building ML.NET for .NET Core 3.0 in the official builds. (#4096)
* Enabling building ML.NET for .NET Core 3.0 in the official builds.

* Update .sln for the new configuration

* Fix native build on windows.

* Fix netfx configuration by working around an sdk bug.

* Fix CpuMath tests on netfx.

Combine the two separate CpuMath UnitTest projects into a single project.

* Install the .NET Core 2.1 runtime so we can execute tests against it.

* Remove crossgen call since the crossgen executable is no longer shipped in the NuGet package.

We don't need many things from the Tools directory, so removing crossgen won't affect build performance very much.
2019-08-13 16:03:15 -05:00
Santiago Fernandez Madero e7c5858a52 Move from phases to jobs and use bring your own cloud pool (#3908)
* Move from phases to jobs and use bring your own cloud pool

* Fix queue

* Fix build
2019-06-26 20:58:52 -07:00
Anirudh Agnihotry 802edc7b4a
Renaming CI legs (#3553)
* reNaming CI legs

* correcting mac build

* adding pascal casing and Version to netfx

* R -> RI for centos
2019-04-26 11:21:46 -07:00
Scott Inglis 3ad489ef77
Updating MKL (#2867)
Enables OpenMP in both ML.Net and in our usage of Intel's MKL library. This also upgrades the reference of MKL to use version 2018.3.10 for Windows, Linux and Mac. The MKL binaries are packaged in MLNetDeps nuget, with this change we are referencing 0.0.9. 
As a result of these changes, there were other changes that needed to happen:
* Build needed to support openmp, for linux, this updated our docker build containers to have openmp installed as part of the clang compiler installation. For mac, openmp was already installed this now occurs though before the build. 
*  Enabled NumberOfThreads parameter for the SymSGDTrainer, NumberOfThreads now defaults to Environment.ProcessorCount
2019-03-18 12:23:03 -07:00
Anirudh Agnihotry d65af0f755 Update build tools version and use netcoreapp3.0 on ubuntu CI (#2687)
* temp

* running ubuntu leg on netcoreapp3.0

* correcting names in build definations

* intrinsics environment variables name corrected
2019-03-01 17:18:32 -06:00
Anirudh Agnihotry 13f6ff9b78
Adding a new ubuntu CI leg (#2516)
* addding a new ci lef for ubuntu

* Adding netcore3.0 windows ci, removing extra compile flags, using proper docker image tag

* correcting the argument number

* using netcoreapp2.1 for ubuntuCI, using getlibcversion and runtime infotmation

* using version to compare version numbers
2019-02-19 14:18:06 -08:00
Anirudh Agnihotry 6b9f5893ea
Added Dry job for benchmarks CI (#2109)
* Added Dry job for benchmarks ci

* Increasing Timeout and not copying large datasets to output directory

* disabling failing benchmark tests

* using GetDataPath function and simplyfying code

* Name changed to datapathprovider, make variables private, make xunit variable public

* making can execute static

* Removed Datasets from output and dead import

* Dynamically finding the root of the repo

* using Assembly.load, making getrepo root private, removing touching dependencyTest

* using typeof.assembly, adding summary comment to attribute and adding using statement for TestFramework
2019-01-15 12:28:28 -08:00
Anirudh Agnihotry 64b60d2805
Adding a new CI leg for netframework (#2018)
* adding a new ci leg for netfx

* Update test/Directory.Build.props

Co-Authored-By: Anipik <anirudhagnihotry098@gmail.com>

* configuration corrected in ci defination

* comments added for tracking issues and multi targt temoved for fsharp project
2019-01-10 09:33:42 -08:00
Anirudh Agnihotry 3d33e20f33
Adding a new CI leg for netcoreapp 3.0 (#1700)
* adding new ci leg

* removing .

*  Fixing and disabling tests for netcoreapp3.0

* skipping some more tests and name change

* added comments and conditional fact

* name changed to NotNetCore30

* name changed and reding cli version from the file

* name changed to DotnetCLIVersion.netcoreapp.latest.txt

* contracts assert cprrected for avx=intrinscis

fixing new tests broken by other commits

Fixing Tests
2018-11-28 12:59:44 -08:00
Artidoro Pagnoni 38d3b59f68
Make whitening tests work on hosted macs (#1530) 2018-11-06 10:30:10 -08:00
Artidoro Pagnoni 216f672711
Introducing PR verification x86 CI build (#1309)
* testing installation of tools

* passing all arguments

* fixing syntax

* fixing syntax

* making sure tensorflow tests don't run in x86 pr validation ci leg

* fixing test by adding new regular expression in normalizer

* temporarily skipping benchmark test not working on x86 and fixed packaging issue with tensorflow

* skipping on debug
2018-10-26 09:32:54 -07:00
Artidoro Pagnoni 106a84cf2d
Introducing official x86 CI build (#1306) 2018-10-25 12:56:22 -07:00
Eric Erhardt 3a1fa0f7fa Update build yaml to use official container functionality (#1118) 2018-10-02 22:05:17 -07:00
Eric Erhardt 875ef00440
Add a workaround for the tests hanging while loading MKL. (#1076)
* Add a workaround for the tests hanging while loading MKL.

The workaround is to ensure the MKL library is loaded very early in the test process, so it doesn't cause the deadlock.

Workaround #1073

Another deadlock also occurs when running TestAutoInference and TestPipelineSweeper in parallel. Marking these tests to not run in parallel anymore.

Workaround #1095

Moving back to the Azure Hosted VS2017 pool to run the tests now that we've narrowed the deadlocks down.
2018-10-01 13:20:21 -05:00
Eric Erhardt d7b062de20 Update our Windows CI leg to use the non-Hosted Windows queue (#1030) 2018-09-26 08:14:55 -07:00
Santiago Fernandez Madero 5a6fdedcc2
Move macOS CI legs to VSTS (#727)
* Move macOS CI builds to VSTS:public-CI

* Update README badges
2018-08-29 15:20:27 -07:00
Santiago Fernandez Madero b21094d649
Remove agent.os demands on windows ci (#621)
* Remove agent.os demands as this queue has only Windows_NT machines

* Bump parallel to 4

* Set parallel to 99 per PR feedback
2018-08-02 13:01:38 -07:00
Santiago Fernandez Madero e885b73f9c
Fix Linux CI to actually run inside a docker container (#574) 2018-07-23 14:45:16 -07:00
Santiago Fernandez Madero a862ccc5a8 Move Windows and Linux CI to VSTS (#566)
* Remove Windows and Linux configurations from netci.groovy

* Add end of line to yml files

* Add badges and change leg name to Linux

* Not merge test results

* Add searchFolder to publish test results task
2018-07-23 09:38:21 -07:00