* Packaging cleanup
Originally I was just trying to remove mentions of snupkg, but then
things got a bit carried away. :)
This is trying to remove as much duplication and dead code related to
packaging that I can.
* Apply code review feedback
* Suppress copying indirect references
* Remove unwanted bundled files from AutoML
* Remove leading slash
* Refactor model download
* Correct the packaging path of native symbols
* Rename NoTargets projects from csproj to proj
* Fix build issues around model download and respond to feedback
* Remove NoTargets file extension enforcement
* Rename proj to CSProj, include in SLN
I'd like to ensure all our projects are included in the SLN and don't
rely on separate build steps.
VS prefers *.csproj in the sln so I renamed things back to csproj.
* Respond to PR feedback
* using tensor primitives
* added missing files
* some with indexes changed
* Initial swap for TensorPrimitives done
* Rebased and cleaned code
* more minor cleanup
* added system.numerics.tensors version to props
* build fixes
* added net6 again
* updates from PR comments
* fixed sumabsu
* fixed baseline tests
* test fixes
* fixed test failure for kmeans
* changed decimal comparison
* updated more baselines
* Test fixes.
* template update
* Test Fixes.
* fixed performance test csproj
* added baselines for linux arm/64
* fixed linux arm baselines
* fixed arm baselines
* removed extra files
* arm32 baselines updated
* fixed arm baselines
* Add performance tests
* Add extra tests
* Fix
* Fix typo
* Fix Divide_Int16 and Divide_Int32_Int16 benchmarks
* Fix
* Change csproj file
* Update BenchmarkDotNetVersion to 0.13.5
* Fix
* Change to 0.13.1 because that is what is latest version in our nuget feeds.
---------
Co-authored-by: Jake Radzikowski <JakeRad@Microsoft.com>
* Attempt to retarget tests to .NET 6.0
* Fix places where tests had hardcoded framework versions
* Update build naming
Ideally we should just remove the knowledge of TargetFramework from the
build, but that can happen in a follow up change.
* Update test data for rounding differences on net6.0
* Fix condition on CPUMathNative reference in tests
It should only be included when TargetFramework is not compatible with
netcoreapp3.1.
* Ensure we generate runtimeconfig.dev.json
We need this file to tell the runtime where the NuGet package cache is.
It uses that location when we disable copying of NuGet dependencies.
* Rename test baseline output directory.
Also fix a couple places where ifdefs were hardcoded to the version of .NETCore.
* Arm helix
* yml cross fixes.
* yml cross fixes.
* yml cross fixes.
* yml cross fixes.
* yml cross testing
* Adding docker again.
* queue changes.
* queue changes.
* queue changes.
* arm pre commands
* arm pre commands
* arm pre commands
* arm pre commands
* arm pre commands
* arm pre commands
* arm pre commands
* Arm helix changes.
* Arm helix changes.
* Removing arm so PR can go.
* FIxes from PR comments. Adding arm32 back.
* Fix using statements.
* Namespace fixes.
* Added missed cmake code.
* Cmake fixes
* M1 Additions
* Changes from PR comments.
* x86x64 fact attribute.
* Fixes from PR comments.
* Fixes from PR comments.
* Fixes for Centos 7 fact and Featurizers fact.
* Code Analysis version and Not fail on M1.
* disable m1 tests in prs.
* disable m1 tests in prs.
* disable m1 tests in prs.
* disable m1 tests in prs.
* Change to full url
* Look in osx-arm64 folder for baseline files.
* Fixed -mthumb flag issue.
* Fixes from PR comments.
* Fixes from PR comments.
* Final test fixed.
* Final Fixes from PR comments.
* Initial commit for Arcade migration
* Added omitted files
* Changed strong name signing to use the same key for shipping and test assemblies
* arcade linux build (#5423)
* arcade linux build
* put file execution permission change into source control
* The `-test` command for windows. Nuget packages (#5464)
* working on testing
* testing updates
* tests almost working
* build changes
* all tests should be working
* changes from PR comments
* fixes for .net 3.1
* Fixed extension check. Removed <PackageId> where not needed
* Removed pkg folder and updated paths.
* Added test key. (#5475)
* Added test key.
* Update PublicKey.cs
Removed extra newline.
* Update ComponentCatalog.cs
Fixed 3 spaces to 4.
* Windows CI working (#5477)
* ci testing changes
* comments from pr
* Added Linux & Mac changes for Arcade (#5479)
* Initial Windows, Linux, Macos builds test
* Add Linux/MacOS specific CI requirements
* Run Arcade CI tests on MacOS/Linux
* Fix final package building
* Add benchmark download to benchmars .csporj file
* Print detailed status of each unit test
* Install CentOS & Ubuntu build dependencies
* Use container names to differenciate between Ubuntu & CentOS
* Remove sudo usage in CentOS
* Fix Linux build dependencies
* Add -y param to apt install
* Remove installation of Linux dependencies
* Minor additions
* Rename Benchmarks to PerformanceTests for Arcade
* Changes
* Added benchmark doc changes
* Pre-merge changes
* Fixing failing Arcade Windows Builds (#5482)
* Try Windows build single quote fix
* Remove %20
* Added variable space value
* Using variables for spacing
* Added space values as job parameters
* Try conditional variables again
* fix official builds
* Revert "fix official builds"
This reverts commit 7dbbdc7b94.
* fixing tensorflow rebase issue
* Fixes for many of the CI builds. (#5496)
* yml log changes
* Fix NetFX builds by ensuring assembly version is set correctly and not to Arcade default of 42.42.42.42 (#5503)
* Fixed official builds for Arcade SDK (#5512)
* Added fixes for official builds
* Make .sh files executable
* fix mkl nuget issue
Co-authored-by: Frank Dong <frdong@microsoft.com>
* fix code generator tests failure (#5520)
* Added fixes for official builds
* Make .sh files executable
* fix mkl nuget issue
* fix code generate test fails
* only add necessary dependency
Co-authored-by: Mustafa Bal <5262061+mstfbl@users.noreply.github.com>
* Fixed memory leaks from OnnxTransformer (#5518)
* Fixed memory leak from OnnxTransformer and related x86 build fixes
* Reverting x86 build related fixes to focus only on the memory leaks
* Updated docs
* Reverted OnnxRuntimeOutputCatcher to private class
* Addressed code review comments
* Refactored OnnxTransform back to using MapperBase based on code review comments
* Handle integration tests and nightly build testing (#5509)
* Make -integrationTests work
* Update .yml file
* Added the TargetArchitecture properties
* Try out -integrationTest
* Missed -integrationTest flag
* Renamed FunctionalTestBaseClass to IntegrationTestBaseClass
* Missed rename
* Modified tests to make them more stable
* Fixed leak in object pool (#5521)
Co-authored-by: frank-dong-ms <55860649+frank-dong-ms@users.noreply.github.com>
Co-authored-by: Michael Sharp <51342856+michaelgsharp@users.noreply.github.com>
Co-authored-by: Mustafa Bal <5262061+mstfbl@users.noreply.github.com>
Co-authored-by: Frank Dong <frdong@microsoft.com>
Co-authored-by: Michael Sharp <misharp@microsoft.com>
Co-authored-by: Antonio Velázquez <38739674+antoniovs1029@users.noreply.github.com>
* set max thread to unlimited, see if deadlock cause test host process hanging
* skip TensorFlowImageClassification test as this test is hanging the test host process
* make auto ml test inherient from base test class to identify which test hangs test host process
* undo automl test change, enable diagnostic log
* add new line at end
* undo unnecessary change
* Updated langversion
* Removed comment and other places where LangVersion was being set explicitly
* Moved LangVersion for fsproj to Directory.Build.props
* Enabling building ML.NET for .NET Core 3.0 in the official builds.
* Update .sln for the new configuration
* Fix native build on windows.
* Fix netfx configuration by working around an sdk bug.
* Fix CpuMath tests on netfx.
Combine the two separate CpuMath UnitTest projects into a single project.
* Install the .NET Core 2.1 runtime so we can execute tests against it.
* Remove crossgen call since the crossgen executable is no longer shipped in the NuGet package.
We don't need many things from the Tools directory, so removing crossgen won't affect build performance very much.
* moving IHostEnvironment, IExceptionContext etc from MIcrosoft.ML to Microsoft.Ml.Runtime
moving Loss related types from MIcrosoft.ML to Microsoft.Ml.Trainers
* partial namespace updates
* updating the usings to make the solution build, after the IHost changes
* adding back the semingly unused usings.
the 115 tests in CpuMathUtilsUnitTests pass locally
* restoring some readonly and casts.
Fixing 3 out of 5 analyzer test failures.
* fixing analyzer Contracts tests.
* sse and software implementations
* net461 -> netframwork, automatically copying runtime files, enables software implemenatation path, setting environmentvariables, removed remote executorHandle and Dispose
* Ficing build for netfx and adding comment
* link to the remoteexecutor original code and ordering of parameters corrected
* removing startup object
* Fixing build after mergeconflicts
* sparse vector corrected
* Removind Dead Code, correcting names, adding assert checks to correct place, span overloads and function for common code
* fixing build on unix
* cmake file corrected, if def removed from sse.cpp and unitest name modified
* Performance test corrected, resolved merge conflicts, fma supported added
* implemenatation and unitTests added
* added performance test for matmul and matmulTrans
* load combined with math operation
* add flag removed
* TransPA removed as nobody uses this combination of flags
* removed firstTime and corrected nativePerformanceTests
* removed branch from hot path sseintrinsics
* Refactor CpuMathUtils
- Allow it to take Spans instead of arrays.
- Remove redundant overloads
- When multiple spans are accepted, always use an explicit count parameter instead of one being chosen as having the right length.
Working towards #608
* Use MemoryMarshal.GetReference to avoid perf hit when pinning Span.
* PR feedback
* Stored all native benchmarks in a new class and cleaned up function names
* Style change: turn braces into one-liners
* Changed ShortRun to Default: run the perf tests longer to give more accurate results
* Added back necessary usings
* Implemented AVX intrinsics
* Implemented performance tests for AVX intrinsics, with some fixes to the intrinsics
Note: Building perf tests succeed, but running perf tests for AVX intrinsics ends without results.
* Changes to perf tests in response to feedback
* Fixes across multiple files to make unit tests and perf tests work for all used AVX intrinsics
Note: Except matrix operations
* Implemented new AVX intrinsics that do not involve matrix operations, passing basic unit tests
* Implemented perf tests for AVX via CpuMathUtils class
* Implemented switching logic for Vector128/256Alignment between SSE and AVX support
* Changed perf tests to reveal SSE and AVX intrinsics perf separately
* Fixed access modifiers of private fields
* Implemented all unit tests for AVX intrinsics that do not involve matrix operations with longer input arrays
* Implemented unit tests for AVX intrinsics
* Fixed errors on the RffTransform.CfltAlign const-expression requirement
* Fixed Debug errors by making RffTransform.CfltAlign read-only
* Fixed errors by making CfltAlign static (and read-only)
* Developed two unit tests for netcoreapp and netstandard to deal with different alignments separately, with some style changes to readonly variables
* Kept only the most recent unit tests which are sufficient for both netcoreapp and netstandard
* Respond to PR feedback: Style changes
* Respond to PR feedback: More style changes
* Implemented class inheritance in perf tests to reduce overlapping code
* Respond to PR feedback: Changed Sse/AvxIntrinsics from public to interal, adding InternalsVisibleTo attributes
* Respond to PR feedback: Used env vars to determine whether to use AVX/SSE
* Respond to PR feedback: Included 0 into consideration for parsing env vars
* Respond to PR feedback: env vars, InternalsVisibleTo, and abstract
* Respond to PR feedback: Added new comparer class specifically for MatMul
* Respond to PR feedback: Changes to intrinsics
* Respond to PR comment: Makes alignment checking consistent in external and internal calls
* Respond to PR feedback: Refactored Sse/AvxIntrinsics helper functions
* Made two Sse/AvxIntrinsics helper functions about AlignedArray inline in hopes of improving perf
* Respond to PR feedback: styles for Vector256Alignment and Avx.GetLowerHalf
* update to latest BenchmarkDotNet, x3 perf improvement
* pass the arguments to BenchmarkSwitcher to get advantage of more BDN features
* we must execute the IEnumerable to measure the actual execution time
* run not 3, but up to 20 iterations
* add README.md for the benchmarks
* it's not mandatory to run against local dotnet cli, simplify the docs
* move the BenchmarkDotNet version to Dependencies.props
* Implemented all remaining active SSE intrinsics
* Moved CpuMathUtils' architecture-dependent members into SseIntrinsics
* Implemented all new referenced SSE intrinsics, with software fallbacks, passing unit tests, and performance tests
Note: Performance tests for functions that involve AlignedArray are not implemented.
* Minor clean-up before submitting PR
* Minor changes
* Respond to PR feedback, except for implementing new unit tests (coming soon)
* Respond to PR feedback: Implemented new unit tests
* Respond to PR feedback: Style changes
* Implemented SSE support and software fallbacks for key intrinsics
* Implemented unit tests for key intrinsics with passing results
* Implemented performance tests on some key intrinsics with BenchmarkDotNet
* Fixed array pinning issues and solved unreported latency of NativeDotSU
* Minor syntax change for style consistency in fixed statements
* Implemented performance tests for all key intrinsics
* Simulated user performance with large inputs
* Allow CpuMath to reference C# Hardware Intrinsics APIs.
Need to multi-target CpuMath for netstandard and netcoreapp2.1. Also, since we are going to move CpuMath into its own NuGet package, remove the dependency from CpuMath to the ML.Core project.
* Added files for the hierarchical framework to prepare for multi-targetting.
Note: It will not compile until Microsoft.ML.CpuMath.csproj is changed to adapt to multi-targetting.
* Removed the redundant CpuMathUtils.cs file.
* Cleaned up the primitive build constant for featuring intrinsics
* Created a new helper class holding C# implementations of SSE intrinsics to simplify CpuMathUtils.DotNetCoreApp.cs
* Minor change in naming of variables
* Implemented more SSE intrinsics
* Changed version number of .NET Core App as target framework
* Cleaned up unit test file that needs to be split into two for multi-targeting
* Fixed seed in performance tests
* Cleaned up unreferenced namespaces
* Split unit tests into two projects for multi-targetting
* Cleaned up new intrinsics that are not yet tested to prepare for PR
* Minor style changes
* Added the solution package that includes multi-targeting with UseIntrinsics attribute
* Included all files in the CpuMath project for display in Visual Studio regardless of target framework
* Removed irrelevant build line from CpuMath
- due to working in Mac OSX
* Response to PR review
* Removed deprecated src\Native\CpuMath working folder
* Removed unnecessary references in unit tests
* Minor style changes
* Fixed SLN file
* Fixed build error with netcoreapp3.0 not supported
* Minor style fixes
* Skip netcoreapp3.0 projects when not building for intrinsics
* Exclude netcoreapp3.0 tests from running by overriding VSTest target
* Second response to PR feedback
* Removed NETCoreAppMaximumVersion tags with modification
* Moved VSTest targets to Empty.targets, and parsed -Intrinsics configs for Native build
* Modified VectorSum to fix perf results
* Modified VectorSum to comply with latest C# language updates
* Response to PR feedback: added a comment and removed unnecessary MSBuild tags
* Made private functions for SSE intrinsics inline
Eric StJohn
Michael Sharp
Aleksei Smirnov
Harish Kulkarni
frank-dong-ms
Mustafa Bal
Sam Harwell
Eric Erhardt
Senja Filipi
Anirudh Agnihotry
Zeeshan Siddiqui
Ivan Matantsev
Tanner Gooding
Xiangyang (Mark) Guo
Eric Erhardt
Jon Wood
Brian Lui
Adam Sitnik