* Packaging cleanup
Originally I was just trying to remove mentions of snupkg, but then
things got a bit carried away. :)
This is trying to remove as much duplication and dead code related to
packaging that I can.
* Apply code review feedback
* Suppress copying indirect references
* Remove unwanted bundled files from AutoML
* Remove leading slash
* Refactor model download
* Correct the packaging path of native symbols
* Rename NoTargets projects from csproj to proj
* Fix build issues around model download and respond to feedback
* Remove NoTargets file extension enforcement
* Rename proj to CSProj, include in SLN
I'd like to ensure all our projects are included in the SLN and don't
rely on separate build steps.
VS prefers *.csproj in the sln so I renamed things back to csproj.
* Respond to PR feedback
* arm testing
* initial commit with build working on arm64
* windows changes
* build fixes for arm/arm64 with cross compilation
* cross build instructions added
* renamed arm to Arm. Changed TargetArchitecture to default to OS architecture
* fixed some formatting
* fixed capitilization
* fixed Arm Capitilization
* Fix cross-compilation if statement
* building on apple silicon
* removed non build related files
* Changes from PR comments. Removal of FastTreeNative flag.
* Changes from pr comments.
* Fixes from PR comments.
* Changed how we are excluding files.
* Initial commit for Arcade migration
* Added omitted files
* Changed strong name signing to use the same key for shipping and test assemblies
* arcade linux build (#5423)
* arcade linux build
* put file execution permission change into source control
* The `-test` command for windows. Nuget packages (#5464)
* working on testing
* testing updates
* tests almost working
* build changes
* all tests should be working
* changes from PR comments
* fixes for .net 3.1
* Fixed extension check. Removed <PackageId> where not needed
* Removed pkg folder and updated paths.
* Added test key. (#5475)
* Added test key.
* Update PublicKey.cs
Removed extra newline.
* Update ComponentCatalog.cs
Fixed 3 spaces to 4.
* Windows CI working (#5477)
* ci testing changes
* comments from pr
* Added Linux & Mac changes for Arcade (#5479)
* Initial Windows, Linux, Macos builds test
* Add Linux/MacOS specific CI requirements
* Run Arcade CI tests on MacOS/Linux
* Fix final package building
* Add benchmark download to benchmars .csporj file
* Print detailed status of each unit test
* Install CentOS & Ubuntu build dependencies
* Use container names to differenciate between Ubuntu & CentOS
* Remove sudo usage in CentOS
* Fix Linux build dependencies
* Add -y param to apt install
* Remove installation of Linux dependencies
* Minor additions
* Rename Benchmarks to PerformanceTests for Arcade
* Changes
* Added benchmark doc changes
* Pre-merge changes
* Fixing failing Arcade Windows Builds (#5482)
* Try Windows build single quote fix
* Remove %20
* Added variable space value
* Using variables for spacing
* Added space values as job parameters
* Try conditional variables again
* fix official builds
* Revert "fix official builds"
This reverts commit 7dbbdc7b94.
* fixing tensorflow rebase issue
* Fixes for many of the CI builds. (#5496)
* yml log changes
* Fix NetFX builds by ensuring assembly version is set correctly and not to Arcade default of 42.42.42.42 (#5503)
* Fixed official builds for Arcade SDK (#5512)
* Added fixes for official builds
* Make .sh files executable
* fix mkl nuget issue
Co-authored-by: Frank Dong <frdong@microsoft.com>
* fix code generator tests failure (#5520)
* Added fixes for official builds
* Make .sh files executable
* fix mkl nuget issue
* fix code generate test fails
* only add necessary dependency
Co-authored-by: Mustafa Bal <5262061+mstfbl@users.noreply.github.com>
* Fixed memory leaks from OnnxTransformer (#5518)
* Fixed memory leak from OnnxTransformer and related x86 build fixes
* Reverting x86 build related fixes to focus only on the memory leaks
* Updated docs
* Reverted OnnxRuntimeOutputCatcher to private class
* Addressed code review comments
* Refactored OnnxTransform back to using MapperBase based on code review comments
* Handle integration tests and nightly build testing (#5509)
* Make -integrationTests work
* Update .yml file
* Added the TargetArchitecture properties
* Try out -integrationTest
* Missed -integrationTest flag
* Renamed FunctionalTestBaseClass to IntegrationTestBaseClass
* Missed rename
* Modified tests to make them more stable
* Fixed leak in object pool (#5521)
Co-authored-by: frank-dong-ms <55860649+frank-dong-ms@users.noreply.github.com>
Co-authored-by: Michael Sharp <51342856+michaelgsharp@users.noreply.github.com>
Co-authored-by: Mustafa Bal <5262061+mstfbl@users.noreply.github.com>
Co-authored-by: Frank Dong <frdong@microsoft.com>
Co-authored-by: Michael Sharp <misharp@microsoft.com>
Co-authored-by: Antonio Velázquez <38739674+antoniovs1029@users.noreply.github.com>
* Created the DoubleParser.OptionFlags enum, and made changes on Conversion.cs and TextLoader.ValueCreatorCache so that it can be used by TextLoader.Parser
* Break the singleton pattern on Conversion and ValueCreatorCache, so that they could have more instances with custom DoubleParser.OptionFlags. These new instances would only be used by TextLoader, and the DefaultInstance is used elsewhere in the codebase.
* Typo corrections
* Correct misspelled variable 'supressScoresAndLabels` and updated test as a result of spelling corrections
* Update core_manifest.json
* Update Utils.cs
* Squashed commit of the following:
commit a5e274ef8869576190bbb794360a5f56d998b470
Merge: b7db4fa47 d7f999653
Author: Keren Fuentes <dkeren@seas.upenn.edu>
Date: Thu Nov 14 14:51:21 2019 -0800
Merge branch 'onnx_bin_classifiers' of https://github.com/Lynx1820/machinelearning into onnx_bin_classifiers
commit b7db4fa475
Author: Harish Kulkarni <harishsk@users.noreply.github.com>
Date: Thu Nov 14 17:41:12 2019 +0000
Added onnx export support for KeyToValueMappingTransformer (#4455)
commit f3e0f6b3a5
Author: Eric Erhardt <eric.erhardt@microsoft.com>
Date: Thu Nov 14 07:22:12 2019 -0600
Fix a flaky Extensions.ML test. (#4458)
* Fix a flaky Extensions.ML test.
Make the reload model tests more resistant to timing changes.
* PR feedback.
commit c1e190af4c
Author: Harish Kulkarni <harishsk@users.noreply.github.com>
Date: Thu Nov 14 05:24:14 2019 +0000
Added onnx export support for OptionalColumnTransform (#4454)
* Initial work for adding onnx export support for OptionalColumnTransform
* Implemented support for optional initializers in OnnxTranformer to support OptionalColumnTransform
* Fixed handling of double values and non-long numeric types
* Removed redundant line
* Updated review comment
commit f96761b3ca
Author: Harish Kulkarni <harishsk@users.noreply.github.com>
Date: Thu Nov 14 03:17:12 2019 +0000
Fixed model saving and loading of OneVersusAllTrainer to include SoftMax (#4472)
* Fixed model saving and loading of OneVersusAllTrainer to include SoftMax
* Modified existing test to include SoftMax option
* Modified test to verify both cases: when UseSoftmax is true and false
commit d45cc8a03f
Author: Jake <31937616+JakeRadMSFT@users.noreply.github.com>
Date: Wed Nov 13 17:26:49 2019 -0800
Add InternalsVisibleTo in AutoML and CodeGenerator for the assembly Microsoft.ML.ModelBuilder.AutoMLService.Gpu (#4474)
commit 5e83e235a9
Author: Eric Erhardt <eric.erhardt@microsoft.com>
Date: Wed Nov 13 16:09:05 2019 -0600
CpuMathNative assembly is not getting copied when using packages.config. (#4465)
When we refactored CpuMath to support netcoreapp3.0, we broke the packages.config support to copy the native assembly. This fixes it again by copying the file from the correct location.
Fix#93
commit 693250b9c6
Author: Harish Kulkarni <harishsk@users.noreply.github.com>
Date: Wed Nov 13 21:58:07 2019 +0000
Added onnx export support for WordTokenizingTransformer and NgramExtractingTransformer (#4451)
* Added onnx export support for string related transforms
* Updated baseline test files
A large portion of this commit is upgrading the baseline test files. The rest of the fixes deal with build breaks resulting from the upgrade of ORT version.
* Fixed bugs in ValueToKeyMappingTransformer and added additional tests
commit 59109104e6
Author: Antonio Velázquez <38739674+antoniovs1029@users.noreply.github.com>
Date: Mon Nov 11 17:19:39 2019 -0800
Fixes#4292 about using PFI with BPT and CMPB (#4306)
*Changes in PredictionTransformer.cs and Calibrator.cs to fix the problem of the create methods not being called, to make CMP load its internal calibrator and predictor first so to assign the correct paramaters types and runtimes, and added a PredictionTransformerLoadTypeAttribute so that the binary prediction transformer knows what type to assign when loading a CMP as its internal model.
*Added a working sample for using PFI with BPT and CMPB while loading a model from disk. This is based entirely in the original sample.
*Added file CalibratedModelParametersTests.cs with tests that the CMPs modified in this PR are now being correctly loaded from disk.
*Changed a couple of tests in LbfgsTests.cs that failed because they used casts that now return 'null'.
commit bcdac553ae
Author: Brian Stark <54910472+bpstark@users.noreply.github.com>
Date: Mon Nov 11 13:42:42 2019 -0800
Stabilize the LR test (#4446)
* Stabilize the LR test
Found issue with how we were using random for our
ImageClassificationTrainer. This caused instability in our unit test, as
we were not able to control the random seed. Modified the code to now
use the same random object throughout, the trainer, thus allowing us to
control the seed and therefor have predictable output.
commit d7f9996534ce5e33817f81c92d2b277d9becdf88
Author: Keren Fuentes <dkeren@seas.upenn.edu>
Date: Mon Nov 11 11:33:17 2019 -0800
workaround Scores
commit 7fba31c3e8e358469103e61d4cd2b94292bb09a3
Merge: 93388b60b c96d69043
Author: Keren Fuentes <dkeren@seas.upenn.edu>
Date: Mon Nov 11 11:25:28 2019 -0800
merging changes
commit 93388b60b5286b67ad66e6006c3bc9631733a5c0
Author: Keren Fuentes <dkeren@seas.upenn.edu>
Date: Mon Nov 11 11:19:59 2019 -0800
Added extraction of score column before node creation
commit ea71828bdba14af8a0d1d57162fa8c053136c413
Author: Keren Fuentes <dkeren@seas.upenn.edu>
Date: Fri Nov 8 15:53:11 2019 -0800
fix for binary classification trainers export to onnx
commit 6fad293933f6765b0b5444adcc32907f7a47b091
Author: Keren Fuentes <dkeren@seas.upenn.edu>
Date: Thu Oct 31 15:26:43 2019 -0700
Revert "draft regression test"
This reverts commit 1ad45c995516b9d39fc05aca855ce2abe96c407b.
commit 83c1c8053902bfc06281a85244b0f3e4a1f1dd50
Author: Keren Fuentes <dkeren@seas.upenn.edu>
Date: Thu Oct 31 15:24:23 2019 -0700
draft regression test
commit 888416176e
Author: frank-dong-ms <55860649+frank-dong-ms@users.noreply.github.com>
Date: Fri Nov 8 20:20:53 2019 -0800
nightly build pipeline (#4444)
* nightly build pipeline
commit c96d6904333bd1f10f9e895713225f69879b9e54
Author: Keren Fuentes <dkeren@seas.upenn.edu>
Date: Fri Nov 8 15:53:11 2019 -0800
fix for binary classification trainers export to onnx
commit 8100364471a41b79186b18d1f29fab1c735b4bb0
Author: Keren Fuentes <dkeren@seas.upenn.edu>
Date: Thu Oct 31 15:26:43 2019 -0700
Revert "draft regression test"
This reverts commit 1ad45c995516b9d39fc05aca855ce2abe96c407b.
commit 81381e2714f9e5a2f1435a002946df29fc26790d
Author: Keren Fuentes <dkeren@seas.upenn.edu>
Date: Thu Oct 31 15:24:23 2019 -0700
draft regression test
* update name and remove some dependency code
* baseline file changes
* Refactored function
* Changed ComponentCatalog so it would use the most public "initter"
* Changed the visibility of several constructors and create methods so to choose the correct initter when loading models from disk
- take project level dependency on ML.NET components instead of nuget
- set up bestfriends relationship to ML.Core and remove some of the copies of util classes from AutoML.NET (more work needed to fully remove them, work item 4064)
- misc build script changes to address PR comments
* Implement transformer
* Initial draft of porting tree-based featurization
* Internalize something
* Add Tweedie and Ranking cases
* Some small docs
* Customize output column names
* Fix save and load
* Optional output columns
* Fix a test and add some XML docs
* Add samples
* Add a sample
* API docs
* Fix one line
* Add MC test
* Extend a test further
* Address some comments
* Address some comments
* Address comments
* Comment
* Add cache points
* Update test/Microsoft.ML.Tests/TrainerEstimators/TreeEnsembleFeaturizerTest.cs
Co-Authored-By: Justin Ormont <justinormont@users.noreply.github.com>
* Address comment
* Add Justin's test
* Reduce sample size
* Update sample output
Eric StJohn
Michael Sharp
R. G. Esteves
Franklin Tse
Xiaoyun Zhang
Jon Wood
feiyun0112
Antonio Velázquez
Harish Kulkarni
Ye Wang
frank-dong-ms
Mustafa Bal
Mustafa Bal
Sam Harwell
Keren Fuentes
Maher Jendoubi
redmond\dmitrya
Artidoro Pagnoni
Wei-Sheng Chin
Zeeshan Siddiqui
Shahab Moradi
Nat Kershaw (MSFT)