Граф коммитов

247 Коммитов

Автор SHA1 Сообщение Дата
dotnet-maestro[bot] f602581005
[release/3.0] Update dependencies from dotnet/arcade (#6938)
* Update dependencies from https://github.com/dotnet/arcade build 20231220.2

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 8.0.0-beta.23265.1 -> To Version 8.0.0-beta.23620.2

* Fixed version update breaks.

* Update XUnitVersion

* Update MicrosoftMLOnnxRuntimeVersion to 1.16.3

* Rollback OnnxRuntime and suppress warning

* Update to Xunit with fix for https://github.com/xunit/xunit/issues/2821

* Update Centos docker containers

* Fix packaging step

* Try including stdint.h to fix missing uint8_t on centos

* Update Centos test queue

* Attempt to use runtime centos-stream8-helix container for tests

* Use centos-stream8-mlnet-helix container for testing

* Undo changes to test data

* Make NETFRAMEWORK ifdef versionless

* Only use semi-colons for NoWarn

* Fix assert by only accessing idx (#6924)

Asserting on `_rowCount <  Utils.Size(_valueBoundaries)` was catching a
case where `_rowCount`'s update was reordered before `_valueBoundaries`

This was unnecessary, since this method doesn't need to use `_rowCount`.

Instead, make the asserts use only `idx` which will be maintained
consistent with the waiter logic in this cache.
Ensure we only ever use `_rowCount` from the caching thread, so write
reordering won't matter.

* Don't include the SDK in our helix payload (#6918)

* Don't include the SDK in our helix payload

I noticed that the tests included the latest SDK - including the host -
in our helix payloads.

This is a large amount of unnecessary downloads and it also makes it so
we use the latest host on the older frameworks which can fail when the
latest host drops support for distros.

Since our tests shouldn't need the full CLI, remove this from our helix
payloads.

We'll instead get just the runtime we need through `AdditionalDotNetPackage`

* Place Helix downloaded runtime on the PATH

Helix only sets the path when the CLI is included, however we don't
need the CLI.

* Make double assertions compare with tolerance instead of precision (#6923)

Precision might cause small differences to round to a different number.
Instead compare with a tolerance which is not sensitive to rounding.

---------

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: Michael Sharp <misharp@microsoft.com>
Co-authored-by: Eric StJohn <ericstj@microsoft.com>
2024-01-09 10:08:04 -07:00
dotnet-maestro[bot] c5f03e2c42
[main] Update dependencies from dotnet/arcade (#6354)
* Update dependencies from https://github.com/dotnet/arcade build 20220923.4

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 7.0.0-beta.22327.2 -> To Version 7.0.0-beta.22473.4

* Update XUnit version

* Workaround CS0121 with Xunit.Assert(float,float,int)

Workaround issue https://github.com/xunit/xunit/issues/2393

* Fix a couple more CS0121 XUnit issues

* Fix more xunit issues

* Workaround anoter ref parameter issue.

* Fix more XUnit warnings

* Update XUnitRunnerVersion to 2.4.2

* Update one more Xunit reference

* Update dependencies from https://github.com/dotnet/arcade build 20221007.2

Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.SignTool , Microsoft.DotNet.SwaggerGenerator.MSBuild , Microsoft.DotNet.XUnitExtensions
 From Version 7.0.0-beta.22327.2 -> To Version 7.0.0-beta.22507.2

* Update to RC2 SDK to get FSharp fix

* Extend timeout for code-coverage builds

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: Eric StJohn <ericstj@microsoft.com>
2022-10-13 14:06:01 -07:00
Michael Sharp c0d449f65c
FUNCTIONAL BREAKING CHANGE. Transform chooses score scope by default. (#6269) 2022-08-03 12:24:16 -07:00
Michael Sharp 0577957256
dotnet format/spellchecking (#5988) 2021-11-04 21:48:50 -07:00
Michael Sharp 34eb983687
Arm64 build/test/publish. (#5860)
* Arm helix

* yml cross fixes.

* yml cross fixes.

* yml cross fixes.

* yml cross fixes.

* yml cross testing

* Adding docker again.

* queue changes.

* queue changes.

* queue changes.

* arm pre commands

* arm pre commands

* arm pre commands

* arm pre commands

* arm pre commands

* arm pre commands

* arm pre commands

* Arm helix changes.

* Arm helix changes.

* Removing arm so PR can go.

* FIxes from PR comments. Adding arm32 back.

* Fix using statements.

* Namespace fixes.

* Added missed cmake code.

* Cmake fixes

* M1 Additions

* Changes from PR comments.

* x86x64 fact attribute.

* Fixes from PR comments.

* Fixes from PR comments.

* Fixes for Centos 7 fact and Featurizers fact.

* Code Analysis version and Not fail on M1.

* disable m1 tests in prs.

* disable m1 tests in prs.

* disable m1 tests in prs.

* disable m1 tests in prs.

* Change to full url

* Look in osx-arm64 folder for baseline files.

* Fixed -mthumb flag issue.

* Fixes from PR comments.

* Fixes from PR comments.

* Final test fixed.

* Final Fixes from PR comments.
2021-07-12 15:14:54 -07:00
Jon Wood dd318d89ca
Channels await fix (#5313)
* Update to only get result if awaiter has completed

* Add thread sleep

* Add regression scenario to make sure async fix works

* Update from feedback
2020-07-29 11:46:45 -07:00
frank-dong-ms 5d0edf2499
Test fix using breastcancel dataset and test cleanup (#5292)
* fix tests using breastcancer dataset and test clean up

* format fix
2020-07-08 10:21:41 -07:00
Antonio Velázquez 709ec020b7
Combined methods related to splitting data into one single method. Also fixed related issues. (#5227)
* Refactored CreateStratificationColumn and EnsureGroupPreservationColumn, and some code on GetSplitColumn, into a new method called CreateSplitColumn
* Now "splitColumn" is the name of the temporary column we create for splitting, regardless if it's based-off a "samplingKeyColumn" (ML.NET) or a "stratificationColumn" (NimbusML, Maml, legacy TLC naming conventions...)
* Fixed issues related to dropping or not dropping the splitColumn in some cases
2020-06-30 09:12:07 -07:00
Ye Wang c023271bf0
Support more types for HashEstimator (#5104)
* support more types but tests failed

* fix bugs

* bump to ort1.3 pre-release

* correct/skip some tests

* refactor tests

* fix HashKey8V2

* This reverts commit e0c

* add comments

* revert changes on V1

* upgrade to ORT1.3 official

* fix part of conflits

* update

* update

* update

* fix test failure

* fix another test failure
2020-05-19 15:48:59 -07:00
Yael Dekel 6fddd31fd4
Add support for combining hashes in vector columns to HashingTransformer (#4828)
* Add "Combine" option.

* Add tests.

* Add tests, and change data splitting APIs to use the new functionality of HashingEstimator.

* Address code review comment, change missing value handling and add a test for it.

* Fix hashing tests.

* Add test for backwards compatibility of model format.

* Fix build breaks and tests after rebase.

* Combine V2 and V1 implementations of IHashers into one.

* Update SaveAsOnnx.

* fix baseline

* More unit test fixes

* Code review comments.

* Add back the back-compat test.
2020-05-18 09:26:13 +03:00
frank-dong-ms f0a8a76a40
disable test parallelization for ML.Test assembly to avoid crash (#4896)
* disable test parallelization for ML.Test assembly to avoid crash

* refine comments
2020-02-27 14:29:44 -08:00
frank-dong-ms 6cd6081723
fix issue in WaiterWaiter caused by race condition (#4829)
* fix issue in WaiterWaiter

* re-enable tests that affected by the fixed issue

* refine comments

* refactor based on discussion

* take comments
2020-02-13 13:52:54 -08:00
Najeeb Kazmi 7db00a18a2
Update cookbook to latest API (#4706)
* Update cookbook to latest API

* PR feedback

* Add cross reference to new section

* Align naming in cookbook samples tests with changes

* Add cookbook samples test for WithOnFitDelegate
2020-02-05 14:48:48 -08:00
Harish Kulkarni 24c827416d
Changed all MLContext creation to include a fixed seed (#4736)
* Changed all MLContext creation to include a fixed seed

* Reverted fixed seed related changes from the code generator tests
2020-01-31 00:20:21 +00:00
frank-dong-ms 4779b81dff
fix path in test that not working in linux and mac os (#4737)
* fix path in test that not working in linux and mac os

* undo test change
2020-01-30 03:09:43 +00:00
frank-dong-ms 97c4533df0
separate build pipelines (#4688)
* remove flaky test from CI

* refine text

* remove timeout setting

* also remove benchmarktest from inner loop

* fix skip theory

* fix define constants

* skip benchmark properly

* skip tests in compile time

* skip one more tensorflow test from CI
2020-01-27 13:43:08 -08:00
Frank Dong 0d9a3c2acc retry flaky tests 2020-01-16 16:14:59 -08:00
Maher Jendoubi 712c3ec074 Misc misspellings (#4642). Fixes #4638
* fixed some typos:
LastOccurenceWins --> LastOccurrenceWins
accomodate --> accommodate
occured --> occurred
becuase  --> because
begining --> beginning
neccessary  --> necessary
publically  --> publicly
preceeding --> preceding
acheived  --> achieved
definitly  -->  definitely
realy   -->  really
seperate  --> separate
orignal  -->  original
diffrence  -->  difference
untill  -->  until
sucessfully  -->  successfully
mantains  --> maintains
entrypy  -->  entropy
entrypy  -->  entropy

* Update core_ep-list.tsv

* Update core_manifest.json
2020-01-13 14:11:18 -08:00
Sam Harwell 8cc2f6f92b
Make test methods public (#4532)
Fixes IDE0051 (Private member is unused)
2019-12-07 20:08:04 -08:00
frank-dong-ms 36fab9b680
Extract TestFrameworkCommon Project, remove dependency of TestFramework from FT (#4346)
* extract TestFrameworkCommon project and remove reference to TestFramework project from Functional.Test project

* update on TestFrameworkCommon

* update on TestFrameworkCommon project

* minor fix

* fix reference

* fix project reference

* fix CI build error

* sync latest code and resolve

* take comments and made change

* take comments and fix

* modify sln file to build on azure pipeline
2019-10-30 11:19:51 -07:00
Zeeshan Siddiqui 1503b0aa9c
Image classification preview 2. (#4151)
* Image classification preview 2.

* PR feedback.

* Add unit-test.

* Add unit-test.

* Add unit-test.

* Add unit-test.

* Use Path.Combine instead of Join.

* fix test dataset path.

* fix test dataset path.

* Improve test.

* Improve test.

* Increase epochs in tests.

* Disable test on Ubuntu.

* Move test to its own project.

* Move test to its own project.

* Move test to its own project.

* Move test to its own file.

* cleanup.

* Disable parallel execution of tensorflow tests.

* PR feedback.

* PR feedback.

* PR feedback.

* PR feedback.

* Prevent TF test to execute in parallel.

* PR feedback.

* Build error.

* clean up.
2019-08-29 17:59:57 -07:00
Artidoro Pagnoni 59699a5e5e
Farewell to the Static API (#4009) 2019-07-30 13:55:28 -07:00
Zeeshan Siddiqui 2c1cfcac00
Add option to execute only the last transform in TransformWrapper and have WordBagEstimator return transformer chain (#3700)
* Add option to execute only the last transform in TransformWrapper.

* PR feedback.

* cleanup.

* cleanup.

* PR feedback.

* PR feedback.

* PR feedback.

* PR feedback.

* PR feedback.

* PR feedback.

* fix paths.

* PR feedback.

* PR feedback.
2019-05-16 15:25:54 -07:00
Senja Filipi 610ffcb670
Exposing the confusion matrix (#3250)
* Exposing the confusion matrix
2019-04-19 22:57:16 -07:00
Jon Wood d8e046278c Explainability doc (#2901) 2019-04-19 21:49:02 -07:00
Zeeshan Siddiqui fde1ab718b
Move Normalizer extension method from experimental to stable nuget and remove Normalizer generic APIs (#3118)
* Move Normalizer extension method from experimental to stable nuget.

* Cleanup unused method in Normalizer Estimator.

* remove normalizer estimator catalog methods that take enum as parameter.

* Remove Microsoft.ML.Experimental references in CS files.

* merge fix.

* cleanup.

* cleanup.

* PR feedback.
2019-04-01 19:38:00 -07:00
Ivan Matantsev d2bf3e72fa
Restore OVA ability to preserve key names on predicted label (#3101) 2019-04-01 16:49:51 -07:00
Wei-Sheng Chin 8730c87ca1
Better names to calibreated linear classification models (#3034) 2019-03-25 16:47:49 -07:00
Artidoro Pagnoni 5f9be36908
Multicolumn mapping for some estimators (#3066) 2019-03-25 15:41:08 -07:00
Rogan Carr f342403fa9
Create model file V1 scenario tests (#2899)
* Combining ModelFile scenario with ModelLoading.
2019-03-25 11:58:24 -07:00
Eric Erhardt 45b6c43387 Move KeyType, VectorType and VBuffer to ML.DataView (#3022)
* Move KeyType into ML.DataView assembly.
* Rename KeyType to KeyDataViewType.
* Move VBuffer to ML.DataView
* Move VectorType to ML.DataView
* Rename VectorType to VectorDataViewType.
2019-03-22 13:50:42 -07:00
Tom Finley 75fc05504c
Remove model saving/loading inconsistencies (#3044)
* Change the model load/save API to always have ITransformer as central object.
* Keep the with loader save order the same as with schema overload, with ITransformer always first.
* Change ModelLoadingTests to use the MLContext of its new base class.
2019-03-22 10:13:41 -07:00
Rogan Carr db4ecc0135
Move metrics from percentages to [0,1] (#2697)
* Moving NDCG and LogLossReduction to 0-1 from 0-100
2019-03-21 16:49:05 -07:00
Wei-Sheng Chin 5b22420d28
Polish train catalog (renaming only) (#3030) 2019-03-20 22:41:28 -07:00
Yael Dekel 807d813050
Clean up the SchemaDefinition class (#2995)
* Internalize some members of SchemaDefinition, and add tests

* Code review comments

* Fix build after rebase

* Fix failing test

* Fix build after rebase

* Internalize Column ctor

* Fix build after rebase
2019-03-20 10:01:21 -07:00
Wei-Sheng Chin 3af9a5d96a
Make Multiclass Linear Trainers Typed Based on Output Model Types. (#2976)
* Step 1: create two multi-class linear models

Step 2: Make SDCA trainers typed

Finish version 0.1

Delete commented lines

* Add some doc strings

More document

* Handle static extensions

* Rename several maximum entropy models and trainers

* Fix EP test

Fix two tests and address a comment

Add missing piece

* Address comments

* Improve option of MCSDCA

* Address comments

* Update code sample

* Refactorize saving family

* Rename a class following binary SDCA trainer
2019-03-19 17:19:49 -07:00
Abhishek Goswami 08318656ed
Fixing ModelParameter discrepancies (#2968)
* fixing model parameter discrepencies

* multiclass LR singe that refactoring is happening in a parallel PR

* review comments. Added Multiclass to NaiveBayes

* Drop Classification from trainer names - v1  (more trainers to follow)

* multiclass LR will be handled separately

* Drop Classification from trainer names - v2  (all trainers taken care of)

* fix entrypoint file
2019-03-19 15:25:20 +00:00
Artidoro Pagnoni 71693b3ac8
Cleaning TrainCatalog and RecommenderCatalog (#2973) 2019-03-18 23:54:38 -07:00
Zeeshan Siddiqui 8ed72fb9a0
Include the save file action (.ZIP file) as part of model.SaveFile(). (#2993)
* Include the save file action (.ZIP file) as part of model.SaveFile().

* PR feedback.

* fix comments.
2019-03-18 18:01:57 -07:00
Yael Dekel 665a366965
Add save/load APIs for IDataLoader (#2858)
* Add save/load APIs for IDataLoader

* Address some code review comments, add a non-generic base class for calibrated predictor

* use the contravariance of ISingleFeaturePredictionTransformer instead of loading PredictionTransformer<object> from file

* Add API for saving/loading input schema

* Fix build after rebase

* Add API to create PredictionEngine with input schema

* Address code review comments

* Unfriend Functional.Tests

* Add CreatePredictionEngine API back to ModelOperationsCatalog

* Address code review comments

* Fix build

* Fix F# tests

* Remove duplicate CreatePredictionEngine API

* Add test for creating an IDataView from a loaded schema

* Fix build error after rebase

* Add unit tests, and address some code review comments

* Fix build after rebase

* Code review comments
2019-03-18 15:08:02 -07:00
Eric Erhardt 70ef7ecd43
Move IDataView into Microsoft.ML namespace (#2987)
* Move IDataView into Microsoft.ML namespace

Fix #2974

* Fix code analyzer test now that we removed a using line.
2019-03-18 17:06:41 +00:00
Rogan Carr d794383281
Adding Debugging Scenario tests for V1 APIs (#2937)
* Adding Debugging Scenario tests for V1 APIs
2019-03-14 11:40:28 -07:00
Rogan Carr acfe24a5e8
Adding functional tests for all training scenarios (#2921)
* Adding functional tests for training scenarios
2019-03-13 20:26:45 -07:00
Wei-Sheng Chin dde909a71f
More Normalizer Scrubbing (#2888)
* Clean static part for previous projection catalog

Scrubbing normalization again

* Fix a test

* Address #2928

* Address comments

* Rename RandomFourierKernel to ApproximatedKernel everywhere

* Replace Lp with LpNorm.

* Address a comment

* Make sure EP not affected

* Expose normalizer config again

* Expose normalizer's model parameters correctly
2019-03-13 17:25:32 -07:00
Wei-Sheng Chin 91a8703c6e
Polish char- and word-level tokenizers & stopword removers (#2916)
* Polish char-level tokenizers

* Polish word-level tokenizers

* Scrub stopword removers
2019-03-13 13:43:27 -07:00
Senja Filipi c5aab77062
updating namespaces (#2914)
* Microsoft.ML.LightGBM -> changes to Microsoft.ML.Trainers.LightGBM

Microsoft.ML.Transforms.FeatureSelection -> moves to Microsoft.ML.Transforms

* ImageAnalytics -> Transforms.Images
and the ImageLoader transform moved to Microsoft.ML.Data together with the TextLoader.

* LightGBM -> LightGbm for namespace, assembly and package.
2019-03-13 11:37:49 -07:00
Wei-Sheng Chin 9d9a3d946a
Scrub n-gram hashing and n-gram (#2898)
* Address comments and also handle WordBags and HashedWordBags
2019-03-12 23:55:38 -07:00
Artidoro Pagnoni abc2d638f5
Scrubbing of the key related transforms (#2862) 2019-03-12 18:45:34 -07:00
Abhishek Goswami 7f0c1ad26c
Fixing names of trainer estimators (#2903)
* renaming several trainers

* updating some of the trainers with acronyms

* updated names based on the latest pattern

* fix name of MetaTrainer

* fix review comments

* connect acronym for SGD

* updates to cookbook markdown file
2019-03-12 22:19:29 +00:00
Ivan Matantsev acc4ac016b
One type label policy in trainers (#2804) 2019-03-11 17:30:38 -07:00