Граф коммитов

44 Коммитов

Автор SHA1 Сообщение Дата
Brian Martin 3834ffc22c
bug fixes in log CPU fallback (#474)
* upgrade toolset

* update command line help string

* change "toggle" -> "enable" since it's not a toggle and check the value before logging.
2022-05-04 14:15:13 -07:00
Ryan Lai b9999f40f2 Add 19041 sdk 2021-04-19 10:35:47 -07:00
Ryan Lai 100efafeb4 Upgrade to VS2019 and fix winmlrunner build breaks 2021-04-19 10:32:48 -07:00
Ryan Lai 7f9e19120b
Build MicrosoftMLRunner for using WinML nuget and make WinMLRunner only load from System32 (#348)
* move some headers around

* add nuget

* Refactor bindingutilities and outputhelper into .cpp and header files

* remove dllload.cpp

* Remove windows.ai.machinelearning header

* Fix testing

* Don't change sln

* Fix x86 build

* Added different configurations

* Add preprocessor definitions to separate between headers and namespaces

* Change naming of EXE for nuget builds

* Update OutputHelper.cpp

remove pragma in outputhelper.cpp

* remove pragma from cpp

* Switch between native header and update documentation

* use const cast

* rename nuget to NuGet and address PR comments

Co-authored-by: Ryan Lai <ryalai96@gamil.com>
2020-09-09 15:59:26 -07:00
Xiang Zhang 935b3802e2 refactor code 2020-03-04 14:09:57 -08:00
Xiang Zhang bf37d5d779 E_FAIL returned 2020-03-03 16:03:13 -08:00
Xiang Zhang 45599462bb fix winmlrunner CI build 2020-03-03 14:16:08 -08:00
Ryan Lai 78c4b65525
Add batching size 1 and session options (#264)
* check session options version

* Create Session Considering Support for Session Options

* Only set batchsize to 1

* Made batch size override always 1

* Merged

* Update minimum SDK
2019-08-01 17:03:34 -07:00
pmbrown1055 d813cef500 Fix the input image preprocessing FP16 unit test on the GPU. (#263) 2019-07-29 16:03:25 -07:00
pmbrown1055 ed0b176a8b Add -onClientTensorize [Identity|ScaleMeanStdDev] functionality + uni… (#223)
* Add -tensor preprocessing function options [Identity|ScaleMeanStdDev].

* Unify the csv and image reading mechanisms to unify the preprocessing code.
* Eliminate unnecessary copies of the csv data, expect for preprocessing pass.
* Update unit tests, including new DenseNet121 test for functional verification of preprocessing.

* -Tensor Normalize - update command to accept actual means and std devs on command line.  Fixup tests and target csvs with new values.

* Tensorize edits to update spacing, and add a throw if fetching native tensor buffer fails.
2019-07-24 11:17:30 -07:00
Ryan Lai 99e9342593
This change allows user to specify folder of images to bind and evaluate (#237)
* Evaluate folder of images

* Evaluate folder of images

* Updated usage

* Test change

* test changes back to test method cleanup

* Fix build break

* remove test

* Added test

* Fix comment typo

* Don't print out image path if using -terse flag

* Remove commitments
2019-06-18 09:28:50 -07:00
pmbrown1055 25e1ded8c2 Add per TEST_METHOD names to the output directory structure, so parallel unit test runs won't overwrite one another. (#219) 2019-04-19 15:33:42 -07:00
pmbrown1055 a8727a3e5d Add -BaseOutputDir and more robust path support in general. Fix PerIterationPerf issue with Unit Test. (#216) 2019-04-19 14:30:45 -07:00
Ryan Lai f2a21529f1
Add Bind GPU bound Tensor (#213)
* Add feature to bind GPU tensors

* Add test to check for invalid arg

* Spacing

* Make heap properties and resource desc more simple

* Moved GPU upload tensor code under if statement
2019-04-15 16:38:21 -07:00
Ryan Lai 4648837d0b
Add mnist test files (#210) 2019-04-09 18:03:44 -07:00
Ryan Lai 5cf6f977ef
Support Models with Image Denotation Input (#209)
* Added support for models with image denotation

* Added bind image support for models with image denotation

* Added mnist and mnist input files

* Model tensor inputs should have channel 3 or 1 if binding image

* Add mnist tests

* Removed modelbinding.h

* PRcomments

* Fixed is image, is csv input check
2019-04-09 17:33:57 -07:00
Ori Levari 796e4f91d4
Various bug fixes and build features to public samples (#203) 2019-03-26 13:48:39 -07:00
Ryan Lai 26cefe6037
FP16 Tensor output tests are too strict. Add additional fallback validation tests (#188)
* Add extra checking for fp16 output tensor tests

* Fixed var name

* Adjust values

* Made populating tensors more generic and tensor vectors are passed by reference

* Fixed formatting

* PR comments for renaming method

* Added comment
2019-03-08 10:28:06 -08:00
Young Kim 430847ebff
Printing top K values instead of just single max value (#189)
Currently WinMLRunner prints top value in the result float tensor. This change is to output top K values instead:

Here is the sample output:

Creating Session with GPU: AMD FirePro W4100 Graphics Adapter
Binding (device = GPU, iteration = 1, inputBinding = CPU, inputDataType = RGB_Image, deviceCreationLocation = WinML)...[SUCCESS]
Outputting top 5 values
Feature Name: resnetv23_dense0_fwd
index: 409, value: 6.74914
index: 769, value: 5.66233
index: 664, value: 5.62002
index: 585, value: 5.48044
index: 872, value: 5.29839
Evaluating (device = GPU, iteration = 1, inputBinding = CPU, inputDataType = RGB_Image, deviceCreationLocation = WinML)...[SUCCESS]
2019-03-06 16:06:56 -08:00
Young Kim c208a75895 Add clang format without sorting includes (#187) 2019-03-06 09:30:59 -08:00
Ryan Lai 71e9db519c
Change binding data to not memset 0, make fp16 tensor tests more lenient (#180)
* Memset garbage binding input to 2 instead of 0

* Make FP16 tests more lenient

* Refactored Tensor creation to be cleaner. Garbage data is now arbitrary

* Added check when creating commandline args so that user cannot save tensors if no input data is provided
2019-02-28 17:23:49 -08:00
Ryan Lai a3ac3632f4
Removed DXCore from WInMLRunner (#179) 2019-02-27 16:08:11 -08:00
Young Kim 3db448ff7a
Implement and use thread pool to run models concurrently (#178)
* Implement and use thread pool to run models concurrently

* add comments
2019-02-27 12:37:49 -08:00
Ryan Lai 044757481c
Process FP16 output and output to Console and CSV file (#176)
* Added FP16 tensor output support

* Using winmlrunner tests from master

* Added index 0 to for loop when printing out tensor

* Renamed tensor output files

* Added fp16 tests and made CompareTensors helper method more strict for tensor comparisons

* unused helper method

* Changed template for fp16 case

* Fixed spacing

* Adding tensor output expected files

* Addressed PR comments. Removed unneccessary code from template function
2019-02-25 22:54:32 -08:00
Young Kim 014a0924b5 Pass in second value for -SaveTensorData parameter (#172)
* pass in optional second value for parameter -SaveTensorData and fix test cases to compare csv values correctly

* Fix build break and update comments
2019-02-21 11:17:12 -08:00
Ori Levari 34124f0655
allow SamplesTests to be built on x64 and x86 (#170) 2019-02-20 13:02:56 -08:00
Ryan Lai 96110844bd
Merge MCDM GPUAdapterIndex selection into master (#155)
* Update GPUAdapterIndex option

* Support MCDM but use DIRECT command queue to allow GPU tensorization of image

* Test out DXCore

* Compatibility: Use DXGI adapter, 11_0 feature level and DIRECT command queue for regular GPU

* Use delay-load for dxgi and d3d11, use 18298 SDK and use static lib for debug build

* Remove dependency on kernel32.dll

* Support WCOSHeadless which doesn't have DXGI and D3D11

* Handle exception from delayed load

* Generate full PDB for debug build

* Handle delayed load SEH in a CreateDXGIFactorySEH

* Removing workaround for X86 DXCore, fix is grfx_dev 18318 and EEAP rs_prerelease 18323.1000.190119-1745 or newer

* Fix merge erros

* Delayload dxcore

* Throws error message if dxcore isn't supported

* Added extra checking for gpuadapter index flag argument

* Removed unnecessary files

* Added IsApiSetIMplemented

* Static link onecore on other architectures as well

* Added error checking to see if dxcore is available

* Prints out all adapter enumerations

* Added extra warning information about dxcore support.NEeds SDKat least 18323
2019-02-07 13:38:58 -08:00
laks0209 d2b1522aec Changed formatting for Stringify types to be more consistent (#136) (#113) 2019-02-06 09:41:31 -08:00
Ryan Lai 9e94139ee4
Stop automatically outputting performance file for -perf flag (#134)
* Stop outputting perf file for ever perf run

* Fixed indexing for commandline args

* PR feedback, removed WinMLRunnerDLL because this needs more work

* Changed flag from -output to -PerfOutput in WinMLRunnerTests

* Removed const for primitive types in commandlineargs
2019-01-22 15:21:24 -08:00
Ryan Lai 03cdb808f0
Refactor winmlrunner tests to use hresult (#121) 2019-01-08 16:15:25 -08:00
Ryan Lai 523cf91861
Refactor WinMLRunner to build DLL and EXE from static library (#119)
* Separated source code so static lib is built. DLL and EXE links static lib to build. Refactored Tests.
2019-01-08 14:03:44 -08:00
pmbrown1055 e910fd2fe6 Autoscale input images to match model input, with command line arguments and message to indicate autoscaling if image input dimensions mismatch the model inputs. (#86) 2018-11-15 13:53:56 -08:00
Patrice Vignola bb34ea8393 Add CreateDeviceOnClient and CreateDeviceInWinML flags to WinMLRunner (#70)
* Add client device creation to WinMLRunner

* Add CreateDeviceOnClient and CreateDeviceInWinML flags

* Fix outdated comment

* Change CreateDeviceInWinML default
2018-10-30 14:55:17 -07:00
Ryan Lai d2d9c949c5
User/ryanlai2/add arm64 (#74)
* Changed outputhelper reset so that it is more meaningful to only reset bind and eval times.

* Changed output directory for x86 to x86 instead of win32, to make it more uniform with x64 output directory

* Added ARM64 support

* Made release build static link vcruntime140.dll and MSVCP140.dll

* Changed SDK version on README.md

* Removed winml.lib dependency because not needed

* Removed winml.lib from x86 / debug
2018-10-30 13:09:05 -07:00
Patrice Vignola d7492d3da8 Add more configurations to gather better perf metrics (#59)
* Add a lot more perf metrics and granularity

* Add the Tensor Input Data Type

* Make the existing tests pass

* Add dynamic DLL loading at runtime

* Fix warnings

* Fix x86 warning

* Update README.md and UseGPU() function

* Add more tests for garbage input with the new flags

* Add tests for garbage data

* Revert temporary code

* Fixed typo

* Change old code

* Change tabs for spaces

* Fix bad IF statement

* Fix bad IF statement

* Revert non-backward-compatible change

* Revert non-backward-compatible changes

* Fix typo in README.md

* Fix "totalTime" showing "nan" in the CSV

* Retarget the test Windows SDK

If both projects target the same Windows SDK, we don't need to install
an older SDK just to run the tests.

* Fix the random garbage generator

* Fix the last failing test
2018-10-19 16:47:52 -07:00
Kumar Rajeev 759f4c105c
Update SDK build number to 17763 2018-10-12 22:38:04 -07:00
Young Kim 68b535a0f0
More Samples Tests (#51)
* Update README.md

* add more entries to squeezenet and mnist test

* Add style transfer test

* add teardown & fix timeout handling for style transfer test
2018-09-21 17:30:49 -07:00
Ryan Lai 4b19d4da1b
Check in for visual studio tests for winmlrunner (#43)
This PR checks in tests for WinMLRunner and uses Visual Studio test for testing. Tests for failure and success scenarios for using WInML API.
2018-09-18 14:16:57 -07:00
Young Kim e2f0998a99
Add Squeezenet Test and set up for running samples test on azure devops (#42)
This change includes Squeezenet Samples test (CS project) using kitten image.
Both squeezenet test and mnist test will be run with vstest with MSTest adapter.
2018-09-17 15:42:15 -07:00
Young Kim e1f664f919
Mnist wad tests (#36)
* Add mnist Windows Application Driver Tests

* add readme

* rename Tests directory to Testing
2018-09-12 12:44:51 -07:00
Ryan Lai 610e0fb073 Removed testing code 2018-09-11 14:21:52 -07:00
Ryan Lai d1155f6efb Changed version number to 17738 2018-09-11 11:43:01 -07:00
Ryan Lai 9c08c04e4e Added todo to implement tests after CI testing is setup 2018-09-11 11:37:52 -07:00
Ryan Lai 15791dcc54 Added taef test project to build taef test dll 2018-09-11 11:36:12 -07:00