* fix the build for mobile packaging
* update the cmake file as well
* more fixing on dlib related ops
* release the iOS cmake version constraint
* upgrade cmake in Linux CUDA build
* Update Dockerfile.ubuntu_cuda11_8_tensorrt8_6 for typo
* Update ios_packaging.yml for Azure Pipelines
* update the dlib versoin
* update all cases of cmake version
* update the comment for dlb cmake
* Unify the image operations in extensions library
* fix the build configuration issue
* More build fixings
* Fix the native image codec
* fix encode_image
* Add bgr/rgb conversion for encoding image
* parity check
* build break
* update PNG encoding parameters
* build break on Linux
* using MSE to compare images
* fix the discrependency between Linux and Windows
* final code refinement
* one more change
* fix the C++ warnings
---------
Co-authored-by: Sayan Shaw <52221015+sayanshaw24@users.noreply.github.com>
* upgrade ci matrix
* typo
* revert python version
* update python and ort range
* update python range
* update for macos and linux too
---------
Co-authored-by: Sayan Shaw <sayanshaw@microsoft.com>
* initial checkins for mllama image process
* fix some tests
* some fixings
* add more image
* More test assertions
* parity test passed
* code clean up
* code refinement
* Add general regex support
* add case 5 support instead of replacing with s+
* add more test cases
* address comments
* add back gpt2 and llama regex methods for efficiency
---------
Co-authored-by: Sayan Shaw <sayanshaw@microsoft.com>
* add(tutorials): exporting yolo world model
This allows us to export yolo world onnx model which can be later used in mobile inference.
* add(tutorial): make classes optional
---------
Co-authored-by: Scott McKay <skottmckay@gmail.com>
* add compatibility docs
continue updating the doc
updating doc 2
* support sentence-piece add_dummy_prefix for all models
* revert the flag
* initialize the add_dummy_prefx for llama model
This added support for native image decoding on Windows & Apple platforms.
This helps us remove libpng & libjpeg completely on these platforms, and
in the meantime support more image formats thanks to OS vendors,
* initial commit
* Ugm vocab loaded is good
* test passed
* fixes unit test on win32
* finish the parity check
* code refinement
* code refinement for review
* add C++ standard library regex support for GPT2 case
* reorder regex handling
* try without STL
* missing case
* add llama3 regex support
* add custom regex impl
* change regex based on model
* modify tests, add docs, and code cleanup
* add regex test and const strings
---------
Co-authored-by: Sayan Shaw <sayanshaw@microsoft.com>
* Remove OpenCV dependency from C_API model
* fix build on Windows
* switch ci build flag
* try to fix the macOS build issue
* more fixing
* fix the macOS build issue
* list jpeg source
* verified on MacOS
* update the pp_api too
* avoid the codecs library conflicts
* Add the unit tests
* move the codec test
* add the missing dl lib for extensions test
* refine the code
* a smaller fixing for Windows Python
* optimize the tokenizer for efficiency
* fix the unit test failures.
* fix the api test case failures
* removed the unused code.
* More test cases fixings
* One more fixing
* fix macOS build issues
* refine the test
* add more diagnosis info.
* fix unit test in CI Linux
* fix the pp_api test failure
* initial api for tokenizer
* More fixings and test data refinement
* add a simple wrapper for pre-processing APIs
* fix the test issues
* test if the tokenizer is spm based
* fix the failed test cases
* json pointer does not work
* support tokenizer build only in C API mode
* fix the python build.
* fix the selectedops build
---------
Co-authored-by: Sayan Shaw <52221015+sayanshaw24@users.noreply.github.com>
* Upgrade ESRP signing task from v2 to v5
* Upgrade ESRP signing task from v2 to v5 in win
---------
Co-authored-by: Sayan Shaw <52221015+sayanshaw24@users.noreply.github.com>
Wenbing Li
Kyle
Sayan Shaw
Rony Fadel
Akshay Sonawane
Stalin Sabu Thomas
Chester Liu
Scott McKay
Wenbing Li
Prathik Rao
Yi Zhang
Zhipeng Han
vraspar