* Update JDK version to 17 in ci.yml
* Update com.diffplug.spotless to 6.22.0.
* Copy updated scripts to start/stop the emulator from ORT from https://github.com/microsoft/onnxruntime/pull/17903.
Minimize the time the emulator is running as well.
* Fix includes
* Update to JDK 17 in packaging pipelines.
* Fix pool name.
---------
Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
* Add ability to create a 'dummy' nuget package from a build with no ops. This is required on iOS due to the ORT C# package including the ability to register custom ops. That creates a compile time dependency for the RegisterCustomOps symbol to exist. If the user can't (Xamarin) or does not want to use extensions that results in usage of ORT being broken.
For simplicity, consistency and to minimize changes to the build infrastructure we build a nuget package for all platforms so everything is 1:1 with the real package except the build uses an empty config to select the ops to include. Technically we only need something with iOS builds.
We should only need to publish this once unless the RegisterCustomOps API changes.
* Address PR comments
* Update .pipelines/nuget.yml
Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
---------
Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
- Split ci.yml into stages. This makes it easier to only run a subset of the jobs in the pipeline.
- Use existing iOS simulator instead of creating a new one.
---------
Co-authored-by: Wenbing Li <10278425+wenbingl@users.noreply.github.com>
- Update build script to copy over test files and run tests on an Android device.
- Set up a CI job to run tests on the emulator.
- Fix a test failure encountered when running on an actual Android device. Update string_lower to call ToLower on each UTF code point instead of each UTF-8 byte.
---------
Co-authored-by: Wenbing Li <10278425+wenbingl@users.noreply.github.com>
* doc ops (#529)
* Try and make CIs pass with Azure ops enabled by default.
Misc. other cleanups
* Fix some CI issues.
Cleanups some bits and pieces.
* Fix a couple of issues.
* Fix arg to build.bat
* Increase warning in triton client build to make binskim happy (hopefully).
* Try patching the warning level in the triton grpc branch as well. Shouldn't matter but...
* Run triton patch command for windows as well.
* Add patch.exe directly so windows builds work.
* override auth gen for AOAI
* fix build
* switch to windows-static
* update model for azure chat
* document triton invoker
* doc chat endpoint
* document triton invoker
* format
* format
* format
---------
Co-authored-by: Scott McKay <Scott.McKay@microsoft.com>
Co-authored-by: Randy Shuai <rashuai@microsoft.com>
* address comments
* move doc sect
* typo
* typo
---------
Co-authored-by: Scott McKay <Scott.McKay@microsoft.com>
Co-authored-by: Randy Shuai <rashuai@microsoft.com>
* Build fixes
- zlib needs to come from vcpkg if azures ops are being built and opencv isn't enabled
- set the IR version to 8 for some of the azure ops test models so they can be tested when ORT 1.14 is used
- pass through new ort version value so that a consistent version is used to a) pull the ORT package for the c++ unit tests and b) disable azure ops if ORT version is too old.
* Update to automatically chain package to avoid build errors during the install if cmake runs commands in parallel
* Define simplified ORT_FILE for older ORT versions
* - Ensure we log an error message before throwing on Android
- message in exception will be lost due to how the shared libraries are built (both onnxruntime and extensions use static libc++ so there are no shared exception types between them)
- support static or dynamic build of curl/openssl on android
- TBD which we want to use.
- add infra for anything deriving from BaseKernel to log messages using the ORT logger
- ensures messages from custom kernels end up in the same place as messages from ORT
* Refactor setup for Azure ops to try and make common things more re-usable, and for the actual ops to simply layer in the specific input/output constraints for that type of request.
Currently builds on Linux, Windows (x64 only) and Android
Android requires a manual pre-build of openssl and curl.
Linux requires a manual pre-install of openssl.
Windows currently only works for x64. Other targets need the triplet adjusted.
* Address PR comments
* Fix could of android build warnings.
* Update .gitignore to remove old path
* Fix build break from merge
* set before-test
* test cmd
* clean in yml
* restore toml
* add ut for triton endpoints
* reset working path
* rename suffix
* install ort
* pip install
* make env
* add extra env
* make executable
* set dir for linux
* add switch
* set env default
* skip tests
* simplify env
* clean env for official
---------
Co-authored-by: Randy Shuai <rashuai@microsoft.com>
Co-authored-by: Wenbing Li <10278425+wenbingl@users.noreply.github.com>
Pass `-gpu swiftshader_indirect` emulator args as a workaround to get Android emulator running on the MacOS 13 hosted agents.
See https://github.com/actions/runner-images/issues/7671
Co-authored-by: Wenbing Li <10278425+wenbingl@users.noreply.github.com>
* nuget ci package
* disable macos arm64 build for err
* Get the iOS xcframework build working with the split build/pack approach. (#416)
* refine build_xcframework.py
Cleanup/clarify various things
- naming of parameters and files
- consistency
Make handling of additional build args more generic
Update the artifact download dir/extract dir to more intuitive names
Update scripts
- make usage from CI pipeline clearer (e.g. don't hide directory names inside script)
- keep comments in nuspec
- remove unused args
- make additional arg handling more
Co-authored-by: Scott McKay <skottmckay@gmail.com>
Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
* built-in bounding box op
* update boundary check
* assert policy
* more boundary test and check
* XYXY--> X horizon
---------
Co-authored-by: Scott McKay <skottmckay@gmail.com>
* Added initial CustomOp template generator
* Added arg parsing, error check, and modularized methods
* Added test and models for the same plus refactored code in template generator
* Fixed model path issue
* Final changes
* Removed ONNX models and added model generation in test scripts, changed custom ops to random math ops
* Update based on new custom op changes
---------
Co-authored-by: Sayan Shaw <sayanshaw@microsoft.com>
* re-enable Python pipelines
* deprecate the direct setup.py call
* pipeline fixing
* run cmake from visual studio
* remove the self check
* support different ort versions
* remove ort 1.9 and add 1.13
* ci pipeline fixing
* fix the test with latest pytorch
* code refinement
* torch version detection
* make package python version more consistent.
* Move the pre/post processing scripts into the python module.
Update usage/examples.
* Use better version parsing.
* Update tests, docs,
* Address PR comments.
Remove global Settings and pass onnx opset around directly where needed. Make PrePostProcessor the owner of the checker context.
* Add BertTokenizer to iOS package ops config.
* Also register tokenizer ops in com.microsoft.extensions.
Co-authored-by: Wenbing Li <10278425+wenbingl@users.noreply.github.com>
* Added initial CustomOp template generator
* Added arg parsing, error check, and modularized methods
* Added test and models for the same plus refactored code in template generator
* Fixed model path issue
* Final changes
* Removed ONNX models and added model generation in test scripts, changed custom ops to random math ops
authored-by: Sayan Shaw <sayanshaw@microsoft.com>
- Generate AAR with associated artifacts (javadoc.jar, sources.jar, pom).
- Fix JAR/AAR package descriptions.
- Remove `gradle -b build-android.gradle` invocation from CMake. This creates the AAR and it is not necessary to run it for each Android ABI build.
- Support building on Windows.
* Add python based build infrastructure to simplify developer builds for various platforms. Majority was copied from the ORT build script so usage is consistent with that.
Left the existing build.bat/build.sh but ideally the CI can be updated to use the new infrastructure so things are more consistent.
Updated gradle to 7.5.1 and Android gradle tools to 7.3.0.
Validated Windows and cross-compiling Android on Windows including builds with explicitly selected ops.
WASM and iOS builds aren't tested yet and might need minor tweaks.
* Update build.py to require Python 3.7, remove git submodule sync, reorder options.
* Use 'cmake -E remove' to remove file.
* Enable specifying the ORT version to fetch
* Add ability to enable Java bindings.
Co-authored-by: Wenbing Li <10278425+wenbingl@users.noreply.github.com>
Co-authored-by: edgchen1 <18449977+edgchen1@users.noreply.github.com>
* Separate ops in operators/cv2 that do and do not require codecs so they're easier to include/exclude from a build.
Remove jpeg2000 from opencv file formats. It costs 1MB and is (afaict) not a common format.
Add ability to enable/disable cv2 ops to gen_selectedops.py.
* Remove super resolution pre/post process ops that are no longer needed.
* Replace super resolution e2e tutorial
Co-authored-by: Wenbing Li <10278425+wenbingl@users.noreply.github.com>
* - Fix incorrect weights matrix for conversion from YCbCr to BGR
- Fix Resize when input is HWC or CHW by converting to NHWC/NCHW so bilinear interpolation is inferred. trilinear is inferred otherwise.
- Not aware of a use case where we'd want to use trilinear - if one exists the temporary addition of the batch dim would need to be made explicit.
- Allow Resize to use antialias if ONNX opset is 18 or higher
- Skip adding unnecessary Mul if FloatToImageBytes multiplier is 1.0.
- Make Debug step available at the top level
Validated opset 18 Resize vs PT for super resolution and the results are now a really good match.
* Update test data to use image that shows diffs better.
Make super resolution output format configurable
Address PR comments.
* Adjust test for diffs on macos
Initial changes for adding an iOS package.
Add infrastructure to build iOS static framework, xcframework, and CocoaPods package.
Add test project that uses package.
Add package build and test to CI pipeline.
* Initial changes for supporting mobilenet and superresolution.
- Script to update model with pre/post processing
- custom ops for decode/encode
- user just has to provide jpg or png bytes
- superresolution can return the updated image in jpg or png
- models for testing
Updated cmake setup to enable building of the vision pre/post processing ops
- opencv2 is treated as an internal dependency rather than the mechansim for selecting which operators to include.
* Add extra check in decode.
* an android test app for extensions AAR package
* add the pipeline
* fxing the Android CI pipeline
* fix the build issus on macOS
* more fixings
* more fixings
* switch to jdk 11
* gradlew path issue
* update the command lines
* split the test task
* better name
Rachel Guo
Scott McKay
Yi Zhang
Edward Chen
RandySheriffH
Wenbing Li
Sayan Shaw
JiCheng