Граф коммитов

30 Коммитов

Автор SHA1 Сообщение Дата
Wenbing Li b045e66396
make onnx package to be optional. (#653)
* putting onnx package to be optional

* update the ci.yml

* add more message of missing ONNX package
2024-02-15 14:09:04 -08:00
Wenbing Li fb2a8c2841
Enhancing CUDA Support in Python Package Build and Testing (#608)
* initial commit

* Add the cuda support for python package

* formt the code

* refine it a little bit
2023-11-27 15:39:52 -08:00
Wenbing Li 2b50a82171
Add an environment variable to explicitly enable/disable azureop build (#527)
* Update ci.yml for Azure Pipelines

* Fix the command lines

* is requirements-dev.txt

* switch to windows

* Update windows task

* Update ci.yml for Azure Pipelines

* add cmake in path on windows

* Update ci.yml for Azure Pipelines

* add explicit azure python build flag

---------

Co-authored-by: Sayan Shaw <52221015+sayanshaw24@users.noreply.github.com>
2023-08-17 16:54:45 -07:00
Wenbing Li 978ada6d60
Add TrieTokenizer for RWKV-like LLM models (#509)
* Add TrieTokenizer for RWKV-like LLM models

* add more tests

* fix the windows build

* downloading file instead of check in the vocab file

* a small bug fixing
2023-08-08 16:47:38 -07:00
RandySheriffH 06d5a8d781
Enable AzureOp packaging (#495)
* generalize azure op

* implement text endpoint

* add parameter to linux whl pipeline

* tune grammer

* 2

* 3

* 4

* 5

* 6

* 7

* 9

* 10

* 11

* 12

* 13

* 14

* 15

* 16

* cancel azure build in ci

* tune cpu tensor check

* try mem name

* address comments

* fix path

* clean env

---------

Co-authored-by: Randy Shuai <rashuai@microsoft.com>
2023-07-20 14:16:08 -07:00
Wenbing Li 981cb049ff
Add a new API for building data processing graph from Huggingface transformers processor/tokenizer (#482)
* initial checkins

* test pass

* basic impl

* first unit test pass

* merge error

* refine a little bit

* add more unit test

* fix unit test

* Fix the unit test.

* add one more whisper audiodecoder test case

* update the docs

* More updates
2023-07-17 16:50:58 -07:00
Wenbing Li b603c0283a
fixing the universal2 python package for macOS (#448) 2023-05-21 21:40:11 -07:00
Wenbing Li adb8efd62b
support batch > 1 in BpeDecoder (#400)
* support batch > 1 in BpeDecoder

* update the shape in helper function
2023-04-19 14:28:56 -07:00
Wenbing Li 6b88f4e31f
port the fixing from the release branch (#375) 2023-03-02 14:12:31 -08:00
Wenbing Li 1af7a56b28
update Python pipelines for release (#353)
* re-enable Python pipelines

* deprecate the direct setup.py call

* pipeline fixing

* run cmake from visual studio

* remove the self check

* support different ort versions

* remove ort 1.9 and add 1.13

* ci pipeline fixing

* fix the test with latest pytorch

* code refinement

* torch version detection

* make package python version more consistent.
2023-02-06 18:23:56 -08:00
Wenbing Li 13d9e27ccd
switch to amd64 Python package in CI pipeline (#348)
* switch to amd64 Python package in CI pipeline

* try to fix binskim errors

* try new cibuildwheel parameter

* fix some binskim issues

* fix some binskim issues

* try switch to x64 again

* undo setup.py change

* because of the quotes?

* explicit AMD64?

* output the debug info

* try fixing a setuptool issue

* cleanup

* refinement
2023-01-23 13:52:18 -08:00
Wenbing Li 67c77d9fbc
align python package version with version.txt (#345)
* align python package version with version.txt

* Update setup.py

Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>

* remove a line

Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
2023-01-12 14:28:32 -08:00
Wenbing Li 77bddf6419
using the cmake bundled in visual studio (#319)
* using the cmake bundled in visual studio

* script is batch file

* switch to vsdevcmd

* remove the quotation mark not needed

* one more change
2022-11-29 09:54:49 -08:00
Wenbing Li 08659eae90
Initial Java API for the JAR package. (#292)
* more C++ code fixing and polish for release

* fixing for android build

* build flags for android release

* add missing exporting function

* imint

* first versoin

* more C++ code fixing and polish for release (#275)

* more C++ code fixing and polish for release

* fixing for android build

* build flags for android release

* add missing exporting function

* support build_id on Python package building (#281)

* support buildid in package building

* undo the change on build.sh

* build.sh issue on macos

* Add `$schema` to `cgmanifest.json` (#284)

Co-authored-by: Jamie Magee <jamie.magee@microsoft.com>

* test package with a simple java app

* demo app

* some fixing for windows platform

* refine the example app

* fix the missing symobls issue for Linux build

* fix the package package build issue

* typo

* a missing change

* fix PythonOp

* fix Android test issue

* one more Android change

* replace build flags in ci pipeline

* android AAR package build

* refine the code for android package

Co-authored-by: Jamie Magee <jamie.magee@gmail.com>
Co-authored-by: Jamie Magee <jamie.magee@microsoft.com>
2022-10-04 16:22:28 -07:00
Wenbing Li f7151e3ae0
support build_id on Python package building (#281)
* support buildid in package building

* undo the change on build.sh

* build.sh issue on macos
2022-08-24 14:39:33 -07:00
Wenbing Li 909acb7ce4
build and packaging script improvement for release (#218)
* integrate opencv

* small fixing

* Add the opencv includes and libs

* refine a little bit

* standardize the output folder.

* fix ctest on Linux

* fix setup.py on output folder change.

* more fixings for CI pipeline

* more fixing 1

* more fixing 2

* more fixing 3

* ci pipeline fixing 1

* ci pipeline fixing 2

* a silly typo...

* ci pipeline fixing 3

* fixing the file copy issue.

* last fixing.

* re-test the fullpath in build_ext.

* One more try

* extent timeout

* mshost.yml indent

* Update mshost.yaml for Azure Pipelines

* cibuild build python versions

* Update wheels.yml

* only build python 3.8/3.9

* Update wheels.yml for Azure Pipelines

* seperate the ci pipeline
2022-05-11 16:51:59 -07:00
Wenbing Li bfbfa5a304
An end-to-end BERT model with pre-/post- processing. (#224)
* bert demo

* add some comments

* support multiple outputs in ONNX model

* code polishing

* encoding issue on Windows platform.
2022-04-20 16:14:46 -07:00
Wenbing Li 4ebb8667da
Integrate the changes from the 0.4 release branch. (#162)
* generate the release packages on release branch

* no pyd file on Windows

* undo setup.py simplify
2021-09-29 13:18:36 -07:00
Wenbing Li 9f3abe20fd
Prepare for 0.4.0 release (#151)
* new CI configuration

* Set up CI with Azure Pipelines

[skip ci]

* install numpy in cibuildwheel

* add pyproject.toml

* upgrade vmImage

* update the build python versions

* remove the pytest

* move the wheel build files

* enable sdist setup.py as well.

* use git command line

* Update wheels.yml for Azure Pipelines

* disable the pypy package for macos;

* fix the external repo code tag

* fix the ctest problem

* fix the unicode 8217.

* fix the locale base test
2021-09-25 00:40:12 -07:00
Wenbing Li 6c3b496e3f
Support the domain name in the PyOp. (#136) 2021-08-26 11:04:45 -07:00
Wenbing Li fd1216b027
fix some build and env issues. (#111) 2021-06-10 09:57:49 -07:00
Wenbing Li 48be0efea2
more refinement on the release doc and script (#99)
* more refinement on the release doc and script

* fix the build script.
2021-06-01 14:02:55 -07:00
Wenbing Li 3806e29421
rename the package to onnxruntime_extensions (#92)
* rename the package to onnxruntime_extensions

* fix the cmake file
2021-05-12 12:02:57 -07:00
Mojimi 4e78e9651b
Create webassembly pipeline (#78) 2021-04-14 14:01:29 +08:00
Xavier Dupré 5b14bc606f
Update CI to publish wheels (#79)
* update CI to publish wheels
* update cmake
2021-04-12 23:27:10 +02:00
Wenbing Li 33027d2578
Enable C++ end to end test with onnxruntime (#56)
* basic changes

build issues fixing

runing on Windows Platform

deploy the ort library in CMake

update gitignore

* Add C++ shared tests

* enable ctest

* fixing the python build issue

* remove cc test

* why does macos needs openmp package?
2021-02-03 11:17:35 -08:00
Xavier Dupré a98c29f6d2
Implement custom operators for sentancepiece (#41)
* implements sentancepiecetokozenizer
* add ragged to sparse
* move one input to attribute
2021-01-27 23:55:50 +01:00
Wenbing Li 4e0af5c582
A more formal build process and the fixing of unix-like environment. (#39)
* enable directly pip package build.

* some link symbols

* fixing on Windows platform

* update the build instruction

* update the ci pipeline

* Fix the Linux and MacOS build.

* Update mshost.yaml

* updat the ci python version

* update the pipeline

* simplify the instruction.

* update according to the comments.

Co-authored-by: Wenbing Li <wenli@MacM1.local>
2021-01-11 13:44:17 -08:00
Wenbing Li 2dfd95e64b
rename the package name to onnxruntime-customops (#36)
* normalize the root package name in Python

* fixing build on Linux

* update the tutorial as well
2020-12-22 20:16:30 -08:00
Wenbing Li 28b2ab1092
ONNXRuntime PyOP support via its custom op APIs (#1)
* WIP: ONNXRuntime PyOP support via its custom op APIs

* fix the build issue with ort 1.5.1 header files.
2020-10-20 12:53:14 -07:00