4bb3a22c45
The new pre/post processing API, replacing ONNXCompose ( #205 )
...
* traced processing module
* before debugging.
* updates
* temporary
* the trace mode pass
* code adjusting for ci pipeline.
* only torch 1.11 support prim:pythonop
* extending sequence processing module.
2022-03-08 16:32:59 -08:00
f418557d85
Update mshost.yaml ( #195 )
...
* Update mshost.yaml
* fix the build error for Android.
* add the spm tokenizer in the selected ops list
2022-01-24 10:57:18 -08:00
dddd85397d
Add android pipeline ( #183 )
...
* build locally success
* update
* fix pipeline
* fix pipeline
* fix pipeline
* fix tool chain file
* validate quickly
* fix tool chain
* update
* finished
* fix bug
* fix bugs
* remove tree
* update android ndk
* fix bugs
* remove java install
* bring back build
* fix model
* resolve conflict
* remove uncessary file
* remove tensorflow_text version 2.6.0
Co-authored-by: Ze Tao <zetao@microsoft.com>
2021-11-09 10:25:03 -08:00
46d096f1af
Fix ::tolower error when locale is not 'C' ( #174 )
...
* add test and implement tolower
* fix locale
* fix locale
Co-authored-by: Ze Tao <zetao@microsoft.com>
2021-10-20 20:59:29 -07:00
448518534c
Add native test for bert tokenizer ( #173 )
...
* add native test for bert tokenizer
* add python test
* fix unicode category
Co-authored-by: Ze Tao <zetao@microsoft.com>
2021-10-19 11:09:38 -07:00
3542434876
Fix op selection ( #147 )
...
* fix segement_extraction
* update
Co-authored-by: Ze Tao <zetao@microsoft.com>
2021-09-11 01:23:35 +08:00
6d7a865913
Disable c++ exceptions in onnxruntime-extensions. ( #143 )
...
* Disable c++ exceptions in onnxruntime-extensions.
* Remove cxx flags for extensions.
* Remove redundant lines.
Co-authored-by: Zuwei Zhao <zuzhao@microsoft.com>
2021-09-09 08:21:40 +08:00
8649d98839
update ( #145 )
...
Co-authored-by: Ze Tao <zetao@microsoft.com>
Co-authored-by: Wenbing Li <10278425+wenbingl@users.noreply.github.com>
2021-09-08 14:26:26 +08:00
aef5ef1ef1
Add BertTokenizer ( #135 )
...
* init
* update
* update
* update
* update
* update
* update
* Modify relative path of generated cmake file.
* update
* udapte
* fix the bug
* update
* fix bugs
Co-authored-by: Ze Tao <zetao@microsoft.com>
Co-authored-by: Wenbing Li <10278425+wenbingl@users.noreply.github.com>
Co-authored-by: Zuwei Zhao <zuzhao@microsoft.com>
2021-08-26 13:50:03 -07:00
f74770feed
Customize string operators list for cmake build. ( #134 )
...
* Customize operators list for build.
* Modify cmake_helper.py to not depend on onnxruntime-extensions build/install.
* Add new line at last.
* Move cmake_helper.py to ci_build/tools/ folder.
* undo some unwanted changes.
* Update CMakeLists.txt
* Update gen_selectedops.py
Co-authored-by: Zuwei Zhao <zuzhao@microsoft.com>
Co-authored-by: Wenbing Li <wenli@microsoft.com>
Co-authored-by: Wenbing Li <10278425+wenbingl@users.noreply.github.com>
2021-08-24 12:31:05 -07:00
Wenbing Li
Mojimi
Zuwei Zhao