ort-customops/tokenizer
Wenbing Li 33027d2578
Enable C++ end to end test with onnxruntime (#56)
* basic changes

build issues fixing

runing on Windows Platform

deploy the ort library in CMake

update gitignore

* Add C++ shared tests

* enable ctest

* fixing the python build issue

* remove cc test

* why does macos needs openmp package?
2021-02-03 11:17:35 -08:00
..
gpt2tok.cc Enable C++ end to end test with onnxruntime (#56) 2021-02-03 11:17:35 -08:00
sentencepiece_tokenizer.cc Change Fix type of nbtest_size into int64 (#54) 2021-01-29 14:21:50 -05:00
sentencepiece_tokenizer.hpp Implement custom operators for sentancepiece (#41) 2021-01-27 23:55:50 +01:00
unicode.cc Add Huggingface GPT2Tokenizer Support (#35) 2020-12-21 17:12:32 -08:00
unicode.h Add Huggingface GPT2Tokenizer Support (#35) 2020-12-21 17:12:32 -08:00