Patch WASM artifacts to run optimized (wormhole enabled) inference (#68)

* A script to patch the wasm artifacts to use wormhole via
   APIs that instantiate WASM module
* Updated README
* Load just production ready models
* Shallow clone bergamot-models repo since it has such a large history
* Improved wasm test_page
 - test page can load all 5 language pairs
 - Use intgemm.alpha* models
* Refactor the code that patches wasm artifacts to enable wormhole

Co-authored-by: Andre Natal <anatal@gmail.com>
Co-authored-by: Motin <motin@motin.eu>
This commit is contained in:
abhi-agg 2021-03-24 17:10:42 +01:00 коммит произвёл GitHub
Родитель 34228d37bf
Коммит 12e9232066
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
5 изменённых файлов: 28 добавлений и 7 удалений

Просмотреть файл

@ -33,6 +33,10 @@ jobs:
working-directory: build-wasm
run: emmake make -j2
- name: Instantiate simd wormhole
working-directory: build-wasm
run: bash ../wasm/patch-artifacts-enable-wormhole.sh
- name: Check artifacts
working-directory: build-wasm
run: |

Просмотреть файл

@ -46,8 +46,8 @@ Bergamot translator provides a unified API for ([Marian NMT](https://marian-nmt.
If you want to package bergamot project specific models, please follow these instructions:
```bash
mkdir models
git clone https://github.com/mozilla-applied-ml/bergamot-models
cp -rf bergamot-models/* models
git clone --depth 1 --branch main --single-branch https://github.com/mozilla-applied-ml/bergamot-models
cp -rf bergamot-models/prod/* models
gunzip models/*/*
```
@ -75,11 +75,16 @@ Bergamot translator provides a unified API for ([Marian NMT](https://marian-nmt.
emmake make -j
```
The artefacts (.js and .wasm files) will be available in `wasm` folder of build directory ("build-wasm" in this case).
The wasm artifacts (.js and .wasm files) will be available in `wasm` folder of build directory ("build-wasm" in this case).
3. Enable SIMD Wormhole via Wasm instantiation API in generated artifacts
```bash
bash ../wasm/patch-artifacts-enable-wormhole.sh
```
#### Recompiling
As long as you don't update any submodule, just follow steps in `4.ii` to recompile.\
If you update a submodule, execute following command before executing steps in `4.ii` to recompile.
As long as you don't update any submodule, just follow steps in `4.ii` and `4.iii` to recompile.\
If you update a submodule, execute following command before executing steps in `4.ii` and `4.iii` to recompile.
```bash
git submodule update --init --recursive
```

Просмотреть файл

@ -0,0 +1,7 @@
#!/bin/bash
echo "Patching wasm artifacts to enable wormhole via APIs that compile and instantiate wasm module"
sed -i.bak 's/var result = WebAssembly.instantiateStreaming(response, info);/var result = WebAssembly.instantiateStreaming(response, info, {simdWormhole:true});/g' wasm/bergamot-translator-worker.js
sed -i.bak 's/return WebAssembly.instantiate(binary, info);/return WebAssembly.instantiate(binary, info, {simdWormhole:true});/g' wasm/bergamot-translator-worker.js
sed -i.bak 's/var module = new WebAssembly.Module(bytes);/var module = new WebAssembly.Module(bytes, {simdWormhole:true});/g' wasm/bergamot-translator-worker.js
echo "Done"

Просмотреть файл

@ -32,6 +32,9 @@
<label>Choose the model to use</label>
<input type="radio" name="modellang" value="enes"/><label>English to Spanish</label>
<input type="radio" name="modellang" value="esen" checked/><label>Spanish to English</label>
<input type="radio" name="modellang" value="eten" checked/><label>Estonian to English</label>
<input type="radio" name="modellang" value="enet" checked/><label>English to Estonian</label>
<input type="radio" name="modellang" value="ende" checked/><label>English to German</label>
<input type="button" id="load" value="Load Model"/>
</div>
@ -70,7 +73,7 @@ En consecuencia, durante el año 2011 se introdujeron 180 proyectos de ley que r
// Set the Model Configuration as YAML formatted string.
// For available configuration options, please check: https://marian-nmt.github.io/docs/cmd/marian-decoder/
const modelConfig = `models:
- /${languagePair}/model.${languagePair}.npz
- /${languagePair}/model.${languagePair}.intgemm.alphas.bin
vocabs:
- /${vocabLanguagePair}/vocab.${vocabLanguagePair}.spm
- /${vocabLanguagePair}/vocab.${vocabLanguagePair}.spm

Просмотреть файл

@ -1,8 +1,10 @@
#!/bin/bash
echo "Start: Copying artifacts in local folder------"
cp ../../build-wasm/wasm/bergamot-translator-worker.data .
cp ../../build-wasm/wasm/bergamot-translator-worker.js .
cp ../../build-wasm/wasm/bergamot-translator-worker.wasm .
cp ../../build-wasm/wasm/bergamot-translator-worker.worker.js .
npm install
echo "Start httpserver"
node bergamot-httpserver.js