Patch WASM artifacts to run optimized (wormhole enabled) inference (#68)

* A script to patch the wasm artifacts to use wormhole via
   APIs that instantiate WASM module
* Updated README
* Load just production ready models
* Shallow clone bergamot-models repo since it has such a large history
* Improved wasm test_page
 - test page can load all 5 language pairs
 - Use intgemm.alpha* models
* Refactor the code that patches wasm artifacts to enable wormhole

Co-authored-by: Andre Natal <anatal@gmail.com>
Co-authored-by: Motin <motin@motin.eu>

.github/workflows/macos-custom-marian-wasm.yml

@@ -33,6 +33,10 @@ jobs:
         working-directory: build-wasm
         run: emmake make -j2
 
+      - name: Instantiate simd wormhole
+        working-directory: build-wasm
+        run: bash ../wasm/patch-artifacts-enable-wormhole.sh
+
       - name: Check artifacts
         working-directory: build-wasm
         run: |

README.md

@@ -46,8 +46,8 @@ Bergamot translator provides a unified API for ([Marian NMT](https://marian-nmt.
     If you want to package bergamot project specific models, please follow these instructions:
     ```bash
     mkdir models
-    git clone https://github.com/mozilla-applied-ml/bergamot-models
-    cp -rf bergamot-models/* models
+    git clone --depth 1 --branch main --single-branch https://github.com/mozilla-applied-ml/bergamot-models
+    cp -rf bergamot-models/prod/* models
     gunzip models/*/*
     ```
 
@@ -75,11 +75,16 @@ Bergamot translator provides a unified API for ([Marian NMT](https://marian-nmt.
             emmake make -j
             ```
 
-    The artefacts (.js and .wasm files) will be available in `wasm` folder of build directory ("build-wasm" in this case).
+        The wasm artifacts (.js and .wasm files) will be available in `wasm` folder of build directory ("build-wasm" in this case).
+
+    3. Enable SIMD Wormhole via Wasm instantiation API in generated artifacts
+        ```bash
+        bash ../wasm/patch-artifacts-enable-wormhole.sh
+        ```
 
 #### Recompiling
-As long as you don't update any submodule, just follow steps in `4.ii` to recompile.\
-If you update a submodule, execute following command before executing steps in `4.ii` to recompile.
+As long as you don't update any submodule, just follow steps in `4.ii` and `4.iii` to recompile.\
+If you update a submodule, execute following command before executing steps in `4.ii` and `4.iii` to recompile.
 ```bash
 git submodule update --init --recursive
 ```

wasm/patch-artifacts-enable-wormhole.sh

@@ -0,0 +1,7 @@
+#!/bin/bash
+
+echo "Patching wasm artifacts to enable wormhole via APIs that compile and instantiate wasm module"
+sed -i.bak 's/var result = WebAssembly.instantiateStreaming(response, info);/var result = WebAssembly.instantiateStreaming(response, info, {simdWormhole:true});/g' wasm/bergamot-translator-worker.js
+sed -i.bak 's/return WebAssembly.instantiate(binary, info);/return WebAssembly.instantiate(binary, info, {simdWormhole:true});/g' wasm/bergamot-translator-worker.js
+sed -i.bak 's/var module = new WebAssembly.Module(bytes);/var module = new WebAssembly.Module(bytes, {simdWormhole:true});/g' wasm/bergamot-translator-worker.js
+echo "Done"

wasm/test_page/bergamot.html

@@ -32,6 +32,9 @@
     <label>Choose the model to use</label>
     <input type="radio" name="modellang" value="enes"/><label>English to Spanish</label>
     <input type="radio" name="modellang" value="esen" checked/><label>Spanish to English</label>
+    <input type="radio" name="modellang" value="eten" checked/><label>Estonian to English</label>
+    <input type="radio" name="modellang" value="enet" checked/><label>English to Estonian</label>
+    <input type="radio" name="modellang" value="ende" checked/><label>English to German</label>
     <input type="button" id="load" value="Load Model"/>
 </div>
 
@@ -70,7 +73,7 @@ En consecuencia, durante el año 2011 se introdujeron 180 proyectos de ley que r
     // Set the Model Configuration as YAML formatted string.
     // For available configuration options, please check: https://marian-nmt.github.io/docs/cmd/marian-decoder/
     const modelConfig = `models:
-  - /${languagePair}/model.${languagePair}.npz
+  - /${languagePair}/model.${languagePair}.intgemm.alphas.bin
 vocabs:
   - /${vocabLanguagePair}/vocab.${vocabLanguagePair}.spm
   - /${vocabLanguagePair}/vocab.${vocabLanguagePair}.spm

wasm/test_page/start_server.sh

@@ -1,8 +1,10 @@
 #!/bin/bash
-
+echo "Start: Copying artifacts in local folder------"
 cp ../../build-wasm/wasm/bergamot-translator-worker.data .
 cp ../../build-wasm/wasm/bergamot-translator-worker.js .
 cp ../../build-wasm/wasm/bergamot-translator-worker.wasm .
 cp ../../build-wasm/wasm/bergamot-translator-worker.worker.js .
+
 npm install
+echo "Start httpserver"
 node bergamot-httpserver.js