Marco Castelluccio c8c896d640 Add some languages we are planning to support to the comparison (#130) ... * Support evaluating languages that do not yet have a Bergamot model available Fixes #106 * Move cached results to top level evaluation directory * Add Indonesian * Find right NLLB lang code dynamically Fixes #103 * Decrease partition size for Google translate * Update evaluation results [skip ci] * Update model registry [skip ci] * Add Bosnian * Add Danish * Add Swedish * Update evaluation results [skip ci] * Update model registry [skip ci] * Add Galician * Add Hindi * Add Croatian * Update evaluation results [skip ci] * Update model registry [skip ci] * Add Japanese * Install dependencies required for Japanese tokenization in sacrebleu * Install dependencies required for Korean tokenization in sacrebleu * Update evaluation results [skip ci] * Update model registry [skip ci] * Add Romanian * Retry up to 7 times when Google translate's service is unavailable * Update evaluation results [skip ci] * Update model registry [skip ci] * Build a single report with all models and results * Add Slovak * Update evaluation results [skip ci] * Update model registry [skip ci] * Add Vietnamese * Update evaluation results [skip ci] * Add Latvian * Update evaluation results [skip ci] * Update model registry [skip ci] * Add Serbian * Update evaluation results [skip ci] * Update model registry [skip ci] * Add Thai * Update evaluation results [skip ci] * Update model registry [skip ci] * Add Korean * Update evaluation results [skip ci] * Update model registry [skip ci] * Add Chinese * Update evaluation results [skip ci] * Update model registry [skip ci] * Move logic to fill Bergamot supported languages in a separate function		2024-01-30 21:39:07 +01:00
..
flores-dev.argos.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.argos.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.argos.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.bergamot.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.bergamot.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.bergamot.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.google.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.google.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.microsoft.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.microsoft.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.nllb.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.nllb.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.nllb.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.opusmt.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.opusmt.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.opusmt.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.tr	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.tr-en.cometcompare	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.argos.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.argos.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.argos.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.bergamot.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.bergamot.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.bergamot.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.google.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.google.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.microsoft.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.microsoft.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.nllb.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.nllb.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.nllb.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.opusmt.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.opusmt.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.opusmt.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.tr	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.tr-en.cometcompare	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt16.argos.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt16.argos.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt16.argos.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt16.bergamot.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt16.bergamot.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt16.bergamot.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt16.google.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt16.google.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt16.microsoft.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt16.microsoft.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt16.nllb.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt16.nllb.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt16.nllb.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt16.opusmt.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt16.opusmt.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt16.opusmt.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt16.tr	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt16.tr-en.cometcompare	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt17.argos.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt17.argos.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt17.argos.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt17.bergamot.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt17.bergamot.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt17.bergamot.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt17.google.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt17.google.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt17.microsoft.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt17.microsoft.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt17.nllb.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt17.nllb.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt17.nllb.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt17.opusmt.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt17.opusmt.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt17.opusmt.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt17.tr	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt17.tr-en.cometcompare	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt18.argos.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt18.argos.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt18.argos.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt18.bergamot.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt18.bergamot.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt18.bergamot.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt18.google.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt18.google.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt18.microsoft.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt18.microsoft.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt18.nllb.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt18.nllb.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt18.nllb.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt18.opusmt.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt18.opusmt.en.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt18.opusmt.en.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt18.tr	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt18.tr-en.cometcompare	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00