Marco Castelluccio c8c896d640 Add some languages we are planning to support to the comparison (#130) ... * Support evaluating languages that do not yet have a Bergamot model available Fixes #106 * Move cached results to top level evaluation directory * Add Indonesian * Find right NLLB lang code dynamically Fixes #103 * Decrease partition size for Google translate * Update evaluation results [skip ci] * Update model registry [skip ci] * Add Bosnian * Add Danish * Add Swedish * Update evaluation results [skip ci] * Update model registry [skip ci] * Add Galician * Add Hindi * Add Croatian * Update evaluation results [skip ci] * Update model registry [skip ci] * Add Japanese * Install dependencies required for Japanese tokenization in sacrebleu * Install dependencies required for Korean tokenization in sacrebleu * Update evaluation results [skip ci] * Update model registry [skip ci] * Add Romanian * Retry up to 7 times when Google translate's service is unavailable * Update evaluation results [skip ci] * Update model registry [skip ci] * Build a single report with all models and results * Add Slovak * Update evaluation results [skip ci] * Update model registry [skip ci] * Add Vietnamese * Update evaluation results [skip ci] * Add Latvian * Update evaluation results [skip ci] * Update model registry [skip ci] * Add Serbian * Update evaluation results [skip ci] * Update model registry [skip ci] * Add Thai * Update evaluation results [skip ci] * Update model registry [skip ci] * Add Korean * Update evaluation results [skip ci] * Update model registry [skip ci] * Add Chinese * Update evaluation results [skip ci] * Update model registry [skip ci] * Move logic to fill Bergamot supported languages in a separate function		2024-01-30 21:39:07 +01:00
..
flores-dev.argos.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.argos.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.argos.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.en-ja.cometcompare	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.google.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.google.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.google.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.microsoft.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.microsoft.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.microsoft.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.nllb.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.nllb.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-dev.nllb.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.argos.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.argos.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.argos.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.en-ja.cometcompare	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.google.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.google.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.google.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.microsoft.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.microsoft.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.microsoft.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.nllb.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.nllb.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
flores-test.nllb.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
iwslt17.argos.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
iwslt17.argos.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
iwslt17.argos.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
iwslt17.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
iwslt17.en-ja.cometcompare	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
iwslt17.google.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
iwslt17.google.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
iwslt17.google.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
iwslt17.microsoft.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
iwslt17.microsoft.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
iwslt17.microsoft.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
iwslt17.nllb.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
iwslt17.nllb.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
iwslt17.nllb.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt20.argos.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt20.argos.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt20.argos.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt20.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt20.en-ja.cometcompare	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt20.google.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt20.google.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt20.google.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt20.microsoft.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt20.microsoft.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt20.microsoft.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt20.nllb.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt20.nllb.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt20.nllb.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt21.argos.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt21.argos.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt21.argos.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt21.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt21.en-ja.cometcompare	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt21.google.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt21.google.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt21.google.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt21.microsoft.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt21.microsoft.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt21.microsoft.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt21.nllb.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt21.nllb.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt21.nllb.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt22.argos.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt22.argos.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt22.argos.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt22.en	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt22.en-ja.cometcompare	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt22.google.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt22.google.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt22.google.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt22.microsoft.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt22.microsoft.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt22.microsoft.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt22.nllb.ja	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt22.nllb.ja.bleu	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00
wmt22.nllb.ja.comet	Add some languages we are planning to support to the comparison (#130)	2024-01-30 21:39:07 +01:00