62 строки
1.7 KiB
Plaintext
62 строки
1.7 KiB
Plaintext
|
==========================
|
||
|
|
x_name: wmt15.bergamot.en
|
||
|
|
y_name: wmt15.microsoft.en
|
||
|
|
|
||
|
|
Bootstrap Resampling Results:
|
||
|
|
x-mean: 0.5187
|
||
|
|
y-mean: 0.6463
|
||
|
|
ties (%): 0.0000
|
||
|
|
x_wins (%): 0.0000
|
||
|
|
y_wins (%): 1.0000
|
||
|
|
|
||
|
|
Paired T-Test Results:
|
||
|
|
statistic: -21.9009
|
||
|
|
p_value: 0.0000
|
||
|
|
Null hypothesis rejected according to t-test.
|
||
|
|
Scores differ significantly across samples.
|
||
|
|
wmt15.microsoft.en outperforms wmt15.bergamot.en.
|
||
|
|
==========================
|
||
|
|
x_name: wmt15.bergamot.en
|
||
|
|
y_name: wmt15.google.en
|
||
|
|
|
||
|
|
Bootstrap Resampling Results:
|
||
|
|
x-mean: 0.5187
|
||
|
|
y-mean: 0.6359
|
||
|
|
ties (%): 0.0000
|
||
|
|
x_wins (%): 0.0000
|
||
|
|
y_wins (%): 1.0000
|
||
|
|
|
||
|
|
Paired T-Test Results:
|
||
|
|
statistic: -18.7876
|
||
|
|
p_value: 0.0000
|
||
|
|
Null hypothesis rejected according to t-test.
|
||
|
|
Scores differ significantly across samples.
|
||
|
|
wmt15.google.en outperforms wmt15.bergamot.en.
|
||
|
|
==========================
|
||
|
|
x_name: wmt15.microsoft.en
|
||
|
|
y_name: wmt15.google.en
|
||
|
|
|
||
|
|
Bootstrap Resampling Results:
|
||
|
|
x-mean: 0.6463
|
||
|
|
y-mean: 0.6359
|
||
|
|
ties (%): 0.0067
|
||
|
|
x_wins (%): 0.9733
|
||
|
|
y_wins (%): 0.0200
|
||
|
|
|
||
|
|
Paired T-Test Results:
|
||
|
|
statistic: 2.9289
|
||
|
|
p_value: 0.0034
|
||
|
|
Null hypothesis rejected according to t-test.
|
||
|
|
Scores differ significantly across samples.
|
||
|
|
wmt15.microsoft.en outperforms wmt15.google.en.
|
||
|
|
|
||
|
|
Summary
|
||
|
|
If system_x is better than system_y then:
|
||
|
|
Null hypothesis rejected according to t-test with p_value=0.05.
|
||
|
|
Scores differ significantly across samples.
|
||
|
|
system_x \ system_y wmt15.bergamot.en wmt15.microsoft.en wmt15.google.en
|
||
|
|
--------------------- ------------------- -------------------- -----------------
|
||
|
|
wmt15.bergamot.en False False
|
||
|
|
wmt15.microsoft.en True True
|
||
|
|
wmt15.google.en True False
|