62 строки
1.7 KiB
Plaintext
62 строки
1.7 KiB
Plaintext
==========================
|
|
x_name: wmt09.bergamot.en
|
|
y_name: wmt09.microsoft.en
|
|
|
|
Bootstrap Resampling Results:
|
|
x-mean: 0.4093
|
|
y-mean: 0.5364
|
|
ties (%): 0.0000
|
|
x_wins (%): 0.0000
|
|
y_wins (%): 1.0000
|
|
|
|
Paired T-Test Results:
|
|
statistic: -27.2997
|
|
p_value: 0.0000
|
|
Null hypothesis rejected according to t-test.
|
|
Scores differ significantly across samples.
|
|
wmt09.microsoft.en outperforms wmt09.bergamot.en.
|
|
==========================
|
|
x_name: wmt09.bergamot.en
|
|
y_name: wmt09.google.en
|
|
|
|
Bootstrap Resampling Results:
|
|
x-mean: 0.4093
|
|
y-mean: 0.5129
|
|
ties (%): 0.0000
|
|
x_wins (%): 0.0000
|
|
y_wins (%): 1.0000
|
|
|
|
Paired T-Test Results:
|
|
statistic: -21.2985
|
|
p_value: 0.0000
|
|
Null hypothesis rejected according to t-test.
|
|
Scores differ significantly across samples.
|
|
wmt09.google.en outperforms wmt09.bergamot.en.
|
|
==========================
|
|
x_name: wmt09.microsoft.en
|
|
y_name: wmt09.google.en
|
|
|
|
Bootstrap Resampling Results:
|
|
x-mean: 0.5364
|
|
y-mean: 0.5129
|
|
ties (%): 0.0000
|
|
x_wins (%): 1.0000
|
|
y_wins (%): 0.0000
|
|
|
|
Paired T-Test Results:
|
|
statistic: 7.8084
|
|
p_value: 0.0000
|
|
Null hypothesis rejected according to t-test.
|
|
Scores differ significantly across samples.
|
|
wmt09.microsoft.en outperforms wmt09.google.en.
|
|
|
|
Summary
|
|
If system_x is better than system_y then:
|
|
Null hypothesis rejected according to t-test with p_value=0.05.
|
|
Scores differ significantly across samples.
|
|
system_x \ system_y wmt09.bergamot.en wmt09.microsoft.en wmt09.google.en
|
|
--------------------- ------------------- -------------------- -----------------
|
|
wmt09.bergamot.en False False
|
|
wmt09.microsoft.en True True
|
|
wmt09.google.en True False
|