Moses-support Digest, Vol 86, Issue 38

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Regarding Bleu Score (Philipp Koehn)
2. Re: Regarding Bleu Score (Prasanth K)


----------------------------------------------------------------------

Message: 1
Date: Thu, 12 Dec 2013 15:32:06 +0000
From: Philipp Koehn <pkoehn@inf.ed.ac.uk>
Subject: Re: [Moses-support] Regarding Bleu Score
To: Pranjal Das <pranjal4456@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAAFADDD-F36o9uyP6AaueeqAQuoOrdqJGRz+Fr8p=Zt2dZCAbA@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1

Hi,

BLEU scores are generally lower for morphologically richer languages,
both because these add the additional problem of choosing the right
inflection. English has very little inflection, and hence BLEU scores into
English are generally lower.

Please see for instance:
http://matrix.statmt.org/

-phi

On Thu, Dec 12, 2013 at 3:21 PM, Pranjal Das <pranjal4456@gmail.com> wrote:
> Hi all..
> While doing Bengali to English translation i got a bleu score of 7.02
> and doing English to Bengali i got 4.7
>
> why is the difference so high as i am using the same parallel corpus ??
>
>
> Pranjal Das
> Department of Information Technology,
> Institute of Science and Technology,
> Gauhati University,Guwahati,Assam
> Phone- +91-8399879454
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>


------------------------------

Message: 2
Date: Thu, 12 Dec 2013 16:35:30 +0100
From: Prasanth K <prasanthk.ms09@gmail.com>
Subject: Re: [Moses-support] Regarding Bleu Score
To: Pranjal Das <pranjal4456@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CA+n+9-jx_k34eDXmfXPzVzgKvvh_fkH=QbzxyWdN7NPGa42fMQ@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Pranjal,

Well, the co-relation between corpus size and BLEU score is complicated to
attribute them to the corpus size alone, but in this case, yes. That is the
reason why you get low scores.

- Prasanth


On Thu, Dec 12, 2013 at 4:31 PM, Pranjal Das <pranjal4456@gmail.com> wrote:

> Thank you Prasanth....but why i am getting such a low bleu
> score...actually i have a very small corpus..about 2500 sentences..is it
> because of that ???
>
> *Pranjal Das*
> Department of Information Technology,
> Institute of Science and Technology,
> Gauhati University,Guwahati,Assam
> Phone- +91-8399879454
>
>
> On Thu, Dec 12, 2013 at 8:58 PM, Prasanth K <prasanthk.ms09@gmail.com>wrote:
>
>> Hi Pranjal,
>>
>> Its not uncommon to observe such differences when changing the direction
>> of translation. Translation from English to Bengali is relatively harder as
>> Bengali is morphologically rich, making it difficult for the correct
>> surface forms to be generated. Given that BLEU is a pattern matching
>> algorithm comparing surface forms, the drop in the score could be partly
>> attributed to not being able to generate the correct surface forms.
>>
>> You can look at the EuroMatrix, where similar patterns can be observed.
>> Translation from English->Finnish gives better results than the other way
>> around.
>> http://www.statmt.org/matrix/
>>
>> Prasanth
>>
>> On Thu, Dec 12, 2013 at 4:21 PM, Pranjal Das <pranjal4456@gmail.com>wrote:
>>
>>> Hi all..
>>> While doing Bengali to English translation i got a bleu score of 7.02
>>> and doing English to Bengali i got 4.7
>>>
>>> why is the difference so high as i am using the same parallel corpus ??
>>>
>>>
>>> *Pranjal Das*
>>> Department of Information Technology,
>>> Institute of Science and Technology,
>>> Gauhati University,Guwahati,Assam
>>> Phone- +91-8399879454
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>>
>> --
>> "Theories have four stages of acceptance. i) this is worthless nonsense;
>> ii) this is an interesting, but perverse, point of view, iii) this is true,
>> but quite unimportant; iv) I always said so."
>>
>> --- J.B.S. Haldane
>>
>
>


--
"Theories have four stages of acceptance. i) this is worthless nonsense;
ii) this is an interesting, but perverse, point of view, iii) this is true,
but quite unimportant; iv) I always said so."

--- J.B.S. Haldane
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20131212/68fc70ee/attachment-0001.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 86, Issue 38
*********************************************

Tweet