Moses-support Digest, Vol 99, Issue 10

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Using evaluation metrics other than BLEU in tuning
(Christophe Servan)
2. Re: Using evaluation metrics other than BLEU in tuning
(Rajnath Patel)
3. Re: Using evaluation metrics other than BLEU in tuning
(Christophe Servan)


----------------------------------------------------------------------

Message: 1
Date: Tue, 6 Jan 2015 11:18:43 +0100
From: Christophe Servan <christophe.servan@gmail.com>
Subject: Re: [Moses-support] Using evaluation metrics other than BLEU
in tuning
To: Rajnath Patel <patelrajnath@gmail.com>, moses-support
<moses-support@mit.edu>
Cc: Hieu Hoang <hieu.hoang@ed.ac.uk>
Message-ID:
<CAAsGDko=svL+Zpzw6QoxaPdq84XC7=B5mZe4Ar0WHOJ__amNHA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,
you can use the switch --sctype with mert (and the extractor) like this:

--sctype BLEU

You can also specify a combination of metrics (separated with a comma):

--sctype TER,BLEU

specify the weights:

--sctype TER,BLEU --scconfig weights:0.6+0.4

and use it within mert-moses.pl:

--mertargs=" --sctype TER,BLEU --scconfig weights:0.6+0.4 "

Cheers,

Christophe


2015-01-06 8:33 GMT+01:00 Hieu Hoang <Hieu.Hoang@ed.ac.uk>:

> I wouldn't use zmert for moses these days. It's rather old and I'm sure
> Moses' tuning script can use other metrics. Hopefully, someone who knows
> how will tell you shortly
>
> On 6 January 2015 at 12:37, HOANG Cong Duy Vu <duyvuleo@gmail.com> wrote:
>
>> Hi,
>>
>> You can use other metrics by using ZMERT together with Moses:
>> *ZMERT toolkit*: http://cs.jhu.edu/~ozaidan/zmert/
>>
>>
>> --
>> Cheers,
>> Vu
>>
>> On Tue, Jan 6, 2015 at 12:35 PM, Rajnath Patel <patelrajnath@gmail.com>
>> wrote:
>>
>>> Hi All,
>>> As we know, MOSES uses BLEU for evaluation in tuning process . We want
>>> to use evaluation metrics NIST instead of BLEU. Please suggest how it can
>>> be done?
>>>
>>> Thank you.
>>>
>>> --
>>> Regards:
>>> Raj Nath Patel
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
>
> --
> Hieu Hoang
> Research Associate
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150106/6ae29070/attachment-0001.htm

------------------------------

Message: 2
Date: Tue, 6 Jan 2015 16:05:14 +0530
From: Rajnath Patel <patelrajnath@gmail.com>
Subject: Re: [Moses-support] Using evaluation metrics other than BLEU
in tuning
To: Christophe Servan <christophe.servan@gmail.com>
Cc: moses-support <moses-support@mit.edu>, Hieu Hoang
<hieu.hoang@ed.ac.uk>
Message-ID:
<CAE-r4um+4v9r=yN2mNYA0ti2LR0FvHDyMd2H77sw6rfpe=pWnA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Thanks all for your quick response.

Hi Christophe,

Is it not possible to use any other metrics other than BLEU and TER, like
NIST or METEOR? Please suggest.

Thank you.

On Tue, Jan 6, 2015 at 3:48 PM, Christophe Servan <
christophe.servan@gmail.com> wrote:

> Hi,
> you can use the switch --sctype with mert (and the extractor) like this:
>
> --sctype BLEU
>
> You can also specify a combination of metrics (separated with a comma):
>
> --sctype TER,BLEU
>
> specify the weights:
>
> --sctype TER,BLEU --scconfig weights:0.6+0.4
>
> and use it within mert-moses.pl:
>
> --mertargs=" --sctype TER,BLEU --scconfig weights:0.6+0.4 "
>
> Cheers,
>
> Christophe
>
>
> 2015-01-06 8:33 GMT+01:00 Hieu Hoang <Hieu.Hoang@ed.ac.uk>:
>
>> I wouldn't use zmert for moses these days. It's rather old and I'm sure
>> Moses' tuning script can use other metrics. Hopefully, someone who knows
>> how will tell you shortly
>>
>> On 6 January 2015 at 12:37, HOANG Cong Duy Vu <duyvuleo@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> You can use other metrics by using ZMERT together with Moses:
>>> *ZMERT toolkit*: http://cs.jhu.edu/~ozaidan/zmert/
>>>
>>>
>>> --
>>> Cheers,
>>> Vu
>>>
>>> On Tue, Jan 6, 2015 at 12:35 PM, Rajnath Patel <patelrajnath@gmail.com>
>>> wrote:
>>>
>>>> Hi All,
>>>> As we know, MOSES uses BLEU for evaluation in tuning process . We want
>>>> to use evaluation metrics NIST instead of BLEU. Please suggest how it can
>>>> be done?
>>>>
>>>> Thank you.
>>>>
>>>> --
>>>> Regards:
>>>> Raj Nath Patel
>>>>
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>>
>> --
>> Hieu Hoang
>> Research Associate
>> University of Edinburgh
>> http://www.hoang.co.uk/hieu
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>


--
Regards:
Raj Nath Patel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150106/ad295daa/attachment-0001.htm

------------------------------

Message: 3
Date: Tue, 6 Jan 2015 11:48:46 +0100
From: Christophe Servan <christophe.servan@gmail.com>
Subject: Re: [Moses-support] Using evaluation metrics other than BLEU
in tuning
To: Rajnath Patel <patelrajnath@gmail.com>
Cc: moses-support <moses-support@mit.edu>, Hieu Hoang
<hieu.hoang@ed.ac.uk>
Message-ID:
<CAAsGDkqs7MNqKYwcr2-UrxfpJ_pRVRA9N2kO4DRRxrpLi_P8kw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,
As far as I know, you can use these metrics BLEU, BLEUDOC, PER, TER, CDER,
WER, METEOR.
The NIST scorer is not included but you can add it to moses if you wish to.

Christophe

2015-01-06 11:35 GMT+01:00 Rajnath Patel <patelrajnath@gmail.com>:

> Thanks all for your quick response.
>
> Hi Christophe,
>
> Is it not possible to use any other metrics other than BLEU and TER, like
> NIST or METEOR? Please suggest.
>
> Thank you.
>
> On Tue, Jan 6, 2015 at 3:48 PM, Christophe Servan <
> christophe.servan@gmail.com> wrote:
>
>> Hi,
>> you can use the switch --sctype with mert (and the extractor) like this:
>>
>> --sctype BLEU
>>
>> You can also specify a combination of metrics (separated with a comma):
>>
>> --sctype TER,BLEU
>>
>> specify the weights:
>>
>> --sctype TER,BLEU --scconfig weights:0.6+0.4
>>
>> and use it within mert-moses.pl:
>>
>> --mertargs=" --sctype TER,BLEU --scconfig weights:0.6+0.4 "
>>
>> Cheers,
>>
>> Christophe
>>
>>
>> 2015-01-06 8:33 GMT+01:00 Hieu Hoang <Hieu.Hoang@ed.ac.uk>:
>>
>>> I wouldn't use zmert for moses these days. It's rather old and I'm sure
>>> Moses' tuning script can use other metrics. Hopefully, someone who knows
>>> how will tell you shortly
>>>
>>> On 6 January 2015 at 12:37, HOANG Cong Duy Vu <duyvuleo@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> You can use other metrics by using ZMERT together with Moses:
>>>> *ZMERT toolkit*: http://cs.jhu.edu/~ozaidan/zmert/
>>>>
>>>>
>>>> --
>>>> Cheers,
>>>> Vu
>>>>
>>>> On Tue, Jan 6, 2015 at 12:35 PM, Rajnath Patel <patelrajnath@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi All,
>>>>> As we know, MOSES uses BLEU for evaluation in tuning process . We want
>>>>> to use evaluation metrics NIST instead of BLEU. Please suggest how it can
>>>>> be done?
>>>>>
>>>>> Thank you.
>>>>>
>>>>> --
>>>>> Regards:
>>>>> Raj Nath Patel
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Moses-support mailing list
>>>>> Moses-support@mit.edu
>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>>>
>>>
>>>
>>> --
>>> Hieu Hoang
>>> Research Associate
>>> University of Edinburgh
>>> http://www.hoang.co.uk/hieu
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>
>
> --
> Regards:
> Raj Nath Patel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150106/8d133949/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 99, Issue 10
*********************************************

Tweet