Moses-support Digest, Vol 99, Issue 11

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Training transliteration model outside EMS (Rajnath Patel)
2. Re: Using evaluation metrics other than BLEU in tuning
(Rajnath Patel)
3. Re: Training transliteration model outside EMS (Anoop (?????))


----------------------------------------------------------------------

Message: 1
Date: Tue, 6 Jan 2015 16:27:09 +0530
From: Rajnath Patel <patelrajnath@gmail.com>
Subject: [Moses-support] Training transliteration model outside EMS
To: moses-support <moses-support@mit.edu>
Message-ID:
<CAE-r4unQZ_v6BwACrb9=0yf3GubbhMhBHn=tZaGJ_G=ptoW=fg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi all,

I am trying to use instructions given on moses site (
http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc5) to train a
transliteration model using parallel bilingual corpus, but unable to train.
Actually its not clear to me what exactly 'alignment' switch is suppose to
refer? If possible put a dummy script with a sample training data on moses
site. It will be easy to use. Please have a look at following script and
suggest what I am doing wrong?

Thank You.

*Shell Script:*
$SCRIPTS_ROOTDIR/Transliteration/train-transliteration-module.pl \
--corpus-f /home/raj/anuvad/xlit/data/corpus.mar \
--corpus-e /home/raj/anuvad/xlit/data/corpus.hin \
--alignment /home/raj/anuvad/xlit/align \
--moses-src-dir /home/raj/smt/decoder/mosesdecoder \
--external-bin-dir /home/raj/smt/decoder/mgiza-master/mgizapp/bin \
--input-extension mar \
--output-extension hin \
--srilm-dir /home/raj/smt/decoder/srilm/bin \
--out-dir /home/raj/anuvad/xlit/exp1 \
>& /home/raj/anuvad/xlit/training.out &

Note: '*align*' is empty directory in '*alignment*' switch.

--
Regards:
Raj Nath Patel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150106/0d171c1f/attachment-0001.htm

------------------------------

Message: 2
Date: Tue, 6 Jan 2015 16:39:07 +0530
From: Rajnath Patel <patelrajnath@gmail.com>
Subject: Re: [Moses-support] Using evaluation metrics other than BLEU
in tuning
To: Christophe Servan <christophe.servan@gmail.com>
Cc: moses-support <moses-support@mit.edu>, Hieu Hoang
<hieu.hoang@ed.ac.uk>
Message-ID:
<CAE-r4ukwh9U_6To4EdSPuxQfSZPbWDQ8xst9+BQR+dKOaGXF-g@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Thank you!
Any guideline is available to add new scorer like NIST? If not kindly
suggest.

On Tue, Jan 6, 2015 at 4:18 PM, Christophe Servan <
christophe.servan@gmail.com> wrote:

> Hi,
> As far as I know, you can use these metrics BLEU, BLEUDOC, PER, TER, CDER,
> WER, METEOR.
> The NIST scorer is not included but you can add it to moses if you wish to.
>
> Christophe
>
> 2015-01-06 11:35 GMT+01:00 Rajnath Patel <patelrajnath@gmail.com>:
>
>> Thanks all for your quick response.
>>
>> Hi Christophe,
>>
>> Is it not possible to use any other metrics other than BLEU and TER, like
>> NIST or METEOR? Please suggest.
>>
>> Thank you.
>>
>> On Tue, Jan 6, 2015 at 3:48 PM, Christophe Servan <
>> christophe.servan@gmail.com> wrote:
>>
>>> Hi,
>>> you can use the switch --sctype with mert (and the extractor) like this:
>>>
>>> --sctype BLEU
>>>
>>> You can also specify a combination of metrics (separated with a comma):
>>>
>>> --sctype TER,BLEU
>>>
>>> specify the weights:
>>>
>>> --sctype TER,BLEU --scconfig weights:0.6+0.4
>>>
>>> and use it within mert-moses.pl:
>>>
>>> --mertargs=" --sctype TER,BLEU --scconfig weights:0.6+0.4 "
>>>
>>> Cheers,
>>>
>>> Christophe
>>>
>>>
>>> 2015-01-06 8:33 GMT+01:00 Hieu Hoang <Hieu.Hoang@ed.ac.uk>:
>>>
>>>> I wouldn't use zmert for moses these days. It's rather old and I'm sure
>>>> Moses' tuning script can use other metrics. Hopefully, someone who knows
>>>> how will tell you shortly
>>>>
>>>> On 6 January 2015 at 12:37, HOANG Cong Duy Vu <duyvuleo@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> You can use other metrics by using ZMERT together with Moses:
>>>>> *ZMERT toolkit*: http://cs.jhu.edu/~ozaidan/zmert/
>>>>>
>>>>>
>>>>> --
>>>>> Cheers,
>>>>> Vu
>>>>>
>>>>> On Tue, Jan 6, 2015 at 12:35 PM, Rajnath Patel <patelrajnath@gmail.com
>>>>> > wrote:
>>>>>
>>>>>> Hi All,
>>>>>> As we know, MOSES uses BLEU for evaluation in tuning process . We
>>>>>> want to use evaluation metrics NIST instead of BLEU. Please suggest how it
>>>>>> can be done?
>>>>>>
>>>>>> Thank you.
>>>>>>
>>>>>> --
>>>>>> Regards:
>>>>>> Raj Nath Patel
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Moses-support mailing list
>>>>>> Moses-support@mit.edu
>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Moses-support mailing list
>>>>> Moses-support@mit.edu
>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Hieu Hoang
>>>> Research Associate
>>>> University of Edinburgh
>>>> http://www.hoang.co.uk/hieu
>>>>
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>>>
>>>
>>
>>
>> --
>> Regards:
>> Raj Nath Patel
>>
>>
>


--
Regards:
Raj Nath Patel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150106/432b324f/attachment-0001.htm

------------------------------

Message: 3
Date: Tue, 6 Jan 2015 19:05:49 +0530
From: Anoop (?????) <anoop.kunchukuttan@gmail.com>
Subject: Re: [Moses-support] Training transliteration model outside
EMS
To: Rajnath Patel <patelrajnath@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CADXxMYfY2JyuGu4AGTMDOM_jEzsfLcGGj0o_C9c6MmVDpwiHaw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Rajnath,

This was discussed earlier in the following thread:

https://www.mail-archive.com/moses-support%40mit.edu/msg10740.html

Ratish compiled this document on using transliteration without EMS based on
the discussion:

https://docs.google.com/document/d/1G9RjczZXWGHU6byJFORf6uToItph1jU_piL53wQhGXg/edit

I have tried following the documentation and was able to run it
successfully.

Regards
Anoop.

On Tue, Jan 6, 2015 at 4:27 PM, Rajnath Patel <patelrajnath@gmail.com>
wrote:

> Hi all,
>
> I am trying to use instructions given on moses site (
> http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc5) to train a
> transliteration model using parallel bilingual corpus, but unable to train.
> Actually its not clear to me what exactly 'alignment' switch is suppose to
> refer? If possible put a dummy script with a sample training data on moses
> site. It will be easy to use. Please have a look at following script and
> suggest what I am doing wrong?
>
> Thank You.
>
> *Shell Script:*
> $SCRIPTS_ROOTDIR/Transliteration/train-transliteration-module.pl \
> --corpus-f /home/raj/anuvad/xlit/data/corpus.mar \
> --corpus-e /home/raj/anuvad/xlit/data/corpus.hin \
> --alignment /home/raj/anuvad/xlit/align \
> --moses-src-dir /home/raj/smt/decoder/mosesdecoder \
> --external-bin-dir /home/raj/smt/decoder/mgiza-master/mgizapp/bin \
> --input-extension mar \
> --output-extension hin \
> --srilm-dir /home/raj/smt/decoder/srilm/bin \
> --out-dir /home/raj/anuvad/xlit/exp1 \
> >& /home/raj/anuvad/xlit/training.out &
>
> Note: '*align*' is empty directory in '*alignment*' switch.
>
> --
> Regards:
> Raj Nath Patel
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>


--
I claim to be a simple individual liable to err like any other fellow
mortal. I own, however, that I have humility enough to confess my errors
and to retrace my steps.

http://flightsofthought.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150106/f38f131c/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 99, Issue 11
*********************************************

Tweet