Moses-support Digest, Vol 95, Issue 43

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Query regarding unsupervised transliteration support in
moses (Hieu Hoang)
2. Re: Query regarding unsupervised transliteration support in
moses (Philipp Koehn)
3. Re: Query regarding unsupervised transliteration support in
moses (Ratish Puduppully)


----------------------------------------------------------------------

Message: 1
Date: Fri, 26 Sep 2014 18:21:26 +0100
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] Query regarding unsupervised
transliteration support in moses
To: Ratish Puduppully <ratish123@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAEKMkbhUsd4gnM0=DPAzAQmSzwXpdYnzkGJh6kbUzCohHrWavg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

The moses.ini file looks ok to me. How do you know that the transliteration
pt isn't being consulted? Are there any entries in the transliteration pt?

On 26 September 2014 12:43, Ratish Puduppully <ratish123@gmail.com> wrote:

> Hi,
> Thanks for introducing support for Unsupervised transliteration in moses.
> I was able to execute the same as mentioned using EMS.
> I wish to implement it using moses training scripts. I performed the
> following steps:
>
> - Set up srilm, mgiza
> - Execute train-transliteration-module.pl
> - It generated phrase table for transliteration
> - I then train moses for translation. I add two arguments: -post-decoding-translit
> yes, -transliteration-phrase-table <path to transliteration phrase table>
>
> Accordingly, my moses.ini has additional entries
> But during decoding, my transliteration phrase table is not being
> consulted. Thus the OOV words are not being transliterated.
> I have attached my moses.ini file
> Please help resolve the issue.
>
> --
> Ratish Puduppully
> Research Engineer
> Center for Indian Language Technology <http://www.cfilt.iitb.ac.in/>
> Computer Science and Engineering Department <http://www.cse.iitb.ac.in/>
> IIT Bombay <http://www.iitb.ac.in/>
> email: ratishp@cse.iitb.ac.in
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>


--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140926/f35fe09c/attachment-0001.htm

------------------------------

Message: 2
Date: Fri, 26 Sep 2014 13:41:03 -0400
From: Philipp Koehn <pkoehn@inf.ed.ac.uk>
Subject: Re: [Moses-support] Query regarding unsupervised
transliteration support in moses
To: Ratish Puduppully <ratish123@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDAMPYF_s0hdzR6dCwAu_S1iCW1OLOzHjKWEV5o5hiGwvg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,

did you run post-decoding transliteration?

The original Moses run does not perform the transliteration, it only
produces
additional output that is used in the post-decoding transliterations step.

That step is done by running $MOSES/scripts/Transliteration/
post-decoding-transliteration.pl

Post-decoding transliterations is fully integrated into EMS, but you can
also
run it by hand. Unfortunately there is no detailed documentation about the
script, but its options are pretty self-explanatory.

-phi

On Fri, Sep 26, 2014 at 7:43 AM, Ratish Puduppully <ratish123@gmail.com>
wrote:

> Hi,
> Thanks for introducing support for Unsupervised transliteration in moses.
> I was able to execute the same as mentioned using EMS.
> I wish to implement it using moses training scripts. I performed the
> following steps:
>
> - Set up srilm, mgiza
> - Execute train-transliteration-module.pl
> - It generated phrase table for transliteration
> - I then train moses for translation. I add two arguments: -post-decoding-translit
> yes, -transliteration-phrase-table <path to transliteration phrase table>
>
> Accordingly, my moses.ini has additional entries
> But during decoding, my transliteration phrase table is not being
> consulted. Thus the OOV words are not being transliterated.
> I have attached my moses.ini file
> Please help resolve the issue.
>
> --
> Ratish Puduppully
> Research Engineer
> Center for Indian Language Technology <http://www.cfilt.iitb.ac.in/>
> Computer Science and Engineering Department <http://www.cse.iitb.ac.in/>
> IIT Bombay <http://www.iitb.ac.in/>
> email: ratishp@cse.iitb.ac.in
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140926/a22d9fa8/attachment-0001.htm

------------------------------

Message: 3
Date: Sat, 27 Sep 2014 16:19:40 +0530
From: Ratish Puduppully <ratish123@gmail.com>
Subject: Re: [Moses-support] Query regarding unsupervised
transliteration support in moses
To: Philipp Koehn <pkoehn@inf.ed.ac.uk>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAEb1jG76r6=kiow4M3wmcAZPWPCLYh+nr5CHMqZvQL0S5MfxNg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Thanks Prof.
I had missed the post-decoding-transliteration.pl step.
I executed the same and got the output. The output is looking good and
there is BLEU score improvement.
Compliments to the efforts of the team.

Also created a small doc which goes through the steps for Unsupervised
Transliteration without ems
https://docs.google.com/document/d/1G9RjczZXWGHU6byJFORf6uToItph1jU_piL53wQhGXg/edit?usp=sharing

Regards,

On Fri, Sep 26, 2014 at 11:11 PM, Philipp Koehn <pkoehn@inf.ed.ac.uk> wrote:

> Hi,
>
> did you run post-decoding transliteration?
>
> The original Moses run does not perform the transliteration, it only
> produces
> additional output that is used in the post-decoding transliterations step.
>
> That step is done by running $MOSES/scripts/Transliteration/
> post-decoding-transliteration.pl
>
> Post-decoding transliterations is fully integrated into EMS, but you can
> also
> run it by hand. Unfortunately there is no detailed documentation about the
> script, but its options are pretty self-explanatory.
>
> -phi
>
> On Fri, Sep 26, 2014 at 7:43 AM, Ratish Puduppully <ratish123@gmail.com>
> wrote:
>
>> Hi,
>> Thanks for introducing support for Unsupervised transliteration in moses.
>> I was able to execute the same as mentioned using EMS.
>> I wish to implement it using moses training scripts. I performed the
>> following steps:
>>
>> - Set up srilm, mgiza
>> - Execute train-transliteration-module.pl
>> - It generated phrase table for transliteration
>> - I then train moses for translation. I add two arguments: -post-decoding-translit
>> yes, -transliteration-phrase-table <path to transliteration phrase table>
>>
>> Accordingly, my moses.ini has additional entries
>> But during decoding, my transliteration phrase table is not being
>> consulted. Thus the OOV words are not being transliterated.
>> I have attached my moses.ini file
>> Please help resolve the issue.
>>
>> --
>> Ratish Puduppully
>> Research Engineer
>> Center for Indian Language Technology <http://www.cfilt.iitb.ac.in/>
>> Computer Science and Engineering Department <http://www.cse.iitb.ac.in/>
>> IIT Bombay <http://www.iitb.ac.in/>
>> email: ratishp@cse.iitb.ac.in
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>


--
Ratish Puduppully
Research Engineer
Center for Indian Language Technology <http://www.cfilt.iitb.ac.in/>
Computer Science and Engineering Department <http://www.cse.iitb.ac.in/>
IIT Bombay <http://www.iitb.ac.in/>
email: ratishp@cse.iitb.ac.in
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140927/44ade166/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 95, Issue 43
*********************************************

Tweet