Moses-support Digest, Vol 126, Issue 15

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Adding new aligned phrases to the existing phrase table
(Hieu Hoang)
2. Re: Adding new aligned phrases to the existing phrase table
(Matthias Huck)
3. tuning error (dai xin)
4. Re: tuning error (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Tue, 11 Apr 2017 09:48:22 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Adding new aligned phrases to the
existing phrase table
To: sriram <sriramchaudhury@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbg2UxMgqNJuoVY3F86imxjSV4GR4W7SChWMH2PxKQU-og@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

from the webpage:
http://www.statmt.org/moses/?n=Advanced.Models#ntoc7
Add 2 phrase-table to the [feature] section


[feature]
PhraseDictionaryMemory path=/my-dir/table1 ...
PhraseDictionaryMemory path=/my-dir/table2 ...

Add an entry to the [mapping] section
[mapping]
0 T 0
1 T 1

Add weights to the [weight] section

[weight]
PhraseDictionaryMemory0= 0 0 1 0
PhraseDictionaryMemory1= 0 0 1 0

You don't need to use PhraseDictionaryGroup



* Looking for MT/NLP opportunities *
Hieu Hoang
http://moses-smt.org/


On 9 April 2017 at 05:21, sriram <sriramchaudhury@gmail.com> wrote:

> Hi Hieu,
>
> Thanks for the suggestion.
>
> In regard to point 2 . How can I use multiple phrase table inside moses?
>
>
> Regards,
> Sriram
>
> On Fri, Apr 7, 2017 at 5:44 PM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>
>> there's no tools to do this but you can write it yourself. You need to
>> make up some scores to give each phrase.
>>
>> The other methods to use your phrases are:
>> 1. Add it to the training data and retrain your model.
>> 2. Create a 2nd phrase-table with just your phrases and get the
>> decoder to use it, in addition to the existing phrase-table
>>
>> * Looking for MT/NLP opportunities *
>> Hieu Hoang
>> http://moses-smt.org/
>>
>>
>> On 6 April 2017 at 19:14, sriram <sriramchaudhury@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I have some good aligned phrases collection and I want to add to the
>>> existing phrase table. Is there any existing tool to add the same in Moses.
>>>
>>> Thanks,
>>> Sriram
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>
>
> --
>
> Open source English-Hindi MT system
> http://anusaaraka.iiit.ac.in/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170411/249afb20/attachment-0001.html

------------------------------

Message: 2
Date: Tue, 11 Apr 2017 14:00:57 +0200
From: Matthias Huck <mhuck@cis.lmu.de>
Subject: Re: [Moses-support] Adding new aligned phrases to the
existing phrase table
To: Hieu Hoang <hieuhoang@gmail.com>, sriram
<sriramchaudhury@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <1491912057.31015.432.camel@cis.lmu.de>
Content-Type: text/plain; charset="UTF-8"

Hi,

It might be better to do phrase table fill-up.

You would add entries from a second phrase table ("background phrase
table") to your first phrase table ("foreground phrase table") only if
they're not present yet. You end up with a single table without
duplicates. Added background phrases can be distinguished from
foreground phrases via a binary feature.

There's a tool for this in Moses:

contrib/combine-ptables/combine-ptables.pl

(Use `--mode fillup` for what I've described above. The script provides
further functionality which you can also try if you want.)

Cf.

Arianna Bisazza, Nick Ruiz, and Marcello Federico. 2011. Fill-up versus
Interpolation Methods for Phrase-based SMT Adaptation. In Proc. of the
Int. Workshop on Spoken Language Translation (IWSLT), pages 136?143,
San Francisco, CA, USA, December.

or

Jan Niehues and Alex Waibel. 2012. Detailed Analysis of Different
Strategies for Phrase Table Adaptation in SMT. In Proc. of the Conf. of
the Assoc. for Machine Translation in the Americas (AMTA), San Diego,
CA, USA, October/November.

Cheers,
Matthias


On Tue, 2017-04-11 at 09:48 +0100, Hieu Hoang wrote:
> from the webpage:
> http://www.statmt.org/moses/?n=Advanced.Models#ntoc7
> Add 2 phrase-table to the [feature] section
>
>
> [feature]
> PhraseDictionaryMemory path=/my-dir/table1 ...
> PhraseDictionaryMemory path=/my-dir/table2 ...
>
> Add an entry to the [mapping] section
> [mapping]
> 0 T 0
> 1 T 1
>
> Add weights to the [weight] section
>
> [weight]
> PhraseDictionaryMemory0= 0 0 1 0
> PhraseDictionaryMemory1= 0 0 1 0
>
> You don't need to use PhraseDictionaryGroup
>
>
>
> * Looking for MT/NLP opportunities *
> Hieu Hoang
> http://moses-smt.org/
>
>
> On 9 April 2017 at 05:21, sriram <sriramchaudhury@gmail.com> wrote:
>
> > Hi Hieu,
> >
> > Thanks for the suggestion.
> >
> > In regard to point 2 . How can I use multiple phrase table inside moses?
> >
> >
> > Regards,
> > Sriram
> >
> > On Fri, Apr 7, 2017 at 5:44 PM, Hieu Hoang <hieuhoang@gmail.com> wrote:
> >
> > > there's no tools to do this but you can write it yourself. You need to
> > > make up some scores to give each phrase.
> > >
> > > The other methods to use your phrases are:
> > > 1. Add it to the training data and retrain your model.
> > > 2. Create a 2nd phrase-table with just your phrases and get the
> > > decoder to use it, in addition to the existing phrase-table
> > >
> > > * Looking for MT/NLP opportunities *
> > > Hieu Hoang
> > > http://moses-smt.org/
> > >
> > >
> > > On 6 April 2017 at 19:14, sriram <sriramchaudhury@gmail.com> wrote:
> > >
> > > > Hi,
> > > >
> > > > I have some good aligned phrases collection and I want to add to the
> > > > existing phrase table. Is there any existing tool to add the same in Moses.
> > > >
> > > > Thanks,
> > > > Sriram
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > Moses-support mailing list
> > > > Moses-support@mit.edu
> > > > http://mailman.mit.edu/mailman/listinfo/moses-support
> > > >
> > > >
> > >
> >
> >
> > --
> >
> > Open source English-Hindi MT system
> > http://anusaaraka.iiit.ac.in/
> >
> >
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support


------------------------------

Message: 3
Date: Tue, 11 Apr 2017 15:53:44 +0200
From: dai xin <wingsuestc@gmail.com>
Subject: [Moses-support] tuning error
To: moses-support@mit.edu
Message-ID:
<CADEDxC_xGLerCGXhg42ONF8Y4LYPSMGjT9XCb2UK8c6c3LfH_w@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Dear experts,

when I use EMS to do my experiments, it crashed at tuning/extract.

The error massage is in the attachment.

Anyone has any ideas about the possible reason why this happen?

Thanks in advance for helping. Hoping for some reply.

Best regards.

Xin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170411/1d3c3b56/attachment-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: extract.err
Type: application/octet-stream
Size: 534 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20170411/1d3c3b56/attachment-0001.obj

------------------------------

Message: 4
Date: Tue, 11 Apr 2017 16:28:25 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] tuning error
To: dai xin <wingsuestc@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbjSkJ_QgE_FKZ4si0v9VckH2+Z8P8x76rifm8zmLFqSnw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

quite difficult to say. It may be that the tuning set is incorrect, eg
there isn't the same number of sentences in the source and target language.

Check your tuning set

* Looking for MT/NLP opportunities *
Hieu Hoang
http://moses-smt.org/


On 11 April 2017 at 14:53, dai xin <wingsuestc@gmail.com> wrote:

> Dear experts,
>
> when I use EMS to do my experiments, it crashed at tuning/extract.
>
> The error massage is in the attachment.
>
> Anyone has any ideas about the possible reason why this happen?
>
> Thanks in advance for helping. Hoping for some reply.
>
> Best regards.
>
> Xin
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170411/2f2e0aa9/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 126, Issue 15
**********************************************

Tweet