Moses-support Digest, Vol 103, Issue 26

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Multiple Translation Tables and Back-off Models in Moses
(Tomas Fulajtar)


----------------------------------------------------------------------

Message: 1
Date: Thu, 7 May 2015 11:53:46 +0000
From: Tomas Fulajtar <TomasFu@moravia.com>
Subject: Re: [Moses-support] Multiple Translation Tables and Back-off
Models in Moses
To: Philipp Koehn <phi@jhu.edu>, "xinzyang@mail.ustc.edu.cn"
<xinzyang@mail.ustc.edu.cn>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<BY1PR0201MB0965A59E41B8E13B3A1622D5ADDF0@BY1PR0201MB0965.namprd02.prod.outlook.com>

Content-Type: text/plain; charset="utf-8"

Hi,

Thanks for an excellent explanation. Does this ?default-scores? parameter work in 2.x release as well?

Tomas


From: Philipp Koehn [mailto:phi@jhu.edu]
Sent: Thursday, May 7, 2015 3:22 AM
To: xinzyang@mail.ustc.edu.cn
Cc: moses-support@mit.edu
Subject: Re: [Moses-support] Multiple Translation Tables and Back-off Models in Moses

Hi,

I added a section to the Moses documentation on how to deal with this situation:

http://www.statmt.org/moses/?n=Advanced.Models#ntoc5

-phi
Caveat: Multiple Translation Tables and Lexicalized Reordering You may specify any number of lexicalized reordering models. Each of them will score any translation option, no matter where it comes from. If a lexicalized reordering table does not have an entry for a translation options, it will not assign any score to it. In other words, such a translation option is given the probability 1 no matter how it is reordered. This may not be the way you want to handle it.
For instance, if you have an "in-domain" translation table and an out-of-domain translation table, you can also provide an in-domain reordering table and anout-of-domain reordering table. If a phrase pair occurs in both translation tables, it will be scored by both reordering tables. However, if a phrase pairs occurs only in one of the phrase tables (and hence reordering tables), it will be only score by one of them and get a free ride with the other. this will have the undesirable effect of discouraging phrase pairs that occur in both tables.
To avoid this, you can add default scores to the reordering table:

LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff [...] default-scores=0.5,0.3,0.2,0.5,0.3,0.2

LexicalReordering name=LexicalReordering1 num-features=6 type=wbe-msd-bidirectional-fe-allff [...] default-scores=0.5,0.3,0.2,0.5,0.3,0.2



On Wed, May 6, 2015 at 3:00 AM, <xinzyang@mail.ustc.edu.cn<mailto:xinzyang@mail.ustc.edu.cn>> wrote:
Hi,

I am studying moses to do some research abou SMT. I have read advanced models in moses at the website http://www.statmt.org/moses/?n=Advanced.Models#ntoc5 . I have some questions about the introduction of "Multiple Translation Tables and Back-off Models".

>From the website, I know that moses allows the use of multiple translation tables. But the lexicalized reordering model is not mentioned in the introduction of "Multiple Translation Tables and Back-off Models". So, could I use two translation tables, which I denote translation table1 and translation table2 respectively, and one lexicalized reordering table according to the introduction of "Multiple Translation Tables and Back-off Models"? The phrase pairs in reordering table is the same with translation table1, but not the same with translation table2.

When I use multiple translation tables and only one reordering table, a phrase pair may exist in only one translation table and don't exist in the reordering model, how to calculate the probability of lexicalized reordering model?

I'm sorry for bothering you. Thanks and Best Regards,

Zhenxin

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150507/85490d80/attachment-0001.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 103, Issue 26
**********************************************

Tweet