Moses-support Digest, Vol 121, Issue 33

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Does PhraseDictionaryMultiModel require all models to
contain all phrases? (Michael Denkowski)
2. Build failure (Tyler Wilbers)
3. Re: Build failure (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Wed, 16 Nov 2016 12:34:58 -0500
From: Michael Denkowski <michael.j.denkowski@gmail.com>
Subject: Re: [Moses-support] Does PhraseDictionaryMultiModel require
all models to contain all phrases?
To: Vito Mandorino <vito.mandorino@linguacustodia.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CA+-GegKY4_Tx+mn-nvXuUp0s8PX0CjaYVtgB8gaPKU8z2=PUfQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Lane,

As Vito mentioned, PhraseDictionaryMultiModel is originally for linear
interpolation. It has an option to output all scores from all models
rather than interpolating but I ended up writing PhraseDictionaryGroup to
have the specific functionality plus some additions. One of the additions
is the "default-average-others" option specifically for disjoint phrase
tables. It will fill in scores for missing phrase pairs in any model by
averaging the scores from other models. Empirically, this works better
than zero-filling the scores or using a default score.

Another addition (which was never fully documented) is the
"model-bitmap-count" option that adds indicator features for which models
contain a phrase. For instance, if this option is enabled for 2 models, A
and B, you have 3 features that will have the following scores when:
- A phrase is in model A but *not* model B: 1 0 0
- A phrase is in *not* model A but is in model B: 0 1 0
- A phrase is in *both* models A and B: 0 0 1
Empirically, these help as well and batch MIRA does a fine job of finding
weights for the expanded feature set.

Best,
Michael

On Wed, Nov 16, 2016 at 11:32 AM, Vito Mandorino <
vito.mandorino@linguacustodia.com> wrote:

> Hi Lane,
>
> as far as I know the PhraseDictionaryMultiModel does a linear
> interpolation of two or more phrase-tables at decoding time and is
> equivalent to linearly interpolate the phrase-table beforehand, except
> maybe for some pruning-related corner cases.
>
> If a translation option is not present in one of the phrase-tables, it is
> considered as having zero (or very low, e.g. 10e-6) probability for that
> phrase-table. I guess that the answer to your last question is yes, at
> least in the general case (phrase-tables coming from training on a
> bilingual corpus).
>
> Vito
>
>
> 2016-11-16 16:41 GMT+01:00 Lane Schwartz <dowobeha@gmail.com>:
>
>> Hi,
>>
>> I'm potentially interested in using PhraseDictionaryMultiModel, and how
>> it differs from PhraseDictionaryGroup.
>>
>> With PhraseDictionaryMultiModel, is it OK to have disjoint phrase tables?
>> That is, can PhraseDictionaryMultiModel handle the situation where a
>> translation option is present in one table but not the other(s)?
>>
>> Thanks,
>> Lane
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
>
> --
> *M**. Vito MANDORINO -- Chief Scientist*
>
>
> [image: Description : Description : lingua_custodia_final full logo]
>
> *The Translation Trustee*
>
> *1, Place Charles de Gaulle, **78180 Montigny-le-Bretonneux*
>
> *Tel : +33 1 30 44 04 23 Mobile : +33 6 84 65 68 89
> <%2B33%206%2084%2065%2068%2089>*
>
> *Email :* *vito.mandorino@linguacustodia.com
> <massinissa.ahmim@linguacustodia.com>*
>
> *Website :*
> *www.linguacustodia.finance <http://www.linguacustodia.com/>*
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161116/a245b942/attachment-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 4421 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20161116/a245b942/attachment-0001.jpg

------------------------------

Message: 2
Date: Wed, 16 Nov 2016 16:35:39 -0500
From: Tyler Wilbers <twilbers@gmail.com>
Subject: [Moses-support] Build failure
To: moses-support@mit.edu
Message-ID: <DE1C2FD0-115F-4818-9BC0-B1051478D939@gmail.com>
Content-Type: text/plain; charset="utf-8"

Dear Moses Community?

I have to install Moses for a SMT class this semester. I am on using macOS Sierra and installed dependencies with homebrew. I used the following build command:

./bjam ?with-boost=/usr/local/Cellar/boost/1.62.0 --with-/usr/local/bin/cmph --with-xmlrpc-c=/usr/local/Cellar/xmlrpc-c/1.39.07/ --with-irstlm=~/SMT-dev/irstlm/irstlm-5.80.08/trunk --with-mm --with-probing-pt -j5 toolset=clang -q -d2

It produces the following build errors after failure (build log attached).

Best,

-T
-------------- next part --------------
A non-text attachment was scrubbed...
Name: build.log.gz
Type: application/x-gzip
Size: 237 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20161116/288aab85/attachment-0001.gz

------------------------------

Message: 3
Date: Thu, 17 Nov 2016 10:05:02 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Build failure
To: Tyler Wilbers <twilbers@gmail.com>, moses-support@mit.edu
Message-ID: <6a02d955-5dc2-8de1-b39f-79f6bef3bfcb@gmail.com>
Content-Type: text/plain; charset="windows-1252"

Please see my advise about using Moses with OSX

https://www.mail-archive.com/moses-support@mit.edu/msg14529.html


On 16/11/2016 21:35, Tyler Wilbers wrote:
> Dear Moses Community?
>
> I have to install Moses for a SMT class this semester. I am on using macOS Sierra and installed dependencies with homebrew. I used the following build command:
>
> ./bjam ?with-boost=/usr/local/Cellar/boost/1.62.0 --with-/usr/local/bin/cmph --with-xmlrpc-c=/usr/local/Cellar/xmlrpc-c/1.39.07/ --with-irstlm=~/SMT-dev/irstlm/irstlm-5.80.08/trunk --with-mm --with-probing-pt -j5 toolset=clang -q -d2
>
> It produces the following build errors after failure (build log attached).
>
> Best,
>
> -T
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161117/f17f090c/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 121, Issue 33
**********************************************

Tweet