Moses-support Digest, Vol 109, Issue 64

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: different versions of moses yielding different
translations (Barry Haddow)


----------------------------------------------------------------------

Message: 1
Date: Wed, 25 Nov 2015 20:23:58 +0000
From: Barry Haddow <bhaddow@inf.ed.ac.uk>
Subject: Re: [Moses-support] different versions of moses yielding
different translations
To: Vito Mandorino <vito.mandorino@linguacustodia.com>, Hieu Hoang
<hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <565618DE.2030806@inf.ed.ac.uk>
Content-Type: text/plain; charset="windows-1252"

Hi Vito

The 0.2 difference is after retuning? That's normal then.

But a difference of 5 bleu without retuning suggests a bug. Did you say
that this only happens with PhraseDictionaryMultiModel?

cheers - Barry

On 25/11/15 13:53, Vito Mandorino wrote:
> Thank you. In our tests it seems that with the OnDisk table the
> quality is basically the same between the two versions of Moses
> (average 0.2 difference in score Bleu) but for the CompactPhraseTable
> the difference is larger (2 points Bleu loss in average after
> re-tuning with the new version of Moses, and more than 5 points Bleu
> in average without re-tuning).
> Do you think a better quality would be obtained by running a complete
> re-training of the model with the new version of Moses?
>
>
> Best regards,
> Vito
>
> 2015-11-24 16:31 GMT+01:00 Hieu Hoang <hieuhoang@gmail.com
> <mailto:hieuhoang@gmail.com>>:
>
> There was a change in the underlying datastructure for stacks, it
> changed from std::set (ordered) to boost::unordered_set.
> https://github.com/moses-smt/mosesdecoder/commit/6b182ee5e987a5b2823aea7eaaa7ef0457c6a30d
> This got some speed gains
>
> 1 5 10 15 20 25 30 35
> 56 real4m57.795s real1m19.005s real0m51.636s real0m49.624s
> real0m49.869s real0m52.475s real0m53.806s real 0m54.957s
> 13/10 baseline user4m41.255s user5m45.086s user6m34.053s
> user8m12.430s user8m10.667s user8m16.486s user8m10.592s user
> 8m13.859s
>
> sys0m16.514s sys0m35.494s sys0m54.513s sys1m10.643s
> sys1m18.449s sys1m21.738s sys1m23.133s sys 1m25.048s
>
>
>
>
>
>
>
>
>
>
> 57 real4m41.148s real1m16.002s real0m50.747s real0m48.711s
> real0m49.130s real0m51.473s real0m53.141s real 0m54.513s
> (56) + unordered set stack user4m23.968s user5m30.356s
> user6m26.167s user7m39.286s user7m56.229s user7m52.669s
> user7m56.978s user 7m56.216s
>
> sys0m17.231s sys0m35.063s sys0m54.081s sys1m10.137s
> sys1m17.194s sys1m22.912s sys1m25.948s sys 1m26.247s
>
>
> However, the hypotheses are now added to the stack in a different
> order so there will be slight differences in results
>
>
> On 24/11/2015 13:53, Vito Mandorino wrote:
>> Hi,
>>
>> in some of our tests a recent version of Moses (pulled from
>> github last week) and an older one do not give the same
>> translations on the same source segment (with the same moses.ini).
>> Here is the 5-best list for the translation of 'test' with the
>> last week version:
>>
>> 0 ||| test ||| LexicalReordering0= -1.1969 0 0 0 0 0
>> Distortion0= 0 LM0= -51.1788 WordPenalty0= -1 PhrasePenalty0= 1
>> PhraseDictionaryMultiModel0= -3.03811 -2.5834 -2.08503 -1.83075
>> ||| -1.27754
>> 0 ||| testing ||| LexicalReordering0= 0 0 0 0 0 0 Distortion0= 0
>> LM0= -35.1495 WordPenalty0= -1 PhrasePenalty0= 1
>> PhraseDictionaryMultiModel0= -5.21045 -5.04877 -4.71131 -4.66382
>> ||| -1.70337
>> 0 ||| funds ||| LexicalReordering0= -3.1355 0 0 0 0 0
>> Distortion0= 0 LM0= -11.3753 WordPenalty0= -1 PhrasePenalty0= 1
>> PhraseDictionaryMultiModel0= -10.8209 -10.6835 -5.14555 -5.73388
>> ||| -1.77009
>> 0 ||| known as a ||| LexicalReordering0= -3.1355 0 0 0 0 0
>> Distortion0= 0 LM0= -58.8877 WordPenalty0= -3 PhrasePenalty0= 1
>> PhraseDictionaryMultiModel0= -4.42285 -11.9339 -5.14555 -18.0392
>> ||| -1.89152
>> 0 ||| as a ||| LexicalReordering0= -3.1355 0 0 0 0 0
>> Distortion0= 0 LM0= -35.5353 WordPenalty0= -2 PhrasePenalty0= 1
>> PhraseDictionaryMultiModel0= -9.34698 -11.9339 -5.14555 -9.14874
>> ||| -1.89159
>>
>> and with the older version of Moses:
>>
>> 0 ||| funds ||| LexicalReordering0= -3.1355 0 0 0 0 0
>> Distortion0= 0 LM0= -11.3753 WordPenalty0= -1 PhrasePenalty0= 1
>> PhraseDictionaryMultiModel0= -2.52548 -2.52544 -2.45544 -2.48609
>> ||| -0.815668
>> 0 ||| as a ||| LexicalReordering0= -3.1355 0 0 0 0 0
>> Distortion0= 0 LM0= -35.5353 WordPenalty0= -2 PhrasePenalty0= 1
>> PhraseDictionaryMultiModel0= -2.52464 -2.52565 -2.45544 -2.5244
>> ||| -0.953799
>> 0 ||| as ||| LexicalReordering0= -3.1355 0 0 0 0 0 Distortion0=
>> 0 LM0= -34.1633 WordPenalty0= -1 PhrasePenalty0= 1
>> PhraseDictionaryMultiModel0= -2.5256 -2.52565 -2.45544 -2.48609
>> ||| -1.07254
>> 0 ||| known as a ||| LexicalReordering0= -3.1355 0 0 0 0 0
>> Distortion0= 0 LM0= -58.8877 WordPenalty0= -3 PhrasePenalty0= 1
>> PhraseDictionaryMultiModel0= -2.38597 -2.52565 -2.45544 -2.52573
>> ||| -1.07536
>> 0 ||| is known as a ||| LexicalReordering0= -3.1355 0 0 0 0 0
>> Distortion0= 0 LM0= -80.8518 WordPenalty0= -4 PhrasePenalty0= 1
>> PhraseDictionaryMultiModel0= -2.37158 -2.52565 -2.45544 -2.52573
>> ||| -1.18753
>>
>> This looks very strange. The only difference is in the
>> phrase-table scores. Do you have any idea of what is going on?
>> The only possibility which come to mind is maybe a different
>> handling of the PhraseDictionaryMultiModel feature.
>> The moses.ini is in attachment.
>>
>> Best regards,
>>
>> Vito
>>
>>
>>
>> --
>> *M**. Vito MANDORINO -- Chief Scientist*
>>
>> Description : Description : lingua_custodia_final full logo
>>
>> */The Translation Trustee/*
>>
>> *1, Place Charles de Gaulle, **78180 Montigny-le-Bretonneux*
>>
>> *Tel : +33 1 30 44 04 23 Mobile : +33 6 84 65 68 89
>> <tel:%2B33%206%2084%2065%2068%2089>*
>>
>> *Email :****<mailto:massinissa.ahmim@linguacustodia.com>vito.mandorino@linguacustodia.com
>> <mailto:vito.mandorino@linguacustodia.com>***
>>
>> *Website :****www.linguacustodia.com
>> <http://www.linguacustodia.com/> - www.thetranslationtrustee.com
>> <http://www.thetranslationtrustee.com/>*
>>
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>
> --
> Hieu Hoang
> http://www.hoang.co.uk/hieu
>
>
>
>
> --
> *M**. Vito MANDORINO -- Chief Scientist*
>
> Description : Description : lingua_custodia_final full logo
>
> */The Translation Trustee/*
>
> *1, Place Charles de Gaulle, **78180 Montigny-le-Bretonneux*
>
> *Tel : +33 1 30 44 04 23 Mobile : +33 6 84 65 68 89
> <tel:%2B33%206%2084%2065%2068%2089>*
>
> *Email :****vito.mandorino@linguacustodia.com
> <mailto:massinissa.ahmim@linguacustodia.com>***
>
> *Website :****www.linguacustodia.com
> <http://www.linguacustodia.com/> - www.thetranslationtrustee.com
> <http://www.thetranslationtrustee.com/>*
>
>
>
> This body part will be downloaded on demand.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151125/2ae28a8e/attachment.html
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151125/2ae28a8e/attachment.pl

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 109, Issue 64
**********************************************

Tweet