Moses-support Digest, Vol 126, Issue 20

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Corrupted best100 output when tuning (Dingyuan Wang)
2. "-ttable-limit" (Gary)
3. Tuning for PT2? (Mike Ladwig)


----------------------------------------------------------------------

Message: 1
Date: Sat, 15 Apr 2017 16:57:55 +0800
From: Dingyuan Wang <abcdoyle888@gmail.com>
Subject: Re: [Moses-support] Corrupted best100 output when tuning
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <396f48a5-5968-194f-abe9-124e2d92d73b@gmail.com>
Content-Type: text/plain; charset="utf-8"

I think there is something wrong in the printing of sparse feature
parameters. I can reproduce it on two computers, about one error out of
ten batches. I checked the n-best output using attached script.

2017-04-12 22:25, Dingyuan Wang:
> I don't find anything wrong of this sentence in the test set. Other
> candidates of this sentence is good in the same batch of output. This
> problem occurs randomly (random sentence and candidate) during tuning.
>
> 2017-04-12 21:48, Hieu Hoang:
>> It looks like there is a phrase that is length 0, hence ' = 1'.
>>
>> Check your data has been cleaned and encoded correctly
>>
>> * Looking for MT/NLP opportunities *
>> Hieu Hoang
>> http://moses-smt.org/
>>
>>
>> On 12 April 2017 at 13:36, Dingyuan Wang <abcdoyle888@gmail.com
>> <mailto:abcdoyle888@gmail.com>> wrote:
>>
>> Dear all,
>>
>> I come across the exactly same problem a year ago (follow the thread):
>>
>> https://www.mail-archive.com/moses-support@mit.edu/msg13673.html
>> <https://www.mail-archive.com/moses-support@mit.edu/msg13673.html>
>>
>> The moses constantly and randomly output corrupted best100 out that
>> crashes further kbmira tuning. Like:
>>
>> 45 ||| ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
>> ? ? ?
>> ? ? ||| LexicalReordering0= -6.1176 0 0 -6.58298 0 0 Distortion0= 0
>> LM0= -115.094 TWI_?= 0 SWD_OTHER= 2 WT_?~?= 2 WT_OTHER~OTHER= 12
>> WT_?~OTHER= 1 WT_?~?= 1 WT_?~?= 1 WT_?~OTHER= 1 WT_OTHER~?= 1
>> WT_OTHER~?= 0 WT_OTHER~?= 1 WT_??~?= 1 WT_?~?= 1 WT_?~?= 1
>> WT_?~?= 1
>> WT_?~?= 0 = 1 WT_OTHER~?= 1 WT_?~?= 1 WT_?~OTHER= 0 PL_t2= 5
>> PL_s2= 4
>> PL_1,2= 2 PL_3,4= 0 PL_s3= 1 WordPenalty0= -26 PhrasePenalty0= 21
>> TranslationModel0= -66.0904 -70.4587 -24.5341 -28.4086 ||| -15.012
>>
>> There is an error in "WT_?~?= 0 = 1". Then kbmira:
>>
>> kbmira with c=0.01 decay=0.999 no_shuffle=0
>> Initialising random seed from system clock
>> terminate called after throwing an instance of
>> 'MosesTuning::FileFormatException'
>> what(): Error in line "-6.1176 0 0 -6.58298 0 0 0 -115.094 1 -26 21
>> -66.0904 -70.4587 -24.5341 -28.4086 SWD_OTHER=2 WT_?~?=1
>> WT_OTHER~OTHER=12 PL_t2=5 PL_s3=1 PL_s2=4 PL_1,2=2 WT_?~?=1 WT_?~?=1
>> WT_??~?=1 WT_?~?=1 WT_OTHER~?=1 WT_?~?=2 WT_OTHER~?=1
>> WT_OTHER~?=1
>> WT_?~?=1 WT_?~?=1 WT_?~OTHER=1 WT_?~OTHER=1 " of run1.features.dat
>> Aborted (core dumped)
>>
>> System is Debian 9 (stretch/testing) with GCC 6.3.0, moses latest git
>> checkout.
>>
>> --
>> Dingyuan Wang
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>> <http://mailman.mit.edu/mailman/listinfo/moses-support>
>>
>>
>

--
Dingyuan Wang
-------------- next part --------------
A non-text attachment was scrubbed...
Name: validatenbest.py
Type: text/x-python
Size: 315 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20170415/b924f05d/attachment-0001.py

------------------------------

Message: 2
Date: Sat, 15 Apr 2017 10:04:42 +0000
From: Gary <garyhess999@hotmail.com>
Subject: [Moses-support] "-ttable-limit"
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<VI1PR1001MB1119D2784F69337637CFCDEDF6040@VI1PR1001MB1119.EURPRD10.PROD.OUTLOOK.COM>

Content-Type: text/plain; charset="iso-8859-1"

Hi --


I am getting started with Moses and working through the Moses phrased based tutorial (http://www.statmt.org/moses/?n=Moses.Tutorial). There is the following command which doesn't work because "-ttable-limit" is now deprecated:


echo 'das ist ein kleines haus' | moses -f phrase-model/moses.ini -ttable-limit 0 -v 2


Is there a command-line equivalent for "-ttable-limit 0", or would I have to do that in moses.ini?


Thanks,

Gary Hess


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170415/b3bf9ce0/attachment-0001.html

------------------------------

Message: 3
Date: Sat, 15 Apr 2017 08:21:53 -0400
From: Mike Ladwig <mdladwig@gmail.com>
Subject: [Moses-support] Tuning for PT2?
To: Moses Support <moses-support@mit.edu>
Message-ID:
<CAB3VaD0YZ4mtZfgYszWL_DRx2DAdELMkVia1br+gbtMEWyfh6Q@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

I've been looking, without success, for any experiences tuning PT2 with the
various controls (num-lex-scores, prune, num-scores, etc.)

Can anyone share size-speed-quality experiences or suggest good starting
points?

Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170415/95ad89d8/attachment-0001.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 126, Issue 20
**********************************************

Tweet