Moses-support Digest, Vol 126, Issue 16

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Corrupted best100 output when tuning (Dingyuan Wang)
2. Re: Corrupted best100 output when tuning (Hieu Hoang)
3. Re: Corrupted best100 output when tuning (Dingyuan Wang)


----------------------------------------------------------------------

Message: 1
Date: Wed, 12 Apr 2017 20:36:32 +0800
From: Dingyuan Wang <abcdoyle888@gmail.com>
Subject: [Moses-support] Corrupted best100 output when tuning
To: moses-support <moses-support@mit.edu>
Message-ID: <dcc37f9f-fe13-a714-9e93-58c9161af65f@gmail.com>
Content-Type: text/plain; charset=utf-8

Dear all,

I come across the exactly same problem a year ago (follow the thread):

https://www.mail-archive.com/moses-support@mit.edu/msg13673.html

The moses constantly and randomly output corrupted best100 out that
crashes further kbmira tuning. Like:

45 ||| ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
? ? ||| LexicalReordering0= -6.1176 0 0 -6.58298 0 0 Distortion0= 0
LM0= -115.094 TWI_?= 0 SWD_OTHER= 2 WT_?~?= 2 WT_OTHER~OTHER= 12
WT_?~OTHER= 1 WT_?~?= 1 WT_?~?= 1 WT_?~OTHER= 1 WT_OTHER~?= 1
WT_OTHER~?= 0 WT_OTHER~?= 1 WT_??~?= 1 WT_?~?= 1 WT_?~?= 1 WT_?~?= 1
WT_?~?= 0 = 1 WT_OTHER~?= 1 WT_?~?= 1 WT_?~OTHER= 0 PL_t2= 5 PL_s2= 4
PL_1,2= 2 PL_3,4= 0 PL_s3= 1 WordPenalty0= -26 PhrasePenalty0= 21
TranslationModel0= -66.0904 -70.4587 -24.5341 -28.4086 ||| -15.012

There is an error in "WT_?~?= 0 = 1". Then kbmira:

kbmira with c=0.01 decay=0.999 no_shuffle=0
Initialising random seed from system clock
terminate called after throwing an instance of
'MosesTuning::FileFormatException'
what(): Error in line "-6.1176 0 0 -6.58298 0 0 0 -115.094 1 -26 21
-66.0904 -70.4587 -24.5341 -28.4086 SWD_OTHER=2 WT_?~?=1
WT_OTHER~OTHER=12 PL_t2=5 PL_s3=1 PL_s2=4 PL_1,2=2 WT_?~?=1 WT_?~?=1
WT_??~?=1 WT_?~?=1 WT_OTHER~?=1 WT_?~?=2 WT_OTHER~?=1 WT_OTHER~?=1
WT_?~?=1 WT_?~?=1 WT_?~OTHER=1 WT_?~OTHER=1 " of run1.features.dat
Aborted (core dumped)

System is Debian 9 (stretch/testing) with GCC 6.3.0, moses latest git
checkout.

--
Dingyuan Wang


------------------------------

Message: 2
Date: Wed, 12 Apr 2017 14:48:34 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Corrupted best100 output when tuning
To: Dingyuan Wang <abcdoyle888@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbhM+GBrBWY6OXCr7ar_Hwv4qHOkB1G8K4yW9KuhChqzOg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

It looks like there is a phrase that is length 0, hence ' = 1'.

Check your data has been cleaned and encoded correctly

* Looking for MT/NLP opportunities *
Hieu Hoang
http://moses-smt.org/


On 12 April 2017 at 13:36, Dingyuan Wang <abcdoyle888@gmail.com> wrote:

> Dear all,
>
> I come across the exactly same problem a year ago (follow the thread):
>
> https://www.mail-archive.com/moses-support@mit.edu/msg13673.html
>
> The moses constantly and randomly output corrupted best100 out that
> crashes further kbmira tuning. Like:
>
> 45 ||| ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
> ? ? ||| LexicalReordering0= -6.1176 0 0 -6.58298 0 0 Distortion0= 0
> LM0= -115.094 TWI_?= 0 SWD_OTHER= 2 WT_?~?= 2 WT_OTHER~OTHER= 12
> WT_?~OTHER= 1 WT_?~?= 1 WT_?~?= 1 WT_?~OTHER= 1 WT_OTHER~?= 1
> WT_OTHER~?= 0 WT_OTHER~?= 1 WT_??~?= 1 WT_?~?= 1 WT_?~?= 1 WT_?~?= 1
> WT_?~?= 0 = 1 WT_OTHER~?= 1 WT_?~?= 1 WT_?~OTHER= 0 PL_t2= 5 PL_s2= 4
> PL_1,2= 2 PL_3,4= 0 PL_s3= 1 WordPenalty0= -26 PhrasePenalty0= 21
> TranslationModel0= -66.0904 -70.4587 -24.5341 -28.4086 ||| -15.012
>
> There is an error in "WT_?~?= 0 = 1". Then kbmira:
>
> kbmira with c=0.01 decay=0.999 no_shuffle=0
> Initialising random seed from system clock
> terminate called after throwing an instance of
> 'MosesTuning::FileFormatException'
> what(): Error in line "-6.1176 0 0 -6.58298 0 0 0 -115.094 1 -26 21
> -66.0904 -70.4587 -24.5341 -28.4086 SWD_OTHER=2 WT_?~?=1
> WT_OTHER~OTHER=12 PL_t2=5 PL_s3=1 PL_s2=4 PL_1,2=2 WT_?~?=1 WT_?~?=1
> WT_??~?=1 WT_?~?=1 WT_OTHER~?=1 WT_?~?=2 WT_OTHER~?=1 WT_OTHER~?=1
> WT_?~?=1 WT_?~?=1 WT_?~OTHER=1 WT_?~OTHER=1 " of run1.features.dat
> Aborted (core dumped)
>
> System is Debian 9 (stretch/testing) with GCC 6.3.0, moses latest git
> checkout.
>
> --
> Dingyuan Wang
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170412/67da7b50/attachment-0001.html

------------------------------

Message: 3
Date: Wed, 12 Apr 2017 22:25:48 +0800
From: Dingyuan Wang <abcdoyle888@gmail.com>
Subject: Re: [Moses-support] Corrupted best100 output when tuning
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <ea11f606-3207-d76f-fc2b-59dc8bef811d@gmail.com>
Content-Type: text/plain; charset=utf-8

I don't find anything wrong of this sentence in the test set. Other
candidates of this sentence is good in the same batch of output. This
problem occurs randomly (random sentence and candidate) during tuning.

2017-04-12 21:48, Hieu Hoang:
> It looks like there is a phrase that is length 0, hence ' = 1'.
>
> Check your data has been cleaned and encoded correctly
>
> * Looking for MT/NLP opportunities *
> Hieu Hoang
> http://moses-smt.org/
>
>
> On 12 April 2017 at 13:36, Dingyuan Wang <abcdoyle888@gmail.com
> <mailto:abcdoyle888@gmail.com>> wrote:
>
> Dear all,
>
> I come across the exactly same problem a year ago (follow the thread):
>
> https://www.mail-archive.com/moses-support@mit.edu/msg13673.html
> <https://www.mail-archive.com/moses-support@mit.edu/msg13673.html>
>
> The moses constantly and randomly output corrupted best100 out that
> crashes further kbmira tuning. Like:
>
> 45 ||| ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
> ? ? ?
> ? ? ||| LexicalReordering0= -6.1176 0 0 -6.58298 0 0 Distortion0= 0
> LM0= -115.094 TWI_?= 0 SWD_OTHER= 2 WT_?~?= 2 WT_OTHER~OTHER= 12
> WT_?~OTHER= 1 WT_?~?= 1 WT_?~?= 1 WT_?~OTHER= 1 WT_OTHER~?= 1
> WT_OTHER~?= 0 WT_OTHER~?= 1 WT_??~?= 1 WT_?~?= 1 WT_?~?= 1
> WT_?~?= 1
> WT_?~?= 0 = 1 WT_OTHER~?= 1 WT_?~?= 1 WT_?~OTHER= 0 PL_t2= 5
> PL_s2= 4
> PL_1,2= 2 PL_3,4= 0 PL_s3= 1 WordPenalty0= -26 PhrasePenalty0= 21
> TranslationModel0= -66.0904 -70.4587 -24.5341 -28.4086 ||| -15.012
>
> There is an error in "WT_?~?= 0 = 1". Then kbmira:
>
> kbmira with c=0.01 decay=0.999 no_shuffle=0
> Initialising random seed from system clock
> terminate called after throwing an instance of
> 'MosesTuning::FileFormatException'
> what(): Error in line "-6.1176 0 0 -6.58298 0 0 0 -115.094 1 -26 21
> -66.0904 -70.4587 -24.5341 -28.4086 SWD_OTHER=2 WT_?~?=1
> WT_OTHER~OTHER=12 PL_t2=5 PL_s3=1 PL_s2=4 PL_1,2=2 WT_?~?=1 WT_?~?=1
> WT_??~?=1 WT_?~?=1 WT_OTHER~?=1 WT_?~?=2 WT_OTHER~?=1
> WT_OTHER~?=1
> WT_?~?=1 WT_?~?=1 WT_?~OTHER=1 WT_?~OTHER=1 " of run1.features.dat
> Aborted (core dumped)
>
> System is Debian 9 (stretch/testing) with GCC 6.3.0, moses latest git
> checkout.
>
> --
> Dingyuan Wang
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
> <http://mailman.mit.edu/mailman/listinfo/moses-support>
>
>

--
Dingyuan Wang


------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 126, Issue 16
**********************************************

Tweet