Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: kbmira died with SIGABRT when tuning (Dingyuan Wang)
2. Re: kbmira died with SIGABRT when tuning (Barry Haddow)
3. Re: kbmira died with SIGABRT when tuning (Dingyuan Wang)
----------------------------------------------------------------------
Message: 1
Date: Mon, 18 Jan 2016 18:51:04 +0800
From: Dingyuan Wang <abcdoyle888@gmail.com>
Subject: Re: [Moses-support] kbmira died with SIGABRT when tuning
To: Barry Haddow <bhaddow@inf.ed.ac.uk>, Hieu Hoang
<hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <569CC398.8090207@gmail.com>
Content-Type: text/plain; charset=utf-8
Hi Barry,
The whole file is too large to attach. The error message is
kbmira with c=0.01 decay=0.999 no_shuffle=0
Initialising random seed from system clock
Found 14118 initial sparse features
terminate called after throwing an instance of
'MosesTuning::FileFormatException'
what(): Error in line "-5.44027 0 0 -5.34901 0 0 0 -224.872 1 1 1 -39
18 -26.2331 -40.6736 -44.3698 -82.5072 WT_?~?=3 WT_?~?=1 WT_?~?=1
WT_?~?=1 WT_?~?=1 PL_s3=5 PL_3,2=2 PL_3,3=3 PL_2,3=4 PL_t3=7 PL_s1=5
PL_1,2=2 PL_1,1=3 PL_t1=4 PL_2,2=3 PL_t2=7 PL_s2=8 PL_2,1=1 WT_?~?=1
WT_?~?=1 WT_?~?=1 WT_?~?=1 WT_?~?=1 WT_?~?=1 WT_?~?=1 WT_?~
?=1 WT_??~?=1 WT_??~?=1 WT_?~?=1 WT_?~?=1 WT_?~??=1 WT_?~
?=1 WT_?~?=1 WT_?~??=1 WT_?~??=1 WT_?~?=1 WT_?~?=1 WT_?~?
?=1 WT_?~?=1 WT_?~??=1 WT_?~?=1 WT_?~??=1 WT_?~??=1 WT_?
?~??=1 WT_?~??=1 WT_?~?=1 WT_?~?=1 WT_?~?=1 WT_?~??=1 WT_
?~??=1 WT_??~??=1 " of run7.features.dat
Aborted
I don't think there is any weird characters in it. I'm always using UTF-8.
? 2016?01?18? 16:43, Barry Haddow ??:
> Hi Dingyuan
>
> Is it possible to attach the features.dat file that is causing the
> error? Almost certainly Moses is failing to parse the line because of
> the Asian characters in the feature names,
>
> cheers - Barry
>
> On 16/01/16 15:58, Dingyuan Wang wrote:
>> I ran
>>
>> ~/software/moses/bin/kbmira -J 75 --dense-init run7.dense --sparse-init
>> run7.sparse-weights --ffile run1.features.dat --ffile run2.features.dat
>> --ffile run3.features.dat --ffile run4.features.dat --ffile
>> run5.features.dat --ffile run6.features.dat --ffile run7.features.dat
>> --scfile run1.scores.dat --scfile run2.scores.dat --scfile
>> run3.scores.dat --scfile run4.scores.dat --scfile run5.scores.dat
>> --scfile run6.scores.dat --scfile run7.scores.dat -o /tmp/mert.out
>>
>> in the tuning/tmp.1 directory, which will certainly replicate the error.
>>
>> ? 2016?01?16? 23:42, Hieu Hoang ??:
>>> The mert script prints out every command it runs. You should be able to
>>> replicate the error by running the last command
>>>
>>> On 16 Jan 2016 14:18, "Dingyuan Wang" <abcdoyle888@gmail.com
>>> <mailto:abcdoyle888@gmail.com>> wrote:
>>>
>>> Sorry, but I can't reliably replicate the same problem when
>>> running
>>> TUNING_tune.1 alone. There is no character '_' in the test set
>>> or top50
>>> list.
>>>
>>> I'm using sparse-features = "target-word-insertion top 50,
>>> source-word-deletion top 50, word-translation top 50 50,
>>> phrase-length"
>>>
>>> I've attached some related files from EMS and the EMS config.
>>>
>>>
>>> https://mega.nz/#!xs0SFKxL!M_RTBp1JGX24-b4xlYYLP-bLXKiC_Sl-p96x55avAB4
>>>
>>> ? 2016?01?16? 02:45, Hieu Hoang ??:
>>> > could you make your model files available for download so I can
>>> > replicate this problem.
>>> >
>>> > it seems like you're using a feature function with sparse
>>> scores. I
>>> > think the character '_' must be escaped.
>>> >
>>> >
>>> > On 12/01/16 04:00, Dingyuan Wang wrote:
>>> >> Hi all,
>>> >>
>>> >> I'm using EMS for doing experiments. Every time the kbmira
>>> died with
>>> >> SIGABRT when turning on one direction, while tuning on the
>>> opposite
>>> >> direction (same config and test set) was successful.
>>> >>
>>> >> The mert.log (stderr) shows follows:
>>> >>
>>> >>
>>> >> kbmira with c=0.01 decay=0.999 no_shuffle=0
>>> >> Initialising random seed from system clock
>>> >> Found 15323 initial sparse features
>>> >> ....terminate called after throwing an instance of
>>> >> 'MosesTuning::FileFormatException'
>>> >> what(): Error in line "-4.51933 0 0 -6.09733 0 0 0
>>> -121.556 2
>>> -20 12
>>> >> -31.6201 -38.5211 -26.5112 -60.6166 WT_?~?=2 WT_?~?=1
>>> PL_s1=4
>>> >> PL_s3=1 PL_3,3=1 PL_2,2=3 PL_1,2=1 PL_2,1=3 PL_t1=6 PL_t2=4
>>> PL_t3=2
>>> >> PL_2,3=1 PL_s2=7 PL_1,1=3 WT_?~??=1 WT_?~??=1 WT_?~?=1
>>> WT_?~?
>>> >> ?=1 WT_?~?=1 WT_?~?=2 WT_?~?=1 WT_?~?=1 WT_?~?
>>> ?=1 WT_
>>> ?~?=1
>>> >> WT_?~??=1 WT_?~?=1 WT_?~??=1 WT_?~??=1 WT_?~?
>>> ?=1 WT_?~
>>> >> ?=1 WT_?~??=1 " of run7.features.dat
>>> >> Aborted
>>> >>
>>> >>
>>> >> I think since run7.scores.dat is generated by some scripts, I
>>> wouldn't
>>> >> be responsible for making the bad format. Last time it also
>>> died, I
>>> >> removed the likely offending line in the test set, but this
>>> time
>>> another
>>> >> line appears.
>>> >>
>>> >> --
>>> >> Dingyuan Wang
>>> >> _______________________________________________
>>> >> Moses-support mailing list
>>> >> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>> >> http://mailman.mit.edu/mailman/listinfo/moses-support
>>> >
>>>
>>> --
>>> Dingyuan Wang (gumblex)
>>>
>
>
--
Dingyuan Wang (gumblex)
------------------------------
Message: 2
Date: Mon, 18 Jan 2016 11:03:10 +0000
From: Barry Haddow <bhaddow@inf.ed.ac.uk>
Subject: Re: [Moses-support] kbmira died with SIGABRT when tuning
To: Dingyuan Wang <abcdoyle888@gmail.com>, Hieu Hoang
<hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <569CC66E.1010400@inf.ed.ac.uk>
Content-Type: text/plain; charset=utf-8; format=flowed
Hi Dingyuan
In fact it is not the sparse features nor the Asian characters that are
the problem. The offending line has 17 dense features, yet your model
has 14 dense features.
The string "1 1 1" appears directly after the language model feature in
line 1694, in your attachment, adding the extra 3 features. Note that
this is not the line you mentioned in your earlier email.
I have no idea why there are extra features. Have you made changes to
any of the core Moses features?
best wishes
Barry
The offending line:
what(): Error in line "-5.44027 0 0 -5.34901 0 0 0 -224.872 1 1 1 -39
18 -26.2331 -40.6736 -44.3698 -82.5072 WT_?~?=3 WT_?~?=1 WT_?~?=1
WT_?~?=1 WT_?~?=1 PL_s3=5 PL_3,2=2 PL_3,3=3 PL_2,3=4 PL_t3=7 PL_s1=5
PL_1,2=2 PL_1,1=3 PL_t1=4 PL_2,2=3 PL_t2=7 PL_s2=8 PL_2,1=1 WT_?~?=1
WT_?~?=1 WT_?~?=1 WT_?~?=1 WT_?~?=1 WT_?~?=1 WT_?~?=1 WT_?~
?=1 WT_??~?=1 WT_??~?=1 WT_?~?=1 WT_?~?=1 WT_?~??=1 WT_?~
?=1 WT_?~?=1 WT_?~??=1 WT_?~??=1 WT_?~?=1 WT_?~?=1 WT_?~?
?=1 WT_?~?=1 WT_?~??=1 WT_?~?=1 WT_?~??=1 WT_?~??=1 WT_?
?~??=1 WT_?~??=1 WT_?~?=1 WT_?~?=1 WT_?~?=1 WT_?~??=1 WT_
?~??=1 WT_??~??=1 " of ...
On 18/01/16 10:37, Dingyuan Wang wrote:
> Hi,
>
> I've attached that. The line number is 1694.
>
> ? 2016?01?18? 16:43, Barry Haddow ??:
>> Hi Dingyuan
>>
>> Is it possible to attach the features.dat file that is causing the
>> error? Almost certainly Moses is failing to parse the line because of
>> the Asian characters in the feature names,
>>
>> cheers - Barry
>>
>> On 16/01/16 15:58, Dingyuan Wang wrote:
>>> I ran
>>>
>>> ~/software/moses/bin/kbmira -J 75 --dense-init run7.dense --sparse-init
>>> run7.sparse-weights --ffile run1.features.dat --ffile run2.features.dat
>>> --ffile run3.features.dat --ffile run4.features.dat --ffile
>>> run5.features.dat --ffile run6.features.dat --ffile run7.features.dat
>>> --scfile run1.scores.dat --scfile run2.scores.dat --scfile
>>> run3.scores.dat --scfile run4.scores.dat --scfile run5.scores.dat
>>> --scfile run6.scores.dat --scfile run7.scores.dat -o /tmp/mert.out
>>>
>>> in the tuning/tmp.1 directory, which will certainly replicate the error.
>>>
>>> ? 2016?01?16? 23:42, Hieu Hoang ??:
>>>> The mert script prints out every command it runs. You should be able to
>>>> replicate the error by running the last command
>>>>
>>>> On 16 Jan 2016 14:18, "Dingyuan Wang" <abcdoyle888@gmail.com
>>>> <mailto:abcdoyle888@gmail.com>> wrote:
>>>>
>>>> Sorry, but I can't reliably replicate the same problem when running
>>>> TUNING_tune.1 alone. There is no character '_' in the test set
>>>> or top50
>>>> list.
>>>>
>>>> I'm using sparse-features = "target-word-insertion top 50,
>>>> source-word-deletion top 50, word-translation top 50 50,
>>>> phrase-length"
>>>>
>>>> I've attached some related files from EMS and the EMS config.
>>>>
>>>>
>>>> https://mega.nz/#!xs0SFKxL!M_RTBp1JGX24-b4xlYYLP-bLXKiC_Sl-p96x55avAB4
>>>>
>>>> ? 2016?01?16? 02:45, Hieu Hoang ??:
>>>> > could you make your model files available for download so I can
>>>> > replicate this problem.
>>>> >
>>>> > it seems like you're using a feature function with sparse
>>>> scores. I
>>>> > think the character '_' must be escaped.
>>>> >
>>>> >
>>>> > On 12/01/16 04:00, Dingyuan Wang wrote:
>>>> >> Hi all,
>>>> >>
>>>> >> I'm using EMS for doing experiments. Every time the kbmira
>>>> died with
>>>> >> SIGABRT when turning on one direction, while tuning on the
>>>> opposite
>>>> >> direction (same config and test set) was successful.
>>>> >>
>>>> >> The mert.log (stderr) shows follows:
>>>> >>
>>>> >>
>>>> >> kbmira with c=0.01 decay=0.999 no_shuffle=0
>>>> >> Initialising random seed from system clock
>>>> >> Found 15323 initial sparse features
>>>> >> ....terminate called after throwing an instance of
>>>> >> 'MosesTuning::FileFormatException'
>>>> >> what(): Error in line "-4.51933 0 0 -6.09733 0 0 0
>>>> -121.556 2
>>>> -20 12
>>>> >> -31.6201 -38.5211 -26.5112 -60.6166 WT_?~?=2 WT_?~?=1
>>>> PL_s1=4
>>>> >> PL_s3=1 PL_3,3=1 PL_2,2=3 PL_1,2=1 PL_2,1=3 PL_t1=6 PL_t2=4
>>>> PL_t3=2
>>>> >> PL_2,3=1 PL_s2=7 PL_1,1=3 WT_?~??=1 WT_?~??=1 WT_?~?=1
>>>> WT_?~?
>>>> >> ?=1 WT_?~?=1 WT_?~?=2 WT_?~?=1 WT_?~?=1 WT_?~??=1
>>>> WT_
>>>> ?~?=1
>>>> >> WT_?~??=1 WT_?~?=1 WT_?~??=1 WT_?~??=1 WT_?~?
>>>> ?=1 WT_?~
>>>> >> ?=1 WT_?~??=1 " of run7.features.dat
>>>> >> Aborted
>>>> >>
>>>> >>
>>>> >> I think since run7.scores.dat is generated by some scripts, I
>>>> wouldn't
>>>> >> be responsible for making the bad format. Last time it also
>>>> died, I
>>>> >> removed the likely offending line in the test set, but this time
>>>> another
>>>> >> line appears.
>>>> >>
>>>> >> --
>>>> >> Dingyuan Wang
>>>> >> _______________________________________________
>>>> >> Moses-support mailing list
>>>> >> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>>> >> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>> >
>>>>
>>>> --
>>>> Dingyuan Wang (gumblex)
>>>>
>>
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
------------------------------
Message: 3
Date: Mon, 18 Jan 2016 19:24:01 +0800
From: Dingyuan Wang <abcdoyle888@gmail.com>
Subject: Re: [Moses-support] kbmira died with SIGABRT when tuning
To: Barry Haddow <bhaddow@inf.ed.ac.uk>, Hieu Hoang
<hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <569CCB51.50701@gmail.com>
Content-Type: text/plain; charset=utf-8
Hi Barry,
I have rerun the ems after the first email, and then posted the recent
results, so the line changed.
I just use the latest code, and the EMS script. Pretty much are default
settings. The EMS setting is:
sparse-features = "target-word-insertion top 50, source-word-deletion
top 50, word-translation top 50 50, phrase-length"
I suspect there is something unexpected in the extractor.
? 2016?01?18? 19:03, Barry Haddow ??:
> Hi Dingyuan
>
> In fact it is not the sparse features nor the Asian characters that are
> the problem. The offending line has 17 dense features, yet your model
> has 14 dense features.
>
> The string "1 1 1" appears directly after the language model feature in
> line 1694, in your attachment, adding the extra 3 features. Note that
> this is not the line you mentioned in your earlier email.
>
> I have no idea why there are extra features. Have you made changes to
> any of the core Moses features?
>
> best wishes
> Barry
>
> The offending line:
> what(): Error in line "-5.44027 0 0 -5.34901 0 0 0 -224.872 1 1 1 -39
> 18 -26.2331 -40.6736 -44.3698 -82.5072 WT_?~?=3 WT_?~?=1 WT_?~?=1
> WT_?~?=1 WT_?~?=1 PL_s3=5 PL_3,2=2 PL_3,3=3 PL_2,3=4 PL_t3=7 PL_s1=5
> PL_1,2=2 PL_1,1=3 PL_t1=4 PL_2,2=3 PL_t2=7 PL_s2=8 PL_2,1=1 WT_?~?=1
> WT_?~?=1 WT_?~?=1 WT_?~?=1 WT_?~?=1 WT_?~?=1 WT_?~?=1 WT_?~
> ?=1 WT_??~?=1 WT_??~?=1 WT_?~?=1 WT_?~?=1 WT_?~??=1 WT_?~
> ?=1 WT_?~?=1 WT_?~??=1 WT_?~??=1 WT_?~?=1 WT_?~?=1 WT_?~?
> ?=1 WT_?~?=1 WT_?~??=1 WT_?~?=1 WT_?~??=1 WT_?~??=1 WT_?
> ?~??=1 WT_?~??=1 WT_?~?=1 WT_?~?=1 WT_?~?=1 WT_?~??=1 WT_
> ?~??=1 WT_??~??=1 " of ...
>
>
> On 18/01/16 10:37, Dingyuan Wang wrote:
>> Hi,
>>
>> I've attached that. The line number is 1694.
>>
>> ? 2016?01?18? 16:43, Barry Haddow ??:
>>> Hi Dingyuan
>>>
>>> Is it possible to attach the features.dat file that is causing the
>>> error? Almost certainly Moses is failing to parse the line because of
>>> the Asian characters in the feature names,
>>>
>>> cheers - Barry
>>>
>>> On 16/01/16 15:58, Dingyuan Wang wrote:
>>>> I ran
>>>>
>>>> ~/software/moses/bin/kbmira -J 75 --dense-init run7.dense
>>>> --sparse-init
>>>> run7.sparse-weights --ffile run1.features.dat --ffile
>>>> run2.features.dat
>>>> --ffile run3.features.dat --ffile run4.features.dat --ffile
>>>> run5.features.dat --ffile run6.features.dat --ffile run7.features.dat
>>>> --scfile run1.scores.dat --scfile run2.scores.dat --scfile
>>>> run3.scores.dat --scfile run4.scores.dat --scfile run5.scores.dat
>>>> --scfile run6.scores.dat --scfile run7.scores.dat -o /tmp/mert.out
>>>>
>>>> in the tuning/tmp.1 directory, which will certainly replicate the
>>>> error.
>>>>
>>>> ? 2016?01?16? 23:42, Hieu Hoang ??:
>>>>> The mert script prints out every command it runs. You should be
>>>>> able to
>>>>> replicate the error by running the last command
>>>>>
>>>>> On 16 Jan 2016 14:18, "Dingyuan Wang" <abcdoyle888@gmail.com
>>>>> <mailto:abcdoyle888@gmail.com>> wrote:
>>>>>
>>>>> Sorry, but I can't reliably replicate the same problem when
>>>>> running
>>>>> TUNING_tune.1 alone. There is no character '_' in the test set
>>>>> or top50
>>>>> list.
>>>>>
>>>>> I'm using sparse-features = "target-word-insertion top 50,
>>>>> source-word-deletion top 50, word-translation top 50 50,
>>>>> phrase-length"
>>>>>
>>>>> I've attached some related files from EMS and the EMS config.
>>>>>
>>>>>
>>>>> https://mega.nz/#!xs0SFKxL!M_RTBp1JGX24-b4xlYYLP-bLXKiC_Sl-p96x55avAB4
>>>>>
>>>>> ? 2016?01?16? 02:45, Hieu Hoang ??:
>>>>> > could you make your model files available for download so I
>>>>> can
>>>>> > replicate this problem.
>>>>> >
>>>>> > it seems like you're using a feature function with sparse
>>>>> scores. I
>>>>> > think the character '_' must be escaped.
>>>>> >
>>>>> >
>>>>> > On 12/01/16 04:00, Dingyuan Wang wrote:
>>>>> >> Hi all,
>>>>> >>
>>>>> >> I'm using EMS for doing experiments. Every time the kbmira
>>>>> died with
>>>>> >> SIGABRT when turning on one direction, while tuning on the
>>>>> opposite
>>>>> >> direction (same config and test set) was successful.
>>>>> >>
>>>>> >> The mert.log (stderr) shows follows:
>>>>> >>
>>>>> >>
>>>>> >> kbmira with c=0.01 decay=0.999 no_shuffle=0
>>>>> >> Initialising random seed from system clock
>>>>> >> Found 15323 initial sparse features
>>>>> >> ....terminate called after throwing an instance of
>>>>> >> 'MosesTuning::FileFormatException'
>>>>> >> what(): Error in line "-4.51933 0 0 -6.09733 0 0 0
>>>>> -121.556 2
>>>>> -20 12
>>>>> >> -31.6201 -38.5211 -26.5112 -60.6166 WT_?~?=2 WT_?~?=1
>>>>> PL_s1=4
>>>>> >> PL_s3=1 PL_3,3=1 PL_2,2=3 PL_1,2=1 PL_2,1=3 PL_t1=6 PL_t2=4
>>>>> PL_t3=2
>>>>> >> PL_2,3=1 PL_s2=7 PL_1,1=3 WT_?~??=1 WT_?~??=1 WT_?~
>>>>> ?=1
>>>>> WT_?~?
>>>>> >> ?=1 WT_?~?=1 WT_?~?=2 WT_?~?=1 WT_?~?=1 WT_?~??=1
>>>>> WT_
>>>>> ?~?=1
>>>>> >> WT_?~??=1 WT_?~?=1 WT_?~??=1 WT_?~??=1 WT_?~?
>>>>> ?=1 WT_?~
>>>>> >> ?=1 WT_?~??=1 " of run7.features.dat
>>>>> >> Aborted
>>>>> >>
>>>>> >>
>>>>> >> I think since run7.scores.dat is generated by some scripts, I
>>>>> wouldn't
>>>>> >> be responsible for making the bad format. Last time it also
>>>>> died, I
>>>>> >> removed the likely offending line in the test set, but
>>>>> this time
>>>>> another
>>>>> >> line appears.
>>>>> >>
>>>>> >> --
>>>>> >> Dingyuan Wang
>>>>> >> _______________________________________________
>>>>> >> Moses-support mailing list
>>>>> >> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>>>> >> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>> >
>>>>>
>>>>> --
>>>>> Dingyuan Wang (gumblex)
>>>>>
>>>
>
>
--
Dingyuan Wang (gumblex)
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 111, Issue 51
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 111, Issue 51"
Post a Comment