Moses-support Digest, Vol 89, Issue 72

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Mosesserver fails with "core dumped" (Barry Haddow)


----------------------------------------------------------------------

Message: 1
Date: Thu, 27 Mar 2014 14:03:52 +0000
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] Mosesserver fails with "core dumped"
To: "Ertu?rul YILMAZ (B?LGEM)" <yilmaz.ertugrul@tubitak.gov.tr>, Hieu
Hoang <Hieu.Hoang@ed.ac.uk>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <53342FC8.6070909@staffmail.ed.ac.uk>
Content-Type: text/plain; charset=UTF-8; format=flowed

Hi

From this message:

> Check nextPos != string::npos failed in moses/Phrase.cpp:214

It looks as though there is a format error in your phrase table, such as
a stray pipe (|) or square bracket. Did you use the Moses tokeniser to
prepare the corpus? It will escape the Moses reserved characters. If you
can use a more recent version of Moses then it may give more
information. My version of the source code has this error message at
about the same place:

UTIL_THROW_IF2(nextPos == string::npos,
"Incorrect formatting of non-terminal. Should have 2
non-terms, eg. [X][X]. "
<< "Current string: " << annotatedWord);


cheers - Barry

On 27/03/14 13:25, Ertu?rul YILMAZ (B?LGEM) wrote:
> Hi,
>
> Well, I get the same failure with command line moses too :(
> The interesting thing is that, I could use the exact same moses.ini
> with the only difference of using a smaller phrase table filtered for
> an evaluation set of ours (only changing the line for
> PhraseDictionaryMemory) and both moses and mosesserver works fine.
> The one that works is about 12MB compressed and the one that fails is
> about 596MB compressed, would there be some sort of a setting to play
> with to work with larger phrase tables?
>
> I do have four scores in the translation table (system was trained
> with one of latest versions of Moses).
>
> Thanks,
>
> ---
> /usr/local/smt/moses_2013_11/bin/moses -threads 8 -dl 5
> -minimum-bayes-risk -v 0 -f moses.tuned.ini.5.mod
> /usr/local/smt/moses_2013_11/bin
> line=UnknownWordPenalty
> line=WordPenalty
> line=PhrasePenalty
> line=PhraseDictionaryMemory name=TranslationModel0 table-limit=20
> num-features=4
> path=/media/pala12/burak/aren/ems2/model/phrase-table-sigtest-filter.5.gz
> input-factor=0 output-factor=0
> line=LexicalReordering name=LexicalReordering0 num-features=6
> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
> path=/media/pala12/burak/aren/ems2/model/reordering-table-sigtest-filter.5.wbe-msd-bidirectional-fe.gz
> Initializing LexicalReordering..
> line=Distortion
> line=KENLM lazyken=0 name=LM0 factor=0
> path=/media/pala12/burak/aren/ems2/lm/nist12.binlm.1 order=4
> line=KENLM lazyken=0 name=LM1 factor=0
> path=/media/pala12/burak/aren/ems2/lm/isi.binlm.1 order=4
> line=KENLM lazyken=0 name=LM2 factor=0
> path=/media/pala12/burak/aren/ems2/lm/un.binlm.1 order=4
> line=KENLM lazyken=0 name=LM3 factor=0
> path=/media/pala06/ilknur/pala05/gigaword_lm/gigaword5gr.tcafteraps3.irstlm.bin
> order=5
> Loading table into memory...done.
> Start loading text SCFG phrase table. Moses format : [235.000] seconds
> Check nextPos != string::npos failed in moses/Phrase.cpp:214
> Aborted (core dumped)
>
>
>
> ------------------------------------------------------------------------
> *Kimden: *"Hieu Hoang" <Hieu.Hoang@ed.ac.uk>
> *Kime: *"Ertu?rul YILMAZ (B?LGEM)" <yilmaz.ertugrul@tubitak.gov.tr>
> *Kk: *"moses-support" <moses-support@mit.edu>
> *G?nderilenler: *27 Mart Per?embe 2014 13:55:53
> *Konu: *Re: [Moses-support] Mosesserver fails with "core dumped"
>
> Does the ini file work when you use the command line moses?
>
> How many scores are there for each rule in the translation table?
> There should be 4 but if you had trained it with Moses v.1 then there
> will be 5.
>
> In that case, just delete the last score, which is just a constant 2.718
>
>
> On 26 March 2014 10:11, Ertu?rul YILMAZ (B?LGEM)
> <yilmaz.ertugrul@tubitak.gov.tr
> <mailto:yilmaz.ertugrul@tubitak.gov.tr>> wrote:
>
> Hi All,
>
> I am working on running a phrase-based MT system with mosesserver
> resulting in the attached failure.
> I was able to run the same system with binarized phrase and
> reordering tables, but it fails to run them in memory.
>
> Did any of you run into a similar problem or have any ideas on the
> resolution for this? Attaching my moses.ini also.
>
> Thanks,
>
> ertugrul@pala01$ /usr/local/smt/moses_2013_11/bin/mosesserver
> -threads 8 -dl 5 -minimum-bayes-risk -v 0 -f moses.tuned.ini.5.mod
> /usr/local/smt/moses_2013_11/bin
> line=UnknownWordPenalty
> line=WordPenalty
> line=PhrasePenalty
> line=PhraseDictionaryMemory name=TranslationModel0 table-limit=20
> num-features=4
> path=/media/pala12/burak/aren/ems2/model/phrase-table-sigtest-filter.5.gz
> input-factor=0 output-factor=0
> line=LexicalReordering name=LexicalReordering0 num-features=6
> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
> path=/media/pala12/burak/aren/ems2/model/reordering-table-sigtest-filter.5.wbe-msd-bidirectional-fe.gz
> Initializing LexicalReordering..
> line=Distortion
> line=KENLM lazyken=0 name=LM0 factor=0
> path=/media/pala12/burak/aren/ems2/lm/nist12.binlm.1 order=4
> line=KENLM lazyken=0 name=LM1 factor=0
> path=/media/pala12/burak/aren/ems2/lm/isi.binlm.1 order=4
> line=KENLM lazyken=0 name=LM2 factor=0
> path=/media/pala12/burak/aren/ems2/lm/un.binlm.1 order=4
> line=KENLM lazyken=0 name=LM3 factor=0
> path=/media/pala06/ilknur/pala05/gigaword_lm/gigaword5gr.tcafteraps3.irstlm.bin
> order=5
> Loading table into memory...done.
> Start loading text SCFG phrase table. Moses format : [233] seconds
> Check nextPos != string::npos failed in moses/Phrase.cpp:214
> Aborted (core dumped)
>
>
> *----------------------------------------------------------
> *
> *Ertu?rul YILMAZ*
> Senior Researcher
>
> Speech and Natural Language Processing Group
> T?B?TAK B?LGEM
> 41470 GEBZE / KOCAEL?, TURKEY
> T +90262 648 1000 <callto:262%20648%201000> - 2216
> F +90262 648 1100 <callto:262%20648%201100>
> www.bilgem.tubitak.gov.tr <http://www.bilgem.tubitak.gov.tr>
> yilmaz.ertugrul@tubitak.gov.tr <http://www.bilgem.tubitak.gov.tr/>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
>
> --
> Hieu Hoang
> Research Associate
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support


--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 89, Issue 72
*********************************************

Tweet