Moses-support Digest, Vol 86, Issue 20

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: error during testing (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Sat, 7 Dec 2013 15:51:47 +0000
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] error during testing
To: amir haghighi <amir.haghighi.64@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbh+3tCNDW4PSX+MbXUowyatyF1ACM+OW3QBh5GuhMNn_Q@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

are you sure the parallel data is encoded in UTF8? Was it tokenized,
cleaned and escaped by the Moses scripts or by another external script?

Can you please send me you EMS config file too


On 7 December 2013 14:03, amir haghighi <amir.haghighi.64@gmail.com> wrote:

> Hi,
>
> I have also the same problem in evaluation step with EMS and I would be
> thankful if you could help me.
> the lexical reordering file is emtpy and the log of the output in
> evaluation_test_filter.2.stderr is:
>
> Using SCRIPTS_ROOTDIR:
> /opt/tools/mosesdecoder-RELEASE-1.0/mosesdecoder-RELEASE-1.0/scripts
> (9) create moses.ini @ Sat Dec 7 04:50:15 PST 2013
> Executing: mkdir -p /opt/tools/workingEms/evaluation/test.filtered.2
> Considering factor 0
> Considering factor 0
> filtering /opt/tools/workingEms/model/phrase-table.2 ->
> /opt/tools/workingEms/evaluation/test.filtered.2/phrase-table.0-0.1.1...
> 0 of 2197240 phrases pairs used (0.00%) - note: max length 10
> binarizing...cat
> /opt/tools/workingEms/evaluation/test.filtered.2/phrase-table.0-0.1.1 |
> LC_ALL=C sort -T /opt/tools/workingEms/evaluation/test.filtered.2 |
>
> /opt/tools/mosesdecoder-RELEASE-1.0/mosesdecoder-RELEASE-1.0/bin/processPhraseTable
> -ttable 0 0 - -nscores 5 -out
> /opt/tools/workingEms/evaluation/test.filtered.2/phrase-table.0-0.1.1
> processing ptree for stdin
> Segmentation fault (core dumped)
> filtering
> /opt/tools/workingEms/model/reordering-table.2.wbe-msd-bidirectional-fe.gz
> ->
>
> /opt/tools/workingEms/evaluation/test.filtered.2/reordering-table.2.wbe-msd-bidirectional-fe...
> 0 of 2197240 phrases pairs used (0.00%) - note: max length 10
>
> binarizing.../opt/tools/mosesdecoder-RELEASE-1.0/mosesdecoder-RELEASE-1.0/bin/processLexicalTable
> -in
>
> /opt/tools/workingEms/evaluation/test.filtered.2/reordering-table.2.wbe-msd-bidirectional-fe
> -out
>
> /opt/tools/workingEms/evaluation/test.filtered.2/reordering-table.2.wbe-msd-bidirectional-fe
> processLexicalTable v0.1 by Konrad Rawlik
> processing
>
> /opt/tools/workingEms/evaluation/test.filtered.2/reordering-table.2.wbe-msd-bidirectional-fe
> to
>
> /opt/tools/workingEms/evaluation/test.filtered.2/reordering-table.2.wbe-msd-bidirectional-fe.*
> ERROR: empty lexicalised reordering file
>
>
>
> Barry Haddow <bhaddow@...> writes:
>
> >
> > Hi Irene
> >
> > > But the output is empty. And the errors are 1. segmentation fault
> > 2. error: empty lexicalized
> > > reordering file
> >
> > Is this lexicalised reordering file empty then?
> >
> > It would be helpful if you could post the full log of the output when
> > your run the filter command,
> >
> > cheers - Barry
> >
> > On 26/10/12 17:59, Irene Huang wrote:
> > > Hi, I have trained and tuned the model, now I am using
> > >
> > > ~/mosesdecoder/scripts/training/filter-model-given-input.pl
> > > <http://filter-model-given-input.pl> filtered-newstest2011
> > > mert-work/moses.ini ~/corpus/newstest2011.true.fr
> > > <http://newstest2011.true.fr> \
> > > -Binarizer ~/mosesdecoder/bin/processPhraseTable
> > >
> > > to filter the phrase table.
> > >
> > > But the output is empty. And the errors are 1. segmentation fault
> > > 2. error: empty lexicalized reordering file
> > >
> > > So does this mean it's out of memory error?
> > >
> > > Thanks
> > >
> > >
> > > _______________________________________________
> > > Moses-support mailing list
> > > Moses-support@...
> > > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>


--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20131207/decf140f/attachment-0001.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 86, Issue 20
*********************************************

Tweet