Moses-support Digest, Vol 116, Issue 9

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Moses "died with error 11" error in factored training
(Hieu Hoang)
2. Core generated while tuning using METEOR (Carlos Escolano)
3. Re: Moses "died with error 11" error in factored training
(Sa?o Kuntaric)
4. Re: Moses "died with error 11" error in factored training
(Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Sun, 12 Jun 2016 20:08:51 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Moses "died with error 11" error in
factored training
To: Sa?o Kuntaric <saso.kuntaric@gmail.com>, moses-support@mit.edu
Message-ID: <fd5732e4-92e5-0988-a914-90c0187ec035@gmail.com>
Content-Type: text/plain; charset=windows-1252; format=flowed

judging by the source code in mgiza's getSentence.cpp line 366,

cerr << "ERROR: Forbidden zero sentence length " <<
sent.sentenceNo << endl;
the 0 in your output is the line number.

It may be that your corpora was produced on windows and has a BOM at the
beginning


On 12/06/2016 10:40, Sa?o Kuntaric wrote:
> Forbidden zero sentence



------------------------------

Message: 2
Date: Sun, 12 Jun 2016 22:35:44 +0200
From: Carlos Escolano <carlos.e.p93@gmail.com>
Subject: [Moses-support] Core generated while tuning using METEOR
To: moses-support@mit.edu
Message-ID:
<CACvrzNTsp1EXGQ6pOjxe=UZQeitZ3EUHwkZJnDe=8nJsP+JhdQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi all,

I'm trying to tune a model using METEOR as metric but everytime I'm
encountering an error at the begining of the tuning.

I'm using mert with the following parameters:

$MOSES_SCRIPTS/training/mert-moses.pl $WORKING_DIR/dev/devSingle.zh
$WORKING_DIR/dev/devSingle.es
/veu4/usuaris24/xtrans/mosesdecoder/moses-cmd/bin/gcc-4.8.5/release/link-static/threading-multi/moses

$WORKING_DIR/baselineMeteor/model/moses.ini
--nbest 100
--working-dir $WORKING_DIR/baselineMeteor/tuning/
--rootdir /veu4/usuaris24/xtrans/mosesdecoder/scripts/
--mertdir /veu4/usuaris24/xtrans/mosesdecoder/bin/
-threads 24
--filtercmd '/veu4/usuaris24/xtrans/mosesdecoder/scripts/training/
filter-model-given-input.pl'
--decoder-flags "-drop-unknown -mbr -threads 24 -mp -v 0"
--mertargs="--sctype METEOR --scconfig
jar:/veu4/usuaris24/cescola/meteor-1.5/meteor-1.5.jar"

And this is the content of the extract.err after the error.

Binary write mode is NOT selected
Scorer type: METEOR
name: jar value: /veu4/usuaris24/cescola/meteor-1.5/meteor-1.5.jar
name: case value: true
Executing: java -Xmx1G -jar
/veu4/usuaris24/cescola/meteor-1.5/meteor-1.5.jar - - -stdio -lower -t tune
-l en
Data::m_score_type METEOR
Data::Scorer type from Scorer: METEOR
loading nbest from run1.best100.out.gz
Loaded N-best lists : [Wall 98.2086 CPU 6.90767] seconds.
Binary write mode is NOT selected
saving the array into run1.features.dat
saving the array into run1.scores.dat
Stopping... : [Wall 100.662 CPU 9.36354] seconds.
Could not close file 4
/veu4/usuaris24/cescola/tfg/1-corpus/baselineMeteor/tuning/extractor.sh:
l?nea 3: 30494 Abortado (`core' generado)
/veu4/usuaris24/xtran
s/mosesdecoder/bin/extractor --sctype METEOR --scconfig
jar:/veu4/usuaris24/cescola/meteor-1.5/meteor-1.5.jar,case:true --scfile
run1.scores.dat --ffil
e run1.features.dat -r
/veu4/usuaris24/cescola/tfg/1-corpus//dev/devSingle.es -n
run1.best100.out.gz

Thanks to you all I'd really apreciate your help.

Best regards,

Carlos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160612/5dc132d2/attachment-0001.html

------------------------------

Message: 3
Date: Mon, 13 Jun 2016 08:51:52 +0200
From: Sa?o Kuntaric <saso.kuntaric@gmail.com>
Subject: Re: [Moses-support] Moses "died with error 11" error in
factored training
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support@mit.edu
Message-ID:
<CANsquDoQWo4h=BfMEv21bu5KL7vq+iim5N=Sw_SyFuqfJvmNXA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Thanks for the tip, however in my case the problem was that after tagging
the files with mxpost and post-processing I had some standalone |PRP tags
in the source file. Once I removed those, training resumed.

Which leads me to another question. Since mxpost was used for the Moses
tutorial, I was wondering how did you create the input files for Moses
after tagging? Was there any post-processing done or can mxpost use the
pipes (|) instead of underlines? And one more thing, how can lemmas be
added, was a custom tagger project made or is there a parameter which tells
mxpost to do it?

Best regards,

Sa?o

2016-06-12 21:08 GMT+02:00 Hieu Hoang <hieuhoang@gmail.com>:

> judging by the source code in mgiza's getSentence.cpp line 366,
>
> cerr << "ERROR: Forbidden zero sentence length " << sent.sentenceNo
> << endl;
> the 0 in your output is the line number.
>
> It may be that your corpora was produced on windows and has a BOM at the
> beginning
>
>
> On 12/06/2016 10:40, Sa?o Kuntaric wrote:
>
>> Forbidden zero sentence
>>
>
>


--
lp,

Sa?o
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160613/998ade32/attachment-0001.html

------------------------------

Message: 4
Date: Mon, 13 Jun 2016 11:12:20 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Moses "died with error 11" error in
factored training
To: Sa?o Kuntaric <saso.kuntaric@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbih1i6d+2V0aJ-MwFvymH8BB9Oq=dpdJ2qrS4POvJSDQQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hieu Hoang
http://www.hoang.co.uk/hieu

On 13 June 2016 at 07:51, Sa?o Kuntaric <saso.kuntaric@gmail.com> wrote:

> Thanks for the tip, however in my case the problem was that after tagging
> the files with mxpost and post-processing I had some standalone |PRP tags
> in the source file.
>
that suggest the corpus file has not been cleaned. eg. there may be
multiple white spaces ' '


> Once I removed those, training resumed.
>
> Which leads me to another question. Since mxpost was used for the Moses
> tutorial, I was wondering how did you create the input files for Moses
> after tagging? Was there any post-processing done or can mxpost use the
> pipes (|) instead of underlines? And one more thing, how can lemmas be
> added, was a custom tagger project made or is there a parameter which tells
> mxpost to do it?
>
not sure what you mean

>
> Best regards,
>
> Sa?o
>
> 2016-06-12 21:08 GMT+02:00 Hieu Hoang <hieuhoang@gmail.com>:
>
>> judging by the source code in mgiza's getSentence.cpp line 366,
>>
>> cerr << "ERROR: Forbidden zero sentence length " <<
>> sent.sentenceNo << endl;
>> the 0 in your output is the line number.
>>
>> It may be that your corpora was produced on windows and has a BOM at the
>> beginning
>>
>>
>> On 12/06/2016 10:40, Sa?o Kuntaric wrote:
>>
>>> Forbidden zero sentence
>>>
>>
>>
>
>
> --
> lp,
>
> Sa?o
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160613/b1b96476/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 116, Issue 9
*********************************************

Tweet