Moses-support Digest, Vol 152, Issue 1

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Tree-based training, with syntax parsing, died with signal 11
(Sakenov, Renat)


----------------------------------------------------------------------

Message: 1
Date: Sat, 1 Jun 2019 13:11:48 +0000
From: "Sakenov, Renat" <sakenovr@dhbw-loerrach.de>
Subject: [Moses-support] Tree-based training, with syntax parsing,
died with signal 11
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <1559394711452.87136@dhbw-loerrach.de>
Content-Type: text/plain; charset="iso-8859-1"

Hello everyone,


I have tried to train a tree-based model. The corpus was previously annotated with syntax information accordingly to the manual.

I use the parallel corpus newcommentary-v10 in german - english, and used the Berkeley Parser for german

and the collins parser for english text (with the wrapper scripts from Moses).

After parsing I trained the model with following command:


moses/ubuntu-17.04/moses/scripts/training/train-model.perl \
-mgiza \
--mgiza-cpus 4 \
--root-dir train-short \
--corpus corpus/nc.parsed.short \
--f de --e en \
--lm 0:5:$PWD/lm/europarl-v7.de-en.blm.en:8 \
--hierarchical \
--glue-grammar \
--source-syntax \
--target-syntax \
--extract-options="--MaxSpan 999" \
-external-bin-dir /home/renat_sakenov/moses/ubuntu-17.04/training-tools -cores=4


which led to following error:


==========================================================
Hmm Training Started at: Sat Jun 1 10:50:26 2019

-----------
Hmm: Iteration 1
Dump files 0 it 1 noIterations 5 dumpFreq 0
ERROR: Execution of: /home/renat_sakenov/moses/ubuntu-17.04/training-tools/mgiza -CoocurrenceFile /home/renat_sakenov/moses-2/tree-based-parsed/train-short/giza.de-en/de-en.cooc -c /home/renat_sakenov/moses-2/tree-based-parsed/train-short/corpus/de-en-int-train.snt -m1 5 -m2 0 -m3 3 -m4 3 -model1dumpfrequency 1 -model4smoothfactor 0.4 -ncpus 4 -nodumps 1 -nsmooth 4 -o /home/renat_sakenov/moses-2/tree-based-parsed/train-short/giza.de-en/de-en -onlyaldumps 1 -p0 0.999 -s /home/renat_sakenov/moses-2/tree-based-parsed/train-short/corpus/en.vcb -t /home/renat_sakenov/moses-2/tree-based-parsed/train-short/corpus/de.vcb
died with signal 11, without coredump


This seems similar to this issue: https://www.mail-archive.com/moses-support@mit.edu/msg11270.html
I am using Moses Release 4.0, Collins Parser Version 1.0, MXPOST Version 1.0 and Berkeley Parser Version 1.1
I have attached the complete output from the trainings procedure.

At some point this message appears:

Reading more sentence pairs into memory ...
ERROR: Forbidden zero sentence length 0
Corpus fits in memory, corpus has: 9256 sentence pairs.

But I'm certain there are no empty lines in the trainings-corpora.


Is anyone familiar with this problem?


Thank you in advance


Best,


Renat Sakenov


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20190601/4e1c374f/attachment.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nohup.out
Type: application/octet-stream
Size: 25931 bytes
Desc: nohup.out
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20190601/4e1c374f/attachment.obj

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 152, Issue 1
*********************************************

Tweet