Moses-support Digest, Vol 104, Issue 11

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Moses infinite loop (Hieu Hoang)
2. Re: Error while training the translation system (Philipp Koehn)
3. Re: kbmira error (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Mon, 8 Jun 2015 20:27:18 +0400
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Moses infinite loop
To: "Bc. Robert ADAMEC" <adamecr@students.zcu.cz>, moses-support
<moses-support@mit.edu>
Message-ID:
<CAEKMkbgg2gqA8W8rfbwHLE-C8b4xVBmwUPfOS8uPFYFSuBvtDA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

What is the exact command you're executing? Can u please send me the debug
messages that Moses output
On 8 Jun 2015 19:54, "Bc. Robert ADAMEC" <adamecr@students.zcu.cz> wrote:

> Hello,
>
> I made the compact versions of the phrase tables and reordering models,
> the language models were already binarised. However it did not help, the
> tool still "freezes" on the same spot and stays there.
>
> I have let it run for almost 24 hours and it moved nowhere. I really do
> not think that it would change with time, the experiments that I used
> earlier run about 24 hours total, for this experiment I added factored
> corpus for both sides, previously it was only for the source language, and
> I do not think that it should impact the performance so heavily.
>
> I have tried to put some debug messages into moses itself, namely into the
> DecodeStepGeneration class around the lines 149 - 170 (the for (size_t
> currIter = 0 ; currIter < numIteration ; currIter++) cycle) and I found out
> that the tool freezes because it gets stuck in this cycle forever after a
> while (for the hypothesis that you can see as last in the log from the
> previous e-mail) and never leaves it.
>
> I do not know what is the reason and cannot find it out.
>
> Any ideas?
>
> Best regards,
>
> Robert Adamec
>
>
>
> ?ter?, 2 ?erven, 2015 14:05 CEST, Hieu Hoang <hieuhoang@gmail.com>
> napsal:
>
> > please subscribe to the mailing list before posting to it. You can
> > subscribe here:
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> > To answer your question - how do you know it is in an inifinite loop,
> > rather than just taking a very long time? From the moses.ini file, it
> > seems the phrase tables are not binarized. This will cause loading to
> > take a very long time and use a lot of memory. You should binarize your
> > phrase-table, reordering model and LM.
> >
> > Also, if you are new to moses, please work through the tutorial here to
> > get a better understanding of what moses is doing
> > http://www.statmt.org/moses/?n=Moses.Baseline
> >
> >
> > On 29/05/2015 14:35, Bc. Robert ADAMEC wrote:
> > > Hello,
> > >
> > > I have encountered some trouble while working with Moses 3.0
> optimization.
> > >
> > > When I use the configuration which you can see in the attached
> moses.ini the Moses tool runs infinitely long in the run 1 and is stuck on
> the Loading table into memory line. Do you know what could be the problem?
> For the "-v 100" log see the atached log.txt
> > >
> > > When I tweak some of the config options in the moses.ini file I get
> segfault, for example if I change the GenerationModel from current 1 -> 0
> to 0,1 -> 0.
> > >
> > > The source corpus has the "word|factor" format for both used languages
> so there should be no problems, the factor is an int number.
> > >
> > > Can you imagine any reason why it could be behaving this way and what
> could have gone wrong?
> > >
> > > I launch the optimization via this command:
> > >
> > > /opt/moses-3.0/scripts/training/mert-moses.pl
> /root/faktor_6/corpus/heldout.src /root/faktor_6/corpus/heldout.ref \
> > > /opt/moses-3.0/bin/moses /root/faktor_6/experiment/model/moses.ini
> \
> > > --working-dir /root/faktor_6/tmp \
> > > --decoder-flags "-threads 6 -v 0" \
> > > --rootdir /opt/moses-3.0/scripts -mertdir /opt/moses-3.0/bin \
> > > --maximum-iterations 10
> > >
> > > Best regards,
> > >
> > > Robert Adamec
> >
> > --
> > Hieu Hoang
> > Researcher
> > New York University, Abu Dhabi
> > http://www.hoang.co.uk/hieu
> >
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150608/86e8b74d/attachment-0001.htm

------------------------------

Message: 2
Date: Mon, 8 Jun 2015 14:51:05 -0400
From: Philipp Koehn <phi@jhu.edu>
Subject: Re: [Moses-support] Error while training the translation
system
To: Harshit Gupta <harshitgupta165@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDDXdqNA36HAvaDAEa0nnj9zhePqH0e3WX_YEpfguku3gg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

Hi,

first, please avoid relative paths like "-root-dir train" and
references to the home directory as "~".

If it still fails, please describe in more detail what (a) error
messages you get and (b) which files are created.

-phi

On Wed, Jun 3, 2015 at 9:02 AM, Harshit Gupta <harshitgupta165@gmail.com> wrote:
> This is the command given in moses baseline tutorial for training the
> corpora.
>
> mkdir ~/working
> cd ~/working
> nohup nice ~/mosesdecoder/scripts/training/train-model.perl -root-dir train
> \
> -corpus ~/corpus/news-commentary-v8.fr-en.clean
> \
> -f fr -e en -alignment grow-diag-final-and -reordering msd-bidirectional-fe
> \
> -lm 0:3:$HOME/lm/news-commentary-v8.fr-en.blm.en:8
> \
> -external-bin-dir ~/mosesdecoder/tools >& training.out &
>
> I followed all the steps for preparing the corpora, so I have followed the
> same directory structure.
>
> However, when I am running this command, I am not getting the model folder
> and moses.ini file in ~/working/train/model .
>
> On terminal, it just displays a number after I execute this command. Also,
> the training.out file generated in ~/working/train
>
> contains most of the lines as ERROR : parameter does not exist and
> unrecognized attribute.
>
> Please help me through it.
>
>
> --
> Harshit Gupta
> Third Year Undergraduate
> Electrical Engineering
> IIT Madras
> P Please don't print this e-mail unless you really need to
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>


------------------------------

Message: 3
Date: Tue, 09 Jun 2015 18:07:13 +0400
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] kbmira error
To: moses-support@mit.edu
Message-ID: <5576F311.6070202@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed

ah thanks.

sentence 147 is missing 'cos the hard constraint pruned everything out.
I guess the tuning doesn't encounter that very often


On 08/06/2015 13:38, Rico Sennrich wrote:
> Hieu Hoang <hieuhoang@...> writes:
>
>>
>>
>> Hi All
>> Does anyone know why I get this error? # $MOSES_DIR/bin/kbmira
> --dense-init run3.dense --sparse-init run3.sparse-weights --ffile
> run1.features.dat --ffile run2.features.dat --ffile run3.features.dat
> --scfile run1.scores.dat --scfile run2.scores.dat --scfile run3.scores.dat
> -o mert.out....Found 401 initial sparse features.......Error: Feature file 2
> ended prematurely
>
>
> Hi Hieu,
>
> grep FEATURES_TXT_BEGIN run1.features.dat | wc -l
> 751
> FEATURES_TXT_BEGIN run2.features.dat | wc -l
> 751
> grep FEATURES_TXT_BEGIN run3.features.dat | wc -l
> 750
>
> is it possible that run3.features.dat got corrupted somehow? Sentence 147
> seems to be missing.
>
> best wishes,
> Rico
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

--
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu



------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 104, Issue 11
**********************************************

Tweet