Moses-support Digest, Vol 94, Issue 8

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: moses version 2.1 very slow loading and decoding (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Tue, 5 Aug 2014 16:02:05 +0100
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] moses version 2.1 very slow loading and
decoding
To: Marie Sayadchi <marie.sayadchi@gmail.com>, moses-support
<moses-support@mit.edu>
Message-ID:
<CAEKMkbhAm1tYP4SkHehOishjNADww5MV1SSdkt7_R5yV88v1ow@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

hi marie


On 5 August 2014 13:58, Marie Sayadchi <marie.sayadchi@gmail.com> wrote:

> hi
> thanks for your reply.
>
>
> *Are the binary files in filesystem cache?*
> im not sure how to put them there or how to check it! I would appreciate
> if you could supply more details on this.
>
before you run the decoder, run this first
cat
/home/lingua/smt/moses-binaries/test-new-moses/new/binariesed/phrase-table/*.dat
> /dev/null
cat
/home/lingua/smt/moses-binaries/test-new-moses/new/binariesed/reordering-table.bin*
> /dev/null
cat
/home/lingua/smt/Multisensor/news-models/news.en.truecase.irstlm.blm >
/dev/null

>
>
> * what happens if you fun the decoding again with same sentence and
> moses-bin.ini ?*
> this log is with moses-bin.ini file, yes with binaries it is ok in
> loading, but still slow in translation, and also I would like to know why
> normal model(non-binaries) are so slow in loading compared to older
> versions?
>
i haven't monitored the loading time of the memory phrase table because i
think most people are more interested in the binary version. Running with
the binary version isn't much slower and is sometimes faster, according to
my tests:
http://www.statmt.org/moses/RELEASE-2.1/Mosesv2.1.1releasenotes.pdf


> thanks
>
>
> On Tue, Aug 5, 2014 at 2:45 PM, Hieu Hoang <Hieu.Hoang@ed.ac.uk> wrote:
>
>> from the log file, the loading seemed to have taken 1 second, which is
>> fast and normal.
>>
>> the translation of each sentence took 30+ sec each. Are the binary files
>> in filesystem cache? what happens if you fun the decoding again with same
>> sentence and moses-bin.ini ?
>>
>>
>>
>>
>> On 5 August 2014 13:33, Marie Sayadchi <marie.sayadchi@gmail.com> wrote:
>>
>>> Dear all
>>>
>>> I am using moses RELEASE-2.1 binaries.
>>> my training data is very small corpus of 170k sentences.
>>> I following instructions from "
>>> http://www.statmt.org/moses/?n=Moses.Baseline" for training and testing.
>>> the problem is that when I test the system, loading time is very long
>>> and after 30min the phrase-table is still loading .. the models are small
>>> and it should not take that long, also translation itself is very slow.
>>> same model loads relatively fast and works fine with my older version of
>>> moses(before 2012). both were using only one cpu and almost same amount of
>>> memory.
>>> my question is :
>>> how can I get the same decoding speed as before; with the moses 2.1 ,
>>> and what is wrong with my current settings?
>>>
>>>
>>> binarising the models helped with initial loading, but the translation
>>> is still much faster with my old moses.
>>> please see my moses.ini and translation log files in the attachment.
>>> the command that I am using:
>>> training:
>>> nohup nice
>>> /home/lingua/smt/moses-binaries/www.statmt.org_64bit/moses/RELEASE-2.1/binaries/linux-64bit/scripts/training/train-model.perl
>>> -root-dir new/ -corpus ./all -f de -e en -max-phrase-length 5 -alignment
>>> grow-diag-final-and -reordering msd-bidirectional-fe -lm
>>> 0:3:/home/lingua/smt/Multisensor/news-models/news.en.truecase.irstlm.blm
>>> -external-bin-dir
>>> /home/lingua/smt/moses-binaries/www.statmt.org_64bit/moses/RELEASE-2.1/binaries/linux-64bit/training-tools
>>> -mgiza mgiza -snt2cooc snt2cooc >& ./new/training.out
>>>
>>> binarising:
>>> /home/lingua/smt/moses-binaries/www.statmt.org_64bit/moses/RELEASE-2.1/binaries/linux-64bit/bin/processLexicalTable
>>> -in ./new/model/reordering-table.wbe-msd-bidirectional-fe.gz -out
>>> ./new/binariesed/reordering-table
>>>
>>> /home/lingua/smt/moses-binaries/www.statmt.org_64bit/moses/RELEASE-2.1/binaries/linux-64bit/bin/processLexicalTable
>>> -in ./new/model/reordering-table.wbe-msd-bidirectional-fe.gz -out
>>> ./new/binariesed/reordering-table
>>>
>>> testing:
>>> nohup nice
>>> /home/lingua/smt/moses-binaries/www.statmt.org_64bit/moses/RELEASE-2.1/binaries/linux-64bit/bin/moses
>>> -f ./new/model/moses-bin.ini <./newstest2013.tok.norm.de > out 2> log
>>>
>>> Best,
>>> Marieh
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>>
>> --
>> Hieu Hoang
>> Research Associate
>> University of Edinburgh
>> http://www.hoang.co.uk/hieu
>>
>>
>


--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140805/7153e660/attachment-0001.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 94, Issue 8
********************************************

Tweet