Moses-support Digest, Vol 88, Issue 60

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: ERROR: Can't generate symmetrized alignment file
(cyrine.nasri@univ-lorraine.fr)
2. Re: Moses training performance (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Wed, 26 Feb 2014 18:25:55 +0100
From: "cyrine.nasri@univ-lorraine.fr" <cyrine.nasri@gmail.com>
Subject: Re: [Moses-support] ERROR: Can't generate symmetrized
alignment file
To: Massinissa Ahmim <massinissa.ahmim@linguacustodia.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAPg_V0jdR=Brhz369KsDMZfFB=gPxL46EUqiUiTgxr4w_MzkLw@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Thank you Massinissa<https://plus.google.com/u/0/106330732665975122814?prsrc=4>
,

I try to do the command that you propose, but it stills the same problem

The symal script is in anothe directory..


2014-02-26 17:46 GMT+01:00 Massinissa Ahmim <
massinissa.ahmim@linguacustodia.com>:

> Hi Cyrine,
>
> Try to run this command instead :
>
>
> ...train-model.perl -corpus /home/corpus/Apprentissage/
> train..tok.true.clean.low1 -f en -e fr -alignment grow-diag-final-and
> -reordering msd-bidirectional-fe -lm 0:3:/home/corpus/Apprentissage/FR3GR.lm:8
> -external-bin-dir /home/moses/bin"
>
> You don't have to call the files twice just the extensions, for instance
> if your parallel files are named corpus.en and corpus.fr use the path
> ~/whatever/corpus -f en -e fr and make sure both are located in the same
> directory
>
> Regards
>
> Massinissa
>
>
> 2014-02-26 16:47 GMT+01:00 cyrine.nasri@univ-lorraine.fr <
> cyrine.nasri@gmail.com>:
>
>> Hello,
>>
>> I try to train phrase model with this command :
>> " train-model.perl -corpus
>> /home/corpus/Apprentissage/train..tok.true.clean.low1
>> /home/corpus/Apprentissage/train..tok.true.clean.low1 -f en -e fr
>> -alignment grow-diag-final-and -reordering msd-bidirectional-fe -lm
>> 0:3:/home/corpus/Apprentissage/FR3GR.lm:8 -external-bin-dir /home/moses/bin"
>>
>> But it indicates this message "
>>
>> "Executing: mkdir -p ./model
>> Executing: /home/moses/mosesdecoder/scripts/training/giza2bal.pl -d
>> "gzip -cd ./giza.fr-en/fr-en.A3.final.gz" -i "gzip -cd
>> ./giza.en-fr/en-fr.A3.final.gz"
>> |/home/moses/mosesdecoder/scripts/../bin/symal -alignment="grow"
>> -diagonal="yes" -final="yes" -both="yes" >
>> ./model/aligned.grow-diag-final-and
>> sh: 1: /home/moses/mosesdecoder/scripts/../bin/symal: not found
>> Exit code: 127
>> ERROR: Can't generate symmetrized alignment file
>>
>> Any idea to solve this problem?
>>
>> Thank you in advance
>>
>>
>>
>> --
>>
>> *Cyrine Ph.D. Student in Computer Science*
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
>
> --
>
> [image: Description : Description : lingua_custodia_final full logo]
>
> *The Translation Trustee*
>
> *1, Place Charles de Gaulle*
>
> *78180 Montigny-le-Bretonneux*
>
> *Tel : +33 1 30 44 04 23 Mobile : +33 7 61 44 40 84*
>
> *Email :* *massinissa.ahmim@linguacustodia.com
> <massinissa.ahmim@linguacustodia.com>*
>
> *Website :* *www.linguacustodia.com <http://www.linguacustodia.com/> -
> www.thetranslationtrustee.com <http://www.thetranslationtrustee.com>*
>
> ? Pensez ? l'environnement, n'imprimez ce courriel que si n?cessaire.
>
> Please do not print this email unless it is absolutely necessary. Spread
> environmental awareness.
>



--

*Cyrine NASRIPh.D. Student in Computer Science*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140226/6197d560/attachment-0001.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 4421 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20140226/6197d560/attachment-0001.jpg

------------------------------

Message: 2
Date: Wed, 26 Feb 2014 17:36:09 +0000
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] Moses training performance
To: Andrzej Zydron <azydron@xtm-intl.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbivoT93js5F8rJ=2RcVgLv9pVLBO1G-V8Eofi4uDiKPtA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Is there any other tasks running on either machines, especially the Xeon?
If there is, the comparison is invalid.

Other processes may not only hog cpu, but disk io and filesystem cache.

To get good comparison, you need to run it twice, 1st to make sure
everything is read from disk & in cache, 2nd to do the timing



On 25 February 2014 20:37, Andrzej Zydron <azydron@xtm-intl.com> wrote:

> Many thanks Hieu,
>
> I did specify "-mgiza-cpus 4" for the Mac and "-mgiza-cpus 12" for the
> Xeon server. Interestingly "-mgiza-cpus 10" gave slightly better
> performance (5 mins). Looking at the io stats mgiza did not appear to be io
> bound.
>
> Best Regards,
>
>
> Andrzej Zydro?
>
> ---------------------------------------
>
> CTO
>
> *XTM International Ltd.*
>
> PO Box 2167, Gerrards Cross, SL9 8XF, UK
>
> email: azydron@xtm-intl.com
>
> Tel: +44 (0) 1753 480 479
>
> Mob: +44 (0) 7966 477 181
>
> skype: Zydron
>
> www.xtm-intl.com
>
>
>
>
>
>
>
>
>
> On 25/02/2014 18:19, Hieu Hoang wrote:
>
> Strange and interesting.
>
> I can think of 2 issues:
> 1. The number of cores isn't relevant unless you explicitly ask mgiza &
> the various extraction steps to use multiple cores.
> 2. It looks like mgiza is the issue
> 3. I'm not sure how io-bound mgiza is. However, in my test with virtual
> machines, io-bound processes are slow
>
> http://www.hanselman.com/blog/VMPerformanceChecklistBeforeYouComplainThatYourVirtualMachineIsSlow.aspx
>
> This may be the case with ram-disk
>
>
> On 25 February 2014 18:01, Andrzej Zydron <azydron@xtm-intl.com> wrote:
>
>> Dear Support,
>>
>> I realize that there may not be a simple answer, but I would like to
>> understand why running training on a 9300 segment corpus takes nearly
>> three times as long on a 12 core Xeon E5-1650v2 128GB RAM Running CentOS
>> 6.5, than on my MacBook Pro 4 core i7 3720QM 8GB RAM running Mavericks.
>> I am at a loss to explain. On the Xeon server I used a 28GB RAMDISK to
>> simulate an SSD to make things more equal. I have used mgiza throughout.
>> I have used the same data nad identical settings throughout on both
>> machines and I have used the official Moses 2.1 Git distribution and
>> compiled and linked on the machine.
>>
>> These are the timings in minutes for the MacBook Pro 4 core i7 3720QM
>> 8Gb RAM SSD:
>>
>> Start End Time taken
>> mkls 10:18:50 10:19:23 00:00:33
>> snt2cooc 10:19:23 10:19:25 00:00:02
>> mgiza 10:19:25 10:31:58 00:12:33
>> extract 10:31:58 10:32:04 00:00:06
>> score 10:32:04 10:32:14 00:00:10
>> reordering 10:32:14 10:32:17 00:00:03
>>
>> Total 00:13:27
>>
>> and these for the 12 core Xeon E5-1650v2 128GB RAM using 28GB RAMDISKfor
>> all the data:
>>
>> Start End Time taken
>> mkls 09:44:24 09:49:00 00:04:36
>> snt2cooc 09:49:00 09:49:23 00:00:23
>> mgiza 09:49:23 10:23:32 00:34:09
>> extract 10:23:32 10:24:20 00:00:48
>> score 10:24:20 10:26:08 00:01:48
>> reordering 10:26:08 10:26:20 00:00:12
>>
>> Total 00:41:56
>>
>> I know that the Mac is a superb machine (the best I have ever put my
>> hands on), but I find it difficult to understand why it should be so
>> much faster than a state of the art Xeon server for Moses training.
>>
>> Email signature standard
>>
>> Best Regards,
>>
>>
>> Andrzej Zydro?
>>
>> ---------------------------------------
>>
>> CTO
>>
>> *XTM International Ltd.*
>>
>> PO Box 2167, Gerrards Cross, SL9 8XF, UK
>>
>> email: azydron@xtm-intl.com <mailto:azydron@xtm-intl.com>
>>
>> Tel: +44 (0) 1753 480 479
>>
>> Mob: +44 (0) 7966 477 181
>>
>> skype: Zydron
>>
>> www.xtm-intl.com <http://www.xtm-intl.com/>
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>
>
>
> --
> Hieu Hoang
> Research Associate
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>
>
>


--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140226/22e25ff1/attachment.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: xtm_logo.png
Type: image/png
Size: 5245 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20140226/22e25ff1/attachment.png

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 88, Issue 60
*********************************************

Tweet