Moses-support Digest, Vol 87, Issue 31

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Get word-to-word alignments from mosesserver (Jyotesh Choudhari)
2. Re: Memory leak with large input files? (Barry Haddow)
3. Re: When using moses, cpu is are not stable. (Hansung Cho)


----------------------------------------------------------------------

Message: 1
Date: Mon, 13 Jan 2014 13:44:11 +0000 (UTC)
From: Jyotesh Choudhari <jyoteshrc@gmail.com>
Subject: [Moses-support] Get word-to-word alignments from mosesserver
To: moses-support@mit.edu
Message-ID: <loom.20140113T144325-651@post.gmane.org>
Content-Type: text/plain; charset=us-ascii

Hi,
According to Moses tutorial
(http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc30), mosesserver
returns only phrase alignments. I want word alignments too. How can I get these?

Thanks and regards,
Jyotesh Choudhari



------------------------------

Message: 2
Date: Wed, 15 Jan 2014 07:05:52 +0000
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] Memory leak with large input files?
To: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <20140115070552.13595kk9yri6vgg0@www.staffmail.ed.ac.uk>
Content-Type: text/plain; charset=ISO-8859-1; DelSp="Yes";
format="flowed"

Hi Marcin

If you give Moses a file to translate, then it will read in the whole
file in one thread, whilst translation begins in another thread. I
guess this is a bug if you have very large files.

cheers - Barry

Quoting Marcin Junczys-Dowmunt <junczys@amu.edu.pl> on Tue, 14 Jan
2014 23:27:58 +0100:

> Hi,
> I think I have noticed some weird behaviour. If a huge input file
> (millions of sentences) is processed by moses on stdin, memory usage
> keeps growing and growing. At the beginning usage is about 2GB, after
> roughly 20000 sentences it reaches around 8GB and it keeps growing.
>
> However, if I give moses only those 20000 sentences at stdin (cut off
> with head -n 20000) memory usage stays below 3 GB. it seems the input is
> somehow being buffered before translation?
>
> I came across this by retranslating the training corpus, looks buggy.
>
> Best,
> Marcin
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>



--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.




------------------------------

Message: 3
Date: Wed, 15 Jan 2014 16:24:39 +0900
From: Hansung Cho <gyuber@gmail.com>
Subject: Re: [Moses-support] When using moses, cpu is are not stable.
To: moses-support@mit.edu
Cc: Tom Hoar <tahoar@precisiontranslationtools.com>
Message-ID:
<CA+G32fcFt9eKWQ=YdTnb0xJq1DUv8zDDdWNOPiaC49s64jvg5g@mail.gmail.com>
Content-Type: text/plain; charset="euc-kr"

hello.

To change the option "-threads 64" then monitoring at server CPUs status.
but, result is the same..
Try the other way?

Attach the CPUs status picture

thank you
Hansung Cho

[image: ?? ??? 1]


2014/1/15 Tom Hoar <tahoar@precisiontranslationtools.com>

> To change the threads, run with '-threads x' added to your command line:
>
> moses -f /path/to/moses.ini -threads 20
>
>
>
>
> On 01/14/2014 11:41 PM, Philipp Koehn wrote:
>
> Hi,
>
> you can try specifying more threads than the number of CPUs.
> I have no idea if this speeds things up or causes blockage in
> your setup. Just try it out and see what happens.
>
> -phi
>
>
> On Tue, Jan 14, 2014 at 4:31 AM, Hansung Cho <gyuber@gmail.com> wrote:
>
>> My server configuration are multiple CPUs 16 core and running thread 16.
>>
>> How do I adjust the moses running thread?
>>
>> I need your help...
>>
>> Thank you
>>
>>
>> 2014/1/14 Tom Hoar <tahoar@precisiontranslationtools.com>
>>
>>> With a system as complex as Moses, why would you think the CPU usage
>>> would be steady? Each sentence task goes through many stages with different
>>> CPU demands which probably map to the variation in each cycle.
>>>
>>> You can also look first at your threading configuration. If your system
>>> has multiple CPUs (cores) and Moses is running multi-threaded, I suspect
>>> the cycles' frequencies are related to the activities on each thread.
>>>
>>>
>>>
>>> On 01/14/2014 01:09 PM, Hansung Cho wrote:
>>>
>>> hello..
>>>
>>> I'm using mosesdecoding
>>>
>>> When monitoring a physical server, Server cpu is not very stable.
>>> However, the state is constantly moving cpu
>>>
>>> Look at the picture below
>>> [image: ?? ??? 1]
>>>
>>> Is it because the moses system cache??
>>> Should any part option of the adjustment?
>>>
>>> My moses commit a1584c608f8bbd83921896bac9eac97d9effc0c8
>>> Options related to the cache are "clean-lm-cache" ( default ==1 )
>>>
>>> Will be waiting for your input.
>>>
>>> Thank you
>>> Hansung cho
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
>
> _______________________________________________
> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140115/e8cb9882/attachment.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 4432 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20140115/e8cb9882/attachment.png
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 4887 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20140115/e8cb9882/attachment-0001.png

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 87, Issue 31
*********************************************

Tweet