Moses-support Digest, Vol 105, Issue 50

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Performance issues using Moses Server with Moses 3
(Barry Haddow)


----------------------------------------------------------------------

Message: 1
Date: Thu, 23 Jul 2015 15:19:00 +0100
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] Performance issues using Moses Server
with Moses 3
To: Oren <mooshified@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <55B0F7D4.3010900@staffmail.ed.ac.uk>
Content-Type: text/plain; charset="utf-8"

Hi Oren

You can fit a lot of model in 50G RAM. It's worth looking at the compact
phrase and reordering models, pruning options, and kenlm quantisation,
and you might fit the models on one machine.

As to the slowdown, a delay of 20s or so suggests that it's waiting on
I/O, or perhaps xmlrpc-c is queueing up requests or connections.

cheers - Barry

On 23/07/15 15:06, Oren wrote:
> More details...
>
> Our NFS is mounted at /mnt/storage .
>
> The command to run moses server is:
>
> /mnt/storage/Common/mosesdecoder3/bin/mosesserver -f
> /mnt/storage/Common/models/translation_models/moses3_model/mert-work/moses.ini
> mark-unknown -xml-input -threads 18 exclusive
>
> Here is the complete moses.ini (excluding comments):
>
>> [input-factors]
>> 0
>>
>> [mapping]
>> 0 T 0
>>
>> [distortion-limit]
>> 6
>>
>> [feature]
>> UnknownWordPenalty
>> WordPenalty
>> PhrasePenalty
>> PhraseDictionaryMemory name=TranslationModel0 num-features=4
>> path=/mnt/storage/Common/models/translation_models/moses3_model/train/model/phrase-table.gz
>> input-factor=0 output-factor=0
>> LexicalReordering name=LexicalReordering0 num-features=6
>> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
>> path=/mnt/storage/Common/models/translation_models/moses3_model/train/model/reordering-table.wbe-msd-bidirectional-fe.gz
>> Distortion
>> KENLM lazyken=0 name=LM0
>> path=/mnt/storage/Common/models/language_models/langmod.5gram.blmorder=5
>>
>> [threads]
>> 6
>>
>> [weight]
>>
>> LexicalReordering0= 0.0268311 -0.0146878 0.0261305 0.0380759
>> 0.0118265 0.07479
>> Distortion0= 0.074665
>> LM0= 0.0972206
>> WordPenalty0= -0.18469
>> PhrasePenalty0= -0.212528
>> TranslationModel0= 0.0184105 0.091358 0.112618 0.0161682
>> UnknownWordPenalty0= 1
>
> On Wednesday, July 22, 2015, Oren <mooshified@gmail.com
> <mailto:mooshified@gmail.com>> wrote:
>
> Yes, we use a lot of RAM in our setup. But the improved response
> time justifies it.
>
> Our language is on a nfs, bit we've been working this way with
> moses 1 for some time with no problems (ten different machines
> using the same language model file over nfs). Same for the
> reordering model. Neither of them is in memory. Raising these
> models into memory will require raising our already excessive RAM
> requirements...
>
> Thanks again for the help.
>
> On Wednesday, July 22, 2015, Barry Haddow
> <bhaddow@staffmail.ed.ac.uk
> <javascript:_e(%7B%7D,'cvml','bhaddow@staffmail.ed.ac.uk');>> wrote:
>
> Hi Oren
>
> I'm not aware of any threading problems with
> PhraseDictionaryMemory, but not many people use it since it
> takes up too much memory. Moses command line runs
> multi-threaded using a thread pool.
>
> Is your language model on a local file system? Running it over
> nfs can be bad. What about your reordering model? Are you
> using the compact or the memory version?
>
> cheers - Barry
>
> On 22/07/15 15:09, Oren wrote:
>> We have no swapping issues. I was asking if the use of
>> an in-memory translation model might cause multithreading
>> problems.
>>
>>
>> I'm not sure how to replicate the problem on cmd moses, since
>> it's purely a multithreading issue. Ca you run cmd moses
>> multithreaded?
>>
>> I can't attach the complete moses.ini because it's on a
>> separate network... But I copied below the stuff that looked
>> relevant. I also tried to change the [threads] setting to 18,
>> with no apparent effect.
>>
>> [input-factors]
>> 0
>>
>> [mapping]
>> 0 T 0
>>
>> [distortion-limit]
>> 6
>>
>> [feature]
>> UnknownWordPenalty
>> WordPenalty
>> PhrasePenalty
>> PhraseDictionaryMemory name=TranslationModel0 num-features=4
>> path=<path>/phrase-table.gz input-factor=0 output-factor=0
>> LexicalReordering <parameters>
>> Distortion
>> KENLM lazyken=0 name=LM0 path=<path> order=5
>>
>> [threads]
>> 6
>>
>> [weight]
>>
>> <weight parameters>
>>
>> On Tuesday, July 21, 2015, Hieu Hoang <hieuhoang@gmail.com>
>> wrote:
>>
>> is it possible you can make your moses.ini file available
>> for us to see?
>>
>> do you know if the same problem occurs if you use the
>> command line moses, rather than mosesserver?
>>
>>
>> Hieu Hoang
>> Researcher
>> New York University, Abu Dhabi
>> http://www.hoang.co.uk/hieu
>>
>> On 21 July 2015 at 18:07, Barry Haddow
>> <bhaddow@staffmail.ed.ac.uk> wrote:
>>
>>
>> On 21/07/15 14:51, Oren wrote:
>>> I am using the in-memory mode, using about 50GB of
>>> RAM. (No swap issues as far as I can tell.) Could
>>> that cause issues?
>>
>> Yes, swapping would definitely cause issues - was
>> that your question?
>>
>>
>>>
>>> I looked at the commit you linked to, but it doesn't
>>> seem to be something configurable beyond the
>>> -threads switch. Am I missing something?
>>
>> The commit enables you to set the maximum number of
>> connections to be the same as the maximum number of
>> threads.
>>
>>>
>>> On Tuesday, July 21, 2015, Barry Haddow
>>> <bhaddow@staffmail.ed.ac.uk> wrote:
>>>
>>> Hi Oren
>>>
>>> Does your host have 18 threads available? It
>>> could also be that xmlrpc-c is limiting the
>>> number of connections - this can now be configured:
>>> https://github.com/moses-smt/mosesdecoder/commit/b3baade7f022edbcea2969679a40616683f63523
>>>
>>> Slowdowns in Moses are often caused by disk
>>> access bottlenecks. You can use --minphr-memory
>>> and --minlexr-memory to make sure that the
>>> phrase and reordering tables are loaded in to
>>> memory, rather than being access on-demand. Make
>>> sure your host has enough RAM and is not
>>> swapping. As I mentioned before there are
>>> various ways to make your models smaller
>>> (http://www.statmt.org/moses/?n=Advanced.RuleTables),
>>> which can make a big difference to speed
>>> depending on your setup.
>>>
>>> cheers - Barry
>>>
>>> On 21/07/15 09:30, Oren wrote:
>>>> Hi Barry,
>>>>
>>>> Thanks for the quick response.
>>>>
>>>> I added the switch "-threads 18" to the command
>>>> to raise moses server. The slowness issue
>>>> persists but in a different form. Most requests
>>>> return right away, even under heavy load, but
>>>> some requests (about 5%) take far longer -
>>>> about 15-20seconds.
>>>>
>>>> Perhaps there are other relevant switches?
>>>>
>>>> Thanks again.
>>>>
>>>> On Monday, July 20, 2015, Barry Haddow
>>>> <bhaddow@staffmail.ed.ac.uk> wrote:
>>>>
>>>> Hi Oren
>>>>
>>>> The threading model is different. In v1,
>>>> the server created a new thread for every
>>>> request, v3 uses a thread pool. Try
>>>> increasing the number of threads.
>>>>
>>>> Also, make sure you use the compact phrase
>>>> table and KenLM as they are normally
>>>> faster, and pre-pruning your phrase table
>>>> can help,
>>>>
>>>> cheers - Barry
>>>>
>>>> On 20/07/15 12:01, Oren wrote:
>>>>> Hi all,
>>>>>
>>>>> We are in the process of migrating from
>>>>> Moses 1 to Moses 3. We have noticed a
>>>>> significant slowdown when sending many
>>>>> requests at once to Moses Server. The
>>>>> first request will actually finish about
>>>>> 25% faster that a single request using
>>>>> Moses 1, but as more requests accumulate
>>>>> there is a marked slowdown, until requests
>>>>> take 5 times longer or more.
>>>>>
>>>>> Is this a known issue? Is it specific to
>>>>> Moses Server? What can we do about it?
>>>>>
>>>>> Thanks!
>>>>>
>>>>> Oren.
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Moses-support mailing list
>>>>> Moses-support@mit.edu
>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>
>>
>> The University of Edinburgh is a charitable body,
>> registered in
>> Scotland, with registration number SC005336.
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150723/81865ad6/attachment.htm
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150723/81865ad6/attachment.bat

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 105, Issue 50
**********************************************

Tweet