Moses-support Digest, Vol 126, Issue 2

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Moses vs Moses2 in its multi-threading (Hieu Hoang)
2. Support Moses and GPU on cloud (liling tan)


----------------------------------------------------------------------

Message: 1
Date: Tue, 4 Apr 2017 02:25:17 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Moses vs Moses2 in its multi-threading
To: Ivan Zapreev <ivan.zapreev@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbjzUOwtqM7b2hz00wu9xUoLi=3yDVYZmDr85JJVNnXQKA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

hi Ivan

Can you clarify a few things for me.
1. How many sentences, words were in the test set you used to measure
decoding speed? Are there many duplicate sentence - ie. did you create a
large test set by concatenating the same small test set multiple times?
2. Are the model sizes you quoted the gzipped text files or unzipped, or
the model size as it is when loaded into memory?
3. Can you please reformat this graph

https://github.com/ivan-zapreev/Basic-Translation-Infrastructure/blob/master/doc/images/experiments/servers/stats.time.tools.log.png
as #threads v. words per second. Ie. don't use log, don't use
decoding time.


* Looking for MT/NLP opportunities *
Hieu Hoang
http://moses-smt.org/


On 16 March 2017 at 15:51, Ivan Zapreev <ivan.zapreev@gmail.com> wrote:

> Awesome! Thanks, we will keep in touch.
>
> On Mar 16, 2017 16:49, "Hieu Hoang" <hieuhoang@gmail.com> wrote:
>
>>
>>
>> * Looking for MT/NLP opportunities *
>> Hieu Hoang
>> http://moses-smt.org/
>>
>>
>> On 16 March 2017 at 15:23, Ivan Zapreev <ivan.zapreev@gmail.com> wrote:
>>
>>> Dear Hieu,
>>>
>>> Thank you for you time and the clear answers.
>>>
>>> >>> ok, if you are using plain text models, then this is the 1st
>>> difference to my results. I don't optimize the plain text models, I don't
>>> wanna wait for 13 minutes and this is not how most users use the decoder.
>>>
>>> Well, perhaps, but for me as a user it should not matter, loading the
>>> models happens just once, when the system is started. So in production this
>>> is not a big issue, only if one wants to start things up and shut them down
>>> periodically and quite often.
>>>
>> good point. It come down to how you're using the decoder and your
>> priorities.
>>
>> For completeness, please evaluate with binary models. I'll test my models
>> with plain text models too
>>
>>
>>>
>>>
>>> >>> But if you see bad scaling in Moses2 with binary models, please let
>>> me know
>>>
>>> Is not that so that when the text models are loaded into memory they get
>>> binarized and then the binary model is just the memory snapshot of the
>>> loaded text model? I would expect so because otherwise one would need to
>>> use different data structures for storing binary and text models. Moreover,
>>> this would also mean different execution times on these two model types. So
>>> I suspect that having a text model should only influence the loading time
>>> of the model but not the decoding times...
>>>
>>> >>> If you do load the plain text files, you should check that it
>>> doesn't use up all memory and has to disk swap
>>>
>>> I have 5 times more memory than all the models take all together, so
>>> swapping is not an issue.
>>>
>>> >>> I know it's faster on 1 thread, it should be much faster on lots of
>>> threads.
>>>
>>> Yes, I was expecting the same results, but somehow I get a different
>>> trend .... so this is why I am asking.
>>>
>>> >>> no, just pre-load the binary files
>>>
>>> Ok, thanks!
>>>
>>> Kind regards,
>>>
>>> Dr. Ivan S. Zapreev
>>>
>>>
>>>
>>>> On Thu, Mar 16, 2017 at 3:11 PM, Hieu Hoang <hieuhoang@gmail.com>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> * Looking for MT/NLP opportunities *
>>>>> Hieu Hoang
>>>>> http://moses-smt.org/
>>>>>
>>>>>
>>>>> On 16 March 2017 at 13:16, Ivan Zapreev <ivan.zapreev@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Dear Hieu,
>>>>>>
>>>>>> Thank you for a prompt and detailed reply!
>>>>>>
>>>>>> >>> So your server has 20 cores (40 hyperthreads) and 16GB RAM? If
>>>>>> that's correct, then the RAM size would be a problem - you need as much RAM
>>>>>> as the total size of your models, plus more for working memory and the OS.
>>>>>>
>>>>>> The amount of memory is 256 Gb and not 16. There are a number of 16
>>>>>> Gb plates installed.
>>>>>> To my knowledge the machine is not hyperthreaded but just has 40
>>>>>> cores, although I am now getting a bit doubtful about that.
>>>>>>
>>>>> 256GB is good. 20/40 core/hyperthreads is not important for the
>>>>> moment, but you should find out exactly what it is
>>>>>
>>>>>
>>>>>>
>>>>>> >> Do you run Moses command line, or the server? My timings are based
>>>>>> on the command line, the server is a little slower.
>>>>>>
>>>>>> Both Moses and Moses2 are run in the console mode (not server). The
>>>>>> model loading time is excluded from the measurements. I could not manage to
>>>>>> get the asynchronous XML-RPC to work, so for my experiments that would be
>>>>>> as if I used Moses/Moses2 in a single-thread mode. Therefore I used the
>>>>>> command-line version.
>>>>>>
>>>>>> >>> Do you run Moses directly, or is another evaluation process
>>>>>> running it? Are you sure that evaluation process is working as it should?
>>>>>>
>>>>>> Moses is run from command time under the "time" command of Linux, and
>>>>>> so are other systems we used in comarison. We look at the runtime and not
>>>>>> the CPU times, but we perform a number of experiments to measure the
>>>>>> average times and control the standard deviations.
>>>>>>
>>>>>> >>> Do you minimise the effect of disk read by pre-loading the models
>>>>>> into filesystem cache? This is usually done by running this before running
>>>>>> the decoder cat [binary model files] > /dev/null
>>>>>>
>>>>>> Nop, we did not do pre-loading, for none of the tools but perhaps
>>>>>> this is not an issue as we just measure the average model loading times and
>>>>>> subtract them from the average run-time with decoding. So the model loading
>>>>>> times are excluded from the results. Our goal was to measure and compare
>>>>>> the decoding times and how they scale in the number of threads.
>>>>>>
>>>>> You're using the Probing phrase-table with integrated reordering
>>>>> model, and a binary KenLM, right?
>>>>>
>>>>> If so, the loading time will be minimal (1-2 secs) since the binary
>>>>> format just memory map the whole data but doesn't actually load them into
>>>>> memory. However, the overwhelming amount of the time taken for decoding
>>>>> will be page faults while doing LM and pt random lookups.
>>>>>
>>>>> It would be no surprise that decoding speed for Moses and Moses2 would
>>>>> be similar without pre-loading - they are looking up the same data.
>>>>>
>>>>>
>>>>>>
>>>>>> >>> it may take a while, but I can't replicate your results without
>>>>>> it. Alternatively, I can provide you with my models so you can try &
>>>>>> replicate my results.
>>>>>>
>>>>>> The experiments are run on an internal server which is not visible
>>>>>> from outside. I shall explore the possibilities of sharing the models, but
>>>>>> I am doubtful it is possible. The university network is very restricted.
>>>>>> Yet, I am definitely open to re-running your experiments. If possible.
>>>>>>
>>>>> i can make it available. But the results will be the same unless you
>>>>> sort out your pre-loading
>>>>>
>>>>>>
>>>>>> Kind regards,
>>>>>>
>>>>>> Ivan
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>>
>>>> Ivan
>>>>
>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>>
>>> Ivan
>>> <http://www.tainichok.ru/>
>>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170403/2cbc08bf/attachment-0001.html

------------------------------

Message: 2
Date: Tue, 4 Apr 2017 15:30:36 +0800
From: liling tan <alvations@gmail.com>
Subject: [Moses-support] Support Moses and GPU on cloud
To: moses-support <moses-support@mit.edu>
Message-ID:
<CAKzPaJJep2GvTXgTcwABDO95+KPNAwrnZ+GGds2oG50PhQUvOw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Dear Moses community,

Amittai had written a nice package and setup guide for Moses on AWS. But to
do some NMT on GPU, the instances wouldn't usually have enough RAM for
Moses.

Does anyone have experiencing deploying Moses on GPU instances on AWS or
any other cloud servers?

Regards,
Liling
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170404/c8c0ac98/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 126, Issue 2
*********************************************

Tweet