Moses-support Digest, Vol 125, Issue 34

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Moses vs Moses2 in its multi-threading (Hieu Hoang)
2. Re: Moses vs Moses2 in its multi-threading (Ivan Zapreev)


----------------------------------------------------------------------

Message: 1
Date: Thu, 16 Mar 2017 14:11:14 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Moses vs Moses2 in its multi-threading
To: Ivan Zapreev <ivan.zapreev@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbjxq4rB4CpB_fy1pHwG-mZeZy77Y=SPYoFszsBV6e8Tmw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

* Looking for MT/NLP opportunities *
Hieu Hoang
http://moses-smt.org/


On 16 March 2017 at 13:16, Ivan Zapreev <ivan.zapreev@gmail.com> wrote:

> Dear Hieu,
>
> Thank you for a prompt and detailed reply!
>
> >>> So your server has 20 cores (40 hyperthreads) and 16GB RAM? If that's
> correct, then the RAM size would be a problem - you need as much RAM as the
> total size of your models, plus more for working memory and the OS.
>
> The amount of memory is 256 Gb and not 16. There are a number of 16 Gb
> plates installed.
> To my knowledge the machine is not hyperthreaded but just has 40 cores,
> although I am now getting a bit doubtful about that.
>
256GB is good. 20/40 core/hyperthreads is not important for the moment, but
you should find out exactly what it is


>
> >> Do you run Moses command line, or the server? My timings are based on
> the command line, the server is a little slower.
>
> Both Moses and Moses2 are run in the console mode (not server). The model
> loading time is excluded from the measurements. I could not manage to get
> the asynchronous XML-RPC to work, so for my experiments that would be as if
> I used Moses/Moses2 in a single-thread mode. Therefore I used the
> command-line version.
>
> >>> Do you run Moses directly, or is another evaluation process running
> it? Are you sure that evaluation process is working as it should?
>
> Moses is run from command time under the "time" command of Linux, and so
> are other systems we used in comarison. We look at the runtime and not the
> CPU times, but we perform a number of experiments to measure the average
> times and control the standard deviations.
>
> >>> Do you minimise the effect of disk read by pre-loading the models into
> filesystem cache? This is usually done by running this before running the
> decoder cat [binary model files] > /dev/null
>
> Nop, we did not do pre-loading, for none of the tools but perhaps this is
> not an issue as we just measure the average model loading times and
> subtract them from the average run-time with decoding. So the model loading
> times are excluded from the results. Our goal was to measure and compare
> the decoding times and how they scale in the number of threads.
>
You're using the Probing phrase-table with integrated reordering model, and
a binary KenLM, right?

If so, the loading time will be minimal (1-2 secs) since the binary format
just memory map the whole data but doesn't actually load them into memory.
However, the overwhelming amount of the time taken for decoding will be
page faults while doing LM and pt random lookups.

It would be no surprise that decoding speed for Moses and Moses2 would be
similar without pre-loading - they are looking up the same data.


>
> >>> it may take a while, but I can't replicate your results without it.
> Alternatively, I can provide you with my models so you can try & replicate
> my results.
>
> The experiments are run on an internal server which is not visible from
> outside. I shall explore the possibilities of sharing the models, but I am
> doubtful it is possible. The university network is very restricted. Yet, I
> am definitely open to re-running your experiments. If possible.
>
i can make it available. But the results will be the same unless you sort
out your pre-loading

>
> Kind regards,
>
> Ivan
>
> <http://www.tainichok.ru/>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170316/a32212cc/attachment-0001.html

------------------------------

Message: 2
Date: Thu, 16 Mar 2017 15:51:33 +0100
From: Ivan Zapreev <ivan.zapreev@gmail.com>
Subject: Re: [Moses-support] Moses vs Moses2 in its multi-threading
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAOwV4isG+Oc63KLeiZDjfhAxyhtkKHZQP8MRxB7EsOWsSMspTQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Dear Hieu,

Well, whether it is 20 or 40 cores is irrelevant as the trend on up to 20
threads still contradicts the data provided on the Moses2 page.

Regarding using the binary models, I shall check this out, as far as I
remember I use plain text models. I will look it up later today. But I see
that loading the models takes about 763 seconds, so it is 12-13 minutes ...
does not look binary to me. At least not all of them.

By the way, did you look at the data I provided? Moses2 and Moses do NOT
have the same speed using single thread. Moses2 is about 2 times faster and
by reaching 40 threads it is just 1.5 faster than Moses. This means that
Moses2 is by itself using faster algorithms or data structures, and its
better performance in my experiments comes from there and not from a better
multi-threading. You can consider also the Moses2 speed-up plots with
respect to single thread (provided in my experiments). Moses2 seems to
scale worse than Moses in the number of threads. I use the same models for
Moses and Moses2 so any possible memory hits, unless the data structures
are different, should be the same and thus not influencing the overall
picture of scaling with the increased number of threads.

By the way, should pre-loading of plain text models help? As far as I see
it should not make any difference ...

Kind regards,

Dr. Ivan S. Zapreev

On Thu, Mar 16, 2017 at 3:11 PM, Hieu Hoang <hieuhoang@gmail.com> wrote:

>
>
> * Looking for MT/NLP opportunities *
> Hieu Hoang
> http://moses-smt.org/
>
>
> On 16 March 2017 at 13:16, Ivan Zapreev <ivan.zapreev@gmail.com> wrote:
>
>> Dear Hieu,
>>
>> Thank you for a prompt and detailed reply!
>>
>> >>> So your server has 20 cores (40 hyperthreads) and 16GB RAM? If that's
>> correct, then the RAM size would be a problem - you need as much RAM as the
>> total size of your models, plus more for working memory and the OS.
>>
>> The amount of memory is 256 Gb and not 16. There are a number of 16 Gb
>> plates installed.
>> To my knowledge the machine is not hyperthreaded but just has 40 cores,
>> although I am now getting a bit doubtful about that.
>>
> 256GB is good. 20/40 core/hyperthreads is not important for the moment,
> but you should find out exactly what it is
>
>
>>
>> >> Do you run Moses command line, or the server? My timings are based on
>> the command line, the server is a little slower.
>>
>> Both Moses and Moses2 are run in the console mode (not server). The model
>> loading time is excluded from the measurements. I could not manage to get
>> the asynchronous XML-RPC to work, so for my experiments that would be as if
>> I used Moses/Moses2 in a single-thread mode. Therefore I used the
>> command-line version.
>>
>> >>> Do you run Moses directly, or is another evaluation process running
>> it? Are you sure that evaluation process is working as it should?
>>
>> Moses is run from command time under the "time" command of Linux, and so
>> are other systems we used in comarison. We look at the runtime and not the
>> CPU times, but we perform a number of experiments to measure the average
>> times and control the standard deviations.
>>
>> >>> Do you minimise the effect of disk read by pre-loading the models
>> into filesystem cache? This is usually done by running this before running
>> the decoder cat [binary model files] > /dev/null
>>
>> Nop, we did not do pre-loading, for none of the tools but perhaps this is
>> not an issue as we just measure the average model loading times and
>> subtract them from the average run-time with decoding. So the model loading
>> times are excluded from the results. Our goal was to measure and compare
>> the decoding times and how they scale in the number of threads.
>>
> You're using the Probing phrase-table with integrated reordering model,
> and a binary KenLM, right?
>
> If so, the loading time will be minimal (1-2 secs) since the binary format
> just memory map the whole data but doesn't actually load them into memory.
> However, the overwhelming amount of the time taken for decoding will be
> page faults while doing LM and pt random lookups.
>
> It would be no surprise that decoding speed for Moses and Moses2 would be
> similar without pre-loading - they are looking up the same data.
>
>
>>
>> >>> it may take a while, but I can't replicate your results without it.
>> Alternatively, I can provide you with my models so you can try & replicate
>> my results.
>>
>> The experiments are run on an internal server which is not visible from
>> outside. I shall explore the possibilities of sharing the models, but I am
>> doubtful it is possible. The university network is very restricted. Yet, I
>> am definitely open to re-running your experiments. If possible.
>>
> i can make it available. But the results will be the same unless you sort
> out your pre-loading
>
>>
>> Kind regards,
>>
>> Ivan
>>
>> <http://www.tainichok.ru/>
>>
>
>


--
Best regards,

Ivan
<http://www.tainichok.ru/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170316/5f1eb473/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 125, Issue 34
**********************************************

Tweet