Moses-support Digest, Vol 106, Issue 14

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Performance issue using Moses Server with Moses 3
(probably same as Oren's) (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Thu, 6 Aug 2015 13:46:35 +0400
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Performance issue using Moses Server with
Moses 3 (probably same as Oren's)
To: Martin Baumg?rtner <martin.baumgaertner@star-group.net>
Cc: moses-support <moses-support@mit.edu>, Barry Haddow
<bhaddow@staffmail.ed.ac.uk>
Message-ID:
<CAEKMkbjOcKoAc6VE8Oc5QiotrR=x+hDqCdXPRh9wyKrMHrzVbg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

cheers. pushed

https://github.com/moses-smt/mosesdecoder/commit/3c682fa8b05af6bff1a09f420141795875cf9685

Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu

On 6 August 2015 at 13:06, Martin Baumg?rtner <
martin.baumgaertner@star-group.net> wrote:

> Hi Hieu,
>
> just sent you the patch for mosesserver, because my attempts to push to
> github failed for some unknown reasons ;-)
>
> Best regards,
> Martin
>
>
>
> Am 05.08.2015 um 14:23 schrieb Hieu Hoang:
>
> It would be good if you can check in your change and take charge of it.
>
> If you're waiting for us academics to fix it, you'll be waiting a long
> time. We rarely use the server, we don't know what the issues are and we
> won't know if we've really fixed it when we change it
>
> Hieu Hoang
> Sent while bumping into things
> On 5 Aug 2015 4:15 pm, "Martin Baumg?rtner" <
> martin.baumgaertner@star-group.net> wrote:
>
>> Hi Oren,
>>
>> we temporarily fixed this issue with the following quick hack for Abyss
>> server's constructor call:
>>
>> xmlrpc_c::serverAbyss myAbyssServer(
>> xmlrpc_c::serverAbyss::constrOpt()
>> .registryP(&myRegistry)
>> .portNumber(port) // TCP port on which to listen
>> .logFileName(logfile)
>> .allowOrigin("*")
>> .maxConn((unsigned int)numThreads*4) // *4 (performance issue,
>> inofficial quick hack)
>> );
>>
>> I'm also looking forward to the official fix, i.e. a configurable value
>> for abyss connections ...
>>
>> Kind regards,
>> Martin
>>
>>
>> Am 04.08.2015 um 09:08 schrieb Oren:
>>
>> Hi Barry and Martin,
>>
>> Has this issue been fixed in the source code? Should I take thr current
>> master branch and compile it myself to avoid this issue?
>>
>> Thanks.
>>
>> On Friday, July 24, 2015, Barry Haddow <bhaddow@staffmail.ed.ac.uk>
>> wrote:
>>
>>> Hi Martin
>>>
>>> So it looks like it was the abyss connection limit that was causing the
>>> problem? I'm not sure why this should be, either it should queue the jobs
>>> up or discard them.
>>>
>>> Probably Moses server should allow users to configure the number of
>>> abyss connections directly rather than tying it to the number of Moses
>>> threads.
>>>
>>> cheers - Barry
>>>
>>> On 24/07/15 14:17, Martin Baumg?rtner wrote:
>>>
>>> Hi Barry,
>>>
>>> thanks for your quick reply!
>>>
>>> We're currently testing on SHA e53ad4085942872f1c4ce75cb99afe66137e1e17
>>> (master, from 2015-07-23). This version includes the fix for mosesserver
>>> recently mentioned by Hieu in the performance thread.
>>>
>>> Following my first intuition, I ran the critical experiments after
>>> having modified mosesserver.cpp just by simply doubling the given --threads
>>> value, but only for abyss server: .maxConn((unsigned int)numThreads*2):
>>>
>>> 2.)
>>> server: --threads: 8 (i.e. abyss: 16)
>>> client: shoots 10 threads => about 11 seconds, server shows busy CPU
>>> workload => OK
>>>
>>> 5.)
>>> server: --threads: 16 (i.e. abyss: 32)
>>> client: shoots 20 threads => about 11 seconds, server shows busy CPU
>>> workload => OK
>>>
>>> Helps. :-)
>>>
>>> Best wishes,
>>> Martin
>>>
>>> Am 24.07.2015 um 13:26 schrieb Barry Haddow:
>>>
>>> Hi Martin
>>>
>>> Thanks for the detailed information. It's a bit strange since
>>> command-line Moses uses the same threadpool, and we always overload the
>>> threadpool since the entire test set is read in and queued.
>>>
>>> The server was refactored somewhat recently - which git revision are you
>>> using?
>>>
>>> In the case where Moses takes a long time, and cpu activity is low, it
>>> could be either waiting on IO, or waiting on locks. If the former, I don't
>>> know why it works fine for command-line Moses, and if the latter then it's
>>> odd how it eventually frees itself.
>>>
>>> Is it possible to run scenario 2, then attach a debugger whilst Moses is
>>> in the low-CPU phase to see what it is doing? (You can do this in gdb with
>>> "info threads")
>>>
>>> cheers - Barry
>>>
>>> On 24/07/15 12:07, Martin Baumg?rtner wrote:
>>>
>>> Hi,
>>>
>>> followed your discussion about mosesserver performance issue with much
>>> interest so far.
>>>
>>> We're having similar behaviour in our perfomance tests with a current
>>> github master clone. Both, mosesserver and complete engine run from same
>>> local machine, i.e. no NFS. Machine is virtualized CentOS 7 using Hyper-V:
>>>
>>> > lscpu
>>>
>>> Architecture: x86_64
>>> CPU op-mode(s): 32-bit, 64-bit
>>> Byte Order: Little Endian
>>> CPU(s): 8
>>> On-line CPU(s) list: 0-7
>>> Thread(s) per core: 1
>>> Core(s) per socket: 8
>>> Socket(s): 1
>>> NUMA node(s): 1
>>> Vendor ID: GenuineIntel
>>> CPU family: 6
>>> Model: 30
>>> Model name: Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz
>>> Stepping: 5
>>> CPU MHz: 2667.859
>>> BogoMIPS: 5335.71
>>> Hypervisor vendor: Microsoft
>>> Virtualization type: full
>>> L1d cache: 32K
>>> L1i cache: 32K
>>> L2 cache: 256K
>>> L3 cache: 8192K
>>>
>>>
>>> Following experiments using an engine with 75000 segments for TM/LM
>>> (--minphr-memory, --minlexr-memory):
>>>
>>> 1.)
>>> server: --threads: 8
>>> client: shoots 8 threads => about 12 seconds, server shows full CPU
>>> workload => OK
>>>
>>> 2.)
>>> server: --threads: 8
>>> client: shoots 10 threads => about 85 seconds, server shows mostly low
>>> activity, full CPU workload only near end of process => NOT OK
>>>
>>> 3.)
>>> server: --threads: 16
>>> client: shoots 10 threads => about 12 seconds, server shows busy CPU
>>> workload => OK
>>>
>>> 4.)
>>> server: --threads: 16
>>> client: shoots 16 threads => about 11 seconds, server shows busy CPU
>>> workload => OK
>>>
>>> 5.)
>>> server: --threads: 16
>>> client: shoots 20 threads => about 40-60 seconds (depending), server
>>> shows mostly low activity, full CPU workload only near end of process =>
>>> NOT OK
>>>
>>>
>>> We've seen a breakdown in performance always when the client threads
>>> exceed the number of threads given by the --threads param.
>>>
>>> Kind regards,
>>> Martin
>>>
>>> --
>>>
>>>
>>> *STAR Group* <http://www.star-group.net>
>>> <http://www.star-group.net/>
>>> *Martin Baumg?rtner*
>>> STAR Language Technology & Solutions GmbH Umberto-Nobile-Stra?e 19 |
>>> 71063 Sindelfingen | Germany Tel. +49 70 31-4 10 92-0 martin.baumgaertner@star-group.net
>>> Fax +49 70 31-4 10 92-70 www.star-group.net Gesch?ftsf?hrer: Oliver
>>> Rau, Bernd Barth
>>> Handelsregister Stuttgart HRB 245654 | St.-Nr. 56098/11677
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>>
>>>
>>> The University of Edinburgh is a charitable body, registered in
>>> Scotland, with registration number SC005336.
>>>
>>>
>>> --
>>>
>>>
>>> *STAR Group* <http://www.star-group.net>
>>> <http://www.star-group.net/>
>>> *Martin Baumg?rtner*
>>> STAR Language Technology & Solutions GmbH Umberto-Nobile-Stra?e 19 |
>>> 71063 Sindelfingen | Germany Tel. +49 70 31-4 10 92-0 martin.baumgaertner@star-group.net
>>> Fax +49 70 31-4 10 92-70 www.star-group.net Gesch?ftsf?hrer: Oliver
>>> Rau, Bernd Barth
>>> Handelsregister Stuttgart HRB 245654 | St.-Nr. 56098/11677
>>>
>>>
>>>
>>>
>>>
>> --
>>
>>
>> *STAR Group* <http://www.star-group.net>
>> <http://www.star-group.net/>
>> *Martin Baumg?rtner*
>> STAR Language Technology & Solutions GmbH Umberto-Nobile-Stra?e 19 |
>> 71063 Sindelfingen | Germany Tel. +49 70 31-4 10 92-0 martin.baumgaertner@star-group.net
>> Fax +49 70 31-4 10 92-70 www.star-group.net Gesch?ftsf?hrer: Oliver Rau,
>> Bernd Barth
>> Handelsregister Stuttgart HRB 245654 | St.-Nr. 56098/11677
>>
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
> --
>
>
> *STAR Group* <http://www.star-group.net>
> <http://www.star-group.net/>
> *Martin Baumg?rtner*
> STAR Language Technology & Solutions GmbH Umberto-Nobile-Stra?e 19 |
> 71063 Sindelfingen | Germany Tel. +49 70 31-4 10 92-0 martin.baumgaertner@star-group.net
> Fax +49 70 31-4 10 92-70 www.star-group.net Gesch?ftsf?hrer: Oliver Rau,
> Bernd Barth
> Handelsregister Stuttgart HRB 245654 | St.-Nr. 56098/11677
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150806/92edd8ef/attachment.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 8030 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20150806/92edd8ef/attachment.gif
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 8030 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20150806/92edd8ef/attachment-0001.gif
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 8030 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20150806/92edd8ef/attachment-0002.gif
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fgadbhbc.gif
Type: image/gif
Size: 8030 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20150806/92edd8ef/attachment-0003.gif

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 106, Issue 14
**********************************************

Tweet