Home » Uncategories » Moses-support Digest, Vol 106, Issue 6

Moses-support Digest, Vol 106, Issue 6

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: blm napalm weights name in ems (John Morgan)
2. Re: blm napalm weights name in ems (Barry Haddow)
3. Re: blm napalm weights name in ems (John Joseph Morgan)
4. Re: Performance issue using Moses Server with Moses 3
(probably same as Oren's) (Oren)


----------------------------------------------------------------------

Message: 1
Date: Mon, 3 Aug 2015 13:55:51 -0400
From: John Morgan <johnjosephmorgan@gmail.com>
Subject: Re: [Moses-support] blm napalm weights name in ems
To: Barry Haddow <bhaddow@inf.ed.ac.uk>
Cc: Moses-support <moses-support@mit.edu>
Message-ID:
<CAMVtbfPNtufpSOSse0iADAVcq8oSUX0sXCpvdrfmLYeUdgit3A@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Thanks Barry,
I think there's still a problem with the feature name.
I think the subroutine get_order_of_scores_from_nbestlist in
mert-moses.pl does not expect a dash in the feature name.
John


On 8/3/15, Barry Haddow <bhaddow@inf.ed.ac.uk> wrote:
> Hi John
>
>> Is there a reason the example weight file has this feature name that I?m
>> missing?
> My fault I'm afraid. I streamlined bilingual-lm in EMS, but didn't
> realise that the example bypassed tuning. I've fixed it now according to
> your suggestion,
>
> cheers - Barry
>
> On 02/08/15 15:46, John Joseph Morgan wrote:
>> Hello,
>> I just wanted to give a heads up that in order to get the ems to run
>> completely with the config.toy.bilinguallm config file I had to make a
>> minor change.
>> Instead of tuning the config file points to a file called
>> weight_bilinguallm.ini.
>> I changed the feature name ?BilingualNPLM0? to BLMbilingual-lm?.
>> I?m running the same setup now with tuning, I?m hoping the feature names
>> get written correctly.
>> Is there a reason the example weight file has this feature name that I?m
>> missing?
>> Thanks,
>> John
>>
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: TUNING_tune.3.STDERR
Type: application/octet-stream
Size: 4553 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20150803/a1a02333/attachment-0001.obj

------------------------------

Message: 2
Date: Mon, 03 Aug 2015 21:19:33 +0100
From: Barry Haddow <bhaddow@inf.ed.ac.uk>
Subject: Re: [Moses-support] blm napalm weights name in ems
To: John Morgan <johnjosephmorgan@gmail.com>
Cc: Moses-support <moses-support@mit.edu>
Message-ID: <55BFCCD5.7070006@inf.ed.ac.uk>
Content-Type: text/plain; charset=UTF-8; format=flowed

I took out the dash - does it work now?

On 03/08/15 18:55, John Morgan wrote:
> Thanks Barry,
> I think there's still a problem with the feature name.
> I think the subroutine get_order_of_scores_from_nbestlist in
> mert-moses.pl does not expect a dash in the feature name.
> John
>
>
> On 8/3/15, Barry Haddow <bhaddow@inf.ed.ac.uk> wrote:
>> Hi John
>>
>>> Is there a reason the example weight file has this feature name that I?m
>>> missing?
>> My fault I'm afraid. I streamlined bilingual-lm in EMS, but didn't
>> realise that the example bypassed tuning. I've fixed it now according to
>> your suggestion,
>>
>> cheers - Barry
>>
>> On 02/08/15 15:46, John Joseph Morgan wrote:
>>> Hello,
>>> I just wanted to give a heads up that in order to get the ems to run
>>> completely with the config.toy.bilinguallm config file I had to make a
>>> minor change.
>>> Instead of tuning the config file points to a file called
>>> weight_bilinguallm.ini.
>>> I changed the feature name ?BilingualNPLM0? to BLMbilingual-lm?.
>>> I?m running the same setup now with tuning, I?m hoping the feature names
>>> get written correctly.
>>> Is there a reason the example weight file has this feature name that I?m
>>> missing?
>>> Thanks,
>>> John
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>> --
>> The University of Edinburgh is a charitable body, registered in
>> Scotland, with registration number SC005336.
>>
>>


--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



------------------------------

Message: 3
Date: Mon, 3 Aug 2015 18:49:11 -0400
From: John Joseph Morgan <johnjosephmorgan@gmail.com>
Subject: Re: [Moses-support] blm napalm weights name in ems
To: Barry Haddow <bhaddow@inf.ed.ac.uk>
Cc: Moses-support <moses-support@mit.edu>
Message-ID: <DD21F7D9-57FB-460D-8575-553B9B4F1DD5@gmail.com>
Content-Type: text/plain; charset=utf-8

I have it working by modifying mert-moses.pl on line 1476.
I replace [0-9a-z] with [-0-9a-z].
Is there a problem with modifying mert-moses.pl like this?


> On Aug 3, 2015, at 4:19 PM, Barry Haddow <bhaddow@inf.ed.ac.uk> wrote:
>
> I took out the dash - does it work now?
>
> On 03/08/15 18:55, John Morgan wrote:
>> Thanks Barry,
>> I think there's still a problem with the feature name.
>> I think the subroutine get_order_of_scores_from_nbestlist in
>> mert-moses.pl does not expect a dash in the feature name.
>> John
>>
>>
>> On 8/3/15, Barry Haddow <bhaddow@inf.ed.ac.uk> wrote:
>>> Hi John
>>>
>>>> Is there a reason the example weight file has this feature name that I?m
>>>> missing?
>>> My fault I'm afraid. I streamlined bilingual-lm in EMS, but didn't
>>> realise that the example bypassed tuning. I've fixed it now according to
>>> your suggestion,
>>>
>>> cheers - Barry
>>>
>>> On 02/08/15 15:46, John Joseph Morgan wrote:
>>>> Hello,
>>>> I just wanted to give a heads up that in order to get the ems to run
>>>> completely with the config.toy.bilinguallm config file I had to make a
>>>> minor change.
>>>> Instead of tuning the config file points to a file called
>>>> weight_bilinguallm.ini.
>>>> I changed the feature name ?BilingualNPLM0? to BLMbilingual-lm?.
>>>> I?m running the same setup now with tuning, I?m hoping the feature names
>>>> get written correctly.
>>>> Is there a reason the example weight file has this feature name that I?m
>>>> missing?
>>>> Thanks,
>>>> John
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>> --
>>> The University of Edinburgh is a charitable body, registered in
>>> Scotland, with registration number SC005336.
>>>
>>>
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>




------------------------------

Message: 4
Date: Tue, 4 Aug 2015 10:08:57 +0300
From: Oren <mooshified@gmail.com>
Subject: Re: [Moses-support] Performance issue using Moses Server with
Moses 3 (probably same as Oren's)
To: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>, Martin
Baumg?rtner <martin.baumgaertner@star-group.net>
Message-ID:
<CAHWEG_VoqdwE+FW3ijWCOwsxO7oq=h92KFZbw2d7cnD78z6L3w@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Barry and Martin,

Has this issue been fixed in the source code? Should I take thr current
master branch and compile it myself to avoid this issue?

Thanks.

On Friday, July 24, 2015, Barry Haddow <bhaddow@staffmail.ed.ac.uk> wrote:

> Hi Martin
>
> So it looks like it was the abyss connection limit that was causing the
> problem? I'm not sure why this should be, either it should queue the jobs
> up or discard them.
>
> Probably Moses server should allow users to configure the number of abyss
> connections directly rather than tying it to the number of Moses threads.
>
> cheers - Barry
>
> On 24/07/15 14:17, Martin Baumg?rtner wrote:
>
> Hi Barry,
>
> thanks for your quick reply!
>
> We're currently testing on SHA e53ad4085942872f1c4ce75cb99afe66137e1e17
> (master, from 2015-07-23). This version includes the fix for mosesserver
> recently mentioned by Hieu in the performance thread.
>
> Following my first intuition, I ran the critical experiments after having
> modified mosesserver.cpp just by simply doubling the given --threads value,
> but only for abyss server: .maxConn((unsigned int)numThreads*2):
>
> 2.)
> server: --threads: 8 (i.e. abyss: 16)
> client: shoots 10 threads => about 11 seconds, server shows busy CPU
> workload => OK
>
> 5.)
> server: --threads: 16 (i.e. abyss: 32)
> client: shoots 20 threads => about 11 seconds, server shows busy CPU
> workload => OK
>
> Helps. :-)
>
> Best wishes,
> Martin
>
> Am 24.07.2015 um 13:26 schrieb Barry Haddow:
>
> Hi Martin
>
> Thanks for the detailed information. It's a bit strange since command-line
> Moses uses the same threadpool, and we always overload the threadpool since
> the entire test set is read in and queued.
>
> The server was refactored somewhat recently - which git revision are you
> using?
>
> In the case where Moses takes a long time, and cpu activity is low, it
> could be either waiting on IO, or waiting on locks. If the former, I don't
> know why it works fine for command-line Moses, and if the latter then it's
> odd how it eventually frees itself.
>
> Is it possible to run scenario 2, then attach a debugger whilst Moses is
> in the low-CPU phase to see what it is doing? (You can do this in gdb with
> "info threads")
>
> cheers - Barry
>
> On 24/07/15 12:07, Martin Baumg?rtner wrote:
>
> Hi,
>
> followed your discussion about mosesserver performance issue with much
> interest so far.
>
> We're having similar behaviour in our perfomance tests with a current
> github master clone. Both, mosesserver and complete engine run from same
> local machine, i.e. no NFS. Machine is virtualized CentOS 7 using Hyper-V:
>
> > lscpu
>
> Architecture: x86_64
> CPU op-mode(s): 32-bit, 64-bit
> Byte Order: Little Endian
> CPU(s): 8
> On-line CPU(s) list: 0-7
> Thread(s) per core: 1
> Core(s) per socket: 8
> Socket(s): 1
> NUMA node(s): 1
> Vendor ID: GenuineIntel
> CPU family: 6
> Model: 30
> Model name: Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz
> Stepping: 5
> CPU MHz: 2667.859
> BogoMIPS: 5335.71
> Hypervisor vendor: Microsoft
> Virtualization type: full
> L1d cache: 32K
> L1i cache: 32K
> L2 cache: 256K
> L3 cache: 8192K
>
>
> Following experiments using an engine with 75000 segments for TM/LM
> (--minphr-memory, --minlexr-memory):
>
> 1.)
> server: --threads: 8
> client: shoots 8 threads => about 12 seconds, server shows full CPU
> workload => OK
>
> 2.)
> server: --threads: 8
> client: shoots 10 threads => about 85 seconds, server shows mostly low
> activity, full CPU workload only near end of process => NOT OK
>
> 3.)
> server: --threads: 16
> client: shoots 10 threads => about 12 seconds, server shows busy CPU
> workload => OK
>
> 4.)
> server: --threads: 16
> client: shoots 16 threads => about 11 seconds, server shows busy CPU
> workload => OK
>
> 5.)
> server: --threads: 16
> client: shoots 20 threads => about 40-60 seconds (depending), server shows
> mostly low activity, full CPU workload only near end of process => NOT OK
>
>
> We've seen a breakdown in performance always when the client threads
> exceed the number of threads given by the --threads param.
>
> Kind regards,
> Martin
>
> --
>
>
> *STAR Group* <http://www.star-group.net>
> <http://www.star-group.net/>
> *Martin Baumg?rtner*
> STAR Language Technology & Solutions GmbH Umberto-Nobile-Stra?e 19 |
> 71063 Sindelfingen | Germany Tel. +49 70 31-4 10 92-0 martin.baumgaertner@star-group.net
> <javascript:_e(%7B%7D,'cvml','martin.baumgaertner@star-group.net');> Fax
> +49 70 31-4 10 92-70 www.star-group.net Gesch?ftsf?hrer: Oliver Rau,
> Bernd Barth
> Handelsregister Stuttgart HRB 245654 | St.-Nr. 56098/11677
>
>
>
>
> _______________________________________________
> Moses-support mailing listMoses-support@mit.edu <javascript:_e(%7B%7D,'cvml','Moses-support@mit.edu');>http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
>
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
> --
>
>
> *STAR Group* <http://www.star-group.net>
> <http://www.star-group.net/>
> *Martin Baumg?rtner*
> STAR Language Technology & Solutions GmbH Umberto-Nobile-Stra?e 19 |
> 71063 Sindelfingen | Germany Tel. +49 70 31-4 10 92-0 martin.baumgaertner@star-group.net
> <javascript:_e(%7B%7D,'cvml','martin.baumgaertner@star-group.net');> Fax
> +49 70 31-4 10 92-70 www.star-group.net Gesch?ftsf?hrer: Oliver Rau,
> Bernd Barth
> Handelsregister Stuttgart HRB 245654 | St.-Nr. 56098/11677
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150804/aec6c050/attachment.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 8030 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20150804/aec6c050/attachment.gif
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 8030 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20150804/aec6c050/attachment-0001.gif

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 106, Issue 6
*********************************************

0 Response to "Moses-support Digest, Vol 106, Issue 6"

Post a Comment

Subscribe to: Post Comments (Atom)