Moses-support Digest, Vol 135, Issue 16

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: speed up translation for one sentence on server (Hieu Hoang)
2. Re: speed up translation for one sentence on server (Hieu Hoang)
3. phrase table cleaning (jean boyer)


----------------------------------------------------------------------

Message: 1
Date: Sun, 14 Jan 2018 17:05:05 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] speed up translation for one sentence on
server
To: Manli Zhu <zhum.osu@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbj62Gutnp_UL1-HGAOFf+i5uKk5i8waSsNFrSw46h+CUQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hieu Hoang
http://moses-smt.org/


On 14 January 2018 at 17:03, Hieu Hoang <hieuhoang@gmail.com> wrote:

>
>
> Hieu Hoang
> http://moses-smt.org/
>
>
> On 14 January 2018 at 15:11, Manli Zhu <zhum.osu@gmail.com> wrote:
>
>> Thank you, Hieu! I still have questions for your suggestion, could you
>> answer my question below?
>>
>> 1. moses will be slow for the 1stfew Sentences
>>
>> If client will submit one sentence at a time, will it always be slow?
>> Is my lm and phrase table too big?
>>
> it will get faster.
>
>>
>> 2. The server runs only 1 thread for each daemon, but it can handle
>> multiple daemons at once.
>>
>> By multiple daemons, do you mean run multiple Maemo.pl simultaneously?
>> The multiple daemons share the same port number?
>>
> yes
>
>>
>> 3. If you want to use more than 16 cores, it's better to use Moses2.
>>
>> Can Moses2 run multiple threads for each daemon?
>>
> the daemon is the same, the server is different. Maybe write your own
> multithreaded daemon
>
Sorry - didn't answer your question correctly. No, Moses2 server behaves in
the same way as moses(1) - it runs 1 thread for each daemon

>
>> Thanks!
>>
>>
>>
>>
>> On Sunday, January 14, 2018, Hieu Hoang <hieuhoang@gmail.com> wrote:
>>
>>> 1. moses will be slow for the 1st few sentences. You can make it faster
>>> by running
>>> http://www.statmt.org/moses/?n=Moses.Optimize#ntoc3
>>> 2. The server runs only 1 thread for each daemon, but it can handle
>>> multiple daemons at once.
>>> 3. If you want to use more than 16 cores, it's better to use Moses2. See
>>> here
>>> http://www.statmt.org/moses/?n=Site.Moses2
>>>
>>> Hieu Hoang
>>> http://moses-smt.org/
>>>
>>>
>>> On 14 January 2018 at 06:20, Manli Zhu <zhum.osu@gmail.com> wrote:
>>>
>>>>
>>>> Hello,
>>>>
>>>> I built a FR-EN translator.
>>>>
>>>> binarised phrase-table.minphr : 13.9G
>>>> binarized recorerig-table.minlexr 6.9G
>>>> language model model.blm.en 17.1G
>>>>
>>>>
>>>> I am using daemon.pl running as moses server.
>>>>
>>>> It successfully open the port and loaded the translation model and
>>>> language mode. I can send sentence to this port and get correct
>>>> translation. My problem is the translation time is too long:
>>>>
>>>> Line 0: Initialize search took 0.001 seconds total
>>>> Line 0: Collecting options took 60.428 seconds at moses/Manager.cpp
>>>> Line 141
>>>> Line 0: Search took 0.641 seconds
>>>>
>>>> So, it took more than one minute to translate one normal sentence.
>>>>
>>>> Moses run on a server with 64 cores and each CPU speec is 1400MHz.
>>>>
>>>> During translation, it only used 100% of one CPU although I set the
>>>> threads: 12 in moses.ini file.
>>>>
>>>> Is there any way to speed up the translation?
>>>>
>>>> Thank you!
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20180114/361ea9ec/attachment-0001.html

------------------------------

Message: 2
Date: Sun, 14 Jan 2018 17:03:21 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] speed up translation for one sentence on
server
To: Manli Zhu <zhum.osu@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbjKg2QTn_XMgbGOVzO-Qf=Ky1wh09RyfRMGU8O+wi5Raw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hieu Hoang
http://moses-smt.org/


On 14 January 2018 at 15:11, Manli Zhu <zhum.osu@gmail.com> wrote:

> Thank you, Hieu! I still have questions for your suggestion, could you
> answer my question below?
>
> 1. moses will be slow for the 1stfew Sentences
>
> If client will submit one sentence at a time, will it always be slow? Is
> my lm and phrase table too big?
>
it will get faster.

>
> 2. The server runs only 1 thread for each daemon, but it can handle
> multiple daemons at once.
>
> By multiple daemons, do you mean run multiple Maemo.pl simultaneously?
> The multiple daemons share the same port number?
>
yes

>
> 3. If you want to use more than 16 cores, it's better to use Moses2.
>
> Can Moses2 run multiple threads for each daemon?
>
the daemon is the same, the server is different. Maybe write your own
multithreaded daemon

>
> Thanks!
>
>
>
>
> On Sunday, January 14, 2018, Hieu Hoang <hieuhoang@gmail.com> wrote:
>
>> 1. moses will be slow for the 1st few sentences. You can make it faster
>> by running
>> http://www.statmt.org/moses/?n=Moses.Optimize#ntoc3
>> 2. The server runs only 1 thread for each daemon, but it can handle
>> multiple daemons at once.
>> 3. If you want to use more than 16 cores, it's better to use Moses2. See
>> here
>> http://www.statmt.org/moses/?n=Site.Moses2
>>
>> Hieu Hoang
>> http://moses-smt.org/
>>
>>
>> On 14 January 2018 at 06:20, Manli Zhu <zhum.osu@gmail.com> wrote:
>>
>>>
>>> Hello,
>>>
>>> I built a FR-EN translator.
>>>
>>> binarised phrase-table.minphr : 13.9G
>>> binarized recorerig-table.minlexr 6.9G
>>> language model model.blm.en 17.1G
>>>
>>>
>>> I am using daemon.pl running as moses server.
>>>
>>> It successfully open the port and loaded the translation model and
>>> language mode. I can send sentence to this port and get correct
>>> translation. My problem is the translation time is too long:
>>>
>>> Line 0: Initialize search took 0.001 seconds total
>>> Line 0: Collecting options took 60.428 seconds at moses/Manager.cpp Line
>>> 141
>>> Line 0: Search took 0.641 seconds
>>>
>>> So, it took more than one minute to translate one normal sentence.
>>>
>>> Moses run on a server with 64 cores and each CPU speec is 1400MHz.
>>>
>>> During translation, it only used 100% of one CPU although I set the
>>> threads: 12 in moses.ini file.
>>>
>>> Is there any way to speed up the translation?
>>>
>>> Thank you!
>>>
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20180114/04aa3449/attachment-0001.html

------------------------------

Message: 3
Date: Mon, 15 Jan 2018 11:48:03 +0000 (UTC)
From: jean boyer <tunitrade@yahoo.fr>
Subject: [Moses-support] phrase table cleaning
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <1370285402.5539177.1516016884183@mail.yahoo.com>
Content-Type: text/plain; charset="utf-8"

Hi,
I want to clean my phrase table fr-en.
There are any solution to clean phrase table and keep only the best translation, like in the example below in green color ?

?

Thank you !
Eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20180115/33b781a7/attachment.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 1516016713146blob.jpg
Type: image/png
Size: 14092 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20180115/33b781a7/attachment.png

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 135, Issue 16
**********************************************

Tweet