Moses-support Digest, Vol 135, Issue 21

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: speed up translation for one sentence on server (Manli Zhu)
2. Re: phrase table cleaning (jean boyer)


----------------------------------------------------------------------

Message: 1
Date: Mon, 15 Jan 2018 20:18:07 -0500
From: Manli Zhu <zhum.osu@gmail.com>
Subject: Re: [Moses-support] speed up translation for one sentence on
server
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAAB_TANficykVbXS+WFV7mFxADtcZ5kP69B+CqLop+vdrz-EYQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Thank you, Hieu!

On Sun, Jan 14, 2018 at 12:03 PM, Hieu Hoang <hieuhoang@gmail.com> wrote:

>
>
> Hieu Hoang
> http://moses-smt.org/
>
>
> On 14 January 2018 at 15:11, Manli Zhu <zhum.osu@gmail.com> wrote:
>
>> Thank you, Hieu! I still have questions for your suggestion, could you
>> answer my question below?
>>
>> 1. moses will be slow for the 1stfew Sentences
>>
>> If client will submit one sentence at a time, will it always be slow?
>> Is my lm and phrase table too big?
>>
> it will get faster.
>
>>
>> 2. The server runs only 1 thread for each daemon, but it can handle
>> multiple daemons at once.
>>
>> By multiple daemons, do you mean run multiple Maemo.pl simultaneously?
>> The multiple daemons share the same port number?
>>
> yes
>
>>
>> 3. If you want to use more than 16 cores, it's better to use Moses2.
>>
>> Can Moses2 run multiple threads for each daemon?
>>
> the daemon is the same, the server is different. Maybe write your own
> multithreaded daemon
>
>>
>> Thanks!
>>
>>
>>
>>
>> On Sunday, January 14, 2018, Hieu Hoang <hieuhoang@gmail.com> wrote:
>>
>>> 1. moses will be slow for the 1st few sentences. You can make it faster
>>> by running
>>> http://www.statmt.org/moses/?n=Moses.Optimize#ntoc3
>>> 2. The server runs only 1 thread for each daemon, but it can handle
>>> multiple daemons at once.
>>> 3. If you want to use more than 16 cores, it's better to use Moses2. See
>>> here
>>> http://www.statmt.org/moses/?n=Site.Moses2
>>>
>>> Hieu Hoang
>>> http://moses-smt.org/
>>>
>>>
>>> On 14 January 2018 at 06:20, Manli Zhu <zhum.osu@gmail.com> wrote:
>>>
>>>>
>>>> Hello,
>>>>
>>>> I built a FR-EN translator.
>>>>
>>>> binarised phrase-table.minphr : 13.9G
>>>> binarized recorerig-table.minlexr 6.9G
>>>> language model model.blm.en 17.1G
>>>>
>>>>
>>>> I am using daemon.pl running as moses server.
>>>>
>>>> It successfully open the port and loaded the translation model and
>>>> language mode. I can send sentence to this port and get correct
>>>> translation. My problem is the translation time is too long:
>>>>
>>>> Line 0: Initialize search took 0.001 seconds total
>>>> Line 0: Collecting options took 60.428 seconds at moses/Manager.cpp
>>>> Line 141
>>>> Line 0: Search took 0.641 seconds
>>>>
>>>> So, it took more than one minute to translate one normal sentence.
>>>>
>>>> Moses run on a server with 64 cores and each CPU speec is 1400MHz.
>>>>
>>>> During translation, it only used 100% of one CPU although I set the
>>>> threads: 12 in moses.ini file.
>>>>
>>>> Is there any way to speed up the translation?
>>>>
>>>> Thank you!
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20180115/74091e6c/attachment-0001.html

------------------------------

Message: 2
Date: Tue, 16 Jan 2018 09:23:53 +0000 (UTC)
From: jean boyer <tunitrade@yahoo.fr>
Subject: Re: [Moses-support] phrase table cleaning
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <1777895160.6700769.1516094633938@mail.yahoo.com>
Content-Type: text/plain; charset="utf-8"

"I would like edit my message "https://www.mail-archive.com/moses-support@mit.edu/msg15788.html" because the accompanying image was not posted."
Hi,
I would like clean my phrase table fr-en.There are any solution to clean phrase table and keep only the bests translations.like in the example below i want keep only the lines that begin with an arrow?

-----------------------------------------------------------------------? sur ce sujet?? ??? ??? ?a?? ??? ??? ?1.21619e-05 2.15066e-09 0.111111 0.0117291?? ??? ??? ?1-0?? ??? ??? ?82224 9 1=>sur ce sujet?? ??? ??? ?on this subject?? ??? ??? ?0.5 0.0104788 0.111111 0.00456608?? ??? ??? ?0-0 1-1 2-2?? ??? ??? ?2 9 1
=>sur ce sujet?? ??? ??? ?on this topic?? ??? ??? ?0.2 0.0416959 0.111111 0.00307945?? ??? ??? ?0-0 1-1 2-2?? ??? ??? ?5 9 1
sur ce sujet?? ??? ??? ?rebellion against?? ??? ??? ?0.0909091 9.07596e-09 0.111111 1.53269e-08?? ??? ??? ?0-1?? ??? ??? ?11 9 1
-------------------------------------------------------------------------------------
Thank you !
Eric

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20180116/cf8104aa/attachment-0001.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 135, Issue 21
**********************************************

Tweet