Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Need help for parallelisation in mosesserver (Shubham Khandelwal)
2. Re: Newbie questions (Ri ki)
3. Re: Need help for parallelisation in mosesserver (Mathias M?ller)
----------------------------------------------------------------------
Message: 1
Date: Thu, 29 Dec 2016 00:53:41 +0530
From: Shubham Khandelwal <skhlnmiit@gmail.com>
Subject: [Moses-support] Need help for parallelisation in mosesserver
To: moses-support <moses-support@mit.edu>
Message-ID:
<CAHweNTuuQL83+Cw8QmVzgN=pZ64PZ7aCK-Uo6H1Qe4u1ZmM49Q@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hello,
As mosesserver accepts only one sentence at a time. So I am creating one
another component in front of mosesserver to handle tokenisation, casing
and splitting taking care of parallelisation.
Following is my procedure to do it, let me know whether am I heading
correctly or not to do this:
*---*
*So suppose, if I have 5 different sentences (as a paragraph) to translate
at once (fr-en). So I will be creating mosesserver on 5 different ports
firstly and pass those 5 different sentences after doing parallely
tokenisaton, casing and splitting on those different ports and then
concatenate the output after recasing and detokenisation parallely. *
*--*
Let me know whether this is correct or not ? If no, then please suggest me
better solution to do this.
Also, I have one more question in this that if a sentence is composed of
around 10 words. Then when I pass this sentence to translate as follows:
-> ~/mosesdecoder/bin/mosesserver -f moses.ini -threads 16 -b 0.000000001
then it takes around 10 seconds to translate. To make it fast, I can run
this on different ports but that is not a good idea I think, as splitting a
single sentence to multiple group of sentence and then translate them on
different ports separately, can give different meaning rather than
translate the whole single sentence at single port.
So basically, my doubt is how to make better splitting in such cases which
can take care of parallelisation aswell ?
--
Yours Sincerely,
Shubham Khandelwal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161228/110fbd96/attachment-0001.html
------------------------------
Message: 2
Date: Wed, 28 Dec 2016 15:38:11 -0500
From: Ri ki <mariasmo54@gmail.com>
Subject: Re: [Moses-support] Newbie questions
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAPWXzC9nretvrhh4mDQ8TfhwrGn1njtcbsCx2WtGv8PDXdY72w@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi Hieu Hoang,
Thanks for your response. Appreciate your candid feedback.
Apologies for not going over the website first. I will surely do that and
might have more questions as I start learning it.
Regarding MOSES on Android, I haven't done much market research(will
definitely do some research as you had suggested) but i am guessing mobile
devices being so ubiquitous these days, an Android app will definitely be
useful. Again that's just a hunch. I could be wrong.
And regarding the foot print, from what you mentioned, I deduce the
following:
Approximately 500MB storage space for models( for one translation model, so
if i use the reverse translation model say for example, spanish-english, I
am guessing the storage requirement would approximately double) and
approximately 1GB RAM. Am I correct in my understanding?
That's a lot of storage and memory requirements. However, the hardware for
mobile devices have been improving at a fast pace these days, so pretty
soon this should be an ok requirement in my opinion.
Apart from the storage and memory, the important metric is speed. Not sure
how fast it would be on a mobile device(of-course it depends on the CPU
speed, # of cores etc. on the target device)
I just checked the MOSES license.It says LGPL. Most of the open source
software I have been using is Apache licensing(like Tensorflow etc.). Not
sure how LGPL differs from Apache. I will have to do some research on this.
Again thanks a lot.
Regards
On Wed, Dec 28, 2016 at 4:49 AM, Hieu Hoang <hieuhoang@gmail.com> wrote:
> If you are new to MT and Moses, please make sure you have read the website
> and done the tutorials before asking too many questions:
> http://www.statmt.org/moses/
>
> Hieu Hoang
> http://moses-smt.org/ <http://www.hoang.co.uk/hieu>
>
> On 27 December 2016 at 20:23, Ri ki <mariasmo54@gmail.com> wrote:
>
>> I forgot to add one other question:
>>
>> 6) What type of license does MOSES have? Apache? BSD? MIT? GPL?
>>
> See website
>
>>
>> Thanks again
>>
>> On Tue, Dec 27, 2016 at 2:36 PM, Ri ki <mariasmo54@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I am new to Machine translation. I was doing some googling and stumbled
>>> upon Moses.
>>>
>>> I have the following questions:
>>>
>>> 1) What is the footprint of Moses(for one bilingual translation, say
>>> english-spanish?(including all the necessary models and binaries)
>>>
>> Total model sizes + around 500MB-1GB RAM
>
>>
>>> 2) Can Moses be run on a mobile device standalone i.e without going to
>>> server at all? My aim is to create an Android app. Is it feasible at all to
>>> run Moses as a standalone application on mobile devices?
>>>
>> It's been tried a few times, eg
> https://github.com/moses-smt/mosesdecoder/tree/android-ui
>
>>
>>> 3) Has anyone tried it? If so can you please provide some input?
>>>
>> Ask yourself - so you put a decoder onto a phone. Then what? What value
> can you add? Who is interested? Who will pay for your time?
>
>>
>>> 4) If it is indeed possible to run Moses standalone on a Mobile
>>> device(probably by pruning down the models to be able to fit on mobile
>>> devices), what is the loss in accuracy when compared to the server version?
>>> BLEU scores would help.
>>>
>>> 5) Are there any other alternatives to MOSES that I can run on mobile
>>> devices?
>>>
>> Let us know if you find out
>
>>
>>>
>>> Any help is greatly appreciated.
>>>
>>> Thanks in advance
>>>
>>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161228/010f3d5b/attachment-0001.html
------------------------------
Message: 3
Date: Wed, 28 Dec 2016 22:51:10 +0100
From: Mathias M?ller <mathias.mueller@uzh.ch>
Subject: Re: [Moses-support] Need help for parallelisation in
mosesserver
To: Shubham Khandelwal <skhlnmiit@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAF2XUm2g35Lj42BUHUwHdzoOiWFa2TUVfasgB_-paTztoYZqxg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi Shubham
There is not necessarily a need to start several server processes running
on different ports. A Moses server can serve more than one client at the
same time, except when the "serial" option is used explicitly. For other
useful server options, see e.g. :
https://github.com/moses-smt/mosesdecoder/blob/master/moses/Parameter.cpp#L219
Parallelization has to kick in in another place, namely in your client code
that requests the translations. One solution is to use multithreading to
submit several requests at the same time and, of course, also do the pre-
and postprocessing in parallel.
You did not really explain to us how exactly you are requesting
translations from the server, though. Also, this part of your message:
*"... as splitting a single sentence to multiple group of sentence and then
translate them on different ports separately, can give different meaning
rather than translate the whole single sentence at single port."*
Is a bit confusing. Currently, decoding always happens sentence-by-sentence
and there is no document context. So, documents can safely be translated
sentence by sentence. If you mean splitting a single sentence into words
and send those to different Moses servers, then yes, that's a bad idea.
At any rate, your questions are very vague and general and it is difficult
to give specific advice.
Regards,
Mathias
?
Mathias M?ller
AND-2-20
Institute of Computational Linguistics
University of Zurich
Switzerland
+41 44 635 75 81
mathias.mueller@uzh.ch
On Wed, Dec 28, 2016 at 8:23 PM, Shubham Khandelwal <skhlnmiit@gmail.com>
wrote:
> Hello,
>
> As mosesserver accepts only one sentence at a time. So I am creating one
> another component in front of mosesserver to handle tokenisation, casing
> and splitting taking care of parallelisation.
>
> Following is my procedure to do it, let me know whether am I heading
> correctly or not to do this:
> *---*
> *So suppose, if I have 5 different sentences (as a paragraph) to translate
> at once (fr-en). So I will be creating mosesserver on 5 different ports
> firstly and pass those 5 different sentences after doing parallely
> tokenisaton, casing and splitting on those different ports and then
> concatenate the output after recasing and detokenisation parallely. *
> *--*
> Let me know whether this is correct or not ? If no, then please suggest me
> better solution to do this.
>
> Also, I have one more question in this that if a sentence is composed of
> around 10 words. Then when I pass this sentence to translate as follows:
> -> ~/mosesdecoder/bin/mosesserver -f moses.ini -threads 16 -b 0.000000001
>
> then it takes around 10 seconds to translate. To make it fast, I can run
> this on different ports but that is not a good idea I think, as splitting a
> single sentence to multiple group of sentence and then translate them on
> different ports separately, can give different meaning rather than
> translate the whole single sentence at single port.
> So basically, my doubt is how to make better splitting in such cases which
> can take care of parallelisation aswell ?
>
> --
> Yours Sincerely,
>
> Shubham Khandelwal
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161228/7d7cdf16/attachment.html
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 122, Issue 40
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 122, Issue 40"
Post a Comment