Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Tutorial correction - Experiment Management System
(Vincent Nguyen)
2. Errors in tuning Moses (Sudip Mahajan)
3. Re: Getting started - Best practices (Hieu Hoang)
4. Re: Performance issues using Moses Server with Moses 3 (Oren)
5. Re: Performance issues using Moses Server with Moses 3
(Barry Haddow)
----------------------------------------------------------------------
Message: 1
Date: Mon, 20 Jul 2015 18:29:36 +0200
From: Vincent Nguyen <vnguyen@neuf.fr>
Subject: [Moses-support] Tutorial correction - Experiment Management
System
To: moses-support <moses-support@mit.edu>
Message-ID: <55AD21F0.4030708@neuf.fr>
Content-Type: text/plain; charset=windows-1252; format=flowed
Hi,
In case you got a chance to make the correction.
the github scripts are asking for the WMT12 files, the tutorial says
WMT10 files
need to replace
wget http://www.statmt.org/wmt12/dev.tgz
wget http://www.statmt.org/wmt12/training-parallel.tgz
plus I think it would be good to specify that you need to edit the
config file for the -mgiza parameter ....
My experiment is running .... crossed finger.
------------------------------
Message: 2
Date: Tue, 21 Jul 2015 10:42:12 +0530
From: Sudip Mahajan <sudipcmahajan@gmail.com>
Subject: [Moses-support] Errors in tuning Moses
To: Moses-support@mit.edu
Message-ID:
<CAEth75tZT2=A+1eoYb-j1FCkDX4FgVpB=ZjO-tfaJfj2Hx01mQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hello,
I am trying to train the 'moses.ini' file with 'mert-moses.pl' script.
The exact command I am running is
"
nohup nice $myPATH/mosesdecoder/scripts/training/mert-moses.pl
--decoder-flags=threads 4 \
$myPATH/working/tuningcorpus/prepared-corpus/news-test2008.clean.fr
$myPATH/working/tuningcorpus/prepared-corpus/news-test2008.clean.en \
$myPATH/working/train/model/moses.ini --mertdir
$myPATH/working/train/model &> mert.out &
"
However I am getting the following error in 'mert.out' file:
"
Using SCRIPTS_ROOTDIR: /media/sudip/linuxdata1/mosesdecoder/scripts
Not executable: /media/sudip/linuxdata1/working/train/model/extractor at
/media/sudip/linuxdata1/mosesdecoder/scripts/training/mert-moses.pl line
362.
"
I don't quite understand this error, please help.
Thanks in advance
-----
Regards,
Sudip C. Mahajan | Ph.D. Student,Dept. of Technology
University of Pune
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150721/97d1a5ff/attachment-0001.htm
------------------------------
Message: 3
Date: Tue, 21 Jul 2015 09:21:38 +0400
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Getting started - Best practices
To: moses-support@mit.edu
Message-ID: <55ADD6E2.8050704@gmail.com>
Content-Type: text/plain; charset=windows-1252; format=flowed
On 19/07/2015 23:06, Vincent Nguyen wrote:
> I finally went through the all Baseline process with the KenLM model.
>
> results are mitigated, so from here what would be the best practices ?
>
> 1) I saw online a bunch of corpus available from the European Union
> should this be used to train the translation system AND the langue
> model or just one of the 2 ?
you cab use the data for creating both the language model and the
translation model. The only thing you have to make sure is that your
training data is not part of the tuning or test data
>
> 2) Is there a benchmark between the different model (Kenlm, Irstlm, ...)
> ie is there a big difference in the observed results ?
> is it worth trying several ones ?
Try it yourself and tell us the results.
>
> 3) I read an article mentioning that the results after the tuning were
> not as good as before ...
> does this make any sense ?
If you report BLEU score without tuning first, you will be crucified,
see this thread:
https://www.mail-archive.com/moses-support@mit.edu/msg12593.html
You MUST tune. Tuning can sometime to difficult. See this post on how to
pick a good tuning set:
https://www.mail-archive.com/moses-support@mit.edu/msg12594.html
>
> Thanks.
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
--
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu
------------------------------
Message: 4
Date: Tue, 21 Jul 2015 11:30:19 +0300
From: Oren <mooshified@gmail.com>
Subject: Re: [Moses-support] Performance issues using Moses Server
with Moses 3
To: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAHWEG_WknzGuLuDYdQGiTRLupZqvPapYbLjz8YTZXZywWwoO4A@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi Barry,
Thanks for the quick response.
I added the switch "-threads 18" to the command to raise moses server. The
slowness issue persists but in a different form. Most requests return right
away, even under heavy load, but some requests (about 5%) take far longer -
about 15-20 seconds.
Perhaps there are other relevant switches?
Thanks again.
On Monday, July 20, 2015, Barry Haddow <bhaddow@staffmail.ed.ac.uk> wrote:
> Hi Oren
>
> The threading model is different. In v1, the server created a new thread
> for every request, v3 uses a thread pool. Try increasing the number of
> threads.
>
> Also, make sure you use the compact phrase table and KenLM as they are
> normally faster, and pre-pruning your phrase table can help,
>
> cheers - Barry
>
> On 20/07/15 12:01, Oren wrote:
>
> Hi all,
>
> We are in the process of migrating from Moses 1 to Moses 3. We have
> noticed a significant slowdown when sending many requests at once to Moses
> Server. The first request will actually finish about 25% faster that a
> single request using Moses 1, but as more requests accumulate there is a
> marked slowdown, until requests take 5 times longer or more.
>
> Is this a known issue? Is it specific to Moses Server? What can we do
> about it?
>
> Thanks!
>
> Oren.
>
>
> _______________________________________________
> Moses-support mailing listMoses-support@mit.edu <javascript:_e(%7B%7D,'cvml','Moses-support@mit.edu');>http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150721/b649c6b9/attachment-0001.htm
------------------------------
Message: 5
Date: Tue, 21 Jul 2015 12:33:56 +0100
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] Performance issues using Moses Server
with Moses 3
To: Oren <mooshified@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <55AE2E24.9000104@staffmail.ed.ac.uk>
Content-Type: text/plain; charset="windows-1252"
Hi Oren
Does your host have 18 threads available? It could also be that xmlrpc-c
is limiting the number of connections - this can now be configured:
https://github.com/moses-smt/mosesdecoder/commit/b3baade7f022edbcea2969679a40616683f63523
Slowdowns in Moses are often caused by disk access bottlenecks. You can
use --minphr-memory and --minlexr-memory to make sure that the phrase
and reordering tables are loaded in to memory, rather than being access
on-demand. Make sure your host has enough RAM and is not swapping. As I
mentioned before there are various ways to make your models smaller
(http://www.statmt.org/moses/?n=Advanced.RuleTables), which can make a
big difference to speed depending on your setup.
cheers - Barry
On 21/07/15 09:30, Oren wrote:
> Hi Barry,
>
> Thanks for the quick response.
>
> I added the switch "-threads 18" to the command to raise moses server.
> The slowness issue persists but in a different form. Most requests
> return right away, even under heavy load, but some requests (about 5%)
> take far longer - about 15-20seconds.
>
> Perhaps there are other relevant switches?
>
> Thanks again.
>
> On Monday, July 20, 2015, Barry Haddow <bhaddow@staffmail.ed.ac.uk
> <mailto:bhaddow@staffmail.ed.ac.uk>> wrote:
>
> Hi Oren
>
> The threading model is different. In v1, the server created a new
> thread for every request, v3 uses a thread pool. Try increasing
> the number of threads.
>
> Also, make sure you use the compact phrase table and KenLM as they
> are normally faster, and pre-pruning your phrase table can help,
>
> cheers - Barry
>
> On 20/07/15 12:01, Oren wrote:
>> Hi all,
>>
>> We are in the process of migrating from Moses 1 to Moses 3. We
>> have noticed a significant slowdown when sending many requests at
>> once to Moses Server. The first request will actually finish
>> about 25% faster that a single request using Moses 1, but as more
>> requests accumulate there is a marked slowdown, until requests
>> take 5 times longer or more.
>>
>> Is this a known issue? Is it specific to Moses Server? What can
>> we do about it?
>>
>> Thanks!
>>
>> Oren.
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu <javascript:_e(%7B%7D,'cvml','Moses-support@mit.edu');>
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150721/df03a238/attachment.htm
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150721/df03a238/attachment.bat
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 105, Issue 40
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 105, Issue 40"
Post a Comment