Moses-support Digest, Vol 105, Issue 44

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: BLEU result on baseline EMS experiment (Vincent Nguyen)
2. Re: Performance issues using Moses Server with Moses 3
(Barry Haddow)


----------------------------------------------------------------------

Message: 1
Date: Wed, 22 Jul 2015 15:23:54 +0200
From: Vincent Nguyen <vnguyen@neuf.fr>
Subject: Re: [Moses-support] BLEU result on baseline EMS experiment
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <55AF996A.30705@neuf.fr>
Content-Type: text/plain; charset="utf-8"


shouldn't the Belu score be more in the 50's for a test set close to the
corpus ?
I meant by "real text" that I have a corpus of translations (fr to eng)
made by translators, typically the kind of text I would like to test
with Moses.

so my question is : should I use these texts to 1) train or 2) tune my
model ?

also in terms of language model, can we make it evolve with new texts to
make it better in time ?





Le 22/07/2015 14:28, Hieu Hoang a ?crit :
> it looks ok, your bleu score is 22.68 for this test set.
>
> I don't know what you mean by real text.
>
> Hieu Hoang
> Researcher
> New York University, Abu Dhabi
> http://www.hoang.co.uk/hieu
>
> On 21 July 2015 at 23:45, Vincent Nguyen <vnguyen@neuf.fr
> <mailto:vnguyen@neuf.fr>> wrote:
>
> here is what I got
>
> make sense ?
>
>
> MT evaluation scorer began on 2015 Jul 20 at 23:27:39
> command line:
> /home/moses/mosesdecoder/scripts/generic/mteval-v13a.pl
> <http://mteval-v13a.pl> -c
> -c -s /home/moses/working/data/dev/newstest2011-src.fr.sgm -r
> /home/moses/working/data/dev/newstest2011-ref.en.sgm -t
> /home/moses/working/evaluation/newstest2011.detokenized.sgm.3
> Evaluation of any-to-en translation using:
> src set "newstest2011" (110 docs, 3003 segs)
> ref set "newstest2011" (1 refs)
> tst set "newstest2011" (1 systems)
>
> length ratio: 0.994844739625875 (74296/74681), penalty (log):
> -0.00518197480348868
> NIST score = 6.8964 BLEU score = 0.2268 for system "Edinburgh"
>
> #
> ------------------------------------------------------------------------
>
> Individual N-gram scoring
> 1-gram 2-gram 3-gram 4-gram 5-gram 6-gram 7-gram
> 8-gram 9-gram
> ------ ------ ------ ------ ------ ------ ------
> ------ ------
> NIST: 5.2752 1.3399 0.2499 0.0273 0.0041 0.0005 0.0000
> 0.0000 0.0000 "Edinburgh"
>
> BLEU: 0.5883 0.2887 0.1636 0.0972 0.0589 0.0364 0.0230
> 0.0146 0.0093 "Edinburgh"
>
> #
> ------------------------------------------------------------------------
> Cumulative N-gram scoring
> 1-gram 2-gram 3-gram 4-gram 5-gram 6-gram 7-gram
> 8-gram 9-gram
> ------ ------ ------ ------ ------ ------ ------
> ------ ------
> NIST: 5.2752 6.6151 6.8650 6.8923 6.8964 6.8969 6.8970
> 6.8970 6.8970 "Edinburgh"
>
> BLEU: 0.5853 0.4100 0.3013 0.2268 0.1730 0.1333 0.1037
> 0.0811 0.0637 "Edinburgh"
> MT evaluation scorer ended on 2015 Jul 20 at 23:28:01
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150722/e954f920/attachment-0001.htm

------------------------------

Message: 2
Date: Wed, 22 Jul 2015 15:11:21 +0100
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] Performance issues using Moses Server
with Moses 3
To: Oren <mooshified@gmail.com>, Hieu Hoang <hieuhoang@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <55AFA489.4010204@staffmail.ed.ac.uk>
Content-Type: text/plain; charset="utf-8"

Hi Oren

I'm not aware of any threading problems with PhraseDictionaryMemory, but
not many people use it since it takes up too much memory. Moses command
line runs multi-threaded using a thread pool.

Is your language model on a local file system? Running it over nfs can
be bad. What about your reordering model? Are you using the compact or
the memory version?

cheers - Barry

On 22/07/15 15:09, Oren wrote:
> We have no swapping issues. I was asking if the use of an in-memory
> translation model might cause multithreading problems.
>
>
> I'm not sure how to replicate the problem on cmd moses, since it's
> purely a multithreading issue. Ca you run cmd moses multithreaded?
>
> I can't attach the complete moses.ini because it's on a separate
> network... But I copied below the stuff that looked relevant. I also
> tried to change the [threads] setting to 18, with no apparent effect.
>
> [input-factors]
> 0
>
> [mapping]
> 0 T 0
>
> [distortion-limit]
> 6
>
> [feature]
> UnknownWordPenalty
> WordPenalty
> PhrasePenalty
> PhraseDictionaryMemory name=TranslationModel0 num-features=4
> path=<path>/phrase-table.gz input-factor=0 output-factor=0
> LexicalReordering <parameters>
> Distortion
> KENLM lazyken=0 name=LM0 path=<path> order=5
>
> [threads]
> 6
>
> [weight]
>
> <weight parameters>
>
> On Tuesday, July 21, 2015, Hieu Hoang <hieuhoang@gmail.com
> <javascript:_e(%7B%7D,'cvml','hieuhoang@gmail.com');>> wrote:
>
> is it possible you can make your moses.ini file available for us
> to see?
>
> do you know if the same problem occurs if you use the command line
> moses, rather than mosesserver?
>
>
> Hieu Hoang
> Researcher
> New York University, Abu Dhabi
> http://www.hoang.co.uk/hieu
>
> On 21 July 2015 at 18:07, Barry Haddow
> <bhaddow@staffmail.ed.ac.uk> wrote:
>
>
> On 21/07/15 14:51, Oren wrote:
>> I am using the in-memory mode, using about 50GB of RAM. (No
>> swap issues as far as I can tell.) Could that cause issues?
>
> Yes, swapping would definitely cause issues - was that your
> question?
>
>
>>
>> I looked at the commit you linked to, but it doesn't seem to
>> be something configurable beyond the -threads switch. Am
>> I missing something?
>
> The commit enables you to set the maximum number of
> connections to be the same as the maximum number of threads.
>
>>
>> On Tuesday, July 21, 2015, Barry Haddow
>> <bhaddow@staffmail.ed.ac.uk> wrote:
>>
>> Hi Oren
>>
>> Does your host have 18 threads available? It could also
>> be that xmlrpc-c is limiting the number of connections -
>> this can now be configured:
>> https://github.com/moses-smt/mosesdecoder/commit/b3baade7f022edbcea2969679a40616683f63523
>>
>> Slowdowns in Moses are often caused by disk access
>> bottlenecks. You can use --minphr-memory and
>> --minlexr-memory to make sure that the phrase and
>> reordering tables are loaded in to memory, rather than
>> being access on-demand. Make sure your host has enough
>> RAM and is not swapping. As I mentioned before there are
>> various ways to make your models smaller
>> (http://www.statmt.org/moses/?n=Advanced.RuleTables),
>> which can make a big difference to speed depending on
>> your setup.
>>
>> cheers - Barry
>>
>> On 21/07/15 09:30, Oren wrote:
>>> Hi Barry,
>>>
>>> Thanks for the quick response.
>>>
>>> I added the switch "-threads 18" to the command to raise
>>> moses server. The slowness issue persists but in a
>>> different form. Most requests return right away, even
>>> under heavy load, but some requests (about 5%) take far
>>> longer - about 15-20seconds.
>>>
>>> Perhaps there are other relevant switches?
>>>
>>> Thanks again.
>>>
>>> On Monday, July 20, 2015, Barry Haddow
>>> <bhaddow@staffmail.ed.ac.uk> wrote:
>>>
>>> Hi Oren
>>>
>>> The threading model is different. In v1, the server
>>> created a new thread for every request, v3 uses a
>>> thread pool. Try increasing the number of threads.
>>>
>>> Also, make sure you use the compact phrase table and
>>> KenLM as they are normally faster, and pre-pruning
>>> your phrase table can help,
>>>
>>> cheers - Barry
>>>
>>> On 20/07/15 12:01, Oren wrote:
>>>> Hi all,
>>>>
>>>> We are in the process of migrating from Moses 1 to
>>>> Moses 3. We have noticed a significant slowdown
>>>> when sending many requests at once to Moses Server.
>>>> The first request will actually finish about 25%
>>>> faster that a single request using Moses 1, but as
>>>> more requests accumulate there is a marked
>>>> slowdown, until requests take 5 times longer or more.
>>>>
>>>> Is this a known issue? Is it specific to Moses
>>>> Server? What can we do about it?
>>>>
>>>> Thanks!
>>>>
>>>> Oren.
>>>>
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>
>
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150722/d4aed3d2/attachment.htm
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150722/d4aed3d2/attachment.bat

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 105, Issue 44
**********************************************

Tweet