Moses-support Digest, Vol 106, Issue 58

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Decoding Speed perfomance - suggestion and question
(Philipp Koehn)
2. error buggy config line (Stefy D.)


----------------------------------------------------------------------

Message: 1
Date: Mon, 31 Aug 2015 10:54:02 -0400
From: Philipp Koehn <phi@jhu.edu>
Subject: Re: [Moses-support] Decoding Speed perfomance - suggestion
and question
To: Vincent Nguyen <vnguyen@neuf.fr>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAAFADDD5kMaveh3umr=ei6cMD82JqDXXOuVgd+0Xe0FGif7_gw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,

the filter script filters an existing phrase table. With the EMS settings,
it would build another phrase table.

Don't worry about the reordering table. It will have excess entries, but
they will not be used.
If you really care, you can used the script
scripts/training/remove-orphan-phrase-pairs-from-reordering-table.perl

-phi

On Mon, Aug 31, 2015 at 10:50 AM, Vincent Nguyen <vnguyen@neuf.fr> wrote:

>
> thanks, will try and post results.
> just to be clear:
> I can re-use the previous extract file
> I have to rebuild the phrase-table with new min score (ie no way to just
> filter the previous one ?)
> do I have to rebuild the reordering table too ?
>
> Vincent
>
>
> Le 31/08/2015 16:44, Philipp Koehn a ?crit :
>
> hI,
>
> 0.0001 should have no impact on translation quality,
> 0.001 will have some impact
> 0.01 is probably a bit too drastic.
>
> But that's the range you should explore.
>
> -phi
>
> On Mon, Aug 31, 2015 at 10:33 AM, Vincent Nguyen <vnguyen@neuf.fr> wrote:
>
>> is there any benchmark on what value / what impact ?
>> what should I start with as a test 0.001 ?
>>
>> the standard value 0.0001 seems really really low to me ....
>> maybe I am not getting what this probability exactly refers to.
>>
>>
>>
>> where FIELDn is the position of the score (typically 2 for the direct
>> phrase probability p(e|f), or 0 for the indirect phrase probability p(f|e))
>> and THRESHOLD the maximum probability allowed. A good setting is 2:0.0001,
>> which removes all rules, where the direct phrase translation probability is
>> below 0.0001.
>>
>>
>>
>> Le 31/08/2015 16:14, Philipp Koehn a ?crit :
>>
>> Hi,
>>
>> I would suspect that with beam sizes <500 the bulk of the time is
>> spent on translation option collection, not decoding. You could speed
>> that up with tighter threshold pruning of the phrase table.
>>
>> See the script scripts/training/threshold-filter.perl or the setting
>> score-settings = "--MinScore 2:0.0001"
>> in EMS.
>>
>> -phi
>>
>> On Mon, Aug 31, 2015 at 3:03 AM, Vincent Nguyen <vnguyen@neuf.fr> wrote:
>>
>>> Hi,
>>>
>>> Here are some results with several values with cube pruning pop limit :
>>>
>>> (pop limit / decoding time for 3000 sentences / BLEU score)
>>>
>>> 5000 - 15m45 - 29.59
>>> 1000 - 4m27 - 29.59
>>> 500 - 3m35 - 29.59
>>> 200 - 3m15 - 29.51
>>> 100 - 3m00 - 29.40
>>>
>>> Therefore I took 400 - 3m19 - 29.58
>>>
>>> If I am not mistaken the default value for Moses is 1000 [read in the
>>> doc] but in the EMS
>>> it is 5000 right now .... which makes the experience so long .....
>>> I suggest to change the EMS default value.
>>>
>>> Is there a way to also use a cube pruning limit in the decoder at Tuning
>>> time ?
>>>
>>> Now with this optimized setting I get a ration of 15 segments per second
>>> in average.
>>> What is the reason for online tools like Google / Bing to be much much
>>> faster.
>>> it's not a machine issue, is it ?
>>>
>>>
>>> Cheers
>>> Vincent
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150831/a1b0c3fa/attachment-0001.html

------------------------------

Message: 2
Date: Mon, 31 Aug 2015 15:20:36 +0000 (UTC)
From: "Stefy D." <tsuki_stefy@yahoo.com>
Subject: [Moses-support] error buggy config line
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<1380854981.3483421.1441034436174.JavaMail.yahoo@mail.yahoo.com>
Content-Type: text/plain; charset="utf-8"

Dear all,

I am using EMS and I want to skip the preprocessing steps for the evaluation set. I use one reference for translation, but I get an error.

Here is a part of my config file:

multiref = no
multi-bleu = "$moses-script-dir/generic/multi-bleu.perl -lc"
multi-bleu-c = $moses-script-dir/generic/multi-bleu.perl

[EVALUATION:test]

### input data
?truecased-input = $toy-data/khresmoi-summary-test.tc.$input-extension

### reference data
?truecased-reference = $toy-data/khresmoi-summary-test.tc.$output-extension

I receive this error when I try to run experiment.perl:

LOAD CONFIG...
BUGGY CONFIG LINE (640):? truecased-input = $toy-data/khresmoi-summary-test.tc.$input-extension
BUGGY CONFIG LINE (648):? truecased-reference = $toy-data/khresmoi-summary-test.tc.$output-extension
2 ERRORs IN CONFIG FILE

Can someone please help me solve this? Thank you very much!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150831/50ce2daf/attachment-0001.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 106, Issue 58
**********************************************

Tweet