Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: Tuning with no language model (Hieu Hoang)
2. Re: Tuning with no language model (Read, James C)
3. Re: Tuning with no language model (Hieu Hoang)
----------------------------------------------------------------------
Message: 1
Date: Wed, 13 Jan 2016 13:56:04 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Tuning with no language model
To: "Read, James C" <jcread@essex.ac.uk>, Moses Support
<moses-support@mit.edu>
Message-ID: <56965774.1080405@gmail.com>
Content-Type: text/plain; charset="windows-1252"
did you delete the line
LM0= 0.5
too?
On 13/01/16 13:53, Read, James C wrote:
>
> The following command works fine for when a language model is
> specified. Deleting the line KENLM lazyken=0 name=LM0 factor=0
> path=$PWD/blm.en order=3
> causes the script to fail with error:
>
>
> ERROR: Failed to run '/pathtomoses/mosesdecoder/bin/moses -config
> /pathtodata/binarised/moses.ini -show-weights'. at
> /pathtomoses/mosesdecoder/scripts/training/mert-moses.pl line 1744.
>
>
> #########################
> ### MOSES CONFIG FILE ###
> #########################
>
> # input factors
> [input-factors]
> 0
>
> # mapping steps
> [mapping]
> 0 T 0
>
> [distortion-limit]
> 6
>
> # feature functions
> [feature]
> UnknownWordPenalty
> WordPenalty
> PhrasePenalty
> PhraseDictionaryCompact name=TranslationModel0 num-features=4
> path=/pathtodata/binarised/phrase-table input-factor=0 output-factor=0
> LexicalReordering name=LexicalReordering0 num-features=6
> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
> path=/pathtodata/binarised/reordering-table
> Distortion
> KENLM lazyken=0 name=LM0 factor=0 path=/pathtodata/blm.en order=3
>
> # dense weights for feature functions
> [weight]
> # The default weights are NOT optimized for translation quality. You
> MUST tune the weights.
> # Documentation for tuning is here:
> http://www.statmt.org/moses/?n=FactoredTraining.Tuning
> UnknownWordPenalty0= 1
> WordPenalty0= -1
> PhrasePenalty0= 0.2
> TranslationModel0= 0.2 0.2 0.2 0.2
> LexicalReordering0= 0.3 0.3 0.3 0.3 0.3 0.3
> Distortion0= 0.3
> LM0= 0.5
>
> /pathtomoses/mosesdecoder/scripts/training/mert-moses.pl
> -no-filter-phrase-table true.fr true.en
> /pathtomoses/mosesdecoder/bin/moses binarised/moses.ini --mertdir
> /pathtomoses/mosesdecoder/bin/
--
Hieu Hoang
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160113/4936c81d/attachment-0001.html
------------------------------
Message: 2
Date: Wed, 13 Jan 2016 13:57:25 +0000
From: "Read, James C" <jcread@essex.ac.uk>
Subject: Re: [Moses-support] Tuning with no language model
To: Hieu Hoang <hieuhoang@gmail.com>, Moses Support
<moses-support@mit.edu>
Message-ID:
<HE1PR06MB1481CF85639151E4CF8BC1E385CB0@HE1PR06MB1481.eurprd06.prod.outlook.com>
Content-Type: text/plain; charset="iso-8859-1"
no
________________________________
From: Hieu Hoang <hieuhoang@gmail.com>
Sent: Wednesday, January 13, 2016 1:56 PM
To: Read, James C; Moses Support
Subject: Re: [Moses-support] Tuning with no language model
did you delete the line
LM0= 0.5
too?
On 13/01/16 13:53, Read, James C wrote:
The following command works fine for when a language model is specified. Deleting the line KENLM lazyken=0 name=LM0 factor=0 path=$PWD/blm.en order=3
causes the script to fail with error:
ERROR: Failed to run '/pathtomoses/mosesdecoder/bin/moses -config /pathtodata/binarised/moses.ini -show-weights'. at /pathtomoses/mosesdecoder/scripts/training/mert-moses.pl line 1744.
#########################
### MOSES CONFIG FILE ###
#########################
# input factors
[input-factors]
0
# mapping steps
[mapping]
0 T 0
[distortion-limit]
6
# feature functions
[feature]
UnknownWordPenalty
WordPenalty
PhrasePenalty
PhraseDictionaryCompact name=TranslationModel0 num-features=4 path=/pathtodata/binarised/phrase-table input-factor=0 output-factor=0
LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 path=/pathtodata/binarised/reordering-table
Distortion
KENLM lazyken=0 name=LM0 factor=0 path=/pathtodata/blm.en order=3
# dense weights for feature functions
[weight]
# The default weights are NOT optimized for translation quality. You MUST tune the weights.
# Documentation for tuning is here: http://www.statmt.org/moses/?n=FactoredTraining.Tuning
[http://www.statmt.org/moses/img/coin-tiny.png]<http://www.statmt.org/moses/?n=FactoredTraining.Tuning>
Moses - FactoredTraining/Tuning<http://www.statmt.org/moses/?n=FactoredTraining.Tuning>
www.statmt.org
Tuning. Overview. During decoding, Moses scores translation hypotheses using a linear model. In the traditional approach, the features of the model are the ...
UnknownWordPenalty0= 1
WordPenalty0= -1
PhrasePenalty0= 0.2
TranslationModel0= 0.2 0.2 0.2 0.2
LexicalReordering0= 0.3 0.3 0.3 0.3 0.3 0.3
Distortion0= 0.3
LM0= 0.5
/pathtomoses/mosesdecoder/scripts/training/mert-moses.pl -no-filter-phrase-table true.fr true.en /pathtomoses/mosesdecoder/bin/moses binarised/moses.ini --mertdir /pathtomoses/mosesdecoder/bin/
--
Hieu Hoang
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160113/5f969c6e/attachment-0001.html
------------------------------
Message: 3
Date: Wed, 13 Jan 2016 13:58:59 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Tuning with no language model
To: "Read, James C" <jcread@essex.ac.uk>, Moses Support
<moses-support@mit.edu>
Message-ID: <56965823.30601@gmail.com>
Content-Type: text/plain; charset="windows-1252"
there lies your problem. There's a weight for a non-existant feature
function
On 13/01/16 13:57, Read, James C wrote:
>
> no
>
>
>
> ------------------------------------------------------------------------
> *From:* Hieu Hoang <hieuhoang@gmail.com>
> *Sent:* Wednesday, January 13, 2016 1:56 PM
> *To:* Read, James C; Moses Support
> *Subject:* Re: [Moses-support] Tuning with no language model
> did you delete the line
> LM0= 0.5
> too?
>
> On 13/01/16 13:53, Read, James C wrote:
>>
>> The following command works fine for when a language model is
>> specified. Deleting the line KENLM lazyken=0 name=LM0 factor=0
>> path=$PWD/blm.en order=3
>> causes the script to fail with error:
>>
>>
>> ERROR: Failed to run '/pathtomoses/mosesdecoder/bin/moses -config
>> /pathtodata/binarised/moses.ini -show-weights'. at
>> /pathtomoses/mosesdecoder/scripts/training/mert-moses.pl line 1744.
>>
>>
>> #########################
>> ### MOSES CONFIG FILE ###
>> #########################
>>
>> # input factors
>> [input-factors]
>> 0
>>
>> # mapping steps
>> [mapping]
>> 0 T 0
>>
>> [distortion-limit]
>> 6
>>
>> # feature functions
>> [feature]
>> UnknownWordPenalty
>> WordPenalty
>> PhrasePenalty
>> PhraseDictionaryCompact name=TranslationModel0 num-features=4
>> path=/pathtodata/binarised/phrase-table input-factor=0 output-factor=0
>> LexicalReordering name=LexicalReordering0 num-features=6
>> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
>> path=/pathtodata/binarised/reordering-table
>> Distortion
>> KENLM lazyken=0 name=LM0 factor=0 path=/pathtodata/blm.en order=3
>>
>> # dense weights for feature functions
>> [weight]
>> # The default weights are NOT optimized for translation quality. You
>> MUST tune the weights.
>> # Documentation for tuning is here:
>> http://www.statmt.org/moses/?n=FactoredTraining.Tuning
>> <http://www.statmt.org/moses/?n=FactoredTraining.Tuning>
>>
>> Moses - FactoredTraining/Tuning
>> <http://www.statmt.org/moses/?n=FactoredTraining.Tuning>
>> www.statmt.org
>> Tuning. Overview. During decoding, Moses scores translation
>> hypotheses using a linear model. In the traditional approach, the
>> features of the model are the ...
>>
>>
>> UnknownWordPenalty0= 1
>> WordPenalty0= -1
>> PhrasePenalty0= 0.2
>> TranslationModel0= 0.2 0.2 0.2 0.2
>> LexicalReordering0= 0.3 0.3 0.3 0.3 0.3 0.3
>> Distortion0= 0.3
>> LM0= 0.5
>>
>> /pathtomoses/mosesdecoder/scripts/training/mert-moses.pl
>> -no-filter-phrase-table true.fr true.en
>> /pathtomoses/mosesdecoder/bin/moses binarised/moses.ini --mertdir
>> /pathtomoses/mosesdecoder/bin/
>
> --
> Hieu Hoang
> http://www.hoang.co.uk/hieu
--
Hieu Hoang
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160113/9cd500f0/attachment.html
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 111, Issue 29
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 111, Issue 29"
Post a Comment