Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. facored LMs in Moses (joerg)
2. Re: Learning the moses.ini v2 format (Hieu Hoang)
3. Re: Using factor in the hierarchical model (Kenneth Heafield)
4. Re: Learning the moses.ini v2 format (Tom Hoar)
----------------------------------------------------------------------
Message: 1
Date: Mon, 2 Feb 2015 11:49:19 +0100
From: joerg <tiedeman@gmail.com>
Subject: [Moses-support] facored LMs in Moses
To: moses-support <moses-support@mit.edu>
Message-ID: <5ADDF205-83AE-4493-8842-21775FBD192F@gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
I would like to have an LM over several factors (in a factored phrase-based model). What would be the proper way to train and integrate such a model in Moses? Is it straightforward to use the factored models in SRILM? Can they be binarized in the same way as other models?
Could I otherwise also train a standard LM (say KenLM) over several factors and the decoder can deal with it? If that is possible, how do I have to format my input for KenLM parameter estimation? Simply concatenating the factors I need with '|'?
Thanks for helping!
J?rg
**********************************************************************************
J?rg Tiedemann http://stp.lingfil.uu.se/~joerg/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150202/6d450b4f/attachment-0001.htm
------------------------------
Message: 2
Date: Mon, 02 Feb 2015 13:56:32 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Learning the moses.ini v2 format
To: Tom Hoar <tahoar@precisiontranslationtools.com>,
moses-support@mit.edu
Message-ID: <54CF8210.9010200@gmail.com>
Content-Type: text/plain; charset="windows-1252"
Hi Tom
On 02/02/15 01:33, Tom Hoar wrote:
> Much of the v2 moses.ini looks self-explanatory, but I'd like to
> confirm my understanding.
>
> The website (http://www.statmt.org/moses/?n=Moses.FeatureFunctions)
> defines three feature/functions without arguments. In the moses.ini
> files made by train-model.perl's step 9, there also appears to be a
> 4th that requires no argument. Can someone confirm this is the case?
> Are there others that could appear without arguments?
Yes, PhrasePenalty is a standard feature function (FF), it doesn't need
any arguments. It used to be the constant 2.718 in the last score in the
phrase-table. But i thought that was silly so move it to it's own
feature function.
>
> [feature]
> UnknownWordPenalty
> WordPenalty
> Distortion
> PhrasePenalty * - not listed on the website (are there more)
>
> Feature/functions in the [feature] section and items in the [weight]
> sections appear to be linked. The feature/functions without arguments
> have corresponding entries linked by the same option name with an
> appended zero in the [weight] section. Since these feature/functions
> have arguments, is it safe to say that they can appear only once in
> both the [feature] and [weight] sections?
You can have multiple feature function of the same type. The most
obvious 1 would be having multiple LM, eg.
[feature]
KENLM path=file1.lm
KENLM path=file2.lm
Each instance of an FF must have a unique name, if you don't give inamet
a , the decoder will name it for you, KENLM0, KENLM1, .... Each instance
must have the corresponding weights (unless it's non-tuneable)
[weight]
KENLM0= 0.4
KENLM1= 0.6
>
> [weight]
> UnknownWordPenalty0= 1
> WordPenalty0= -1
> Distortion0= 0.3
> PhrasePenalty0= 0.2
>
> The feature/functions arguments have corresponding entries liked by
> the "name=" argument as the option name in the [weight] section. Are
> there cases where there will be entries in the [feature] section
> without corresponding entries in the [weight] section or vice-versa?
Yes, if the FF is not tuneable, you don't need to give it weight(s). The
hardcoded weights will be used. UnknownWordPenalty is a non-tuneable FF
and its hardcoded weight is 1, so the line
UnknownWordPenalty0= 1
isn't strictly necessary.
Whether a FF is non-tuneable or not by default is determined in the
code. You can also set the non-tuneable property, eg
[feature]
UnknownWordPenalty tuneable=true
>
> [feature]
> PhraseDictionaryMemory name=*TranslationModel0* num-features=4 ...
> KENLM name=*LM0* factor=0 ...
>
> [weight]
> *TranslationModel0*= 0.2 0.2 0.2 0.2
> *LM0*= 0.5
>
> The sections other than [feature] and [weight], such as
> [input-factors] and [mapping], appear to preserve the v1 moses.ini
> format. Is this true?
yep
>
> The order of lines in the [feature] and [weight] sections is
> irrelevant (as many examples have them in different orders). Also, the
> order of the arguments on a feature/function line is irrelevant
> (examples show them in different orders).
yep. However, the relative order of the phrase-tables is relevant. The
[mapping] section refers to the index of the phrase-table, eg
[mapping]
0 T 0
1 T 1
so if you swap the order of the phrase-tables in the [feature] section,
they will refer to different phrase-tables
>
> Finally, is there a connection between the [input-factors] section's
> value and the input-factor argument value for PhraseDictionaryMemory
> and LexicalReordering feature/functions? Or, are the similar names and
> corresponding values only coincidental?
Your input sentence can contain multiple factors, eg. surface|POS|lemma,
in which case [input-factor] should be
[input-factor]
0
1
2
However, the phase-table may choose to only use the surface form, in
which case
[feature]
PhraseDictionaryMemory input-factors=0
>
> My intention is to build two scripts and contribute these scripts to
> the Moses project. One will convert the v2 moses.ini file to a
> standard form (not associated with the command line syntax) so people
> can easily edit the values. The other will convert the interim form
> back to the native v2 moses.ini format.
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150202/236cbea2/attachment-0001.htm
------------------------------
Message: 3
Date: Mon, 02 Feb 2015 09:31:45 -0500
From: Kenneth Heafield <moses@kheafield.com>
Subject: Re: [Moses-support] Using factor in the hierarchical model
To: moses-support@mit.edu
Message-ID: <54CF8A51.8070604@kheafield.com>
Content-Type: text/plain; charset=windows-1252
Hi,
It should be fixed. Whomever wrote that argument parser (not me) needs
to be reminded to consume only the first = sign.
Kenneth
On 02/02/2015 05:06 AM, ekkim214 wrote:
>
> I want to use some factor (class) in the hierarchical model.
> I am using the ems script and using KenLM for surface and factor both.
> The building of LM is successful with settings "-discount_fallback" but
> I got an error during tuning phase like this.
>
> Exception: moses/LM/Ken.cpp:399 in Moses::LanguageModel*
> Moses::ConstructKenLM(const string&) threw util::Exception because
> `args.size() != 2'.
> Incorrect format of KenLM property:
> path=/home3/ekkim/working/E2K_1501/hierarchical_class/lm/travel22=mkcls.binlm.5
> Exit code: 1
>
> Does not "moses_chart" decoder which is distributed in the package
> support factored training?
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
------------------------------
Message: 4
Date: Mon, 02 Feb 2015 22:20:51 +0700
From: Tom Hoar <tahoar@precisiontranslationtools.com>
Subject: Re: [Moses-support] Learning the moses.ini v2 format
To: moses-support@mit.edu
Message-ID: <54CF95D3.5010502@precisiontranslationtools.com>
Content-Type: text/plain; charset="windows-1252"
Thanks, Hieu.
Here are the two scripts. Add them to the contrib (or other) folder as
you see fit. They have a -h switch. The moses2-to-ini.py script copies
everything from moses.ini v2 format into a traditional INI file format.
The ini-2-moses2.py script restores the moses.ini v2 format.
They change the line order in the [feature] and [weight] sections. The
overall order of section blocks can change. They preserve the order of
lines/values inside the old v1 moses.ini format sections, like
[input-factor], [mapping], etc. Please report any errors.
There two limits. First, the FF name= attribute is not optional because
relying on sequential orders can be risky. Users should declare the name
to match the weight value key. Second, they do not support use cases
where non-tuneable FF does not include weights. If you place dummy
weights, like you did with UnknownWordPenalty, everything is ok.
There's one additional feature. Users can use the --escape-prefix
command-line option to escape a prefix path in FF "path=" attribute
values. This will be helpful in moving models. I noticed that the
clone_moses_model.pl script has not been updated to support the new
moses.ini file format. Maybe someone would like to use this as a
starting point.
I find it easier/faster to manipulate values a traditional INI file
structure. Users can also import classes in the scripts into other
modules. They have a pretty simple API. I thought others might find
these useful.
Tom
On 02/02/2015 08:56 PM, Hieu Hoang wrote:
> Hi Tom
>
> On 02/02/15 01:33, Tom Hoar wrote:
>> Much of the v2 moses.ini looks self-explanatory, but I'd like to
>> confirm my understanding.
>>
>> The website (http://www.statmt.org/moses/?n=Moses.FeatureFunctions)
>> defines three feature/functions without arguments. In the moses.ini
>> files made by train-model.perl's step 9, there also appears to be a
>> 4th that requires no argument. Can someone confirm this is the case?
>> Are there others that could appear without arguments?
> Yes, PhrasePenalty is a standard feature function (FF), it doesn't
> need any arguments. It used to be the constant 2.718 in the last score
> in the phrase-table. But i thought that was silly so move it to it's
> own feature function.
>>
>> [feature]
>> UnknownWordPenalty
>> WordPenalty
>> Distortion
>> PhrasePenalty * - not listed on the website (are there more)
>>
>> Feature/functions in the [feature] section and items in the [weight]
>> sections appear to be linked. The feature/functions without arguments
>> have corresponding entries linked by the same option name with an
>> appended zero in the [weight] section. Since these feature/functions
>> have arguments, is it safe to say that they can appear only once in
>> both the [feature] and [weight] sections?
> You can have multiple feature function of the same type. The most
> obvious 1 would be having multiple LM, eg.
> [feature]
> KENLM path=file1.lm
> KENLM path=file2.lm
> Each instance of an FF must have a unique name, if you don't give
> inamet a , the decoder will name it for you, KENLM0, KENLM1, .... Each
> instance must have the corresponding weights (unless it's non-tuneable)
> [weight]
> KENLM0= 0.4
> KENLM1= 0.6
>>
>> [weight]
>> UnknownWordPenalty0= 1
>> WordPenalty0= -1
>> Distortion0= 0.3
>> PhrasePenalty0= 0.2
>>
>> The feature/functions arguments have corresponding entries liked by
>> the "name=" argument as the option name in the [weight] section. Are
>> there cases where there will be entries in the [feature] section
>> without corresponding entries in the [weight] section or vice-versa?
> Yes, if the FF is not tuneable, you don't need to give it weight(s).
> The hardcoded weights will be used. UnknownWordPenalty is a
> non-tuneable FF and its hardcoded weight is 1, so the line
> UnknownWordPenalty0= 1
> isn't strictly necessary.
>
> Whether a FF is non-tuneable or not by default is determined in the
> code. You can also set the non-tuneable property, eg
> [feature]
> UnknownWordPenalty tuneable=true
>
>>
>> [feature]
>> PhraseDictionaryMemory name=*TranslationModel0* num-features=4 ...
>> KENLM name=*LM0* factor=0 ...
>>
>> [weight]
>> *TranslationModel0*= 0.2 0.2 0.2 0.2
>> *LM0*= 0.5
>>
>> The sections other than [feature] and [weight], such as
>> [input-factors] and [mapping], appear to preserve the v1 moses.ini
>> format. Is this true?
> yep
>>
>> The order of lines in the [feature] and [weight] sections is
>> irrelevant (as many examples have them in different orders). Also,
>> the order of the arguments on a feature/function line is irrelevant
>> (examples show them in different orders).
> yep. However, the relative order of the phrase-tables is relevant. The
> [mapping] section refers to the index of the phrase-table, eg
> [mapping]
> 0 T 0
> 1 T 1
> so if you swap the order of the phrase-tables in the [feature]
> section, they will refer to different phrase-tables
>
>>
>> Finally, is there a connection between the [input-factors] section's
>> value and the input-factor argument value for PhraseDictionaryMemory
>> and LexicalReordering feature/functions? Or, are the similar names
>> and corresponding values only coincidental?
> Your input sentence can contain multiple factors, eg.
> surface|POS|lemma, in which case [input-factor] should be
> [input-factor]
> 0
> 1
> 2
> However, the phase-table may choose to only use the surface form, in
> which case
> [feature]
> PhraseDictionaryMemory input-factors=0
>>
>> My intention is to build two scripts and contribute these scripts to
>> the Moses project. One will convert the v2 moses.ini file to a
>> standard form (not associated with the command line syntax) so people
>> can easily edit the values. The other will convert the interim form
>> back to the native v2 moses.ini format.
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150202/ad4c61f0/attachment.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: moses2-to-ini.py
Type: text/x-python
Size: 6078 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20150202/ad4c61f0/attachment.py
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ini-to-moses2.py
Type: text/x-python
Size: 5738 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20150202/ad4c61f0/attachment-0001.py
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 100, Issue 4
*********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 100, Issue 4"
Post a Comment