Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. lattice input weight (Jorg Tiedemann)
2. Re: lattice input weight (Hieu Hoang)
3. Re: Using factor in the hierarchical model (Lisa Kim)
----------------------------------------------------------------------
Message: 1
Date: Wed, 4 Feb 2015 23:03:06 +0100
From: Jorg Tiedemann <tiedeman@gmail.com>
Subject: [Moses-support] lattice input weight
To: moses-support <moses-support@mit.edu>
Message-ID: <8D106253-4434-41E8-9897-976F604A9324@gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
Hi,
How do I specify the word lattice feature and its weight in the new Moses configuration file format?
Thanks!
J?rg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150204/003cc20f/attachment-0001.htm
------------------------------
Message: 2
Date: Wed, 4 Feb 2015 22:06:48 +0000
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] lattice input weight
To: Jorg Tiedemann <tiedeman@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbjT6cJ1AKk=Nt5_JZtiNhvSsSXWr0bx6=QR8aPct0GJZQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
the regression test for lattice input is a good example
https://github.com/moses-smt/moses-regression-tests/tree/master/tests/phrase.lattice-surface
(lattices only works for phrase-based)
Hieu Hoang
Research Associate (until March 2015)
** searching for interesting commercial MT position **
University of Edinburgh
http://www.hoang.co.uk/hieu
On 4 February 2015 at 22:03, Jorg Tiedemann <tiedeman@gmail.com> wrote:
> Hi,
>
> How do I specify the word lattice feature and its weight in the new Moses
> configuration file format?
> Thanks!
>
> J?rg
>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150204/19c57952/attachment-0001.htm
------------------------------
Message: 3
Date: Thu, 5 Feb 2015 13:42:31 +0900
From: Lisa Kim <ekkim214@gmail.com>
Subject: Re: [Moses-support] Using factor in the hierarchical model
To: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAOmDmvaoyaWU5+qYgWqge1zdxQ71ZKkUbJOr-Oyq8t-JxL3T7w@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Thank you, Hoang.
After updating latest version, I have different error during tuning phase.
:(
For the some issues, I cannot upload the model files and these are part of
my models and configs.
Do I have some mistakes in the ems configuration ?
Error Message :
Start loading text phrase table. Moses format : [0.014] seconds
terminate called after throwing an instance of 'util::Exception'
what(): moses/TranslationModel/RuleTable/PhraseDictionaryOnDisk.cpp:96
in virtual void Moses::PhraseDictionaryOnDisk::InitializeForInput(const
Moses::InputType&) threw util::Exception because
`obj->GetMisc("NumSourceFactors") != m_input.size()'.
On-disk phrase table has 1 source factors.. The ini file specified 2 source
factors
Aborted (core dumped)
Exit code: 134
Moses.ini file :
# input factors
[input-factors]
0
1
# mapping steps
[mapping]
0 T 0
1 T 1
[cube-pruning-pop-limit]
1000
[non-terminals]
X
[search-algorithm]
3
[inputtype]
3
[max-chart-span]
20
1000
# feature functions
[feature]
UnknownWordPenalty
WordPenalty
PhrasePenalty
PhraseDictionaryMemory name=TranslationModel0 num-features=4
path=/home3/ekkim/working/E2K_1502/hierarchical_class/model/phrase-table.1.0,1-0,1
input-factor=0,1 output-factor=0,1
PhraseDictionaryMemory name=TranslationModel1 num-features=1
path=/home3/ekkim/working/E2K_1502/hierarchical_class/model/glue-grammar.1
input-factor=0 output-factor=0
KENLM lazyken=0 name=LM0 factor=0
path=/home3/ekkim/working/E2K_1502/hierarchical_class/lm/travel22.binlm.1
order=5# dense weights for feature functions
KENLM lazyken=0 name=LM1 factor=1
path=/home3/ekkim/working/E2K_1502/hierarchical_class/lm/travel22_mkcls.binlm.1
order=5
# dense weights for feature functions
[weight]
UnknownWordPenalty0= 1
WordPenalty0= -1
PhrasePenalty0= 0.2
TranslationModel0= 0.2 0.2 0.2 0.2
TranslationModel1= 1.0
LM0= 0.5
LM1= 0.5
Part of Phrase Table :
[X][X] any|296 discount|413 [X][X] [X] ||| [X][X] ??|349 ??|176 ?|393
[X][X] [X] ||| 0.0306047 0.00709669 0.0481453 0.000125329 ||| 0-4 1-1 1-3
2-1 2-2 3-0 ||| 0.460664 0.292832 0.292832 ||| |||
[X][X] any|296 discount|413 [X][X] a|327 [X] ||| [X][X] ??|349 ??|176 ?|393
[X][X] [X] ||| 0.00803946 0.000214715 0.0481453 0.000125329 ||| 0-4 1-1 1-3
2-1 2-2 3-0 ||| 0.460664 0.0769231 0.0769231 ||| |||
[X][X] any|296 discount|413 [X][X] use|136 [X] ||| [X][X] ??|349 ??|176
?|393 [X][X] [X] ||| 0.00950118 2.99551e-06 0.0481453 0.000125329 ||| 0-4
1-1 1-3 2-1 2-2 3-0 ||| 0.460664 0.0909091 0.0909091 ||| |||
[X][X] any|296 discount|413 cards|404 [X] ||| ??|349 ?|161 ?|302 ??|184
[X][X] [X] ||| 0.0266944 0.00037872 0.0481453 8.14358e-05 ||| 0-4 1-0 2-0
2-1 3-2 3-3 ||| 0.561111 0.311111 0.311111 ||| |||
[X][X] any|296 discount|413 card|404 [X] ||| ??|349 ?|161 ?|302 ??|184
[X][X] [X] ||| 0.0214509 0.00256232 0.0481453 0.00010193 ||| 0-4 1-0 1-2
2-0 2-1 3-3 ||| 0.561111 0.25 0.25 ||| |||
[X][X] any|296 discount|413 for|487 [X] ||| ??|349 [X][X] [X] |||
0.000905141 0.000836177 0.0202717 0.102804 ||| 0-1 1-0 2-0 3-0 ||| 17.7303
0.791666 0.333333 ||| |||
Addings in ems cofig file for factor training :
input-factors = word mkcls
output-factors = word mkcls
alignment-factors = "word -> word"
translation-factors = "word+mkcls -> word+mkcls"
decoding-steps = "t0"
2015. 2. 4. ?? 9:07? "Hieu Hoang" <Hieu.Hoang@ed.ac.uk>?? ??:
try using the latest version of moses from github.
If it still fails, please make your model files available for download so I
can replicate the problem
Hieu Hoang
Research Associate (until March 2015)
** searching for interesting commercial MT position **
University of Edinburgh
http://www.hoang.co.uk/hieu
On 4 February 2015 at 06:54, Lisa Kim <ekkim214@gmail.com> wrote:
>
> Thank you.
> It is successful to change the LM name without "=" symbol.
>
> However, decoding still fails with following message.
> Could you check the error message?
>
> --------------------
> Start loading text phrase table. Moses format : [0.038] seconds
> terminate called recursively
> terminate called after throwing an instance of 'util::Exception'
> what(): Aborted (core dumped)
> Exit code: 134
> The decoder died. CONFIG WAS -weight-overwrite 'LM1= 0.125000
> PhrasePenalty0= 0.050000 WordPenalty0= -0.250000 TranslationModel0=
> 0.050000 0.050000 0.050000 0.050000 TranslationModel1= 0.250000 LM0=
> 0.125000'
> 2015. 2. 3. ?? 8:12? "Hieu Hoang" <hieuhoang@gmail.com>?? ??:
>
> as ken mentioned, don't use the = sign in the name of your LM file. I
>> think it's fixed in the latest version of moses. Which version do you have,
>> from when ago?
>>
>> moses_chart should handle factors. Again, if you version is from long
>> ago, I suggest your update
>>
>> On 02/02/15 10:06, ekkim214 wrote:
>>
>>
>> I want to use some factor (class) in the hierarchical model.
>> I am using the ems script and using KenLM for surface and factor both.
>> The building of LM is successful with settings "-discount_fallback" but
>> I got an error during tuning phase like this.
>>
>> Exception: moses/LM/Ken.cpp:399 in Moses::LanguageModel*
>> Moses::ConstructKenLM(const string&) threw util::Exception because
>> `args.size() != 2'.
>> Incorrect format of KenLM property:
>> path=/home3/ekkim/working/E2K_1501/hierarchical_class/lm/travel22=mkcls.binlm.5
>> Exit code: 1
>>
>> Does not "moses_chart" decoder which is distributed in the package
>> support factored training?
>>
>>
>> _______________________________________________
>> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150205/48d03888/attachment.htm
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 100, Issue 18
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 100, Issue 18"
Post a Comment