Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: --activate-features in mert-moses.perl not working?
(Hieu Hoang)
2. Re: tuning (weights, normalization) (Hieu Hoang)
3. mosesv2-decoder-error (nadeem khan)
4. Re: mosesv2-decoder-error (Philipp Koehn)
----------------------------------------------------------------------
Message: 1
Date: Wed, 26 Feb 2014 20:49:20 +0000
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] --activate-features in mert-moses.perl
not working?
To: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbgy-wb14zMBEjBcVsA6-cfTmW31-8te3ap-L7ECyjN3dQ@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
we should sort out the tuneable/activate feature issue when you get to
Edinburgh. They seem to overlap and cause problems if we don't sort it out
it may be that we should delete 1 of these options and depend wholly on the
other
On 26 February 2014 08:17, Rico Sennrich <rico.sennrich@gmx.ch> wrote:
> On 26.02.2014 07:24, moses-support-request@mit.edu wrote:
> > Hi Hieu, Rico,
> > this does not seem to be an issue with the ini-file. It actually works
> > as well with stand-alone moses. The issue seems to be the mert-moses.pl
> > script which switches off features that are not returned by the decoder
> > because they are set to tuneable=false.
> >
> > In the function "run_decoder" in mert-moses.perl there is this line:
> >
> > $decoder_config = "-weight-overwrite '" . join(" ", values
> > %model_weights) ."'" unless $___USE_CONFIG_WEIGHTS_FIRST && $run==1;
> >
> > And I suspect -weight-overwrite is with a truncated list of
> > model_weights is causing the issue.
> > Best,
> > Marcin
> does your ini file define weights for all features? mert-moses.pl passes
> the original ini file as an argument to the decoder, and if no weights
> are provided by mert-moses.pl with "-weight-overwrite", the weights from
> the ini are used.
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140226/b0b33c11/attachment-0001.htm
------------------------------
Message: 2
Date: Wed, 26 Feb 2014 21:01:43 +0000
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] tuning (weights, normalization)
To: Jani Dugonik <jani.dugonik@um.si>
Cc: Moses support <moses-support@mit.edu>
Message-ID:
<CAEKMkbid=zM39abipmDWJPYdkn3NuLoZWo-kWjOh4FwOY6YMOw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
On 25 February 2014 09:57, Jani Dugonik <jani.dugonik@um.si> wrote:
> Hi,
>
> I have a few questions about tuning weights.
>
> a) On statmt website it says:
>
> "Good values for the weights for phrase translation table (weight-t,
> short tm), language model (weight-l, short lm), and reordering model (
> weight-d, short d) are 0.1-1, good values for the word penalty (weight-w,
> short w) are -3-3. Negative values for the word penalty favor longer
> output, positive values favor shorter output. "
>
> Are there any lower and upper bounds for these weights?
>
> b) Do these weights need to be normalized? I searched for the answer on
> the Internet and I only came across this answer:
>
> "Also, for the decoder, it doesn't really matter if the weights are
> normalised or not."
>
> The weights are normalized by the mert-moses.pl script. I'm not actually
sure how, but the weights are between -1 and +1.
It shouldn't matter if the weights aren't normalised but in practise, it
does matter. There are some default values for beam widths and other
pruning parameters which have only been tested when the scores are in a
certain range/
> c) Tuning with PRO is not working, I think their website isn't available
> (more info in the attached file pro.out)
>
You can see it working in experiment 6 here:
http://www.statmt.org/moses/RELEASE-2.1/models/en-fr/
>
> Thanks, Jani
>
> --
> Jani Dugonik, mag. in?. ra?. in inf. tehnol.
> Laboratorij za ra?unalni?ke arhitekture in jezike
> In?titut za ra?unalni?tvo
> Fakulteta za elektrotehniko, ra?unalni?tvo in informatiko
> Univerza v Mariboru
> Smetanova 17, 2000 Maribor
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140226/ef4250aa/attachment-0001.htm
------------------------------
Message: 3
Date: Wed, 26 Feb 2014 16:58:41 -0800 (PST)
From: nadeem khan <nad_star06@yahoo.com>
Subject: [Moses-support] mosesv2-decoder-error
To: moses-support <moses-support@mit.edu>
Message-ID:
<1393462721.55756.YahooMailNeo@web162402.mail.bf1.yahoo.com>
Content-Type: text/plain; charset="iso-8859-1"
Hi all;
I am getting this error while running decoder with alignment flags:
FeatureFunction: UnknownWordPenalty0 start: 9 end: 9
line=PhraseDictionaryMemory input-factor=0 output-factor=0 path=/home/legends/work/hin-eng/f5/model/phrase-table.gz num-features=5 table-limit=20
FeatureFunction: PhraseDictionaryMemory0 start: 10 end: 14
Loading SRILM0
/home/legends/work/hin-eng/f5/lm/urd-eng.lm: line 4317: warning: non-zero probability for <unk> in closed-vocabulary LM
Loading Distortion0
Loading LexicalReordering0
Loading table into memory...done.
Loading WordPenalty0
Loading UnknownWordPenalty0
Loading PhraseDictionaryMemory0
Start loading text SCFG phrase table. Moses? format : [8.000] seconds
Reading /home/legends/work/hin-eng/f5/model/phrase-table.gz
----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
****************************************************************************************************
Exception: moses/Word.cpp:112 in void Moses::Word::CreateFromString(Moses::FactorDirection, const std::vector<unsigned int>&, const StringPiece&, bool) threw StrayFactorException because `fit'.
You have configured 1 factors but the word | contains factor delimiter | too many times.
Please help out in? fixing it.?
THANKS
Regards
Nadeem
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140226/d9837a0f/attachment-0001.htm
------------------------------
Message: 4
Date: Thu, 27 Feb 2014 09:15:32 -0500
From: Philipp Koehn <pkoehn@inf.ed.ac.uk>
Subject: Re: [Moses-support] mosesv2-decoder-error
To: nadeem khan <nad_star06@yahoo.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAAFADDC+1Sv3F75zaqMqyKUqMZAOu+QSsGRb8a9AeY-XuOhO-A@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Hi,
as the error message says, please remove all bar characters "|" from your
training corpus when building the phrase table.
-phi
On Wed, Feb 26, 2014 at 7:58 PM, nadeem khan <nad_star06@yahoo.com> wrote:
> Hi all;
> I am getting this error while running decoder with alignment flags:
>
> FeatureFunction: UnknownWordPenalty0 start: 9 end: 9
> line=PhraseDictionaryMemory input-factor=0 output-factor=0
> path=/home/legends/work/hin-eng/f5/model/phrase-table.gz num-features=5
> table-limit=20
> FeatureFunction: PhraseDictionaryMemory0 start: 10 end: 14
> Loading SRILM0
> /home/legends/work/hin-eng/f5/lm/urd-eng.lm: line 4317: warning: non-zero
> probability for <unk> in closed-vocabulary LM
> Loading Distortion0
> Loading LexicalReordering0
> Loading table into memory...done.
> Loading WordPenalty0
> Loading UnknownWordPenalty0
> Loading PhraseDictionaryMemory0
> Start loading text SCFG phrase table. Moses format : [8.000] seconds
> Reading /home/legends/work/hin-eng/f5/model/phrase-table.gz
> ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
> ****************************************************************************************************
> Exception: moses/Word.cpp:112 in void
> Moses::Word::CreateFromString(Moses::FactorDirection, const
> std::vector<unsigned int>&, const StringPiece&, bool) threw
> StrayFactorException because `fit'.
> You have configured 1 factors but the word | contains factor delimiter | too
> many times.
>
>
> Please help out in fixing it.
> THANKS
> Regards
> Nadeem
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 88, Issue 63
*********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 88, Issue 63"
Post a Comment