Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: -target-word-insertion-feature (Barry Haddow)
2. sentence is always too short for cleaning (emna hkiri)
3. Re: sentence is always too short for cleaning (Matthias Huck)
4. How to run EMS (Asad A.Malik)
5. How to Run experiment.perl (Asad A.Malik)
----------------------------------------------------------------------
Message: 1
Date: Fri, 28 Nov 2014 17:55:38 +0000
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] -target-word-insertion-feature
To: Prashant Mathur <prashant@fbk.eu>, Philipp Koehn
<pkoehn@inf.ed.ac.uk>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <5478B71A.7060705@staffmail.ed.ac.uk>
Content-Type: text/plain; charset=windows-1252; format=flowed
Hi Prashant
For the Moses version of these features, this paper:
http://www.statmt.org/wmt13/pdf/WMT12.pdf
but as the paper notes, they are based on earlier work, especially
Chiang et al's 2009 "11001 new features" paper.
cheers - Barry
On 28/11/14 16:25, Prashant Mathur wrote:
> blast from the past!
> Which paper should I cite if I use this feature?
>
>
>
>
> On Fri, Apr 12, 2013 at 8:30 PM, Philipp Koehn <pkoehn@inf.ed.ac.uk
> <mailto:pkoehn@inf.ed.ac.uk>> wrote:
>
> Hi,
>
> I wrote a little longer documentation of the sparse features:
> http://www.statmt.org/moses/?n=Moses.SparseFeatures
>
> -phi
>
> On Fri, Apr 12, 2013 at 6:12 PM, Philipp Koehn
> <pkoehn@inf.ed.ac.uk <mailto:pkoehn@inf.ed.ac.uk>>
> wrote:
> > Hi Francis,
> >
> > thank you for your question - the documentation in this respect has
> > not caught up yet.
> >
> > Th target word insertion feature is a sparse feature, so it
> behaves a
> > bit different from the features that you are used to.
> >
> > What it is intended to do is to learn lexical features, one feature
> > for each word, which indicates if inserting the word in the
> output is
> > a good thing or not. The indication if a word has been inserted is
> > detected from the word alignment within each phrase pair. If the
> > target word is not aligned to any source word, then it is deemed
> to be
> > inserted.
> >
> > The options you have to specify is a factor number (typically 0 for
> > the surface form of the word), and optionally a file that contains a
> > restricted list of words. If such a file is present, then only
> target
> > words that are in the file (one word per line) are considered
> for the
> > feature. In other words: words that are not in the file may be
> > inserted or not, no feature calculation takes place.
> >
> > Sparse lexical features require a special weight file that contains
> > the weight for each instantiation of a feature. So this may look
> like
> > the following:
> > twi_I -0.00529196301346302
> > twi_had -4.16585913937328e-05
> > twi_was -0.00612071371830685
> > [...]
> >
> > Of course, you want to learn these feature weights during tuning,
> > which requires the use of either PRO or kbMIRA - it does not
> work with
> > plain MERT.
> >
> > The moses.ini that is used to run tuning must contain:
> >
> > [report-sparse-features]
> > twi
> >
> > in addition to the
> >
> > [target-word-insertion-feature]
> > 0 /path/to/word/list
> >
> > Let me know if this description helps you.
> >
> > -phi
> >
> > On Fri, Apr 5, 2013 at 2:23 PM, Francis Tyers
> <ftyers@prompsit.com <mailto:ftyers@prompsit.com>>
> wrote:
> >> Hello everyone!
> >>
> >> I'm a bit interested in the -target-word-insertion-feature to
> Moses.
> The
> >> help output is as follows:
> >>
> >> -target-word-insertion-feature: Count feature for each
> unaligned target
> >> word
> >>
> >> I tried calling it without any options and it didn't seem to do
> >> anything, so I checked out the code and found a couple of hints:
> >>
> >> 1) in build-sparse-lexical-features.perl:
> >>
> >> [target-word-insertion-feature]
> >> 0 $file
> >>
> >> 2) in moses/StaticData.cpp:
> >>
> >> UserMessage::Add("Format of target word insertion feature
> parameter is:
> >> --target-word-insertion-feature <factor> [filename]");
> >>
> >> So, this would suggest that it requires a factor, and a filename is
> >> optional. The code instantiates a class TargetWordInsertionFeature.
> >>
> >> If we look at the TargetWordInsertionFeature, it seems to:
> >>
> >> * Load a file with a list of words if it exists
> >> * Make a boolean array of size 16 (I guess this is because of
> the limit
> >> on feature score length in ScoreComponentCollection)
> >> * For each word in the phrase it sets if it is aligned or not
> >> * If the word is unaligned it adds 1 to the score for that word
> >> feature.(?)
> >>
> >> ... this is where I get lost.
> >>
> >> Can anyone give a better description of what this option does,
> and how
> >> it effects the translation (if at all).
> >>
> >> My initial interest was in getting statistics on unaligned
> words that
> >> appeared in the output. Can this option give that ?
> >>
> >> Thanks in advance for any help!
> >>
> >> Fran
> >>
> >> _______________________________________________
> >> Moses-support mailing list
> >> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> >> http://mailman.mit.edu/mailman/listinfo/moses-support
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
------------------------------
Message: 2
Date: Fri, 28 Nov 2014 21:51:51 +0100
From: emna hkiri <emna.hkiri@gmail.com>
Subject: [Moses-support] sentence is always too short for cleaning
To: moses-support@mit.edu
Message-ID:
<CAAp-nZ3gsv_C5kawKLAoMT4tT5x0OWnKHVeeptWsqz_=mHoooQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Dear friends
i need your help please
i have a problem of the cleaning phase of the arabic text
every time moses returns the message sentences number 1562783 is too
short!!!
(in fact it is the last sentence in the text) so i delete it and again and
again he tell me that this new last sentence is too short !!!!
and i do delete the last sentences and i have always the same problem
Can someone please throw some light on this.
Thanks & Regards
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141128/c1293213/attachment-0001.htm
------------------------------
Message: 3
Date: Sat, 29 Nov 2014 01:29:48 +0000
From: Matthias Huck <mhuck@inf.ed.ac.uk>
Subject: Re: [Moses-support] sentence is always too short for cleaning
To: emna hkiri <emna.hkiri@gmail.com>
Cc: moses-support@mit.edu
Message-ID: <1417224588.1396.5.camel@hucklap.site>
Content-Type: text/plain; charset="UTF-8"
Hi,
If this happens in scripts/training/clean-corpus-n.perl then you should
check whether a parallel corpus with the same number of lines on source
and target side is passed to that script. Maybe there's an issue with
your training data or something went wrong in a previous step of the
preprocessing pipeline if the line numbers differ.
Cheers,
Matthias
On Fri, 2014-11-28 at 21:51 +0100, emna hkiri wrote:
> Dear friends
> i need your help please
> i have a problem of the cleaning phase of the arabic text
> every time moses returns the message sentences number 1562783 is too
> short!!!
> (in fact it is the last sentence in the text) so i delete it and again
> and
> again he tell me that this new last sentence is too short !!!!
> and i do delete the last sentences and i have always the same problem
>
> Can someone please throw some light on this.
>
> Thanks & Regards
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
------------------------------
Message: 4
Date: Sat, 29 Nov 2014 05:18:07 +0000 (UTC)
From: "Asad A.Malik" <asad_12204@yahoo.com>
Subject: [Moses-support] How to run EMS
To: Moses-support <moses-support@mit.edu>
Message-ID:
<2080355171.2430497.1417238287854.JavaMail.yahoo@jws10611.mail.bf1.yahoo.com>
Content-Type: text/plain; charset="utf-8"
Hi All,
I am trying to use EMS but I am confused about the Experiment.perl, as it is located in scripts/ems.Also the sample configuration, and where I have to setup the work directory?And the step which says to edit the following path setting in config.toy
?--
Kind Regards,
Mr. Asad Abdul Malik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141129/e0bcaa44/attachment-0001.htm
------------------------------
Message: 5
Date: Sat, 29 Nov 2014 15:00:12 +0000 (UTC)
From: "Asad A.Malik" <asad_12204@yahoo.com>
Subject: [Moses-support] How to Run experiment.perl
To: Moses-support <moses-support@mit.edu>
Message-ID:
<401142113.2501767.1417273212412.JavaMail.yahoo@jws10664.mail.bf1.yahoo.com>
Content-Type: text/plain; charset="utf-8"
Hi All,?
How can I?Run experiment.perl -config config.toy -exec. ????
When I type following command:
$ run experiment.perl -config config.toy -exec
It says no command found.?--
Kind Regards,
Mr. Asad Abdul Malik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141129/81f1fe81/attachment.htm
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 97, Issue 98
*********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 97, Issue 98"
Post a Comment