Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: constrained decoding (Lane Schwartz)
2. Re: constrained decoding (Chris Dyer)
3. Re: constrained decoding (Hieu Hoang)
4. Re: constrained decoding (Christian Federmann)
5. Re: mark-unknown/output-unknown parameter not working (Hieu Hoang)
----------------------------------------------------------------------
Message: 1
Date: Wed, 18 Sep 2013 13:15:10 -0400
From: Lane Schwartz <dowobeha@gmail.com>
Subject: Re: [Moses-support] constrained decoding
To: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Cc: moses-support <moses-support@mit.edu>, Maria Nadejde
<maria.nadejde@gmail.com>, Ondrej Bojar <bojar@ufal.mff.cuni.cz>
Message-ID:
<CABv3vZkjk1TyNAB7GwyAWac7i7eiqvZkb+kR7oyW0Wf97i6nHA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Thanks! What happens if the reference isn't reachable?
On Wed, Sep 18, 2013 at 4:50 AM, Hieu Hoang <Hieu.Hoang@ed.ac.uk> wrote:
> For anyone who's interested, the constrained decoding has been reimplemented
> as a feature function and works for both phrase-based and syntax models.
>
> To constrain the output of the decoder to just the reference sentences, add
> this as a feature:
> [feature]
> ....
> ConstrainedDecoding path=ref.txt
>
>
> --
> Hieu Hoang
> Research Associate
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>
--
When a place gets crowded enough to require ID's, social collapse is not
far away. It is time to go elsewhere. The best thing about space travel
is that it made it possible to go elsewhere.
-- R.A. Heinlein, "Time Enough For Love"
------------------------------
Message: 2
Date: Wed, 18 Sep 2013 13:27:30 -0400
From: Chris Dyer <cdyer@cs.cmu.edu>
Subject: Re: [Moses-support] constrained decoding
To: Lane Schwartz <dowobeha@gmail.com>
Cc: Hieu Hoang <hieu.hoang@ed.ac.uk>, moses-support
<moses-support@mit.edu>, Ondrej Bojar <bojar@ufal.mff.cuni.cz>, Maria
Nadejde <maria.nadejde@gmail.com>
Message-ID:
<CAEHEvxObfLKF=_22NvKhnPhBLpoeT=3xFZs0AMLTTcsPSU6xag@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
http://i0.kym-cdn.com/photos/images/original/000/002/299/ze_house_.jpg
On Wed, Sep 18, 2013 at 1:15 PM, Lane Schwartz <dowobeha@gmail.com> wrote:
> Thanks! What happens if the reference isn't reachable?
>
> On Wed, Sep 18, 2013 at 4:50 AM, Hieu Hoang <Hieu.Hoang@ed.ac.uk> wrote:
>> For anyone who's interested, the constrained decoding has been reimplemented
>> as a feature function and works for both phrase-based and syntax models.
>>
>> To constrain the output of the decoder to just the reference sentences, add
>> this as a feature:
>> [feature]
>> ....
>> ConstrainedDecoding path=ref.txt
>>
>>
>> --
>> Hieu Hoang
>> Research Associate
>> University of Edinburgh
>> http://www.hoang.co.uk/hieu
>>
>
>
>
> --
> When a place gets crowded enough to require ID's, social collapse is not
> far away. It is time to go elsewhere. The best thing about space travel
> is that it made it possible to go elsewhere.
> -- R.A. Heinlein, "Time Enough For Love"
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
------------------------------
Message: 3
Date: Wed, 18 Sep 2013 21:45:35 +0200
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] constrained decoding
To: Ondrej Bojar <bojar@ufal.mff.cuni.cz>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <523A02DF.3010403@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Lane - if it's not reachable, it should give you a blank line and 'NO
BEST TRANSLATION' on stderr. Additonally, there won't be anything in the
nbest list. I don't think i kept your multi-reference implementation, my
have to redo that ;) git pull, it might have, err, crashed on the 1st
iteration.
Ondrej - it's applied during the search, when each hypothesis is added
to the stack. We can think about pruning the translation options before
the search
If the hypothesis doesn't create a substring of the reference, the
feature function will give it a score of -infinity and the stack will
immediately discard it.
I think it's a flexible feature, eg. you can replace the exact match
with word edit distance etc.
Christian - allowing some unknown word is a good idea. I've added a
'max-unknowns' argument to the feature function, eg
[feature]
...
ConstrainedDecoding path=ref.txt max-unknowns=2
this is 0 by default, -1 makes it allow any number of unknowns words.
On 18/09/2013 11:10, Ondrej Bojar wrote:
> Hi,
>
> great! Thanks!
>
> When is the constraint applied? After translation options have been created, or already during their creation? I guess the former, since the latter is somewhat difficult to apply.
it works during search,
> My main concern is the relationship between constraint decoding and factored setups. I guess the current version definitely works with alternative decoding paths of single-factored translation, and it also probably works with any factored setup *modulo* pruning of translation options. (If the needed translation option is available but happens to get pruned, the sentence will fail.)
>
> Please correct me if I am wrong.
>
> Cheers, O.
>
> ----- Original Message -----
>> From: "Hieu Hoang"<Hieu.Hoang@ed.ac.uk>
>> To: "Maria Nadejde"<maria.nadejde@gmail.com>, "Lane Schwartz"<dowobeha@gmail.com>, "Ondrej Bojar"
>> <bojar@ufal.mff.cuni.cz>, "moses-support"<moses-support@mit.edu>
>> Sent: Wednesday, 18 September, 2013 10:50:28 AM
>> Subject: constrained decoding
>>
>> For anyone who's interested, the constrained decoding has been
>> reimplemented as a feature function and works for both phrase-based and
>> syntax models.
>>
>> To constrain the output of the decoder to just the reference sentences, add
>> this as a feature:
>> [feature]
>> ....
>> ConstrainedDecoding path=ref.txt
>>
>>
>> --
>> Hieu Hoang
>> Research Associate
>> University of Edinburgh
>> http://www.hoang.co.uk/hieu
>>
------------------------------
Message: 4
Date: Wed, 18 Sep 2013 23:04:29 +0200 (CEST)
From: Christian Federmann <cfedermann@dfki.de>
Subject: Re: [Moses-support] constrained decoding
To: Hieu Hoang <hieuhoang@gmail.com>, Ondrej Bojar
<bojar@ufal.mff.cuni.cz>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<974947211.13339.1379538269735.JavaMail.open-xchange@ox6.dfki.de>
Content-Type: text/plain; charset=UTF-8
Great, thanks!
Christian
Hieu Hoang <hieuhoang@gmail.com> hat am 18. September 2013 um 21:45 geschrieben:
> Lane - if it's not reachable, it should give you a blank line and 'NO
> BEST TRANSLATION' on stderr. Additonally, there won't be anything in the
> nbest list. I don't think i kept your multi-reference implementation, my
> have to redo that ;) git pull, it might have, err, crashed on the 1st
> iteration.
>
> Ondrej - it's applied during the search, when each hypothesis is added
> to the stack. We can think about pruning the translation options before
> the search
>
> If the hypothesis doesn't create a substring of the reference, the
> feature function will give it a score of -infinity and the stack will
> immediately discard it.
>
> I think it's a flexible feature, eg. you can replace the exact match
> with word edit distance etc.
>
> Christian - allowing some unknown word is a good idea. I've added a
> 'max-unknowns' argument to the feature function, eg
> [feature]
> ...
> ConstrainedDecoding path=ref.txt max-unknowns=2
>
> this is 0 by default, -1 makes it allow any number of unknowns words.
>
> On 18/09/2013 11:10, Ondrej Bojar wrote:
> > Hi,
> >
> > great! Thanks!
> >
> > When is the constraint applied? After translation options have been created,
> > or already during their creation? I guess the former, since the latter is
> > somewhat difficult to apply.
> it works during search,
> > My main concern is the relationship between constraint decoding and factored
> > setups. I guess the current version definitely works with alternative
> > decoding paths of single-factored translation, and it also probably works
> > with any factored setup *modulo* pruning of translation options. (If the
> > needed translation option is available but happens to get pruned, the
> > sentence will fail.)
> >
> > Please correct me if I am wrong.
> >
> > Cheers, O.
> >
> > ----- Original Message -----
> >> From: "Hieu Hoang"<Hieu.Hoang@ed.ac.uk>
> >> To: "Maria Nadejde"<maria.nadejde@gmail.com>, "Lane
> >> Schwartz"<dowobeha@gmail.com>, "Ondrej Bojar"
> >> <bojar@ufal.mff.cuni.cz>, "moses-support"<moses-support@mit.edu>
> >> Sent: Wednesday, 18 September, 2013 10:50:28 AM
> >> Subject: constrained decoding
> >>
> >> For anyone who's interested, the constrained decoding has been
> >> reimplemented as a feature function and works for both phrase-based and
> >> syntax models.
> >>
> >> To constrain the output of the decoder to just the reference sentences, add
> >> this as a feature:
> >> [feature]
> >> ....
> >> ConstrainedDecoding path=ref.txt
> >>
> >>
> >> --
> >> Hieu Hoang
> >> Research Associate
> >> University of Edinburgh
> >> http://www.hoang.co.uk/hieu
> >>
>
--
Dipl.-Inf. Christian Federmann, Researcher, Language Technology Lab
Office +1.09 -- Phone +49-681/857-75-5353, Fax +49-681/857-75-5338
DFKI GmbH, Campus D3 2, Stuhlsatzenhausweg 3, 66123 Saarbruecken
http://www.dfki.de/~cfedermann
-------------------------------------------------------------------
Deutsches Forschungszentrum fuer Kuenstliche Intelligenz GmbH
Trippstadter Strasse 122, D-67663 Kaiserslautern, Germany
Geschaeftsfuehrung:
Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender)
Dr. Walter Olthoff
Vorsitzender des Aufsichtsrats:
Prof. Dr. h.c. Hans A. Aukes
Amtsgericht Kaiserslautern, HRB 2313
-------------------------------------------------------------------
------------------------------
Message: 5
Date: Thu, 19 Sep 2013 09:07:14 +0200
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] mark-unknown/output-unknown parameter not
working
To: moses-support@mit.edu
Message-ID: <523AA2A2.7040201@gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
hi rishabh
i've added support for
-mark-unknown
in hiero model. git pull to get it
https://github.com/moses-smt/mosesdecoder/commit/996740de73efb93430c5e4e99fbc6aad037e9b95
-output-unknowns is slightly more difficult to do.
On 17/09/2013 13:32, Rishabh Srivastava wrote:
> Hi,
>
> We have been working on hierarchical based models. Some words are
> directly substituted as they were in source language, as they were out
> of vocabulary words. We tried options like mark-unknown(mu) and
> output-unknown <file-name> but it is neither marking in the former
> case and not creating the file in the latter.
>
> Kindly help us out.
>
> Thanks,
> Rishabh Srivastava.
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20130919/78dc78ea/attachment.htm
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 83, Issue 32
*********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 83, Issue 32"
Post a Comment