Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Access to set of possible target words from within feature
function (Marcin Junczys-Dowmunt)
2. Re: Access to set of possible target words from within
feature function (Hieu Hoang)
3. Re: Segmentation Fault (Hieu Hoang)
4. Re: Access to set of possible target words from within
feature function (Marcin Junczys-Dowmunt)
5. Re: Problem in extracting rules for syntax based model
(Hieu Hoang)
----------------------------------------------------------------------
Message: 1
Date: Fri, 12 Feb 2016 23:19:28 +0100
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: [Moses-support] Access to set of possible target words from
within feature function
To: moses-support <moses-support@mit.edu>
Message-ID: <56BE5A70.3080903@amu.edu.pl>
Content-Type: text/plain; charset=UTF-8; format=flowed
Hi everybody,
Is there a nice way to access the complete set of translation options
for a sentence from within a feature function? I just need the bag of
possible words before any processing. Let's say from within
"EmptyHypothesisState", there is InputType, but I cannot really find any
hook to get that easily.
Thanks,
Marcin
------------------------------
Message: 2
Date: Fri, 12 Feb 2016 22:32:56 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Access to set of possible target words
from within feature function
To: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbh35ST+F0KuP6ctO85Fatb+6CDud_G51Lc6NMKTsSQqUg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
sounds useful.There's
FeatureFunction::EvaluateTranslationOptionListWithSourceContext()
which is called for for every subphrase. It passes the feature function a
list of translation option for that subphrase. The method is called in
TranslationOptionCollection::EvaluateTranslationOptionListWithSourceContext
Or you add yet another Evaluate() method that passes the feature function
the InputPathList, which really has every translation options in 1 go
Hieu Hoang
http://www.hoang.co.uk/hieu
On 12 February 2016 at 22:19, Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
wrote:
> Hi everybody,
> Is there a nice way to access the complete set of translation options
> for a sentence from within a feature function? I just need the bag of
> possible words before any processing. Let's say from within
> "EmptyHypothesisState", there is InputType, but I cannot really find any
> hook to get that easily.
>
> Thanks,
> Marcin
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160212/98f6cc41/attachment-0001.html
------------------------------
Message: 3
Date: Fri, 12 Feb 2016 22:36:18 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Segmentation Fault
To: Jasneet Sabharwal <jasneet.sabharwal@sfu.ca>, moses-support
<moses-support@mit.edu>
Message-ID: <56BE5E62.6080707@gmail.com>
Content-Type: text/plain; charset="windows-1252"
looks like it's run out of memory.
On 11/02/16 23:23, Jasneet Sabharwal wrote:
> Hi,
>
> I was adding a new feature function in Moses
> (https://github.com/KonceptGeek/mosesdecoder/blob/master/moses/FF/CoarseBiLM.cpp).
> It works fine when I test it for 1-2 sentences, but when I?m trying to
> tune my parameters, I?m getting segmentation faults or sometimes it is
> bad_alloc. Following was one of the commands that was executed during
> the tuning process which caused the Segmentation Fault or bad_alloc:
>
> moses -threads 40 -v 0 -config filtered/moses.ini -weight-overwrite
> 'CoarseLM100= 0.075758 LM0= 0.075758 CoarseBiLMNotClustered= 0.075758
> WordPenalty0= -0.151515 PhrasePenalty0= 0.030303 CoarseBiLMClustered=
> 0.075758 TranslationModel0= 0.030303 0.030303 0.030303 0.030303
> Distortion0= 0.045455 CoarseLM1600= 0.075758 LexicalReordering0=
> 0.045455 0.045455 0.045455 0.045455 0.045455 0.045455' -n-best-list
> run1.best100.out 100 distinct -input-file tune.word.lc.cn
> <http://tune.word.lc.cn>
>
> The log is enclosed in this email.
>
> Any pointers would be very useful.
>
> Thanks,
> Jasneet
>
>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
--
Hieu Hoang
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160212/3ecd432c/attachment-0001.html
------------------------------
Message: 4
Date: Fri, 12 Feb 2016 23:36:32 +0100
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: Re: [Moses-support] Access to set of possible target words
from within feature function
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <56BE5E70.6040006@amu.edu.pl>
Content-Type: text/plain; charset=UTF-8; format=flowed
But this happens during hypothesis expansion? I think I need it before.
I have a classifier for which I want to restrict softmaxing over all
available target words for the complete sentence instead over all known
vocabulary items.
I will take a look at your second suggestion.
On 12.02.2016 23:32, Hieu Hoang wrote:
> sounds useful.There's
> FeatureFunction::EvaluateTranslationOptionListWithSourceContext()
> which is called for for every subphrase. It passes the feature
> function a list of translation option for that subphrase. The method
> is called in
> TranslationOptionCollection::EvaluateTranslationOptionListWithSourceContext
>
> Or you add yet another Evaluate() method that passes the feature
> function the InputPathList, which really has every translation options
> in 1 go
>
> Hieu Hoang
> http://www.hoang.co.uk/hieu
>
> On 12 February 2016 at 22:19, Marcin Junczys-Dowmunt
> <junczys@amu.edu.pl <mailto:junczys@amu.edu.pl>> wrote:
>
> Hi everybody,
> Is there a nice way to access the complete set of translation options
> for a sentence from within a feature function? I just need the bag of
> possible words before any processing. Let's say from within
> "EmptyHypothesisState", there is InputType, but I cannot really
> find any
> hook to get that easily.
>
> Thanks,
> Marcin
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
------------------------------
Message: 5
Date: Fri, 12 Feb 2016 22:39:54 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Problem in extracting rules for syntax
based model
To: Khyati Gupta <gkhyati@rocketmail.com>, "moses-support@mit.edu"
<moses-support@mit.edu>, Philipp Koehn <phi@jhu.edu>
Cc: "invincible.roy@gmail.com" <invincible.roy@gmail.com>
Message-ID: <56BE5F3A.6020006@gmail.com>
Content-Type: text/plain; charset="utf-8"
when you extract the rules during training, you must tell the extract
program that there is parse information on the target side
.../extract-rules --TargetSyntax ....
On 08/02/16 10:31, Khyati Gupta wrote:
> Hi,
>
> I have completed string to tree syntax based model. And have converted
> the output of enju parser to format accepted by moses for synatx based
> training.*But the rule table created consist of phrases like <tree
> label ............ as a result the translated text also contains these
> phrases.*
>
> *Output of enju-*
> <sentence id="s0" parse_status="success" fom="0.991145"><cons id="c0"
> cat="NP" xcat="" head="c1" sem_head="c1"
> schema="empty_spec_head"><cons id="c1" cat="NX" xcat="" head="t0"
> sem_head="t0"><tok id="t0" cat="N" pos="NN" base="broadcasting"
> lexentry="[D<N.3sg>]"
> pred="noun_arg0">broadcasting</tok></cons></cons></sentence>
>
> *Converted format of enju to format accepted by moses-*
> <tree label="TOP"> <tree label="NP"> <tree label="NX"> <tree
> label="N"> broadcasting </tree> </tree> </tree> </tree>
>
> *Rule table created-*
> 42800 [X] ||| <tree [SC] ||| 0.00103349 1 0.0103349 1 ||| 0-0 ||| 10 1
> 1 |||
> [X][ADJP] ?????? [X] ||| <tree [X][ADJP] <tree [NP] ||| 0.00729519 1
> 0.00226863 1 ||| 0-1 1-2 ||| 0.485714 1.5619 0.342857 |||
>
> Where am i going wrong?? Is there a problem in XML Format. ??
> Thanks
> Khyati Gupta
> Trainee
> CDAC Noida
--
Hieu Hoang
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160212/54c8eba6/attachment.html
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 112, Issue 24
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 112, Issue 24"
Post a Comment