Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. SMT decoding complexity (amir haghighi)
2. Re: Access to Key-Value Pairs in Phrase Table? (Hieu Hoang)
3. Re: Access to Key-Value Pairs in Phrase Table? (Shuoyang Ding)
----------------------------------------------------------------------
Message: 1
Date: Sun, 26 Feb 2017 21:46:05 +0330
From: amir haghighi <amir.haghighi.tehrani@gmail.com>
Subject: [Moses-support] SMT decoding complexity
To: Moses-support <moses-support@mit.edu>
Message-ID:
<CAPTzi0fMN8MFNJfhtZmKBPAj_WcrZ40vJQUifC5ua+516svz0Q@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi all,
In the Moses manual and also in SMT textbooks it is mentioned that the
decoding complexity for PB-SMT is exponential in the source sentence
length. If we have a source sentence with length n, in decoding by
hypothesis expansion, we have power(2,n) state, each of them can be
reordered in n! orders, and each state can be translated in power(T,n),
where T is the number of translation options, right?
so the decoder complexity is power(2,n)*n!*power(T,n), so why its mentioned
that the complexity is exponential?
Could someone please explain for me how the decoder complexity is
calculated?
I've read the Knight(1999) paper, but I couldn't understand it. Could you
please introduce another reference?
Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170226/1cdd3d64/attachment-0001.html
------------------------------
Message: 2
Date: Sun, 26 Feb 2017 20:18:17 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Access to Key-Value Pairs in Phrase
Table?
To: Shuoyang Ding <mtsding@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbh3AVpJhAuBjRZUoZXhqHfBMkT40UpcHcujMs2iv=r_0g@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hiya
Hieu Hoang
http://moses-smt.org/
On 25 February 2017 at 23:22, Shuoyang Ding <mtsding@gmail.com> wrote:
> Hi All,
>
> In http://www.statmt.org/moses/?n=FactoredTraining.ScorePhrases, it was
> mentioned that there is a key-value pair field at the end of each entry of
> the phrase table. Now I have a phrase table that looks like the following:
>
> , [X] ||| to the outside world [PP] ||| 0.00202802 0.0962381 1.14287e-06
> 7.14872e-10 0.00284822 0.367879 ||| 0-0 ||| 6 10647 1 ||| ||| {{Tree [PP
> [TO to] [NP [DT the] [JJ outside] [NN world]]]}}
>
> where the last field looks like the key-value pair field as described.
>
> My question is the following:
>
> 1. how can I get access to the contents in this field? e.g. suppose tp is
> a TargetPhrase instance, is tp->GetProperty(?Tree?) the right way to do
> it?
>
yep
> 2. is the content of this field always loaded when initializing the
> decoder?
>
I think so, however, you must use PhraseDictionary or
PhraseDictionaryOnDisk. The other phrase-table implementation doesn't
support it
>
> Thanks!
>
> Regards,
> Shuoyang Ding
>
> Ph.D. Student
> Center for Language and Speech Processing
> Department of Computer Science
> Johns Hopkins University
>
> Hackerman Hall 225A
> 3400 N. Charles St.
> Baltimore, MD 21218
>
> http://cs.jhu.edu/~sding
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170226/5002ad4c/attachment-0001.html
------------------------------
Message: 3
Date: Sun, 26 Feb 2017 16:11:39 -0500
From: Shuoyang Ding <mtsding@gmail.com>
Subject: Re: [Moses-support] Access to Key-Value Pairs in Phrase
Table?
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <E1FFA77F-F632-4F70-87F7-DF456DE0914E@gmail.com>
Content-Type: text/plain; charset="utf-8"
Thanks, Hieu. Here is the answer I ended up finding myself. Please correct me if I?m wrong.
1. yes.
2. I cannot say about other properties, but for Tree property, ChartParser.cpp:120-121 listed several conditions where this property will be loaded:
if (!options()->output.detailed_tree_transrep_filepath.empty() ||
options()->nbest.print_trees || staticData.GetTreeStructure() != NULL) {
...
targetPhrase->SetProperty("Tree", prop);
}
which seems to mean unless you are using TreeStructureFeature, or you turned on some specific options, you cannot access this property through TargetPhrase.
(That?s actually my ?real? problem last night ? tp->GetProperty(?Tree?) returned a null pointer, and adding TreeStructureFeature solved it).
Shuoyang
> On Feb 26, 2017, at 3:18 PM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>
> Hiya
>
> Hieu Hoang
> http://moses-smt.org/ <http://moses-smt.org/>
>
> On 25 February 2017 at 23:22, Shuoyang Ding <mtsding@gmail.com <mailto:mtsding@gmail.com>> wrote:
> Hi All,
>
> In http://www.statmt.org/moses/?n=FactoredTraining.ScorePhrases <http://www.statmt.org/moses/?n=FactoredTraining.ScorePhrases>, it was mentioned that there is a key-value pair field at the end of each entry of the phrase table. Now I have a phrase table that looks like the following:
>
> , [X] ||| to the outside world [PP] ||| 0.00202802 0.0962381 1.14287e-06 7.14872e-10 0.00284822 0.367879 ||| 0-0 ||| 6 10647 1 ||| ||| {{Tree [PP [TO to] [NP [DT the] [JJ outside] [NN world]]]}}
>
> where the last field looks like the key-value pair field as described.
>
> My question is the following:
>
> 1. how can I get access to the contents in this field? e.g. suppose tp is a TargetPhrase instance, is tp->GetProperty(?Tree?) the right way to do it?
> yep
> 2. is the content of this field always loaded when initializing the decoder?
> I think so, however, you must use PhraseDictionary or PhraseDictionaryOnDisk. The other phrase-table implementation doesn't support it
>
> Thanks!
>
> Regards,
> Shuoyang Ding
>
> Ph.D. Student
> Center for Language and Speech Processing
> Department of Computer Science
> Johns Hopkins University
>
> Hackerman Hall 225A
> 3400 N. Charles St.
> Baltimore, MD 21218
>
> http://cs.jhu.edu/~sding <http://cs.jhu.edu/~sding>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support <http://mailman.mit.edu/mailman/listinfo/moses-support>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170226/6ca24dca/attachment.html
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 124, Issue 28
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 124, Issue 28"
Post a Comment