Moses-support Digest, Vol 110, Issue 20

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: MERT's Powell Search (Adam Lopez)
2. Same story again: lm integration into moses (hopefully the
last time) (koormoosh)


----------------------------------------------------------------------

Message: 1
Date: Thu, 10 Dec 2015 09:29:54 +0000
From: Adam Lopez <alopez@inf.ed.ac.uk>
Subject: Re: [Moses-support] MERT's Powell Search
To: liling tan <alvations@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAE-ScguSVfmoO0tObAdK87C5t99Z0Ux_HpOWFP2WyRb5nxe-zA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Liling ?


> We are going through the slides for MT tuning on
> http://mt-class.org/jhu/slides/lecture-tuning.pdf and we couldn't figure
> out what does "?ai + bi" on slide 31 refer to.
>
> What are the values for "ai" and "bi"? Are they numbers from the nbest
> list?
>

[For clarity, I'm going to change the notation slightly here: slide 23 uses
? to refer to the parameter vector (indexed by i), while slide 31 uses it
to refer to a single parameter (i.e. an element of this vector). This is
confusing. Let's use ? as the parameter vector, |?| as its length, and ?j as
its j-th element, which is what we're optimizing in slides 31-36 (since i
is already used on slide 31 to index elements of the n-best list, ?i would
be confusing). I'm also going to use g(ei|f) rather than p(ei|f) since this
is just a linear model; we aren't doing probabilistic inference here.]

We're going to compute g(ei|f) as a function of a single parameter ?j while
holding all other parameters fixed. This is just:

g(ei|f) = ?k? 1,...,|?| ?khk(ei,f) = ?jhj(ei,f) + ?k? 1,...,j-1,j+1,...,|?|
?khk(ei,f)

Hence ai = hj(ei,f) and bi = ?k? 1,...,j-1,j+1,...,|?| ?khk(ei,f). In other
words, ai is just the value of the j-th feature on the i-th element of the
n-best list, and bi is the model score according to all other features and
weights.

According to the algorithm on slide 37 of
> http://mt-class.org/jhu/slides/lecture-tuning.pdf, is line 6 where
> the ?ai + bi computation occurs?
>
> compute line l: parameter value ? score
>
>
Yes.

>From the nbest list we have lines as such:
>
> 0 ||| including options , ????? buy 20 ?????? planes , ??????????? volume
> - ? ??? 26 ?????????? ???????? . ||| LexicalReordering0= -3.12525 0 0
> -7.34869 0 0 Distortion0= 0 LM0= -111.207 WordPenalty0= -18 PhrasePenalty0=
> 17 TranslationModel0= -12.8271 -8.45991 -11.4888 -11.3076 ||| -746.163
>
> Let's say we are tuning the first parameter for LexicalReodering0 for this
> sentence, is it that we only calculate:
>
> ? -3.12525 * -746.163
>
>
> Is ai = 3.12525 for this sentence? Is bi = -746.163? What is bi suppose to
> be?
>

>From the above, b_i is a function of the remaining features and weights; so
you need to know your current weight vector to compute it.

-A
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151210/af758691/attachment-0001.html
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151210/af758691/attachment-0001.pl

------------------------------

Message: 2
Date: Thu, 10 Dec 2015 23:13:00 +1100
From: koormoosh <koormoosh@gmail.com>
Subject: [Moses-support] Same story again: lm integration into moses
(hopefully the last time)
To: moses-support@mit.edu
Message-ID:
<CAN3_CDhLK-RYZCCi0_DF64aDJv-kXsCwJ7My0+qa2GtdGox7TA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi all,

It would be great if the experts (who are familiar with the decoder, and
the srilm integration) could have a look at what I have added in below as
comments to the GetValue function of SRI.cpp, and answer the left questions
(in bold fonts) and add possible corrections. I think this will make things
lot more clear for everyone who is planning to do the integration. So I
really appreciate it if you go over this, spend a few minutes and provide
me your precise feedback. please read all the comments to make sure we are
on the same page.

Thanks, -K.

>From SRI.cpp:

//K: contextFactor is the sequence of words we want to assign a probability
to. Example: "The dog chases"
//K: what is *finalState when "The dog chases" is sent to this function?
LMResult LanguageModelSRI::GetValue(const vector<const Word*>
&contextFactor, State* finalState) const
{
...

//K: count is the size of the sequence. In this example, count is 3
size_t count = contextFactor.size();
...

//K: ngram is an empty array of size 4 in this example
VocabIndex ngram[count + 1];

//K: fills ngram array using "part" of the contextFactor
for (size_t i = 0 ; i < count - 1 ; i++) {
ngram[i+1] = GetLmID((*contextFactor[count-2-i])[factorType]);
}
//K: break-down of the loop:
ngram[0+1] = contextFactor[3-2-0] = dog
ngram[1+1] = contextFactor[3-2-1] = the
//K: the ngram array after the for-loop is: [ , dog, the, ]
//K: very weird ordering for ngram.


* //K: what is the use of Vocab_None?*
ngram[count] = Vocab_None;
//K: ngram after the above steps is: [ , dog, the, Vocab_None ]

...

//K: lmId contains the id of the last word of the sequence which in this
case is the id for "chases"
VocabIndex lmId = GetLmID((*contextFactor[count-1])[factorType]);

//K: getting the probability of the last word "chases", given the context
stored in ngram array [ , dog, the, Vocab_None ]
//K: the ngram+1 is to ignore the empty cell in the beginning of the
ngram array.
ret = GetValue(lmId, ngram+1);

//K: if finalState is not zero.
*//K: What does it mean for finalState not to be zero?*
if (finalState) {
//K: Now the first empty cell of the ngram array gets filled with
"chases"
ngram[0] = lmId;
//K: ngram array is now [chases, dog, the, Vocab_None]

* //K: what is this?*
unsigned int dummy;

//K: an id for the full sequence "the dog chases" is being returned by
srilm.
//K: Id returned by srilm and finalState seems to be the same thing.
*finalState = m_srilmModel->contextID(ngram, dummy);
}

//K: So the function takes a sequence, and returns a score for the
sequence and updates the finalState
//K: now if we call this function again with "The dog chases her", the
finalState basically holds the id for "The dog chases".
*//K: Is this correct?*

return ret;
}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151210/236b7973/attachment-0001.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 110, Issue 20
**********************************************

Tweet