Moses-support Digest, Vol 110, Issue 17

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: FinalState in Moses decoder (David Madl)


----------------------------------------------------------------------

Message: 1
Date: Tue, 8 Dec 2015 11:26:38 +0000
From: David Madl <david@madlnet.net>
Subject: Re: [Moses-support] FinalState in Moses decoder
To: moses-support@mit.edu
Message-ID: <5666BE6E.6070502@madlnet.net>
Content-Type: text/plain; charset="windows-1252"

Hi Huyen,

Look at GetValueGivenState() in LM/Implementation.h, and you see that
"finalState" is a language model state (actually both input state before
and output state after the scoring of a word). An LM state represents
the limited n-gram history of words at a specific Hypothesis some way
through the sentence. The class LanguageModelSRI does not use the input
state, but only produces an output state.

The state eventually ends up being FFState, which is used to compare
(FFState::operator==) and recombine Hypotheses with the same
(limited-size: (n-gram order) - 1) history of words, which makes search
faster.

For SRILM, the state is PointerState created via
LanguageModelSingleFactor::NewState(), and thus seems to work by SRILM
returning the same pointer every time you call Ngram::contextID() with
the same words passed to it. Disclaimer: I do not know SRILM in detail.

The LM scores are log (base e) probabilities.

Hope this helps,
David


On 05.12.2015 04:32, Vu Thuong Huyen wrote:
>
> Hi all,
>
> I?m integrating my LM in Mosesdecoder. I followed the SRI.cpp and
> NeuralLMWrapper.cpp. I don?t know how to assign value for *finalState
> variable. Could you explain for me how does it affect when decoding? I
> logged out the context when the system call ?GetValue? function in my
> LM and in SRILM LM, the number of contexts were different.
>
>
>
> LMResult LanguageModelSRI::GetValue(VocabIndex wordId, VocabIndex
> *context) const
>
> {
>
> LMResult ret;
>
> ret.score = FloorScore(TransformLMScore(m_srilmModel->wordProb(
> wordId, context)));
>
> ret.unknown = (wordId == m_unknownId);
>
> return ret;
>
> }
>
>
>
>
> Which is value of *ret.score*? Log Probability or Probability?
>
>
>
> Best Regards,
>
> Huyen.
>
>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151208/1d3481b8/attachment-0001.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 110, Issue 17
**********************************************

Tweet