Moses-support Digest, Vol 91, Issue 20

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Perplexity KenLM (Sarah Schulz)


----------------------------------------------------------------------

Message: 1
Date: Fri, 16 May 2014 16:53:54 +0200
From: Sarah Schulz <sarah.schulz@ugent.be>
Subject: [Moses-support] Perplexity KenLM
To: moses-support@mit.edu
Message-ID: <53762682.8050500@ugent.be>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

Hello,

I am working with kenlm models and Moses. I have several different
models and want to determine which model fits my data best before I
train Moses.
I know that there are simple command line calls for eg SRILM to
calculate perplexity but I couldn't find anything equivalent for kenlm.

The only thing I could find is a python function model.score(sentence)
but no information about the score itself.

Does anybody know more about the score function of kenlm? Or does
anybody know another way of calculating perplexity of a kenlm language
model with respect to a test corpus?

Thanks in advance,

Sarah

--
Sarah Schulz
LT3 Language and Translation Technology Team
Department of Translation, Interpreting and Communication
Ghent University

Groot-Brittanni?laan 45
9000 Gent
sarah.schulz@ugent.be



------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 91, Issue 20
*********************************************

Tweet