Moses-support Digest, Vol 91, Issue 21

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Perplexity KenLM (Miles Osborne)


----------------------------------------------------------------------

Message: 1
Date: Fri, 16 May 2014 13:05:38 -0400
From: Miles Osborne <miles@inf.ed.ac.uk>
Subject: Re: [Moses-support] Perplexity KenLM
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAPRfTYp9c+i2RnzEHFxgowesowLcAQByquLApceUTys-LW_3=A@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

you can get kenlm to report perplexity as follows:

bin/query foo.arpa <text | tail -n 1

note that you need to be careful with OOVs if you are comparing models
that do not all use the same vocabulary.

(SRILM is broekn in this respect in that an OOV will give you a
probability of one)

Miles

--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.


------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 91, Issue 21
*********************************************

Tweet