Moses-support Digest, Vol 86, Issue 81

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Multi-Threaded Moses with Ken-LM (Kenneth Heafield)
2. Re: role of lexical-weighting (Philipp Koehn)
3. Re: problem in tokenization (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Mon, 30 Dec 2013 13:10:31 -0500
From: Kenneth Heafield <moses@kheafield.com>
Subject: Re: [Moses-support] Multi-Threaded Moses with Ken-LM
To: moses-support@mit.edu
Message-ID: <52C1B717.9060107@kheafield.com>
Content-Type: text/plain; charset=ISO-8859-1

Hi,

KenLM supports multiple threads and always has. Where does it say
otherwise?

Kenneth

On 12/30/13 05:09, Roee Aharoni wrote:
> Hi all,
>
> The website says that multi-threaded moses works only with srilm, though
> we want to use it with irstlm and kenlm for memory optimization. Is it
> correct?
>
> Much thanks,
> Roee
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>


------------------------------

Message: 2
Date: Mon, 30 Dec 2013 15:54:19 -0500
From: Philipp Koehn <pkoehn@inf.ed.ac.uk>
Subject: Re: [Moses-support] role of lexical-weighting
To: Andrew <ravenyj@hotmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDCEWbhwntyHmazA5-oFdYa93dEFUS3acX0aQcoioC2Sfg@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1

Hi,

lexical weighting adds translation probabilities based on word translation.
It is a useful feature because it provides some form of backoff or smoothing
for phrase translation probabilities, which are sometimes estimated on
very sparse data.

You can look at the model/lex* files for the scores that are used.
Scoring is based on the word alignment of the extracted phrase
pair.

-phi


On Tue, Dec 17, 2013 at 12:21 AM, Andrew <ravenyj@hotmail.com> wrote:
> Hello,
>
> I am not very sure what lexical-weighting is about.
> Does it take semantic resemblance into consideration? If so, how? And if
> not, how does it differ from usual alignment?
> How would using no-lexical-weighting option in training affect the
> alignments?
>
> I apologize that the question is rather broad, but I I would greatly
> appreciate your advice.
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>


------------------------------

Message: 3
Date: Tue, 31 Dec 2013 14:05:59 +0000
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] problem in tokenization
To: Arththika Paramanathan <arthiparamanathan@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbj0-hAqUDC9jqomWXSqR5F1-Ubd2V=1SCfA0KOuoPo46w@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

I don't really understand your question.

Can you give a example of a
1. (short) untokenized sentence,
2. the command you run to tokenize it,
3. the output.
4. what you prefer the output to be.

If you created a non-breaking prefix file for Tamil, can you also provide
that too so we can reproduce the problem. Also, do you want to add it to
Moses?



On 29 December 2013 16:30, Arththika Paramanathan <
arthiparamanathan@gmail.com> wrote:

> Hi,
> I have a problem in tokenization step. I created a nonbreaking_prefix.ta
> file for Tamil. So, there wasn't show any error message. But, I faced a
> problem in tokenized file because of the characer called zero with joiner
> in Tamil. Where I want to define this character(in tokenizer.perl?)
>
>
>
> --
> regards,
> P.Arththika
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>


--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20131231/a79661ba/attachment-0001.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 86, Issue 81
*********************************************

Tweet