Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: Problems with segmentation mismatch and many unknown
words for Chinese translation (Vincent_hotmail)
2. Re: Problems with segmentation mismatch and many unknown
words for Chinese translation (Hieu Hoang)
3. Re: Get the probability of a given n-gram in a language model
(Albert Llorens)
4. Coling 2014 - List of Accepted Papers & 1 week to Early
Registration Deadline (COLING 2014 - Registration)
----------------------------------------------------------------------
Message: 1
Date: Fri, 30 May 2014 10:47:06 +0800
From: Vincent_hotmail <Vincentwang0229@hotmail.com>
Subject: Re: [Moses-support] Problems with segmentation mismatch and
many unknown words for Chinese translation
To: <gemdbw@hotmail.com>
Cc: moses-support@mit.edu
Message-ID: <SNT147-DS63114D082967CC072115CB4270@phx.gbl>
Content-Type: text/plain; charset="us-ascii"
Hi Gideon,
I recently also came across the similar on training Chinese-other language
pairs. I wonder if you use the latest version of Moses. I firstly use the
Stanford, NLPIR or my own segmenter to tokenize the sentences, and then use
escape-special-chars.perl < input.seg.zh > out.zh to process some special
chars in them. Finally, the problem seems to be solved. But I never come
across the same problem when using old version of Moses. If you have not
solve it, pls try this one.
Best,
Vincent
May 30, 2014
----------------
Longyue WANG, Vincent
Research Assistant @ NLP2CT
Postgraduate @ University of Macau
Tel: (+853) 8397-8051
Homepage: <http://nlp2ct.cis.umac.mo/~vincent/>
http://nlp2ct.cis.umac.mo/~vincent/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140530/f6a7ee06/attachment-0001.htm
------------------------------
Message: 2
Date: Fri, 30 May 2014 05:11:02 +0100
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] Problems with segmentation mismatch and
many unknown words for Chinese translation
To: Vincent_hotmail <Vincentwang0229@hotmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbjVHU_HxZR2rxw8i=y6=C-JpRwert9MiCDCVxAy7ZeA6w@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
was it due to the new version of Moses? It shouldn't be, if this is the
cause please tell me urgently
On 30 May 2014 03:47, Vincent_hotmail <Vincentwang0229@hotmail.com> wrote:
> Hi Gideon,
>
>
>
> I recently also came across the similar on training Chinese-other language
> pairs. I wonder if you use the latest version of Moses. I firstly use the
> Stanford, NLPIR or my own segmenter to tokenize the sentences, and then use *escape-special-chars.perl
> < input.seg.zh > out.zh* to process some special chars in them. Finally,
> the problem seems to be solved. But I never come across the same problem
> when using old version of Moses. If you have not solve it, pls try this one.
>
>
>
> Best,
>
> Vincent
>
> May 30, 2014
>
>
>
> ----------------
>
> Longyue WANG, Vincent
>
> Research Assistant @ NLP2CT
>
> Postgraduate @ University of Macau
>
> Tel: (+853) 8397-8051
>
> Homepage: http://nlp2ct.cis.umac.mo/~vincent/
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140530/77ed6c80/attachment-0001.htm
------------------------------
Message: 3
Date: Fri, 30 May 2014 08:36:38 +0000
From: Albert Llorens <albert.llorens@lucysoftware.com>
Subject: Re: [Moses-support] Get the probability of a given n-gram in
a language model
To: Kenneth Heafield <moses@kheafield.com>, "moses-support@mit.edu"
<moses-support@mit.edu>
Message-ID:
<833DD166FFE895458E5E7BE06CDEECD42C76338D@MUC-EX.lucysoftware.com>
Content-Type: text/plain; charset="us-ascii"
Excellent. Thanks a lot Kenneth.
Albert
-----Original Message-----
From: Kenneth Heafield [mailto:moses@kheafield.com]
Sent: lunes, 26 de mayo de 2014 20:05
To: Albert Llorens; moses-support@mit.edu
Subject: Re: [Moses-support] Get the probability of a given n-gram in a language model
Hi,
Here's a cheap server for fragment scoring.
socat TCP4-LISTEN:2000,fork EXEC:"bin/fragment lm/test.arpa"
Then in another terminal
socat TCP4-CONNECT:localhost:2000 STDIO <text
Now if you want to translate fragments instead then that's what Moses is for, though keep in mind that it will always prepend <s> and append </s> for translation.
Kenneth
On 05/26/14 02:04, Albert Llorens wrote:
> Thanks, Kenneth.
>
> Yes, I want to score sentence fragments. I want to use Moses for fragment translation, but only for frequent or probable fragments. I'll try what you suggest. Any chance the query could be done remotely, using mosesserver or anything else?
>
> Kind regards.
>
> Albert
>
>
> -----Original Message-----
> From: moses-support-bounces@mit.edu
> [mailto:moses-support-bounces@mit.edu] On Behalf Of Kenneth Heafield
> Sent: viernes, 23 de mayo de 2014 17:34
> To: moses-support@mit.edu
> Subject: Re: [Moses-support] Get the probability of a given n-gram in
> a language model
>
> Hi,
>
> You can use bin/query on an ARPA or KenLM file. Then just type sentences at it (or use a file as stdin). By default it will assume you are scoring sentences. You can pass -n to not wrap in <s> and </s>.
>
> It appears that you are asking to score sentence fragments. The
> leading words will be scored using unigrams, bigrams, etc. from, say,
> a 5-gram model. If you are using Kneser-Ney, these lower-order
> probabilities (unigrams through 4-grams) are conditioned on having
> backed off to them. If you want accurate scores for sentence
> fragments, build a model of order 1, order 2, order 3, etc. then
> combine them using
>
> build_binary -r "1.arpa 2.arpa 3.arpa 4.arpa" 5.arpa 5.rest
>
> You can then use
>
> bin/fragment 5.rest <fragments
>
> to attain log10 frequencies. For more on this rant, read
>
> http://kheafield.com/professional/edinburgh/rest_paper.pdf
>
> Kenneth
>
> On 05/23/14 05:13, Albert Llorens wrote:
>> Hi,
>>
>>
>>
>> Is there a straightforward way I can ask Moses for the probability
>> (or the frequency) of a given n-gram in a given language model? If
>> so, can I do the query through mosesserver?
>>
>>
>>
>> Thanks.
>>
>>
>>
>> Kind regards.
>>
>>
>>
>> Albert
>>
>>
>>
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
------------------------------
Message: 4
Date: Fri, 30 May 2014 10:12:16 +0000
From: COLING 2014 - Registration <COLING2014Reg@keynotepco.ie>
Subject: [Moses-support] Coling 2014 - List of Accepted Papers & 1
week to Early Registration Deadline
To: COLING 2014 - Registration <COLING2014Reg@keynotepco.ie>
Message-ID:
<76a4492bb9f541e5be97975a959bf515@AM3PR03MB546.eurprd03.prod.outlook.com>
Content-Type: text/plain; charset="us-ascii"
Trouble viewing this email? View in Browser<http://www.coling-2014.org/accepted_papers_email.html>
[http://www.coling-2014.org/img/email_header.png]
[http://www.coling-2014.org/img/coling_logo.png]<http://www.coling-2014.org/index.php>
View List of
Accepted Papers Here<accepted-papers.php>
Early Registration Deadline
June 6th 2014
Click here to Register<http://www.coling-2014.org/registration.php>
Register for Main Conference<http://www.coling-2014.org/schedule.php>,
1 or 2 day Workshops<http://www.coling-2014.org/workshops.php> and half day Tutorials!<http://www.coling-2014.org/tutorials.php>
[http://www.coling-2014.org/img/email_1.jpg]
Our Sponsors
[http://www.coling-2014.org/img/sponsors_email.jpg]<http://www.coling-2014.org/sponsors.php>
Ireland Inspires!
Click here to see the Ireland Inspires Video<https://www.youtube.com/watch?v=9wmIZEl1nSo&feature=youtu.be>
Coling 2014
www.coling2014.org<http://www.coling-2014.org/index.php> | coling2014reg@keynotepco.ie | View in Browser<http://www.coling-2014.org/accepted_papers_email.html>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140530/9cc7da2d/attachment.htm
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 91, Issue 51
*********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 91, Issue 51"
Post a Comment