Moses-support Digest, Vol 135, Issue 30

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: phrase table cleaning (Hieu Hoang)
2. Re: Recommendations for preparing Korean parallel corpora?
(Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Fri, 26 Jan 2018 14:11:37 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] phrase table cleaning
To: jean boyer <tunitrade@yahoo.fr>, "moses-support@mit.edu"
<moses-support@mit.edu>
Message-ID: <251ef79e-96a3-fe87-68fb-fda53e95f4a5@gmail.com>
Content-Type: text/plain; charset="utf-8"

the program in

?? contrib/sigtest-filter

does some clean, for example, keeping the best 20 target phrase for each
source phrase. More details here

??? http://www.statmt.org/moses/?n=Advanced.RuleTables

If that's not what you want you may have to write your own.


On 16/01/18 09:23, jean boyer wrote:
> "I would like edit my message "https://www.mail-archive.com/moses-support@mit.edu/msg15788.html" because the accompanying image was not posted."
> Hi,
> I would like clean my phrase table fr-en.
> There are any solution to clean phrase table and keep only the bests translations.
> like in the example below i want keep only the lines that begin with an arrow?
>
> -----------------------------------------------------------------------
> sur ce sujet?? ??? ??? ?a?? ??? ??? ?1.21619e-05 2.15066e-09 0.111111
> 0.0117291?? ??? ??? ?1-0 ??? ?82224 9 1
> =>sur ce sujet?? ??? ??? ?on this subject ?0.5 0.0104788 0.111111
> 0.00456608?? ??? ??? ?0-0 1-1 2-2?? ??? ??? ?2 9 1
> =>sur ce sujet?? ??? ??? ?on this topic ?0.2 0.0416959 0.111111
> 0.00307945?? ??? ??? ?0-0 1-1 2-2?? ??? ??? ?5 9 1
> sur ce sujet?? ??? ??? ?rebellion against ?0.0909091 9.07596e-09
> 0.111111 1.53269e-08 ?0-1?? ??? ??? ?11 9 1
>
> -------------------------------------------------------------------------------------
> Thank you !
> Eric
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

--
Hieu Hoang
http://moses-smt.org/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20180126/0bce005a/attachment-0001.html

------------------------------

Message: 2
Date: Fri, 26 Jan 2018 14:04:56 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Recommendations for preparing Korean
parallel corpora?
To: daideqi <daideqi@yahoo.com>, Moses-support <moses-support@mit.edu>
Message-ID: <63854338-6300-1efc-b05e-993db2df9fae@gmail.com>
Content-Type: text/plain; charset="utf-8"

there was a recent checkin for Korean

https://github.com/moses-smt/mosesdecoder/commit/194964c017d8acb56918bab94f4d7cdd60b9c9b7

Maybe there are also some Korean or Asian-specific tools out there


On 17/01/18 01:01, daideqi wrote:
> Dear Moses-Support,
>
> Some colleagues and I, who are all new to SMT and Moses, have some
> Korean parallel corpora that we want to use to train Moses.
>
> My question is how do we go about preparing/tokenizing the data, and
> can you recommend any specific tools?? I searched the moses-support
> archive and the Interwebs and couldn't find any specific
> recommendations or step-by-step instructions for newbs like us.
>
> We'd be very grateful if you could point us in the right direction.
>
> Thanks in advance!
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

--
Hieu Hoang
http://moses-smt.org/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20180126/af06e3b7/attachment-0001.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 135, Issue 30
**********************************************

Tweet