Moses-support Digest, Vol 101, Issue 59

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Chinese segmentation/tokenization (Jeremy Gwinnup)
2. Re: Chinese segmentation/tokenization
(????????? ????? (Ventsislav Zhechev))


----------------------------------------------------------------------

Message: 1
Date: Fri, 20 Mar 2015 09:39:52 -0400
From: Jeremy Gwinnup <jeremy@gwinnup.org>
Subject: Re: [Moses-support] Chinese segmentation/tokenization
To: moses-support@mit.edu
Message-ID: <B57F230C-B863-415B-90E5-0E1726139420@gwinnup.org>
Content-Type: text/plain; charset=utf-8

We?ve had reasonable luck with the Stanford Chinese segmenter - I think the ctb model did better than the pku one for our use case

> Message: 2
> Date: Fri, 20 Mar 2015 13:19:02 +0100
> From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
> Subject: [Moses-support] Chinese segmentation/tokenization
> To: Moses Support <moses-support@mit.edu>
> Message-ID: <e4d171cb90994cb853a9965facaebc37@amu.edu.pl>
> Content-Type: text/plain; charset="us-ascii"
>
>
>
> Hi,
>
> questions appear from time to time on the list concerning Chinese
> segmentation/tokenization. I saw Barry mention Lingpipe and other tools.
> Is there a favourite tool you guys prefer to use over others?
>
> Thanks,
>
> Marcin




------------------------------

Message: 2
Date: Fri, 20 Mar 2015 14:43:32 +0100
From: "????????? ????? (Ventsislav Zhechev)"
<contact@VentsislavZhechev.eu>
Subject: Re: [Moses-support] Chinese segmentation/tokenization
To: moses-support@mit.edu
Message-ID:
<C520C880-35DA-4A62-AA85-C4D811818B40@VentsislavZhechev.eu>
Content-Type: text/plain; charset="utf-8"

Hi Marcin,

At Autodesk we?ve been successfully using KyTea since 2011. The main reason we chose this specific tool is that it has readily available models for both Chinese and Japanese, which simplified the integration in our workflows.
At least for Japanese, we also evaluated Mecab in 2011, but found KyTea to serve us better.

Keep in mind, though, that we are not very interested in the quality of the segmentation per se; instead we need the MT to be of sufficient quality, regardless if what the segmentation tool does makes sense on its own or not.


Cheers,

Ventzi

???????
Dr. Ventsislav Zhechev
Computational Linguist, Certified ScrumMaster?
Platform Architecture and Technologies
Localisation Services

MAIN +41 32 723 91 22
FAX +41 32 723 93 99

http://VentsislavZhechev.eu

Autodesk, Inc.
Rue de Puits-Godet 6
2000 Neuch?tel, Switzerland
www.autodesk.com




> 20.03.2015 ?., ? 14:32, moses-support-request@mit.edu ???????(?):
>
> Date: Fri, 20 Mar 2015 13:19:02 +0100
> From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
> Subject: [Moses-support] Chinese segmentation/tokenization
> To: Moses Support <moses-support@mit.edu>
> Message-ID: <e4d171cb90994cb853a9965facaebc37@amu.edu.pl>
> Content-Type: text/plain; charset="us-ascii"
>
>
>
> Hi,
>
> questions appear from time to time on the list concerning Chinese
> segmentation/tokenization. I saw Barry mention Lingpipe and other tools.
> Is there a favourite tool you guys prefer to use over others?
>
> Thanks,
>
> Marcin

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150320/93fd338f/attachment.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 14277 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20150320/93fd338f/attachment.jpg

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 101, Issue 59
**********************************************

Tweet