Moses-support Digest, Vol 145, Issue 3

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: German tokenizer may fail with numeric endings (Ozan ?a?layan)
2. Re: German tokenizer may fail with numeric endings (Ergun Bicici)


----------------------------------------------------------------------

Message: 1
Date: Wed, 7 Nov 2018 11:13:43 +0100
From: Ozan ?a?layan <ozancag@gmail.com>
Subject: Re: [Moses-support] German tokenizer may fail with numeric
endings
To: moses-support@mit.edu
Message-ID:
<CAFub=KRhztmpcPFj5y4smgLDuqEEz0JeCEeK8CLxC8vgzDHcZA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Hieu,

Here is it with some test cases in the commit message:
https://github.com/moses-smt/mosesdecoder/pull/204

Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20181107/0214abaf/attachment-0001.html

------------------------------

Message: 2
Date: Wed, 7 Nov 2018 17:14:08 +0300
From: Ergun Bicici <bicici@gmail.com>
Subject: Re: [Moses-support] German tokenizer may fail with numeric
endings
To: Ozan ?a?layan <ozancag@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAB59qTMz=iQajGF-4LHnQprf08W9i5KCoBRr46R3x4ftJ4KoHw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Could there be a workaround with "?!$" added to the prefixes/suffixes so
that if not at the end of a sentence, they will not be split?

Ergun

On Wed, Nov 7, 2018 at 1:20 PM Ozan ?a?layan <ozancag@gmail.com> wrote:

> Hi Hieu,
>
> Here is it with some test cases in the commit message:
> https://github.com/moses-smt/mosesdecoder/pull/204
>
> Thanks.
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>


--

Regards,
Ergun
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20181107/49c026e5/attachment-0001.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 145, Issue 3
*********************************************

Tweet