Moses-support Digest, Vol 122, Issue 43

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Moses-support Digest, Vol 122, Issue 38 (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Fri, 30 Dec 2016 16:30:57 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Moses-support Digest, Vol 122, Issue 38
To: Mike Ladwig <mdladwig@gmail.com>
Cc: Moses Support <moses-support@mit.edu>
Message-ID:
<CAEKMkbg6yfJaXbhzkFumji37T5m4N5a9xW0teixMC9kwHJL-_Q@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

there's some gunk in the training data eg. non-printing characters,
trailing/prefix spaces, double spaces/tabs, non-utf8. You'll find out as
soon as you look at the sentences it refers to

Hieu Hoang
http://moses-smt.org/

On 30 December 2016 at 16:25, Mike Ladwig <mdladwig@gmail.com> wrote:

> On Wed, Dec 28, 2016 at 4:37 AM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>
>> I am getting significantly (~20%) lower bleu scores than with 2.x but I
>>> have a lot of testing before I will know why.
>>>
>> Moses and Moses2 should give very similar results. Please let me know
>> what you find
>>
>
> In looking at training logs, I am getting many messages like this:
>
> WARNING: sentence 540930 has alignment point (4, 3) out of bounds (4, 4)
> T: europe is changing .
> S: europa verandert sich .
> WARNING: sentence 540931 has alignment point (9, 5) out of bounds (9, 10)
> T: that was the slogan of the last european elections .
> S: das war das motto der letzten europa wahlen .
> WARNING: sentence 540932 has alignment point (6, 0) out of bounds (6, 6)
> T: personally , i am convinced .
> S: personlich stimme ich dem zu .
>
> Thoughts?
> mike.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161230/2c5815dd/attachment-0001.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 122, Issue 43
**********************************************

Tweet