Home » Uncategories » Moses-support Digest, Vol 125, Issue 52

Moses-support Digest, Vol 125, Issue 52

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Error in mteval-v13a.pl (liling tan)
2. Re: Error in mteval-v13a.pl (liling tan)
3. Re: Error in mteval-v13a.pl (Dingyuan Wang)


----------------------------------------------------------------------

Message: 1
Date: Wed, 29 Mar 2017 00:36:24 +0800
From: liling tan <alvations@gmail.com>
Subject: Re: [Moses-support] Error in mteval-v13a.pl
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAKzPaJLw6wHuOt3hn37VE0QjaKhSWyCVVe3bwwCzdxQ=vVaNYw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Dingyuan, Hieu,

Thanks for highlighting the issue.

The deprecation warning from mteval has been there since early 2015 on
https://www.mail-archive.com/moses-support@mit.edu/msg12057.html

The fix at https://github.com/moses-smt/mosesdecoder/pull/170 was adhering
to unicode annex on 2016-06-01.

Actually \p{Line_Break:Hyphen} is a little more restricting than
\p{Line_Break}.
Technically, if the script didn't break with \p{Line_Break:Hyphen} , the
superset \p{Line_Break} shouldn't break. Perl unicode properties
documentation has the detailed explanations:
http://perl11.org/pod/perluniprops.html

The current mteval-v13a.pl works fine on:
- Ubuntu 16.04.01 with perl v5.22.1
- Mac OS Sierra with perl v5.18.2

@Dingyuan, a couple of short questions.

(i) Which OS are you using? Which version of the OS?

(ii) What is your perl version? use command "perl -v"

That'll help us to know which OS and/or perl version caused that error.

Regards,
Liling


On Tue, Mar 28, 2017 at 8:52 PM, Hieu Hoang <hieuhoang@gmail.com> wrote:

> If you find that {Line_Break:Hyphen} works, please consider checking it in.
>
> These compatibility issues are difficult to debug alone and depends on the
> exact perl/OS version you're running. Your fix will add a little to the
> body of knowledge
>
> * Looking for MT/NLP opportunities *
> Hieu Hoang
> http://moses-smt.org/
>
>
> On 28 March 2017 at 02:48, Dingyuan Wang <abcdoyle888@gmail.com> wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA256
>>
>> Recently mteval-v13a.pl stopped working, printing:
>>
>> Can't find Unicode property definition "Line_Break" in regex; marked
>> by <-- HERE in m/\p{Line_Break} <-- HERE \p{Zl}/ at
>> /home/gumble/software/moses/scripts/generic/mteval-v13a.pl line 953.
>>
>> I see this commit
>> <https://github.com/moses-smt/mosesdecoder/commit/c6c3bc84b7673618f37948
>> 2cbc6b708f55a9ecd3
>> <https://github.com/moses-smt/mosesdecoder/commit/c6c3bc84b7673618f379482cbc6b708f55a9ecd3>
>> >.
>> I found that changing this to \p{Line_Break: Hyphen} worked. Is this
>> the equivalent of \p{Hyphen}?
>>
>> - --
>> Dingyuan Wang
>> -----BEGIN PGP SIGNATURE-----
>>
>> iQEzBAEBCAAdFiEEjE4PLbCEqfvlC0rjs+TYPj8+X9wFAljZwNYACgkQs+TYPj8+
>> X9xShggAhSjPEEYXsiRPT9wVljRV7XjBmexe/E7EKzl9b/PEnuxlSNSrz/0Estr5
>> 8/H4s+lwKdv9xx1jTxOGOVkToiVC95QkuppXX3WS+BCDjajE8fqWc2Y0IhUWRaqf
>> PAhhotEZmoWAhQC/qVM7lILf29N9OhQ2FStQH9rn+LpD2dkSZweZ0XGJ+CFpCdaP
>> VA7XPWJCJZeEBUsBqrSxl1Cwzr+KQ4pw/NFP6yxJ+smmTkUSyp2FfYCtvalx/L0d
>> 2UZ1fiujzco7NHeJW/0ZYwsb+NNMuM7CljBMhQAWIN+D0f6Wz1/bHH8jhFyxUw0B
>> +/hN/chrAmYX+Kz2j/MKc7eXZuPtmA==
>> =C+jT
>> -----END PGP SIGNATURE-----
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170328/533678f5/attachment-0001.html

------------------------------

Message: 2
Date: Wed, 29 Mar 2017 00:53:33 +0800
From: liling tan <alvations@gmail.com>
Subject: Re: [Moses-support] Error in mteval-v13a.pl
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAKzPaJJdCcN3fyYYd02ivUN+LD_EQsbeO5SNpWv4mKQ1VapX9A@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Interesting, I've just took a look at the perluniprop documentation
carefully, generic \p{Line_Break} didn't exist but Ubuntu and Mac OS let me
slip pass without error/warning. I supposed \p{Line_Break} automatically
goes to \p{Line_Break: Alphabetic} or \p{Line_Break: Ambiguous}

Maybe the perluniprop handling is different for different machines/OS/perl
versions.

\p{Line_Break: Hyphen} might be closer to \p{Hyphen} than
alphbetic/ambiguous \p{Line_Break}


I think we can change it to \p{Line_Break: Hyphen} safely.But first,
@Dingyuan, please do tell us your OS and perl versions and see whether
it's just that the newer perl version handles the \p{Line_Break}
automatically while the older perl doesn't


On Wed, Mar 29, 2017 at 12:36 AM, liling tan <alvations@gmail.com> wrote:

> Hi Dingyuan, Hieu,
>
> Thanks for highlighting the issue.
>
> The deprecation warning from mteval has been there since early 2015 on
> https://www.mail-archive.com/moses-support@mit.edu/msg12057.html
>
> The fix at https://github.com/moses-smt/mosesdecoder/pull/170 was
> adhering to unicode annex on 2016-06-01.
>
> Actually \p{Line_Break:Hyphen} is a little more restricting than \p{Line_Break}.
> Technically, if the script didn't break with \p{Line_Break:Hyphen} ,
> the superset \p{Line_Break} shouldn't break. Perl unicode properties
> documentation has the detailed explanations: http://perl11.
> org/pod/perluniprops.html
>
> The current mteval-v13a.pl works fine on:
> - Ubuntu 16.04.01 with perl v5.22.1
> - Mac OS Sierra with perl v5.18.2
>
> @Dingyuan, a couple of short questions.
>
> (i) Which OS are you using? Which version of the OS?
>
> (ii) What is your perl version? use command "perl -v"
>
> That'll help us to know which OS and/or perl version caused that error.
>
> Regards,
> Liling
>
>
> On Tue, Mar 28, 2017 at 8:52 PM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>
>> If you find that {Line_Break:Hyphen} works, please consider checking it
>> in.
>>
>> These compatibility issues are difficult to debug alone and depends on
>> the exact perl/OS version you're running. Your fix will add a little to the
>> body of knowledge
>>
>> * Looking for MT/NLP opportunities *
>> Hieu Hoang
>> http://moses-smt.org/
>>
>>
>> On 28 March 2017 at 02:48, Dingyuan Wang <abcdoyle888@gmail.com> wrote:
>>
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA256
>>>
>>> Recently mteval-v13a.pl stopped working, printing:
>>>
>>> Can't find Unicode property definition "Line_Break" in regex; marked
>>> by <-- HERE in m/\p{Line_Break} <-- HERE \p{Zl}/ at
>>> /home/gumble/software/moses/scripts/generic/mteval-v13a.pl line 953.
>>>
>>> I see this commit
>>> <https://github.com/moses-smt/mosesdecoder/commit/c6c3bc84b7673618f37948
>>> 2cbc6b708f55a9ecd3
>>> <https://github.com/moses-smt/mosesdecoder/commit/c6c3bc84b7673618f379482cbc6b708f55a9ecd3>
>>> >.
>>> I found that changing this to \p{Line_Break: Hyphen} worked. Is this
>>> the equivalent of \p{Hyphen}?
>>>
>>> - --
>>> Dingyuan Wang
>>> -----BEGIN PGP SIGNATURE-----
>>>
>>> iQEzBAEBCAAdFiEEjE4PLbCEqfvlC0rjs+TYPj8+X9wFAljZwNYACgkQs+TYPj8+
>>> X9xShggAhSjPEEYXsiRPT9wVljRV7XjBmexe/E7EKzl9b/PEnuxlSNSrz/0Estr5
>>> 8/H4s+lwKdv9xx1jTxOGOVkToiVC95QkuppXX3WS+BCDjajE8fqWc2Y0IhUWRaqf
>>> PAhhotEZmoWAhQC/qVM7lILf29N9OhQ2FStQH9rn+LpD2dkSZweZ0XGJ+CFpCdaP
>>> VA7XPWJCJZeEBUsBqrSxl1Cwzr+KQ4pw/NFP6yxJ+smmTkUSyp2FfYCtvalx/L0d
>>> 2UZ1fiujzco7NHeJW/0ZYwsb+NNMuM7CljBMhQAWIN+D0f6Wz1/bHH8jhFyxUw0B
>>> +/hN/chrAmYX+Kz2j/MKc7eXZuPtmA==
>>> =C+jT
>>> -----END PGP SIGNATURE-----
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170328/bf9ff159/attachment-0001.html

------------------------------

Message: 3
Date: Wed, 29 Mar 2017 09:31:05 +0800
From: Dingyuan Wang <abcdoyle888@gmail.com>
Subject: Re: [Moses-support] Error in mteval-v13a.pl
To: liling tan <alvations@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAFt8H775NqOKLvdBzsepHhi5F+nO9YVsdXTt6DnL2S8xNUyxag@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,

The OS is Debian 9 (stretch/testing), and perl is v5.24.1.


2017?3?29? ??12:53?"liling tan" <alvations@gmail.com>???

Interesting, I've just took a look at the perluniprop documentation
carefully, generic \p{Line_Break} didn't exist but Ubuntu and Mac OS let me
slip pass without error/warning. I supposed \p{Line_Break} automatically
goes to \p{Line_Break: Alphabetic} or \p{Line_Break: Ambiguous}

Maybe the perluniprop handling is different for different machines/OS/perl
versions.

\p{Line_Break: Hyphen} might be closer to \p{Hyphen} than
alphbetic/ambiguous \p{Line_Break}


I think we can change it to \p{Line_Break: Hyphen} safely.But first,
@Dingyuan, please do tell us your OS and perl versions and see whether
it's just that the newer perl version handles the \p{Line_Break}
automatically while the older perl doesn't


On Wed, Mar 29, 2017 at 12:36 AM, liling tan <alvations@gmail.com> wrote:

> Hi Dingyuan, Hieu,
>
> Thanks for highlighting the issue.
>
> The deprecation warning from mteval has been there since early 2015 on
> https://www.mail-archive.com/moses-support@mit.edu/msg12057.html
>
> The fix at https://github.com/moses-smt/mosesdecoder/pull/170 was
> adhering to unicode annex on 2016-06-01.
>
> Actually \p{Line_Break:Hyphen} is a little more restricting than \p{Line_Break}.
> Technically, if the script didn't break with \p{Line_Break:Hyphen} ,
> the superset \p{Line_Break} shouldn't break. Perl unicode properties
> documentation has the detailed explanations: http://perl11.or
> g/pod/perluniprops.html
>
> The current mteval-v13a.pl works fine on:
> - Ubuntu 16.04.01 with perl v5.22.1
> - Mac OS Sierra with perl v5.18.2
>
> @Dingyuan, a couple of short questions.
>
> (i) Which OS are you using? Which version of the OS?
>
> (ii) What is your perl version? use command "perl -v"
>
> That'll help us to know which OS and/or perl version caused that error.
>
> Regards,
> Liling
>
>
> On Tue, Mar 28, 2017 at 8:52 PM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>
>> If you find that {Line_Break:Hyphen} works, please consider checking it
>> in.
>>
>> These compatibility issues are difficult to debug alone and depends on
>> the exact perl/OS version you're running. Your fix will add a little to the
>> body of knowledge
>>
>> * Looking for MT/NLP opportunities *
>> Hieu Hoang
>> http://moses-smt.org/
>>
>>
>> On 28 March 2017 at 02:48, Dingyuan Wang <abcdoyle888@gmail.com> wrote:
>>
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA256
>>>
>>> Recently mteval-v13a.pl stopped working, printing:
>>>
>>> Can't find Unicode property definition "Line_Break" in regex; marked
>>> by <-- HERE in m/\p{Line_Break} <-- HERE \p{Zl}/ at
>>> /home/gumble/software/moses/scripts/generic/mteval-v13a.pl line 953.
>>>
>>> I see this commit
>>> <https://github.com/moses-smt/mosesdecoder/commit/c6c3bc84b7673618f37948
>>> 2cbc6b708f55a9ecd3
>>> <https://github.com/moses-smt/mosesdecoder/commit/c6c3bc84b7673618f379482cbc6b708f55a9ecd3>
>>> >.
>>> I found that changing this to \p{Line_Break: Hyphen} worked. Is this
>>> the equivalent of \p{Hyphen}?
>>>
>>> - --
>>> Dingyuan Wang
>>> -----BEGIN PGP SIGNATURE-----
>>>
>>> iQEzBAEBCAAdFiEEjE4PLbCEqfvlC0rjs+TYPj8+X9wFAljZwNYACgkQs+TYPj8+
>>> X9xShggAhSjPEEYXsiRPT9wVljRV7XjBmexe/E7EKzl9b/PEnuxlSNSrz/0Estr5
>>> 8/H4s+lwKdv9xx1jTxOGOVkToiVC95QkuppXX3WS+BCDjajE8fqWc2Y0IhUWRaqf
>>> PAhhotEZmoWAhQC/qVM7lILf29N9OhQ2FStQH9rn+LpD2dkSZweZ0XGJ+CFpCdaP
>>> VA7XPWJCJZeEBUsBqrSxl1Cwzr+KQ4pw/NFP6yxJ+smmTkUSyp2FfYCtvalx/L0d
>>> 2UZ1fiujzco7NHeJW/0ZYwsb+NNMuM7CljBMhQAWIN+D0f6Wz1/bHH8jhFyxUw0B
>>> +/hN/chrAmYX+Kz2j/MKc7eXZuPtmA==
>>> =C+jT
>>> -----END PGP SIGNATURE-----
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170328/301c7757/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 125, Issue 52
**********************************************

0 Response to "Moses-support Digest, Vol 125, Issue 52"

Post a Comment

Subscribe to: Post Comments (Atom)