Home » Uncategories » Moses-support Digest, Vol 125, Issue 50

Moses-support Digest, Vol 125, Issue 50

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Error in mteval-v13a.pl (Dingyuan Wang)
2. Select sentences that maximize BLEU from n-best list
(Marcin Junczys-Dowmunt)
3. ERROR: Lexical reordering scoring failed (Anna Kup?)
4. Re: Error in mteval-v13a.pl (Hieu Hoang)
5. ERROR: Lexical reordering scoring failed (Anna Kup?)


----------------------------------------------------------------------

Message: 1
Date: Tue, 28 Mar 2017 09:48:06 +0800
From: Dingyuan Wang <abcdoyle888@gmail.com>
Subject: [Moses-support] Error in mteval-v13a.pl
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <02136cb6-dbad-fa07-2120-31ea2b94a3db@gmail.com>
Content-Type: text/plain; charset=utf-8

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Recently mteval-v13a.pl stopped working, printing:

Can't find Unicode property definition "Line_Break" in regex; marked
by <-- HERE in m/\p{Line_Break} <-- HERE \p{Zl}/ at
/home/gumble/software/moses/scripts/generic/mteval-v13a.pl line 953.

I see this commit
<https://github.com/moses-smt/mosesdecoder/commit/c6c3bc84b7673618f37948
2cbc6b708f55a9ecd3>.
I found that changing this to \p{Line_Break: Hyphen} worked. Is this
the equivalent of \p{Hyphen}?

- --
Dingyuan Wang
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEjE4PLbCEqfvlC0rjs+TYPj8+X9wFAljZwNYACgkQs+TYPj8+
X9xShggAhSjPEEYXsiRPT9wVljRV7XjBmexe/E7EKzl9b/PEnuxlSNSrz/0Estr5
8/H4s+lwKdv9xx1jTxOGOVkToiVC95QkuppXX3WS+BCDjajE8fqWc2Y0IhUWRaqf
PAhhotEZmoWAhQC/qVM7lILf29N9OhQ2FStQH9rn+LpD2dkSZweZ0XGJ+CFpCdaP
VA7XPWJCJZeEBUsBqrSxl1Cwzr+KQ4pw/NFP6yxJ+smmTkUSyp2FfYCtvalx/L0d
2UZ1fiujzco7NHeJW/0ZYwsb+NNMuM7CljBMhQAWIN+D0f6Wz1/bHH8jhFyxUw0B
+/hN/chrAmYX+Kz2j/MKc7eXZuPtmA==
=C+jT
-----END PGP SIGNATURE-----


------------------------------

Message: 2
Date: Tue, 28 Mar 2017 10:19:09 +0200
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: [Moses-support] Select sentences that maximize BLEU from
n-best list
To: moses-support <moses-support@mit.edu>
Message-ID: <71af4b2d-1f98-6142-daa6-0fc972773fc3@amu.edu.pl>
Content-Type: text/plain; charset=utf-8; format=flowed

Hi list,

does anyone have a tool that takes a moses-format n-best list and can
output the single best sentence per source sentence according to BLEU
and a given reference? Or anything that can be shoehorned into something
like that?

Thanks,

Marcin



------------------------------

Message: 3
Date: Tue, 28 Mar 2017 10:31:28 +0000
From: Anna Kup? <aniakups@gmail.com>
Subject: [Moses-support] ERROR: Lexical reordering scoring failed
To: moses-support@mit.edu
Message-ID:
<CAM-RO89jH2Mbp-tbZ8GP6x-XqXOuWEDE4i-Rf=4kSo5i_O0OiQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,

I got the following error while using Experiment Management System:

==> TRAINING_build-reordering.3 <==

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/snap/bin"
cd /root/working/experiments
echo 'starting at '`date`' on '`hostname`
mkdir -p /root/working/experiments/training

/root/mosesdecoder/scripts/training/train-model.perl -mgiza -mgiza-cpus 8
-dont-zip -first-step 7 -last-step 7 -external-bin-dir
/root/mosesdecoder/tools -f en -e de -alignment grow-diag-final-and
-reordering msd-bidirectional-fe -extract-file
/root/working/experiments/model/extract.2 -reordering-table
/root/working/experiments/model/reordering-table.3

echo 'finished at '`date`
touch /root/working/experiments/steps/3/TRAINING_build-reordering.3.DONE

==> TRAINING_build-reordering.3.INFO <==
lexicalized-reordering = msd-bidirectional-fe
INPUT = USED /root/working/experiments/model/extract.2
# reuse run 2 for TRAINING:extract-phrases

==> TRAINING_build-reordering.3.STDERR <==
Using SCRIPTS_ROOTDIR: /root/mosesdecoder/scripts
using gzip
(7) learn reordering model @ Tue Mar 28 12:19:55 CEST 2017
(7.1) [no factors] learn reordering model @ Tue Mar 28 12:19:55 CEST 2017
(7.2) building tables @ Tue Mar 28 12:19:55 CEST 2017
Executing: /root/mosesdecoder/scripts/../bin/lexical-reordering-score
/root/working/experiments/model/extract.2.o.sorted.gz 0.5
/root/working/experiments/model/reordering-table.3. --model "wbe msd
wbe-msd-bidirectional-fe"
Lexical Reordering Scorer
scores lexical reordering models of several types (hierarchical,
phrase-based and word-based-extraction

==> TRAINING_build-reordering.3.STDOUT <==
starting at Di 28. M?r 12:19:54 CEST 2017 on machine-VirtualBox

==> TRAINING_build-reordering.3.STDERR <==
terminate called after throwing an instance of 'FileFormatException'
what(): phrase-extract/lexical-reordering/score.cpp:260 in void
split_line(const StringPiece&, StringPiece&, StringPiece&, StringPiece&,
StringPiece&, StringPiece&, float&) threw FileFormatException because
`errIndex == next.data()'.
Invalid extract file format: inhaled corticosteroids ||| inhalative
Kortikoichaemic ||| isch?mische ||| mono other
Aborted (core dumped)
Exit code: 134
*ERROR: Lexical reordering scoring failed at
/root/mosesdecoder/scripts/training/train-model.perl line 1924.*

Does someone know how to get rid of that problem?

Thanks in advance and best regards,
Anna

--
Anna Kup?
+48 782 823 850 | www.linkedin.com/in/annakups
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170328/228bfacc/attachment-0001.html

------------------------------

Message: 4
Date: Tue, 28 Mar 2017 13:52:08 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Error in mteval-v13a.pl
To: Dingyuan Wang <abcdoyle888@gmail.com>, liling tan
<alvations@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAEKMkbhZc_nppN7_Tna_f28PfdMyHwNoQb7R0UPnYZAm9BRbmg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

If you find that {Line_Break:Hyphen} works, please consider checking it in.

These compatibility issues are difficult to debug alone and depends on the
exact perl/OS version you're running. Your fix will add a little to the
body of knowledge

* Looking for MT/NLP opportunities *
Hieu Hoang
http://moses-smt.org/


On 28 March 2017 at 02:48, Dingyuan Wang <abcdoyle888@gmail.com> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Recently mteval-v13a.pl stopped working, printing:
>
> Can't find Unicode property definition "Line_Break" in regex; marked
> by <-- HERE in m/\p{Line_Break} <-- HERE \p{Zl}/ at
> /home/gumble/software/moses/scripts/generic/mteval-v13a.pl line 953.
>
> I see this commit
> <https://github.com/moses-smt/mosesdecoder/commit/c6c3bc84b7673618f37948
> 2cbc6b708f55a9ecd3>.
> I found that changing this to \p{Line_Break: Hyphen} worked. Is this
> the equivalent of \p{Hyphen}?
>
> - --
> Dingyuan Wang
> -----BEGIN PGP SIGNATURE-----
>
> iQEzBAEBCAAdFiEEjE4PLbCEqfvlC0rjs+TYPj8+X9wFAljZwNYACgkQs+TYPj8+
> X9xShggAhSjPEEYXsiRPT9wVljRV7XjBmexe/E7EKzl9b/PEnuxlSNSrz/0Estr5
> 8/H4s+lwKdv9xx1jTxOGOVkToiVC95QkuppXX3WS+BCDjajE8fqWc2Y0IhUWRaqf
> PAhhotEZmoWAhQC/qVM7lILf29N9OhQ2FStQH9rn+LpD2dkSZweZ0XGJ+CFpCdaP
> VA7XPWJCJZeEBUsBqrSxl1Cwzr+KQ4pw/NFP6yxJ+smmTkUSyp2FfYCtvalx/L0d
> 2UZ1fiujzco7NHeJW/0ZYwsb+NNMuM7CljBMhQAWIN+D0f6Wz1/bHH8jhFyxUw0B
> +/hN/chrAmYX+Kz2j/MKc7eXZuPtmA==
> =C+jT
> -----END PGP SIGNATURE-----
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170328/b5452b09/attachment-0001.html

------------------------------

Message: 5
Date: Tue, 28 Mar 2017 14:03:39 +0000
From: Anna Kup? <aniakups@gmail.com>
Subject: [Moses-support] ERROR: Lexical reordering scoring failed
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAM-RO8-O_ATWsLuT=Ho+eNwquyq_SMALSppGGTotTKKqg7H90A@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,

I got the following error while using Experiment Management System:

==> TRAINING_build-reordering.3 <==

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/snap/bin"
cd /root/working/experiments
echo 'starting at '`date`' on '`hostname`
mkdir -p /root/working/experiments/training

/root/mosesdecoder/scripts/training/train-model.perl -mgiza -mgiza-cpus 8
-dont-zip -first-step 7 -last-step 7 -external-bin-dir
/root/mosesdecoder/tools -f en -e de -alignment grow-diag-final-and
-reordering msd-bidirectional-fe -extract-file
/root/working/experiments/model/extract.2 -reordering-table
/root/working/experiments/model/reordering-table.3

echo 'finished at '`date`
touch /root/working/experiments/steps/3/TRAINING_build-reordering.3.DONE

==> TRAINING_build-reordering.3.INFO <==
lexicalized-reordering = msd-bidirectional-fe
INPUT = USED /root/working/experiments/model/extract.2
# reuse run 2 for TRAINING:extract-phrases

==> TRAINING_build-reordering.3.STDERR <==
Using SCRIPTS_ROOTDIR: /root/mosesdecoder/scripts
using gzip
(7) learn reordering model @ Tue Mar 28 12:19:55 CEST 2017
(7.1) [no factors] learn reordering model @ Tue Mar 28 12:19:55 CEST 2017
(7.2) building tables @ Tue Mar 28 12:19:55 CEST 2017
Executing: /root/mosesdecoder/scripts/../bin/lexical-reordering-score
/root/working/experiments/model/extract.2.o.sorted.gz 0.5
/root/working/experiments/model/reordering-table.3. --model "wbe msd
wbe-msd-bidirectional-fe"
Lexical Reordering Scorer
scores lexical reordering models of several types (hierarchical,
phrase-based and word-based-extraction

==> TRAINING_build-reordering.3.STDOUT <==
starting at Di 28. M?r 12:19:54 CEST 2017 on machine-VirtualBox

==> TRAINING_build-reordering.3.STDERR <==
terminate called after throwing an instance of 'FileFormatException'
what(): phrase-extract/lexical-reordering/score.cpp:260 in void
split_line(const StringPiece&, StringPiece&, StringPiece&, StringPiece&,
StringPiece&, StringPiece&, float&) threw FileFormatException because
`errIndex == next.data()'.
Invalid extract file format: inhaled corticosteroids ||| inhalative
Kortikoichaemic ||| isch?mische ||| mono other
Aborted (core dumped)
Exit code: 134
*ERROR: Lexical reordering scoring failed at
/root/mosesdecoder/scripts/training/train-model.perl line 1924.*

Does someone know how to get rid of that problem?

Thanks in advance and best regards,
Anna

--
Anna Kup?
+48 782 823 850 | www.linkedin.com/in/annakups
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170328/0b2b9738/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 125, Issue 50
**********************************************

0 Response to "Moses-support Digest, Vol 125, Issue 50"

Post a Comment

Subscribe to: Post Comments (Atom)