Moses-support Digest, Vol 111, Issue 1

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Baseline: Problem with tuned weights (Raphael H?ps)


----------------------------------------------------------------------

Message: 1
Date: Mon, 4 Jan 2016 18:16:33 +0100
From: Raphael H?ps <raphael.hoeps@gmx.net>
Subject: [Moses-support] Baseline: Problem with tuned weights
To: moses-support@mit.edu
Message-ID: <568AA8F1.5060500@gmx.net>
Content-Type: text/plain; charset="windows-1252"

Hello,

I did the moses-baseline tutorial to train and tune and translation
model for English to German. After finishing the system it seemed to
work quite well at first but then I noticed that the tuning step seemed
to actually having made my system worse! I really don't know what I did
wrong. I sticked very close to the tutorial. Here is what I did in detail:

1. Training the TM to working/train/model.
2. Tuning with a corpus that is a cut-down version of news-test2008. The
main result of this process are the weights of the new file
mert-work/moses.ini, right?
3. Filtering of mert-work/moses.ini to a testing corpus (cut-down
version of newstest2011).
4. Translating the testing corpus and calculating BLEU-score. I got a
score of 7.42.
5. In a second test I used the default moses.ini file instead of the
tuned one (and the same filtered and binarized model) and got a score of
8.22 on the same testing corpus!

Something is probably wrong with the tuned moses.ini file. To find out,
I translated the corpus that was used for tuning with both ini-files and
calculated the scores:
Untuned: 7.01
Tuned: 6.70 (!)

Now this is really odd! Furthermore in the tuned moses.ini file there is
the line:
# BLEU 0.0755253 on dev
/home/rh/Studium/aktuell/LSS/moses/mosesdecoder/corpus/dev-small.en
Why do I get a score of 6.7 instead? The files dev-small.en and
dev-small.de where my tuning corpora.

Do you have any idea, what I might have done wrong?

For the tuning step, I used:
cd ~/working
nohup nice ~/mosesdecoder/scripts/training/mert-moses.pl \
~/corpus/dev-small.en ~/corpus/dev-small.de \
~/mosesdecoder/bin/moses train/model/moses.ini --mertdir
~/mosesdecoder/bin/ \ &> mert.out &

I appended mert.log and the tune moses.ini file. Did anyone ever build a
system for English to German and can say something about the trained
weights in moses.ini? Do they seem okay?

Thank you very much for your help!
Greetings,
Raphi





-------------- next part --------------
# MERT optimized configuration
# decoder /home/rh/Studium/aktuell/LSS/moses/mosesdecoder/bin/moses
# BLEU 0.0755253 on dev /home/rh/Studium/aktuell/LSS/moses/mosesdecoder/corpus/dev-small.en
# We were before running iteration 16
# finished Mi 18. Nov 03:42:26 CET 2015
### MOSES CONFIG FILE ###
#########################

# input factors
[input-factors]
0

# mapping steps
[mapping]
0 T 0

[distortion-limit]
6

# feature functions
[feature]
UnknownWordPenalty
WordPenalty
PhrasePenalty
PhraseDictionaryMemory name=TranslationModel0 num-features=4 path=/home/rh/Studium/aktuell/LSS/moses/mosesdecoder/working/train/model/phrase-table.gz input-factor=0 output-factor=0
LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 path=/home/rh/Studium/aktuell/LSS/moses/mosesdecoder/working/train/model/reordering-table.wbe-msd-bidirectional-fe.gz
Distortion
KENLM lazyken=0 name=LM0 factor=0 path=/home/rh/Studium/aktuell/LSS/moses/mosesdecoder/languagemodel/news-commentary-v8.de-en.blm.de order=3

# dense weights for feature functions
[weight]

LexicalReordering0= 0.0249924 0.227188 0.0517476 0.19267 -0.0499753 0.0222798
Distortion0= 0.0758859
LM0= 0.0681075
WordPenalty0= -0.125367
PhrasePenalty0= -0.0569564
TranslationModel0= 0.00738901 0.0353195 0.0442114 0.0179101
UnknownWordPenalty0= 1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mert.log
Type: text/x-log
Size: 2733 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20160104/3be688cf/attachment-0001.bin

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 111, Issue 1
*********************************************

Tweet