Home » Uncategories » Moses-support Digest, Vol 108, Issue 69

Moses-support Digest, Vol 108, Issue 69

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. lmplz error (Hieu Hoang)
2. Re: Issue with tuning - decoder died (Ya-chi Lin)
3. OD: lmplz error (Marcin Junczys-Dowmunt)


----------------------------------------------------------------------

Message: 1
Date: Fri, 23 Oct 2015 00:08:27 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: [Moses-support] lmplz error
To: moses-support <moses-support@mit.edu>
Message-ID: <56296C6B.7080600@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed

hi all

does anyone know how to fix this error from lmplz:
terminate called after throwing an instance of
'lm::builder::BadDiscountException'
what(): ERROR: 4-gram discount out of range for adjusted count 3:
-0.446424
The command I was running is
.../lmplz --text .../Ubuntu.truecased.3 --order 5 --arpa
.../Ubuntu.lm.3 --prune 0 0 1 -T .../lm -S 20%
I guess there's not enough data to create a 5-gram

--
Hieu Hoang
http://www.hoang.co.uk/hieu



------------------------------

Message: 2
Date: Fri, 23 Oct 2015 00:22:12 +0000
From: Ya-chi Lin <yachilin@student.uef.fi>
Subject: Re: [Moses-support] Issue with tuning - decoder died
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<DB5PR01MB13843B0087B97096BD11A149FB260@DB5PR01MB1384.eurprd01.prod.exchangelabs.com>

Content-Type: text/plain; charset="big5"

Hi ALL,


I'm building an English to Traditional Chinese system but have encountered a problem while performing tuning. Below is the content of mert.out file. Can anyone please tell me what went wrong here? Any help will be greatly appreciated.

Using SCRIPTS_ROOTDIR: /opt/mosesdecoder/scripts
Assuming the tables are already filtered, reusing filtered/moses.ini
Using cached features list: ./features.list
MERT starting values and ranges for random generation:
LexicalReordering0 = 0.300 ( 0.00 .. 1.00)
LexicalReordering0 = 0.300 ( 0.00 .. 1.00)
LexicalReordering0 = 0.300 ( 0.00 .. 1.00)
LexicalReordering0 = 0.300 ( 0.00 .. 1.00)
LexicalReordering0 = 0.300 ( 0.00 .. 1.00)
LexicalReordering0 = 0.300 ( 0.00 .. 1.00)
Distortion0 = 0.300 ( 0.00 .. 1.00)
LM0 = 0.500 ( 0.00 .. 1.00)
WordPenalty0 = -1.000 ( 0.00 .. 1.00)
PhrasePenalty0 = 0.200 ( 0.00 .. 1.00)
TranslationModel0 = 0.200 ( 0.00 .. 1.00)
TranslationModel0 = 0.200 ( 0.00 .. 1.00)
TranslationModel0 = 0.200 ( 0.00 .. 1.00)
TranslationModel0 = 0.200 ( 0.00 .. 1.00)
featlist: LexicalReordering0=0.300000
featlist: LexicalReordering0=0.300000
featlist: LexicalReordering0=0.300000
featlist: LexicalReordering0=0.300000
featlist: LexicalReordering0=0.300000
featlist: LexicalReordering0=0.300000
featlist: Distortion0=0.300000
featlist: LM0=0.500000
featlist: WordPenalty0=-1.000000
featlist: PhrasePenalty0=0.200000
featlist: TranslationModel0=0.200000
featlist: TranslationModel0=0.200000
featlist: TranslationModel0=0.200000
featlist: TranslationModel0=0.200000
run 1 start at Fri Oct 23 03:07:31 EEST 2015
Parsing --decoder-flags: ||
Saving new config to: ./run1.moses.ini
Saved: ./run1.moses.ini
Normalizing lambdas: 0.300000 0.300000 0.300000 0.300000 0.300000 0.300000 0.300000 0.500000 -1.000000 0.200000 0.200000 0.200000 0.200000 0.200000
DECODER_CFG = -weight-overwrite 'LM0= 0.108696 LexicalReordering0= 0.065217 0.065217 0.065217 0.065217 0.065217 0.065217 TranslationModel0= 0.043478 0.043478 0.043478 0.043478 PhrasePenalty0= 0.043478 Distortion0= 0.065217 WordPenalty0= -0.217391'
Executing: /opt/mosesdecoder/bin/moses -config filtered/moses.ini -weight-overwrite 'LM0= 0.108696 LexicalReordering0= 0.065217 0.065217 0.065217 0.065217 0.065217 0.065217 TranslationModel0= 0.043478 0.043478 0.043478 0.043478 PhrasePenalty0= 0.043478 Distortion0= 0.065217 WordPenalty0= -0.217391' -n-best-list run1.best100.out 100 distinct -input-file /home/yachilin/working/tune.true.en > run1.out
Executing: /opt/mosesdecoder/bin/moses -config filtered/moses.ini -weight-overwrite 'LM0= 0.108696 LexicalReordering0= 0.065217 0.065217 0.065217 0.065217 0.065217 0.065217 TranslationModel0= 0.043478 0.043478 0.043478 0.043478 PhrasePenalty0= 0.043478 Distortion0= 0.065217 WordPenalty0= -0.217391' -n-best-list run1.best100.out 100 distinct -input-file /home/yachilin/working/tune.true.en > run1.out
1-10.20.2 0.2 0.2 0.20.3 0.3 0.3 0.3 0.3 0.30.30.5(1) run decoder to produce n-best lists
params =
decoder_config = -weight-overwrite 'LM0= 0.108696 LexicalReordering0= 0.065217 0.065217 0.065217 0.065217 0.065217 0.065217 TranslationModel0= 0.043478 0.043478 0.043478 0.043478 PhrasePenalty0= 0.043478 Distortion0= 0.065217 WordPenalty0= -0.217391'
Defined parameters (per moses.ini or switch):
config: filtered/moses.ini
distortion-limit: 6
feature: UnknownWordPenalty WordPenalty PhrasePenalty PhraseDictionaryMemory name=TranslationModel0 num-features=4 path=/home/yachilin/working/mert-work/filtered/phrase-table.0-0.1.1.gz input-factor=0 output-factor=0 LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 path=/home/yachilin/working/mert-work/filtered/reordering-table.wbe-msd-bidirectional-fe Distortion KENLM lazyken=0 name=LM0 factor=0 path=/home/yachilin/lm/train.blm.tch order=3
input-factors: 0
input-file: /home/yachilin/working/tune.true.en
mapping: 0 T 0
n-best-list: run1.best100.out 100 distinct
weight: UnknownWordPenalty0= 1 WordPenalty0= -1 PhrasePenalty0= 0.2 TranslationModel0= 0.2 0.2 0.2 0.2 LexicalReordering0= 0.3 0.3 0.3 0.3 0.3 0.3 Distortion0= 0.3 LM0= 0.5
weight-overwrite: LM0= 0.108696 LexicalReordering0= 0.065217 0.065217 0.065217 0.065217 0.065217 0.065217 TranslationModel0= 0.043478 0.043478 0.043478 0.043478 PhrasePenalty0= 0.043478 Distortion0= 0.065217 WordPenalty0= -0.217391
line=UnknownWordPenalty
FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
line=WordPenalty
FeatureFunction: WordPenalty0 start: 1 end: 1
line=PhrasePenalty
FeatureFunction: PhrasePenalty0 start: 2 end: 2
line=PhraseDictionaryMemory name=TranslationModel0 num-features=4 path=/home/yachilin/working/mert-work/filtered/phrase-table.0-0.1.1.gz input-factor=0 output-factor=0
FeatureFunction: TranslationModel0 start: 3 end: 6
line=LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 path=/home/yachilin/working/mert-work/filtered/reordering-table.wbe-msd-bidirectional-fe
Initializing Lexical Reordering Feature..
FeatureFunction: LexicalReordering0 start: 7 end: 12
line=Distortion
FeatureFunction: Distortion0 start: 13 end: 13
line=KENLM lazyken=0 name=LM0 factor=0 path=/home/yachilin/lm/train.blm.tch order=3
FeatureFunction: LM0 start: 14 end: 14
Loading UnknownWordPenalty0
Loading WordPenalty0
Loading PhrasePenalty0
Loading LexicalReordering0
Loading table into memory...done.
Loading Distortion0
Loading LM0
Loading TranslationModel0
Start loading text phrase table. Moses format : [0.719] seconds
Reading /home/yachilin/working/mert-work/filtered/phrase-table.0-0.1.1.gz
----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
****************************************************************************************************
Exception: moses/Phrase.cpp:217 in void Moses::Phrase::CreateFromString(Moses::FactorDirection, const std::vector<long unsigned int>&, const StringPiece&, Moses::Word**) threw util::Exception because `nextPos == string::npos'.
Incorrect formatting of non-terminal. Should have 2 non-terms, eg. [X][X]. Current string: [HIV]
Exit code: 1
The decoder died. CONFIG WAS -weight-overwrite 'LM0= 0.108696 LexicalReordering0= 0.065217 0.065217 0.065217 0.065217 0.065217 0.065217 TranslationModel0= 0.043478 0.043478 0.043478 0.043478 PhrasePenalty0= 0.043478 Distortion0= 0.065217 WordPenalty0= -0.217391'
[1]+ Exit 2 $MOSES/scripts/training/mert-moses.pl tune.true.en tune.true.tch $MOSES/bin/moses train/model/moses.ini --mertdir $MOSES/bin/ &> mert.out
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151022/b23049bc/attachment-0001.html

------------------------------

Message: 3
Date: Fri, 23 Oct 2015 07:22:22 +0100
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: [Moses-support] OD: lmplz error
To: Hieu Hoang <hieuhoang@gmail.com>, moses-support
<moses-support@mit.edu>
Message-ID: <20151023062228.27024ABA54@pp.amu.edu.pl>
Content-Type: text/plain; charset="us-ascii"

An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151023/3a3f60e3/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 108, Issue 69
**********************************************

0 Response to "Moses-support Digest, Vol 108, Issue 69"

Post a Comment

Subscribe to: Post Comments (Atom)