Home » Uncategories » Moses-support Digest, Vol 137, Issue 13

Moses-support Digest, Vol 137, Issue 13

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Decoder died - during tuning (Debajyoty Banik)


----------------------------------------------------------------------

Message: 1
Date: Sun, 1 Apr 2018 00:55:14 +0530
From: Debajyoty Banik <debajyoty.banik@gmail.com>
Subject: [Moses-support] Decoder died - during tuning
To: moses-support@mit.edu
Message-ID:
<CAK0HAkhdfRp4Pypj9NA7S8Vsb1bgPhwpBedtwk4m29jxDf4ALA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hello,


I am training (factored based SMT) my 1.4M Hindi-English Sentence pairs.
Training is completed, moses.ini is also created but when I run following
command for tuning (1000 sentence pairs Hindi-English):

~/smt/mosesdecoder/scripts/training/mert-moses.pl ~/set_hind
i_to_eng_wplg/tune/corpora_tune.hi ~/set_hindi_to_eng_wplg/tune/corpora_tune.en
~/smt/mosesdecoder/bin/moses
~/set_hindi_to_eng_wplg/working/train/model/moses.ini
--mertdir ~/smt/mosesdecoder/bin



I get following error:



Using SCRIPTS_ROOTDIR: /home/kamal.pcs17/smt/mosesdecoder/scripts

Assuming the tables are already filtered, reusing filtered/moses.ini

Using cached features list: ./features.list

MERT starting values and ranges for random generation:

Distortion0 = 0.300 ( 0.00 .. 1.00)

LM0 = 0.500 ( 0.00 .. 1.00)

LM1 = 0.500 ( 0.00 .. 1.00)

WordPenalty0 = -1.000 ( 0.00 .. 1.00)

PhrasePenalty0 = 0.200 ( 0.00 .. 1.00)

TranslationModel0 = 0.200 ( 0.00 .. 1.00)

TranslationModel0 = 0.200 ( 0.00 .. 1.00)

TranslationModel0 = 0.200 ( 0.00 .. 1.00)

TranslationModel0 = 0.200 ( 0.00 .. 1.00)

TranslationModel1 = 0.200 ( 0.00 .. 1.00)

TranslationModel1 = 0.200 ( 0.00 .. 1.00)

TranslationModel1 = 0.200 ( 0.00 .. 1.00)

TranslationModel1 = 0.200 ( 0.00 .. 1.00)

GenerationModel0 = 0.300 ( 0.00 .. 1.00)

GenerationModel0 = 0.000 ( 0.00 .. 1.00)

run 1 start at Sat Mar 31 14:01:09 IST 2018

featlist: Distortion0=0.300000

featlist: LM0=0.500000

featlist: LM1=0.500000

featlist: WordPenalty0=-1.000000

featlist: PhrasePenalty0=0.200000

featlist: TranslationModel0=0.200000

featlist: TranslationModel0=0.200000

featlist: TranslationModel0=0.200000

featlist: TranslationModel0=0.200000

featlist: TranslationModel1=0.200000

featlist: TranslationModel1=0.200000

featlist: TranslationModel1=0.200000

featlist: TranslationModel1=0.200000

featlist: GenerationModel0=0.300000

featlist: GenerationModel0=0.000000

Parsing --decoder-flags: ||

Saving new config to: ./run1.moses.ini

Saved: ./run1.moses.ini

(1) run decoder to produce n-best lists

params =

Normalizing lambdas: 0.300000 0.500000 0.500000 -1.000000 0.200000 0.200000
0.200000 0.200000 0.200000 0.200000 0.200000 0.200000 0.200000 0.300000
0.000000

DECODER_CFG = -weight-overwrite 'GenerationModel0= 0.068182 0.000000 LM0=
0.113636 LM1= 0.113636 WordPenalty0= -0.227273 PhrasePenalty0= 0.045455
TranslationModel0= 0.045455 0.045455 0.045455 0.045455 Distortion0=
0.068182 TranslationModel1= 0.045455 0.045455 0.045455 0.045455'

decoder_config = -weight-overwrite 'GenerationModel0= 0.068182 0.000000
LM0= 0.113636 LM1= 0.113636 WordPenalty0= -0.227273 PhrasePenalty0=
0.045455 TranslationModel0= 0.045455 0.045455 0.045455 0.045455
Distortion0= 0.068182 TranslationModel1= 0.045455 0.045455 0.045455
0.045455'

Executing: /home/kamal.pcs17/smt/mosesdecoder/bin/moses -config
filtered/moses.ini -weight-overwrite 'GenerationModel0= 0.068182 0.000000
LM0= 0.113636 LM1= 0.113636 WordPenalty0= -0.227273 PhrasePenalty0=
0.045455 TranslationModel0= 0.045455 0.045455 0.045455 0.045455
Distortion0= 0.068182 TranslationModel1= 0.045455 0.045455 0.045455
0.045455' -n-best-list run1.best100.out 100 distinct -input-file
/home/kamal.pcs17/set_hindi_to_eng_wplg/tune/corpora_tune.hi > run1.out

Executing: /home/kamal.pcs17/smt/mosesdecoder/bin/moses -config
filtered/moses.ini -weight-overwrite 'GenerationModel0= 0.068182 0.000000
LM0= 0.113636 LM1= 0.113636 WordPenalty0= -0.227273 PhrasePenalty0=
0.045455 TranslationModel0= 0.045455 0.045455 0.045455 0.045455
Distortion0= 0.068182 TranslationModel1= 0.045455 0.045455 0.045455
0.045455' -n-best-list run1.best100.out 100 distinct -input-file
/home/kamal.pcs17/set_hindi_to_eng_wplg/tune/corpora_tune.hi > run1.out

Defined parameters (per moses.ini or switch):

config: filtered/moses.ini

distortion-limit: 6

feature: UnknownWordPenalty WordPenalty PhrasePenalty
PhraseDictionaryMemory name=TranslationModel0 num-features=4
path=/home/kamal.pcs17/set_hindi_to_eng_wplg/mert-work/filtered/phrase-table.1,2-1.1.1.gz
input-factor=1,2 output-factor=1 PhraseDictionaryMemory
name=TranslationModel1 num-features=4 path=/home/kamal.pcs17/set_hin
di_to_eng_wplg/mert-work/filtered/phrase-table.1,2-2.1.1.gz
input-factor=1,2 output-factor=2 Generation name=GenerationModel0
num-features=2 path=/home/kamal.pcs17/set_hindi_to_eng_wplg/working/train/model/generation.1,2-0.gz
input-factor=1,2 output-factor=0 Distortion IRSTLM name=LM0 factor=0
path=/home/kamal.pcs17/set_hindi_to_eng_wplg/lm/corpora_train.lm.final.arpa.en
order=4 IRSTLM name=LM1 factor=1 path=/home/kamal.pcs17/set_hin
di_to_eng_wplg/lm/corpora_train.pos.lm.final.arpa.en order=4

input-factors: 0 1 2

input-file: /home/kamal.pcs17/set_hindi_to
_eng_wplg/tune/corpora_tune.hi

mapping: 0 T 0 0 T 1 0 G 0

n-best-list: run1.best100.out 100 distinct

weight: UnknownWordPenalty0= 1 WordPenalty0= -1
PhrasePenalty0= 0.2 TranslationModel0= 0.2 0.2 0.2 0.2 TranslationModel1=
0.2 0.2 0.2 0.2 GenerationModel0= 0.3 0 Distortion0= 0.3 LM0= 0.5 LM1= 0.5

weight-overwrite: GenerationModel0= 0.068182 0.000000 LM0=
0.113636 LM1= 0.113636 WordPenalty0= -0.227273 PhrasePenalty0= 0.045455
TranslationModel0= 0.045455 0.045455 0.045455 0.045455 Distortion0=
0.068182 TranslationModel1= 0.045455 0.045455 0.045455 0.045455

line=UnknownWordPenalty

FeatureFunction: UnknownWordPenalty0 start: 0 end: 0

line=WordPenalty

FeatureFunction: WordPenalty0 start: 1 end: 1

line=PhrasePenalty

FeatureFunction: PhrasePenalty0 start: 2 end: 2

line=PhraseDictionaryMemory name=TranslationModel0 num-features=4
path=/home/kamal.pcs17/set_hindi_to_eng_wplg/mert-work/filtered/phrase-table.1,2-1.1.1.gz
input-factor=1,2 output-factor=1

FeatureFunction: TranslationModel0 start: 3 end: 6

line=PhraseDictionaryMemory name=TranslationModel1 num-features=4
path=/home/kamal.pcs17/set_hindi_to_eng_wplg/mert-work/filtered/phrase-table.1,2-2.1.1.gz
input-factor=1,2 output-factor=2

FeatureFunction: TranslationModel1 start: 7 end: 10

line=Generation name=GenerationModel0 num-features=2
path=/home/kamal.pcs17/set_hindi_to_eng_wplg/working/train/model/generation.1,2-0.gz
input-factor=1,2 output-factor=0

FeatureFunction: GenerationModel0 start: 11 end: 12

line=Distortion

FeatureFunction: Distortion0 start: 13 end: 13

line=IRSTLM name=LM0 factor=0 path=/home/kamal.pcs17/set_hin
di_to_eng_wplg/lm/corpora_train.lm.final.arpa.en order=4

FeatureFunction: LM0 start: 14 end: 14

line=IRSTLM name=LM1 factor=1 path=/home/kamal.pcs17/set_hin
di_to_eng_wplg/lm/corpora_train.pos.lm.final.arpa.en order=4

FeatureFunction: LM1 start: 15 end: 15

Loading UnknownWordPenalty0

Loading WordPenalty0

Loading PhrasePenalty0

Loading GenerationModel0

terminate called after throwing an instance of 'std::string'

sh: line 1: 55086 Aborted
/home/kamal.pcs17/smt/mosesdecoder/bin/moses
-config filtered/moses.ini -weight-overwrite 'GenerationModel0= 0.068182
0.000000 LM0= 0.113636 LM1= 0.113636 WordPenalty0= -0.227273
PhrasePenalty0= 0.045455 TranslationModel0= 0.045455 0.045455 0.045455
0.045455 Distortion0= 0.068182 TranslationModel1= 0.045455 0.045455
0.045455 0.045455' -n-best-list run1.best100.out 100 distinct -input-file
/home/kamal.pcs17/set_hindi_to_eng_wplg/tune/corpora_tune.hi > run1.out

Exit code: 134

The decoder died. CONFIG WAS -weight-overwrite 'GenerationModel0= 0.068182
0.000000 LM0= 0.113636 LM1= 0.113636 WordPenalty0= -0.227273
PhrasePenalty0= 0.045455 TranslationModel0= 0.045455 0.045455 0.045455
0.045455 Distortion0= 0.068182 TranslationModel1= 0.045455 0.045455
0.045455 0.045455'



Please Help.


--

Thanks and Regards

Debajyoty
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20180331/a9c79cf7/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 137, Issue 13
**********************************************

0 Response to "Moses-support Digest, Vol 137, Issue 13"

Post a Comment

Subscribe to: Post Comments (Atom)