Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Fw: Re: Decoder died in tunning step! (Arezoo Arjomand)
----------------------------------------------------------------------
Message: 1
Date: Wed, 5 Sep 2018 12:53:13 +0000 (UTC)
From: Arezoo Arjomand <arezooarjomand@yahoo.com>
Subject: [Moses-support] Fw: Re: Decoder died in tunning step!
To: Hieu Hoang <hieuhoang@gmail.com>, Moses-support
<moses-support@mit.edu>, "moses-support-request@mit.edu"
<moses-support-request@mit.edu>, "moses-support-owner@mit.edu"
<moses-support-owner@mit.edu>, "moses-support-bounces@mit.edu"
<moses-support-bounces@mit.edu>, Hieu Hoang <hieu.hoang@ed.ac.uk>
Message-ID: <95504789.433492.1536151993684@mail.yahoo.com>
Content-Type: text/plain; charset="utf-8"
Hi,
Thank you for your reply.??I run the following command before tunning step.
/home/arezoo1/mosesdecoder-master/mosesdecoder-master/bin/CreateProbingPT2 --input-pt /home/arezoo1/emlla/train/model/rule-table.gz? --output-dir /home/arezoo1/emlla/train/model/rule-table.1.gz
and change the moses.ini to the output-dir.
PhraseDictionaryMemory name=TranslationModel0 num-features=4 path=/home/arezoo1/emlla/train/model/rule-table.1.gz input-factor=0 output-factor=0
PhraseDictionaryMemory name=TranslationModel1 num-features=1 path=/home/arezoo1/emlla/train/model/glue-grammar input-factor=0 output-factor=0 tuneable=true
and run tunning step as follows:
/home/arezoo1/mosesdecoder-master/mosesdecoder-master/scripts/training/mert-moses.pl /home/arezoo1/emlla/tune/nc-dev2007.true.fr /home/arezoo1/emlla/tune/nc-dev2007.true.en /home/arezoo1/mosesdecoder-master/mosesdecoder-master/bin/moses2 /home/arezoo1/emlla/train/model/moses.ini --rootdir /home/arezoo1/mosesdecoder-master/moesdecoder-master/scripts --decoder-flags "-v 0" --no-filter-phrase-table --inputtype 3 --batch-mira --return-best-dev --filtercmd '/home/arezoo1/mosesdecoder-master/mosesdecoder-master/scripts/training/filter-model-given-input.pl --batch-mira-args '-J 300' 1>&mert.out
It creates features.list,run1.dense, run1.best100.out, run1.moses.ini, run1.out.
the mart.out is contain:
Using SCRIPTS_ROOTDIR: /home/arezoo1/mosesdecoder-master/mosesdecoder-master/scripts
Assuming --mertdir=/home/arezoo1/mosesdecoder-master/mosesdecoder-master/bin
Using cached features list: ./features.list
MERT starting values and ranges for random generation:
??? LM0 =?? 0.500 ( 0.00 ..? 1.00)
? WordPenalty0 =? -1.000 ( 0.00 ..? 1.00)
? PhrasePenalty0 =?? 0.200 ( 0.00 ..? 1.00)
? TranslationModel0 =?? 0.200 ( 0.00 ..? 1.00)
? TranslationModel0 =?? 0.200 ( 0.00 ..? 1.00)
? TranslationModel0 =?? 0.200 ( 0.00 ..? 1.00)
? TranslationModel0 =?? 0.200 ( 0.00 ..? 1.00)
? TranslationModel1 =?? 1.000 ( 0.00 ..? 1.00)
featlist: LM0=0.500000
featlist: WordPenalty0=-1.000000
featlist: PhrasePenalty0=0.200000
featlist: TranslationModel0=0.200000
featlist: TranslationModel0=0.200000
featlist: TranslationModel0=0.200000
featlist: TranslationModel0=0.200000
featlist: TranslationModel1=1.000000
run 1 start at Sun Sep? 2 07:51:49 EDT 2018
Parsing --decoder-flags: |-threads 8 -v 0|
Saving new config to: ./run1.moses.ini
Saved: ./run1.moses.ini
Normalizing lambdas: 0.500000 -1.000000 0.200000 0.200000 0.200000 0.200000 0.200000 1.000000
DECODER_CFG = -weight-overwrite 'PhrasePenalty0= 0.057143 TranslationModel0= 0.057143 0.057143 0.057143 0.057143 WordPenalty0= -0.285714 LM0= 0.142857 TranslationModel1= 0.285714'
Executing: /home/arezoo1/mosesdecoder-master/mosesdecoder-master/bin/moses2 -threads 8 -v 0? -config /home/arezoo1/emlla/train/model/moses.ini -inputtype 3 -weight-overwrite 'PhrasePenalty0= 0.057143 TranslationModel0= 0.057143 0.057143 0.057143 0.057143 WordPenalty0= -0.285714 LM0= 0.142857 TranslationModel1= 0.285714'? -n-best-list run1.best100.out 100 distinct? -input-file /home/arezoo1/emlla/tune/nc-dev2007.true.fr > run1.out
Executing: /home/arezoo1/mosesdecoder-master/mosesdecoder-master/bin/moses2 -threads 8 -v 0? -config /home/arezoo1/emlla/train/model/moses.ini -inputtype 3 -weight-overwrite 'PhrasePenalty0= 0.057143 TranslationModel0= 0.057143 0.057143 0.057143 0.057143 WordPenalty0= -0.285714 LM0= 0.142857 TranslationModel1= 0.285714'? -n-best-list run1.best100.out 100 distinct? -input-file /home/arezoo1/emlla/tune/nc-dev2007.true.fr > run1.out
1-10.20.2 0.2 0.2 0.210.5(1) run decoder to produce n-best lists
params = -threads 8 -v 0
decoder_config = -weight-overwrite 'PhrasePenalty0= 0.057143 TranslationModel0= 0.057143 0.057143 0.057143 0.057143 WordPenalty0= -0.285714 LM0= 0.142857 TranslationModel1= 0.285714'
Starting...
START featureFunctions.Load()
Loading WordPenalty0
Finished loading WordPenalty0
Loading PhrasePenalty0
Finished loading PhrasePenalty0
Loading LM0
Finished loading LM0
Loading UnknownWordPenalty0
Finished loading UnknownWordPenalty0
Loading TranslationModel0
Finished loading TranslationModel0
Loading TranslationModel1
Finished loading TranslationModel1
START LoadMappings()
END LoadMappings()
END LoadDecodeGraphBackoff()
Loaded : [0.00543796] seconds
RUN BATCH
terminate called after throwing an instance of 'util::Exception'
? what():? moses2/SCFG/nbest/KBestExtractor.cpp:42 in void Moses2::SCFG::KBestExtractor::OutputToStream(std::stringstream&) threw util::Exception because `lastStack.GetColl().size() != 1'.
Only suppose to be 1 hypo coll in last stack
Aborted (core dumped)
Exit code: 134
The decoder died. CONFIG WAS -weight-overwrite 'PhrasePenalty0= 0.057143 TranslationModel0= 0.057143 0.057143 0.057143 0.057143 WordPenalty0= -0.285714 LM0= 0.142857 TranslationModel1= 0.285714'
I really appreciate your help in resolving the problem.
Arezoo
On Thursday, August 23, 2018, 6:22:14 AM EDT, Hieu Hoang <hieuhoang@gmail.com> wrote:
you need to binarize with CreateProbingPT if you use moses2
if you use CreateOnDiskPt then you must use the moses decoder
Hieu Hoanghttp://statmt.org/hieu
On Thu, 23 Aug 2018 at 09:02, Arezoo Arjomand <arezooarjomand@yahoo.com> wrote:
Hi,
I installed successfully moses decoder on server. I run syntax grammer extraction as follow :
?
/home/vps/mosesdecoder/scripts/training/train-model.perl? \
-root-dir train? \
-corpus /home/vps/emlla//ttt/news-commentary-v8.fr-en.clean? \
-f fr -e en -alignment grow-diag-final-and? -hierarchical ???-glue-grammar ??????\? -lm 0:3:/home/vps/emlla//ttt/news-commentary-v8.fr-en.blm.en? \
-mgiza -mgiza-cpus 4? \
-external-bin-dir ?/homevps/mosesdecoder/tools >&training2.out
?
the file which created is :
?
train ? corpus , giza.en-fr, giza.fr-en, model
model ? aligned.grow-diag-final-and, extract.sorted.gz, lex.e2f,? moses.ini, extract.inv.sorted.gz,??????? glue-grammar, lex.f2e? rule-table.gz
?
to tune the model the following command is run:
?
/home/vps/mosesdecoder/scripts/training/mert-moses.pl? /home/vps/emlla/nc-dev2007.en.true.fr? /home/vps/emlla/nc-dev2007.en.true.en? /home/vps/mosesdecoder/bin/moses2 /home/vps/emlla/train/model/moses.ini --rootdir? /home/vps/mosesdecoder/scripts --decoder-flags "-v 0" --no-filter-phrase-table --inputtype 3???? --batch-mira --return-best-dev? --filtercmd '/home/vps/mosesdecoder/scripts/training/filter-model-given-input.pl -Binarizer "CreateOnDiskPt 1 1 5 100 2" ' --batch-mira-args '-J 300' --decoder-flags '-threads 8 -v 0' 1>&mert.out????????????????????????????????????
but it dies in following step: ????????????????????????????????????????????????????????????????????????? ???????????????????????
Finished loading LM0
Loading UnknownWordPenalty0
Finished loading UnknownWordPenalty0
Loading TranslationModel0
Finished loading TranslationModel0
Loading TranslationModel1
Finished loading TranslationModel1
START LoadMappings()
END LoadMappings()
END LoadDecodeGraphBackoff()
Loaded : [0.0490292] seconds
RUN BATCH
terminate called after throwing an instance of 'util::Exception'
? what():? moses2/SCFG/nbest/KBestExtractor.cpp:42 in void Moses2::SCFG::KBestE$
Only suppose to be 1 hypo coll in last stack
Aborted (core dumped)
Exit code: 134
The decoder died. CONFIG WAS -weight-overwrite 'TranslationModel1= 0.285714 Phr$
how can I fix it?
Thank? you for your attention
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20180905/cd76a6da/attachment.html
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 143, Issue 3
*********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 143, Issue 3"
Post a Comment