Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. On-Disk Phrase table error (Apurva Joshi)
2. Re: On-Disk Phrase table error (Apurva Joshi)
----------------------------------------------------------------------
Message: 1
Date: Tue, 3 Nov 2015 11:55:32 +0530
From: Apurva Joshi <apurvajoshi1992@gmail.com>
Subject: [Moses-support] On-Disk Phrase table error
To: moses-support@mit.edu
Message-ID:
<CAMfCXbXwA1vNxrWRzSbHDEOFHob2zJwxmfC76CMhp4cXaOK=AQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
hello all,
initially my moses.ini file contains :
PhraseDictionaryMemory name=TranslationModel0 num-features=4
path=/home/techmahindra/working/train/model/phrase-table.gz input-factor=0
output-factor=0
Now to binarise phrase table , we need to convert the rule table into a
binary prefix format. This is done with the command CreateOnDiskPt:
so i executed following command on terminal
/home/techmahindra/mosesdecoder-RELEASE-3.0/bin/CreateOnDiskPt 1 1 4 100 2
/home/techmahindra/working/train/model/phrase-table.gz
phrase-table.1.folder
my "CreateOnDiskPt" file is in ==
/home/techmahindra/mosesdecoder-RELEASE-3.0/bin/
inital phrse table "phrase-table.gz" is in ==
/home/techmahindra/working/train/model/
after executing "CreateOnDiskPt" file === "phrase-table.1.folder" is
generated in my home directory ( i.e. /home/techmahindra)
this "phrase-table.1.folder" == contains 5 files named ==
Misc.dat
Source.dat
TargetColl.dat
TargetInd.dat
Vocab.dat
this is correct as per given by manual...
Now , next i have made following changes in moses.ini
before executing "CreateOnDiskPt" ...there was
PhraseDictionaryMemory name=TranslationModel0 num-features=4
path=/home/techmahindra/working/train/model/phrase-table.gz input-factor=0
output-factor=0
i commented above with # and added following ...
PhraseDictionaryOnDisk name=TranslationModel0 num-features=4
path=/home/techmahindra/phrase-table.1.folder input-factor=1 output-factor=1
and saved moses.ini
now i goto terminal and as usual process of starting moses...
i ran moses -f moses.ini
then surprisingly with less time loading has completed ....
my output on terminal after loading "moses.ini" is ===
Defined parameters (per moses.ini or switch):
config: /home/techmahindra/working/train/model/moses.ini
distortion-limit: 6
feature: UnknownWordPenalty WordPenalty PhrasePenalty
PhraseDictionaryOnDisk name=TranslationModel0 num-features=4
path=/home/techmahindra/phrase-table.1.folder input-factor=1
output-factor=1 LexicalReordering name=LexicalReordering0 num-features=6
type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
path=/home/techmahindra/working/train/model/reordering-table.wbe-msd-bidirectional-fe.gz
Distortion KENLM lazyken=0 name=LM0 factor=0
path=/home/techmahindra/lm/corpora.blm.hi order=3
input-factors: 0
mapping: 0 T 0
weight: UnknownWordPenalty0= 1 WordPenalty0= -1 PhrasePenalty0= 0.2
TranslationModel0= 0.2 0.2 0.2 0.2 LexicalReordering0= 0.3 0.3 0.3 0.3 0.3
0.3 Distortion0= 0.3 LM0= 0.5
line=UnknownWordPenalty
FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
line=WordPenalty
FeatureFunction: WordPenalty0 start: 1 end: 1
line=PhrasePenalty
FeatureFunction: PhrasePenalty0 start: 2 end: 2
line=PhraseDictionaryOnDisk name=TranslationModel0 num-features=4
path=/home/techmahindra/phrase-table.1.folder input-factor=1 output-factor=1
FeatureFunction: TranslationModel0 start: 3 end: 6
line=LexicalReordering name=LexicalReordering0 num-features=6
type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
path=/home/techmahindra/working/train/model/reordering-table.wbe-msd-bidirectional-fe.gz
FeatureFunction: LexicalReordering0 start: 7 end: 12
Initializing LexicalReordering..
line=Distortion
FeatureFunction: Distortion0 start: 13 end: 13
line=KENLM lazyken=0 name=LM0 factor=0
path=/home/techmahindra/lm/corpora.blm.hi order=3
FeatureFunction: LM0 start: 14 end: 14
Loading UnknownWordPenalty0
Loading WordPenalty0
Loading PhrasePenalty0
Loading LexicalReordering0
Loading table into memory...done.
Loading Distortion0
Loading LM0
Loading TranslationModel0
Created input-output object : [115.699] seconds
NOW, when i give input(e.g. "hello" , i expect hindi translation of
"hello")....but it is giving following error ....
hello
Translating: hello
Line 0: Initialize search took 0.300 seconds total
terminate called after throwing an instance of 'util::Exception'
what(): OnDiskPt/OnDiskWrapper.cpp:230 in OnDiskPt::Word*
OnDiskPt::OnDiskWrapper::ConvertFromMoses(const std::vector<long unsigned
int>&, const Moses::Word&) const threw util::Exception because `factor ==
__null'.
Expecting factor 1
Aborted (core dumped)
plzz help ...plz do reply on apurvajoshi1992@gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151103/c086f219/attachment-0001.html
------------------------------
Message: 2
Date: Tue, 3 Nov 2015 12:21:30 +0530
From: Apurva Joshi <apurvajoshi1992@gmail.com>
Subject: Re: [Moses-support] On-Disk Phrase table error
To: moses-support@mit.edu
Message-ID:
<CAMfCXbXiw7+dPbjRzUrbCB6k5km3U33e53_mxuKX_80BzhCO2w@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
hello all i got solution ....
intially input - factor and output factor are 0 and 0 resp...like
PhraseDictionaryMemory name=TranslationModel0 num-features=4
path=/home/techmahindra/working/train/model/phrase-table.gz input-factor=0
output-factor=0
then i changed to
PhraseDictionaryOnDisk name=TranslationModel0 num-features=4
path=/home/techmahindra/phrase-table.1.folder input-factor=1
output-factor=1
here factor are 1 1...i changed it to 0 0...
its working
On Tue, Nov 3, 2015 at 11:55 AM, Apurva Joshi <apurvajoshi1992@gmail.com>
wrote:
> hello all,
> initially my moses.ini file contains :
>
> PhraseDictionaryMemory name=TranslationModel0 num-features=4
> path=/home/techmahindra/working/train/model/phrase-table.gz input-factor=0
> output-factor=0
>
> Now to binarise phrase table , we need to convert the rule table into a
> binary prefix format. This is done with the command CreateOnDiskPt:
>
> so i executed following command on terminal
>
> /home/techmahindra/mosesdecoder-RELEASE-3.0/bin/CreateOnDiskPt 1 1 4 100 2
> /home/techmahindra/working/train/model/phrase-table.gz
> phrase-table.1.folder
>
> my "CreateOnDiskPt" file is in ==
> /home/techmahindra/mosesdecoder-RELEASE-3.0/bin/
>
> inital phrse table "phrase-table.gz" is in ==
> /home/techmahindra/working/train/model/
>
> after executing "CreateOnDiskPt" file === "phrase-table.1.folder" is
> generated in my home directory ( i.e. /home/techmahindra)
>
>
> this "phrase-table.1.folder" == contains 5 files named ==
>
> Misc.dat
> Source.dat
> TargetColl.dat
> TargetInd.dat
> Vocab.dat
>
> this is correct as per given by manual...
>
> Now , next i have made following changes in moses.ini
>
> before executing "CreateOnDiskPt" ...there was
>
> PhraseDictionaryMemory name=TranslationModel0 num-features=4
> path=/home/techmahindra/working/train/model/phrase-table.gz input-factor=0
> output-factor=0
>
>
>
> i commented above with # and added following ...
>
> PhraseDictionaryOnDisk name=TranslationModel0 num-features=4
> path=/home/techmahindra/phrase-table.1.folder input-factor=1 output-factor=1
>
>
> and saved moses.ini
>
>
> now i goto terminal and as usual process of starting moses...
>
> i ran moses -f moses.ini
>
> then surprisingly with less time loading has completed ....
>
> my output on terminal after loading "moses.ini" is ===
>
>
> Defined parameters (per moses.ini or switch):
> config: /home/techmahindra/working/train/model/moses.ini
> distortion-limit: 6
> feature: UnknownWordPenalty WordPenalty PhrasePenalty
> PhraseDictionaryOnDisk name=TranslationModel0 num-features=4
> path=/home/techmahindra/phrase-table.1.folder input-factor=1
> output-factor=1 LexicalReordering name=LexicalReordering0 num-features=6
> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
> path=/home/techmahindra/working/train/model/reordering-table.wbe-msd-bidirectional-fe.gz
> Distortion KENLM lazyken=0 name=LM0 factor=0
> path=/home/techmahindra/lm/corpora.blm.hi order=3
> input-factors: 0
> mapping: 0 T 0
> weight: UnknownWordPenalty0= 1 WordPenalty0= -1 PhrasePenalty0= 0.2
> TranslationModel0= 0.2 0.2 0.2 0.2 LexicalReordering0= 0.3 0.3 0.3 0.3 0.3
> 0.3 Distortion0= 0.3 LM0= 0.5
> line=UnknownWordPenalty
> FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
> line=WordPenalty
> FeatureFunction: WordPenalty0 start: 1 end: 1
> line=PhrasePenalty
> FeatureFunction: PhrasePenalty0 start: 2 end: 2
> line=PhraseDictionaryOnDisk name=TranslationModel0 num-features=4
> path=/home/techmahindra/phrase-table.1.folder input-factor=1 output-factor=1
> FeatureFunction: TranslationModel0 start: 3 end: 6
> line=LexicalReordering name=LexicalReordering0 num-features=6
> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
> path=/home/techmahindra/working/train/model/reordering-table.wbe-msd-bidirectional-fe.gz
> FeatureFunction: LexicalReordering0 start: 7 end: 12
> Initializing LexicalReordering..
> line=Distortion
> FeatureFunction: Distortion0 start: 13 end: 13
> line=KENLM lazyken=0 name=LM0 factor=0
> path=/home/techmahindra/lm/corpora.blm.hi order=3
> FeatureFunction: LM0 start: 14 end: 14
> Loading UnknownWordPenalty0
> Loading WordPenalty0
> Loading PhrasePenalty0
> Loading LexicalReordering0
> Loading table into memory...done.
> Loading Distortion0
> Loading LM0
> Loading TranslationModel0
> Created input-output object : [115.699] seconds
>
>
> NOW, when i give input(e.g. "hello" , i expect hindi translation of
> "hello")....but it is giving following error ....
>
>
> hello
> Translating: hello
> Line 0: Initialize search took 0.300 seconds total
> terminate called after throwing an instance of 'util::Exception'
> what(): OnDiskPt/OnDiskWrapper.cpp:230 in OnDiskPt::Word*
> OnDiskPt::OnDiskWrapper::ConvertFromMoses(const std::vector<long unsigned
> int>&, const Moses::Word&) const threw util::Exception because `factor ==
> __null'.
> Expecting factor 1
> Aborted (core dumped)
>
>
>
>
> plzz help ...plz do reply on apurvajoshi1992@gmail.com
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151103/948d337b/attachment-0001.html
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 109, Issue 9
*********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 109, Issue 9"
Post a Comment