Moses-support Digest, Vol 102, Issue 11

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Can't read binarised reordering-table (Muhammad Danial Raza)
2. Re: Can't read binarised reordering-table (Hokage Sama)


----------------------------------------------------------------------

Message: 1
Date: Wed, 8 Apr 2015 13:23:04 +0500
From: Muhammad Danial Raza <muhammad.danial.raza@gmail.com>
Subject: Re: [Moses-support] Can't read binarised reordering-table
To: Hokage Sama <nvncbol@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEQzcOfG1MqnuvPZnAvV6kpxWF=Q7A1EF+j8TW7bZ=juOof-zA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Are the binarised models and non-binarised models in the same folder?

On Wed, Apr 8, 2015 at 1:14 PM, Hokage Sama <nvncbol@gmail.com> wrote:

> Appreciate the response. I get this now if I change it to
> PhraseDictionaryCompact:
>
> line=PhraseDictionaryCompact name=TranslationModel0 num-features=4
> path=/home/hilton/working/binarised-model/phrase-table input-factor=0
> output-factor=0
> Exception: moses/FF/Factory.cpp:331 in void
> Moses::FeatureRegistry::Construct(const string&, const string&) threw
> UnknownFeatureException because `i == registry_.end()'.
> Feature name PhraseDictionaryCompact is not registered.
>
>
> On 8 April 2015 at 03:07, Muhammad Danial Raza <
> muhammad.danial.raza@gmail.com> wrote:
>
>> I faced the same issue earlier, when following Baseline System. I solved
>> it by changing PhraseDictionaryBinary to PhraseDictionaryCompact. I
>> remember reading somewhere that now moses.ini should contain
>> PhraseDictionaryCompact instead of PhraseDictionaryBinary. The
>> documentation has not been updated at Baseline System.
>>
>> P.S.: Sorry for not giving the actual link to the forum where I read it,
>> it was a month ago when I faced this issue.
>>
>> Regards,
>> Danial
>>
>> On Wed, Apr 8, 2015 at 12:54 PM, Marcin Junczys-Dowmunt <
>> junczys@amu.edu.pl> wrote:
>>
>>> hm, weird, that should work. Can you do "ls -lah" in the directory
>>> that holds the reordering-table and post the result? Also please post your
>>> ini file.
>>>
>>> W dniu 08.04.2015 o 09:48, Hokage Sama pisze:
>>>
>>> Thanks for the prompt reply. However I'm still having the problem.
>>> Getting this now:
>>> Can't read /home/hilton/working/binarised-model/reordering-table
>>>
>>> I tried adding the extension to make it "reordering-table.minlexr",
>>> and I get this:
>>> Loading LexicalReordering0
>>> Loading table into memory...Exception: vector::_M_range_check
>>>
>>> On 8 April 2015 at 02:31, Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
>>> wrote:
>>>
>>>> Hi,
>>>> replace "$HOME" with a real, absolute path.
>>>> Best,
>>>> Marcin
>>>>
>>>> W dniu 08.04.2015 o 09:23, Hokage Sama pisze:
>>>>
>>>> Hello I've been following the Moses manual to create a Baseline
>>>> System but I am now stuck at the Testing section. I get a message saying it
>>>> can't read the reordering-table as shown below. Appreciate any help thanks.
>>>>
>>>> $ ~/mosesdecoder/bin/moses -f ~/working/binarised-model/moses.ini
>>>> Defined parameters (per moses.ini or switch):
>>>> config: /home/hilton/working/binarised-model/moses.ini
>>>> distortion-limit: 6
>>>> feature: UnknownWordPenalty WordPenalty PhrasePenalty
>>>> PhraseDictionaryBinary name=TranslationModel0 num-features=4
>>>> path=$HOME/working/binarised-model/phrase-table input-factor=0
>>>> output-factor=0 LexicalReordering name=LexicalReordering0 num-features=6
>>>> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
>>>> path=$HOME/working/binarised-model/reordering-table Distortion KENLM
>>>> lazyken=0 name=LM0 factor=0
>>>> path=/home/hilton/lm/news-commentary-v8.fr-en.blm.en order=3
>>>> input-factors: 0
>>>> mapping: 0 T 0
>>>> weight: LexicalReordering0= 0.07048 0.00134696 0.0375093 0.0623156
>>>> 0.00105617 0.0594233 Distortion0= 0.0693777 LM0= 0.0598649 WordPenalty0=
>>>> -0.221146 PhrasePenalty0= 0.22473 TranslationModel0= 0.0301386 0.0472199
>>>> 0.0481032 0.067288 UnknownWordPenalty0= 1
>>>> line=UnknownWordPenalty
>>>> FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
>>>> line=WordPenalty
>>>> FeatureFunction: WordPenalty0 start: 1 end: 1
>>>> line=PhrasePenalty
>>>> FeatureFunction: PhrasePenalty0 start: 2 end: 2
>>>> line=PhraseDictionaryBinary name=TranslationModel0 num-features=4
>>>> path=$HOME/working/binarised-model/phrase-table input-factor=0
>>>> output-factor=0
>>>> FeatureFunction: TranslationModel0 start: 3 end: 6
>>>> line=LexicalReordering name=LexicalReordering0 num-features=6
>>>> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
>>>> path=$HOME/working/binarised-model/reordering-table
>>>> FeatureFunction: LexicalReordering0 start: 7 end: 12
>>>> Initializing Lexical Reordering Feature..
>>>> line=Distortion
>>>> FeatureFunction: Distortion0 start: 13 end: 13
>>>> line=KENLM lazyken=0 name=LM0 factor=0
>>>> path=/home/hilton/lm/news-commentary-v8.fr-en.blm.en order=3
>>>> FeatureFunction: LM0 start: 14 end: 14
>>>> Loading UnknownWordPenalty0
>>>> Loading WordPenalty0
>>>> Loading PhrasePenalty0
>>>> Loading LexicalReordering0
>>>> Can't read $HOME/working/binarised-model/reordering-table
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150408/d8fcbfdb/attachment-0001.htm

------------------------------

Message: 2
Date: Wed, 8 Apr 2015 03:27:25 -0500
From: Hokage Sama <nvncbol@gmail.com>
Subject: Re: [Moses-support] Can't read binarised reordering-table
To: Muhammad Danial Raza <muhammad.danial.raza@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAD3ogMbLzCQrZRNaz=aOdTjdkvyk7PXi=kuzOinQoFLFNiLvQA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

No they are in separate folders:
binarised: working/binarised-model/
non-binarised: working/train/model/

On 8 April 2015 at 03:23, Muhammad Danial Raza <
muhammad.danial.raza@gmail.com> wrote:

> Are the binarised models and non-binarised models in the same folder?
>
> On Wed, Apr 8, 2015 at 1:14 PM, Hokage Sama <nvncbol@gmail.com> wrote:
>
>> Appreciate the response. I get this now if I change it to
>> PhraseDictionaryCompact:
>>
>> line=PhraseDictionaryCompact name=TranslationModel0 num-features=4
>> path=/home/hilton/working/binarised-model/phrase-table input-factor=0
>> output-factor=0
>> Exception: moses/FF/Factory.cpp:331 in void
>> Moses::FeatureRegistry::Construct(const string&, const string&) threw
>> UnknownFeatureException because `i == registry_.end()'.
>> Feature name PhraseDictionaryCompact is not registered.
>>
>>
>> On 8 April 2015 at 03:07, Muhammad Danial Raza <
>> muhammad.danial.raza@gmail.com> wrote:
>>
>>> I faced the same issue earlier, when following Baseline System. I solved
>>> it by changing PhraseDictionaryBinary to PhraseDictionaryCompact. I
>>> remember reading somewhere that now moses.ini should contain
>>> PhraseDictionaryCompact instead of PhraseDictionaryBinary. The
>>> documentation has not been updated at Baseline System.
>>>
>>> P.S.: Sorry for not giving the actual link to the forum where I read it,
>>> it was a month ago when I faced this issue.
>>>
>>> Regards,
>>> Danial
>>>
>>> On Wed, Apr 8, 2015 at 12:54 PM, Marcin Junczys-Dowmunt <
>>> junczys@amu.edu.pl> wrote:
>>>
>>>> hm, weird, that should work. Can you do "ls -lah" in the directory
>>>> that holds the reordering-table and post the result? Also please post your
>>>> ini file.
>>>>
>>>> W dniu 08.04.2015 o 09:48, Hokage Sama pisze:
>>>>
>>>> Thanks for the prompt reply. However I'm still having the problem.
>>>> Getting this now:
>>>> Can't read /home/hilton/working/binarised-model/reordering-table
>>>>
>>>> I tried adding the extension to make it "reordering-table.minlexr",
>>>> and I get this:
>>>> Loading LexicalReordering0
>>>> Loading table into memory...Exception: vector::_M_range_check
>>>>
>>>> On 8 April 2015 at 02:31, Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>> replace "$HOME" with a real, absolute path.
>>>>> Best,
>>>>> Marcin
>>>>>
>>>>> W dniu 08.04.2015 o 09:23, Hokage Sama pisze:
>>>>>
>>>>> Hello I've been following the Moses manual to create a Baseline
>>>>> System but I am now stuck at the Testing section. I get a message saying it
>>>>> can't read the reordering-table as shown below. Appreciate any help thanks.
>>>>>
>>>>> $ ~/mosesdecoder/bin/moses -f ~/working/binarised-model/moses.ini
>>>>> Defined parameters (per moses.ini or switch):
>>>>> config: /home/hilton/working/binarised-model/moses.ini
>>>>> distortion-limit: 6
>>>>> feature: UnknownWordPenalty WordPenalty PhrasePenalty
>>>>> PhraseDictionaryBinary name=TranslationModel0 num-features=4
>>>>> path=$HOME/working/binarised-model/phrase-table input-factor=0
>>>>> output-factor=0 LexicalReordering name=LexicalReordering0 num-features=6
>>>>> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
>>>>> path=$HOME/working/binarised-model/reordering-table Distortion KENLM
>>>>> lazyken=0 name=LM0 factor=0
>>>>> path=/home/hilton/lm/news-commentary-v8.fr-en.blm.en order=3
>>>>> input-factors: 0
>>>>> mapping: 0 T 0
>>>>> weight: LexicalReordering0= 0.07048 0.00134696 0.0375093 0.0623156
>>>>> 0.00105617 0.0594233 Distortion0= 0.0693777 LM0= 0.0598649 WordPenalty0=
>>>>> -0.221146 PhrasePenalty0= 0.22473 TranslationModel0= 0.0301386 0.0472199
>>>>> 0.0481032 0.067288 UnknownWordPenalty0= 1
>>>>> line=UnknownWordPenalty
>>>>> FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
>>>>> line=WordPenalty
>>>>> FeatureFunction: WordPenalty0 start: 1 end: 1
>>>>> line=PhrasePenalty
>>>>> FeatureFunction: PhrasePenalty0 start: 2 end: 2
>>>>> line=PhraseDictionaryBinary name=TranslationModel0 num-features=4
>>>>> path=$HOME/working/binarised-model/phrase-table input-factor=0
>>>>> output-factor=0
>>>>> FeatureFunction: TranslationModel0 start: 3 end: 6
>>>>> line=LexicalReordering name=LexicalReordering0 num-features=6
>>>>> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
>>>>> path=$HOME/working/binarised-model/reordering-table
>>>>> FeatureFunction: LexicalReordering0 start: 7 end: 12
>>>>> Initializing Lexical Reordering Feature..
>>>>> line=Distortion
>>>>> FeatureFunction: Distortion0 start: 13 end: 13
>>>>> line=KENLM lazyken=0 name=LM0 factor=0
>>>>> path=/home/hilton/lm/news-commentary-v8.fr-en.blm.en order=3
>>>>> FeatureFunction: LM0 start: 14 end: 14
>>>>> Loading UnknownWordPenalty0
>>>>> Loading WordPenalty0
>>>>> Loading PhrasePenalty0
>>>>> Loading LexicalReordering0
>>>>> Can't read $HOME/working/binarised-model/reordering-table
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Moses-support mailing list
>>>>> Moses-support@mit.edu
>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>>
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150408/5a38b228/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 102, Issue 11
**********************************************

Tweet