Moses-support Digest, Vol 102, Issue 15

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Can't read binarised reordering-table (Abdelfetah Boumerdas)
2. Re: Can't read binarised reordering-table (Marcin Junczys-Dowmunt)


----------------------------------------------------------------------

Message: 1
Date: Wed, 8 Apr 2015 13:45:04 +0100
From: Abdelfetah Boumerdas <aa_boumerdas@esi.dz>
Subject: Re: [Moses-support] Can't read binarised reordering-table
To: Hokage Sama <nvncbol@gmail.com>
Cc: moses-support@mit.edu
Message-ID:
<CABJLC3eO5a+KciUXtO8U5MZsdUNOde2DEjXVwkAJoUKyMSCGNw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hello everyone,
Here's what i have done exactly and still couldn't translate a single
sentence:
1- I followed the tutorial in the moses manual
2- after the tuning i tried to test weather it works or not so i executed
moses with the moses.ini file produced just after tuning and the decoder
consumed all my memory (8 GB) and blocked my laptop
3- when i asked i was told to try and compact the phrase table and the
reordering table
4- so i recompiled moses with the cmph library
5- i sorted the two tables and then compacted them using the
ProcessPhraseTableMin and ProcessLexicalTableMin
6- i tried to change in the moses.ini file from "PhraseDictionaryMemory" to
"PhraseDictionaryCompact" i got an error that tells me that this name is
not registred so changed it to PhraseDictionaryBinary
7- i executed moses with this new moses.ini file and now it gives me the
possibility to provide the sentence i want to translate but after i enter
the sentence i get a segementation fault message.

I think this is the same problem posted by Hokage Sama. So can anyone tell
what's exactly wrong?

Thanks everyone
?

2015-04-08 12:54 GMT+01:00 Hieu Hoang <hieuhoang@gmail.com>:

> oops sorry. I've updated the webpage
>
>
> On 08/04/2015 12:07, Muhammad Danial Raza wrote:
>
> I faced the same issue earlier, when following Baseline System. I solved
> it by changing PhraseDictionaryBinary to PhraseDictionaryCompact. I
> remember reading somewhere that now moses.ini should contain
> PhraseDictionaryCompact instead of PhraseDictionaryBinary. The
> documentation has not been updated at Baseline System.
>
> P.S.: Sorry for not giving the actual link to the forum where I read it,
> it was a month ago when I faced this issue.
>
> Regards,
> Danial
>
> On Wed, Apr 8, 2015 at 12:54 PM, Marcin Junczys-Dowmunt <
> junczys@amu.edu.pl> wrote:
>
>> hm, weird, that should work. Can you do "ls -lah" in the directory that
>> holds the reordering-table and post the result? Also please post your ini
>> file.
>>
>> W dniu 08.04.2015 o 09:48, Hokage Sama pisze:
>>
>> Thanks for the prompt reply. However I'm still having the problem.
>> Getting this now:
>> Can't read /home/hilton/working/binarised-model/reordering-table
>>
>> I tried adding the extension to make it "reordering-table.minlexr", and
>> I get this:
>> Loading LexicalReordering0
>> Loading table into memory...Exception: vector::_M_range_check
>>
>> On 8 April 2015 at 02:31, Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
>> wrote:
>>
>>> Hi,
>>> replace "$HOME" with a real, absolute path.
>>> Best,
>>> Marcin
>>>
>>> W dniu 08.04.2015 o 09:23, Hokage Sama pisze:
>>>
>>> Hello I've been following the Moses manual to create a Baseline
>>> System but I am now stuck at the Testing section. I get a message saying it
>>> can't read the reordering-table as shown below. Appreciate any help thanks.
>>>
>>> $ ~/mosesdecoder/bin/moses -f ~/working/binarised-model/moses.ini
>>> Defined parameters (per moses.ini or switch):
>>> config: /home/hilton/working/binarised-model/moses.ini
>>> distortion-limit: 6
>>> feature: UnknownWordPenalty WordPenalty PhrasePenalty
>>> PhraseDictionaryBinary name=TranslationModel0 num-features=4
>>> path=$HOME/working/binarised-model/phrase-table input-factor=0
>>> output-factor=0 LexicalReordering name=LexicalReordering0 num-features=6
>>> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
>>> path=$HOME/working/binarised-model/reordering-table Distortion KENLM
>>> lazyken=0 name=LM0 factor=0
>>> path=/home/hilton/lm/news-commentary-v8.fr-en.blm.en order=3
>>> input-factors: 0
>>> mapping: 0 T 0
>>> weight: LexicalReordering0= 0.07048 0.00134696 0.0375093 0.0623156
>>> 0.00105617 0.0594233 Distortion0= 0.0693777 LM0= 0.0598649 WordPenalty0=
>>> -0.221146 PhrasePenalty0= 0.22473 TranslationModel0= 0.0301386 0.0472199
>>> 0.0481032 0.067288 UnknownWordPenalty0= 1
>>> line=UnknownWordPenalty
>>> FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
>>> line=WordPenalty
>>> FeatureFunction: WordPenalty0 start: 1 end: 1
>>> line=PhrasePenalty
>>> FeatureFunction: PhrasePenalty0 start: 2 end: 2
>>> line=PhraseDictionaryBinary name=TranslationModel0 num-features=4
>>> path=$HOME/working/binarised-model/phrase-table input-factor=0
>>> output-factor=0
>>> FeatureFunction: TranslationModel0 start: 3 end: 6
>>> line=LexicalReordering name=LexicalReordering0 num-features=6
>>> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
>>> path=$HOME/working/binarised-model/reordering-table
>>> FeatureFunction: LexicalReordering0 start: 7 end: 12
>>> Initializing Lexical Reordering Feature..
>>> line=Distortion
>>> FeatureFunction: Distortion0 start: 13 end: 13
>>> line=KENLM lazyken=0 name=LM0 factor=0
>>> path=/home/hilton/lm/news-commentary-v8.fr-en.blm.en order=3
>>> FeatureFunction: LM0 start: 14 end: 14
>>> Loading UnknownWordPenalty0
>>> Loading WordPenalty0
>>> Loading PhrasePenalty0
>>> Loading LexicalReordering0
>>> Can't read $HOME/working/binarised-model/reordering-table
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
>
> _______________________________________________
> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>


--
BOUMERDAS Abdelfetah
5?me Ann?e Option Syst?mes Informatiques (SIQ)
Ecole nationale Sup?rieure d'Informatique ESI (ex INI)
BP 68 M Oued Smar 16309 - ALGER
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150408/2522ffc2/attachment-0001.htm

------------------------------

Message: 2
Date: Wed, 08 Apr 2015 15:40:12 +0200
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: Re: [Moses-support] Can't read binarised reordering-table
To: Abdelfetah Boumerdas <aa_boumerdas@esi.dz>
Cc: moses-support@mit.edu
Message-ID: <5e265b236dd345bedd2d915e5f246ebd@amu.edu.pl>
Content-Type: text/plain; charset="utf-8"



Hi,

The compact phrase table will NOT work with PhraseDictionyBinary, you
need to use PhraseDictionaryCompact. That feature should be registered
if you compile moses correctly with the --with-cmph option. Can you give
us the exact compilation command? Can you recompile with the "-a"
option? Make sure you use the Moses version that has been compiled to
with --with-cmph for translation. That sounds like a mix-up of moses
versions. Or someone on our side messed up the build system, I haven't
checked yet.

W dniu 2015-04-08 14:45, Abdelfetah Boumerdas napisa?(a):

> Hello everyone,
> Here's what i have done exactly and still couldn't translate a single sentence:
> 1- I followed the tutorial in the moses manual
> 2- after the tuning i tried to test weather it works or not so i executed moses with the moses.ini file produced just after tuning and the decoder consumed all my memory (8 GB) and blocked my laptop
> 3- when i asked i was told to try and compact the phrase table and the reordering table
> 4- so i recompiled moses with the cmph library
> 5- i sorted the two tables and then compacted them using the ProcessPhraseTableMin and ProcessLexicalTableMin
> 6- i tried to change in the moses.ini file from "PhraseDictionaryMemory" to "PhraseDictionaryCompact" i got an error that tells me that this name is not registred so changed it to PhraseDictionaryBinary
> 7- i executed moses with this new moses.ini file and now it gives me the possibility to provide the sentence i want to translate but after i enter the sentence i get a segementation fault message.
>
> I think this is the same problem posted by Hokage Sama. So can anyone tell what's exactly wrong?
>
> Thanks everyone
> ?
>
> 2015-04-08 12:54 GMT+01:00 Hieu Hoang <hieuhoang@gmail.com>:
>
> oops sorry. I've updated the webpage
>
> On 08/04/2015 12:07, Muhammad Danial Raza wrote:
> I faced the same issue earlier, when following Baseline System. I solved it by changing PhraseDictionaryBinary to PhraseDictionaryCompact. I remember reading somewhere that now moses.ini should contain PhraseDictionaryCompact instead of PhraseDictionaryBinary. The documentation has not been updated at Baseline System.
>
> P.S.: Sorry for not giving the actual link to the forum where I read it, it was a month ago when I faced this issue.
>
> Regards,
> Danial
>
> On Wed, Apr 8, 2015 at 12:54 PM, Marcin Junczys-Dowmunt <junczys@amu.edu.pl> wrote:
>
> hm, weird, that should work. Can you do "ls -lah" in the directory that holds the reordering-table and post the result? Also please post your ini file.
>
> W dniu 08.04.2015 o 09:48, Hokage Sama pisze:
>
> Thanks for the prompt reply. However I'm still having the problem. Getting this now:
> Can't read /home/hilton/working/binarised-model/reordering-table
>
> I tried adding the extension to make it "reordering-table.minlexr", and I get this:
>
> Loading LexicalReordering0
> Loading table into memory...Exception: vector::_M_range_check
>
> On 8 April 2015 at 02:31, Marcin Junczys-Dowmunt <junczys@amu.edu.pl> wrote:
>
> Hi,
> replace "$HOME" with a real, absolute path.
> Best,
> Marcin
>
> W dniu 08.04.2015 o 09:23, Hokage Sama pisze:
>
> Hello I've been following the Moses manual to create a Baseline System but I am now stuck at the Testing section. I get a message saying it can't read the reordering-table as shown below. Appreciate any help thanks.
>
> $ ~/mosesdecoder/bin/moses -f ~/working/binarised-model/moses.ini
> Defined parameters (per moses.ini or switch):
> config: /home/hilton/working/binarised-model/moses.ini
> distortion-limit: 6
> feature: UnknownWordPenalty WordPenalty PhrasePenalty PhraseDictionaryBinary name=TranslationModel0 num-features=4 path=$HOME/working/binarised-model/phrase-table input-factor=0 output-factor=0 LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 path=$HOME/working/binarised-model/reordering-table Distortion KENLM lazyken=0 name=LM0 factor=0 path=/home/hilton/lm/news-commentary-v8.fr-en.blm.en order=3
> input-factors: 0
> mapping: 0 T 0
> weight: LexicalReordering0= 0.07048 0.00134696 0.0375093 0.0623156 0.00105617 0.0594233 Distortion0= 0.0693777 LM0= 0.0598649 WordPenalty0= -0.221146 PhrasePenalty0= 0.22473 TranslationModel0= 0.0301386 0.0472199 0.0481032 0.067288 UnknownWordPenalty0= 1
> line=UnknownWordPenalty
> FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
> line=WordPenalty
> FeatureFunction: WordPenalty0 start: 1 end: 1
> line=PhrasePenalty
> FeatureFunction: PhrasePenalty0 start: 2 end: 2
> line=PhraseDictionaryBinary name=TranslationModel0 num-features=4 path=$HOME/working/binarised-model/phrase-table input-factor=0 output-factor=0
> FeatureFunction: TranslationModel0 start: 3 end: 6
> line=LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 path=$HOME/working/binarised-model/reordering-table
> FeatureFunction: LexicalReordering0 start: 7 end: 12
> Initializing Lexical Reordering Feature..
> line=Distortion
> FeatureFunction: Distortion0 start: 13 end: 13
> line=KENLM lazyken=0 name=LM0 factor=0 path=/home/hilton/lm/news-commentary-v8.fr-en.blm.en order=3
> FeatureFunction: LM0 start: 14 end: 14
> Loading UnknownWordPenalty0
> Loading WordPenalty0
> Loading PhrasePenalty0
> Loading LexicalReordering0
> Can't read $HOME/working/binarised-model/reordering-table
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support [1]
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support [1]

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support [1]

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support [1]

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support [1]

--

BOUMERDAS Abdelfetah
5?me Ann?e Option Syst?mes Informatiques (SIQ)
Ecole nationale Sup?rieure d'Informatique ESI (ex INI)
BP 68 M Oued Smar 16309 - ALGER

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support [1]



Links:
------
[1] http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150408/9f184dcf/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 102, Issue 15
**********************************************

Tweet