Moses-support Digest, Vol 89, Issue 55

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Loading a binary rule table (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Fri, 21 Mar 2014 19:49:11 +0000
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] Loading a binary rule table
To: Massinissa Ahmim <massinissa.ahmim@linguacustodia.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAEKMkbjGSjhF01FDdgmxBsjGG67xZw2fosRL+kA8ucGtBJ1kSg@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

i've never seen that error before. Did you change anything else, other than
rebinarizing the phrase-table? What is the exact command you ran when you
got this error? Can I have a look at your moses.ini file


On 21 March 2014 18:32, Massinissa Ahmim <
massinissa.ahmim@linguacustodia.com> wrote:

> Thank you Hieu,
>
> I've binarised my rule-table again and it seems to be successfully loaded
> but I'm getting this message now :
>
> Reading /disque2/Preparation/syntactic/hierarchical_PSCT/rules-table
>
> ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
>
> ****************************************************************************************************
> Exception: <fd:3>: Is a directory from zlib
>
> What can I do?
>
> Thanks !
>
> Regards
>
> Massinissa
>
>
> 2014-03-21 11:18 GMT+01:00 Hieu Hoang <Hieu.Hoang@ed.ac.uk>:
>
> everything seems to be ok. Are you sure the server you are running on has
>> access to the directory
>> /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table
>>
>> Also, you should consider updating your version of Moses to the current
>> github version, or version 2.1.1
>>
>>
>> On 20 March 2014 11:20, Massinissa Ahmim <
>> massinissa.ahmim@linguacustodia.com> wrote:
>>
>>> Dear all,
>>>
>>> I've managed to train a hierarchical model using the following command :
>>>
>>> nohup /mosesdecoder/scripts/training/train-model.perl --hierarchical
>>> --glue-grammar --score-options="--GoodTuring" -root-dir
>>> /disque2/Preparation/syntactic/hierarchical_PSCT -corpus
>>> /disque2/Preparation/backoff/PSCT.tok.uni.low -f fr -e en -lm
>>> 0:5:/disque2/Preparation/syntactic/hierarchical/LM.sur.en.blm
>>> -external-bin-dir /root/external-bin-dir/ -mgiza -mgiza-cpus 30 >&
>>> training.out &
>>>
>>> The resulting model works great but it is very slow.
>>>
>>> So I used CreatOnDiskPt to binarise the rule table as follows :
>>>
>>> /home/Moses/mosesdecoder/bin/CreateOnDiskPt 1 1 5 20 2
>>> model/rule-table.gz rules-table
>>>
>>> This outputs the following files
>>>
>>> -rw-r--r-- 1 root root 85 20 mars 11:35 Misc.dat
>>> -rw-r--r-- 1 root root 781509322 20 mars 11:35 Source.dat
>>> -rw-r--r-- 1 root root 1758588137 20 mars 11:35 TargetColl.dat
>>> -rw-r--r-- 1 root root 1555353459 20 mars 11:35 TargetInd.dat
>>> -rw-r--r-- 1 root root 400714 20 mars 11:35 Vocab.dat
>>>
>>> and I've updated my moses.ini as follows :
>>>
>>> 6 0 0 1 /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table
>>> 6 0 0 1
>>> /disque2/Preparation/syntactic/hierarchical_PSCT/model/glue-grammar
>>>
>>> But as I try to use it, I'm getting this :
>>>
>>> Defined parameters (per moses.ini or switch):
>>> config: moses.ini
>>> cube-pruning-pop-limit: 1000
>>> input-factors: 0
>>> inputtype: 3
>>> lmodel-file: 0 0 5
>>> /disque2/Preparation/syntactic/hierarchical/LM.sur.en.blm
>>> mapping: 0 T 0 1 T 1
>>> max-chart-span: 20 1000
>>> non-terminals: X
>>> search-algorithm: 3
>>> ttable-file: 6 0 0 1
>>> /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table 6 0 0 1
>>> /disque2/Preparation/syntactic/hierarchical_PSCT/model/glue-grammar
>>> ttable-limit: 20
>>> weight-l: 0.5000
>>> weight-t: 0.20 0.20 0.20 0.20 0.20 1.0
>>> weight-w: -1
>>> /mosesdecoder/bin
>>> ScoreProducer: WordPenalty start: 0 end: 1
>>> ScoreProducer: !UnknownWordPenalty start: 1 end: 2
>>> Loading lexical distortion models...have 0 models
>>> Start loading LanguageModel
>>> /disque2/Preparation/syntactic/hierarchical/LM.sur.en.blm : [0.000] seconds
>>> /disque2/Preparation/syntactic/hierarchical/LM.sur.en.blm: line 80492:
>>> reached EOF before \end\
>>> ScoreProducer: LM start: 2 end: 3
>>> Finished loading LanguageModels : [0.056] seconds
>>> Using uniform ttable-limit of 20 for all translation tables.
>>> Start loading PhraseTable
>>> /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table : [0.056]
>>> seconds
>>> filePath: /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table
>>> ScoreProducer: PhraseModel start: 3 end: 4
>>> Start loading PhraseTable
>>> /disque2/Preparation/syntactic/hierarchical_PSCT/model/glue-grammar :
>>> [0.056] seconds
>>> filePath:
>>> /disque2/Preparation/syntactic/hierarchical_PSCT/model/glue-grammar
>>> ScoreProducer: PhraseModel:2 start: 4 end: 5
>>> Finished loading phrase tables : [0.056] seconds
>>> max-chart-span: 20
>>> max-chart-span: 1000
>>> Start loading phrase table from
>>> /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table : [0.056]
>>> seconds
>>> Can't read /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table
>>>
>>> So I wondering if I did something wrong with my training
>>> command/binarisation or with the parameters in the moses.ini
>>>
>>>
>>> Many thanks
>>>
>>> Regads
>>>
>>> MA
>>>
>>>
>>>
>>>
>>> --
>>>
>>> [image: Description : Description : lingua_custodia_final full logo]
>>>
>>> *The Translation Trustee*
>>>
>>> *1, Place Charles de Gaulle*
>>>
>>> *78180 Montigny-le-Bretonneux*
>>>
>>> *Tel : +33 1 30 44 04 23 Mobile : +33 7 61 44 40 84*
>>>
>>> *Email :* *massinissa.ahmim@linguacustodia.com
>>> <massinissa.ahmim@linguacustodia.com>*
>>>
>>> *Website :* *www.linguacustodia.com <http://www.linguacustodia.com/> -
>>> www.thetranslationtrustee.com <http://www.thetranslationtrustee.com>*
>>>
>>> ? Pensez ? l'environnement, n'imprimez ce courriel que si n?cessaire.
>>>
>>> Please do not print this email unless it is absolutely necessary. Spread
>>> environmental awareness.
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>>
>> --
>> Hieu Hoang
>> Research Associate
>> University of Edinburgh
>> http://www.hoang.co.uk/hieu
>>
>>
>
>
> --
>
> [image: Description : Description : lingua_custodia_final full logo]
>
> *The Translation Trustee*
>
> *1, Place Charles de Gaulle*
>
> *78180 Montigny-le-Bretonneux*
>
> *Tel : +33 1 30 44 04 23 Mobile : +33 7 61 44 40 84*
>
> *Email :* *massinissa.ahmim@linguacustodia.com
> <massinissa.ahmim@linguacustodia.com>*
>
> *Website :* *www.linguacustodia.com <http://www.linguacustodia.com/> -
> www.thetranslationtrustee.com <http://www.thetranslationtrustee.com>*
>
> ? Pensez ? l'environnement, n'imprimez ce courriel que si n?cessaire.
>
> Please do not print this email unless it is absolutely necessary. Spread
> environmental awareness.
>



--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140321/e9651561/attachment.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 4421 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20140321/e9651561/attachment.jpg

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 89, Issue 55
*********************************************

Tweet