Moses-support Digest, Vol 89, Issue 54

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Loading a binary rule table (Massinissa Ahmim)


----------------------------------------------------------------------

Message: 1
Date: Fri, 21 Mar 2014 19:32:09 +0100
From: Massinissa Ahmim <massinissa.ahmim@linguacustodia.com>
Subject: Re: [Moses-support] Loading a binary rule table
To: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CANN0mWYqzyK-037ia0=xNrfj5o_vjaQNnuPNTAf5ojzcsFVG8Q@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Thank you Hieu,

I've binarised my rule-table again and it seems to be successfully loaded
but I'm getting this message now :

Reading /disque2/Preparation/syntactic/hierarchical_PSCT/rules-table
----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
****************************************************************************************************
Exception: <fd:3>: Is a directory from zlib

What can I do?

Thanks !

Regards

Massinissa


2014-03-21 11:18 GMT+01:00 Hieu Hoang <Hieu.Hoang@ed.ac.uk>:

> everything seems to be ok. Are you sure the server you are running on has
> access to the directory
> /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table
>
> Also, you should consider updating your version of Moses to the current
> github version, or version 2.1.1
>
>
> On 20 March 2014 11:20, Massinissa Ahmim <
> massinissa.ahmim@linguacustodia.com> wrote:
>
>> Dear all,
>>
>> I've managed to train a hierarchical model using the following command :
>>
>> nohup /mosesdecoder/scripts/training/train-model.perl --hierarchical
>> --glue-grammar --score-options="--GoodTuring" -root-dir
>> /disque2/Preparation/syntactic/hierarchical_PSCT -corpus
>> /disque2/Preparation/backoff/PSCT.tok.uni.low -f fr -e en -lm
>> 0:5:/disque2/Preparation/syntactic/hierarchical/LM.sur.en.blm
>> -external-bin-dir /root/external-bin-dir/ -mgiza -mgiza-cpus 30 >&
>> training.out &
>>
>> The resulting model works great but it is very slow.
>>
>> So I used CreatOnDiskPt to binarise the rule table as follows :
>>
>> /home/Moses/mosesdecoder/bin/CreateOnDiskPt 1 1 5 20 2
>> model/rule-table.gz rules-table
>>
>> This outputs the following files
>>
>> -rw-r--r-- 1 root root 85 20 mars 11:35 Misc.dat
>> -rw-r--r-- 1 root root 781509322 20 mars 11:35 Source.dat
>> -rw-r--r-- 1 root root 1758588137 20 mars 11:35 TargetColl.dat
>> -rw-r--r-- 1 root root 1555353459 20 mars 11:35 TargetInd.dat
>> -rw-r--r-- 1 root root 400714 20 mars 11:35 Vocab.dat
>>
>> and I've updated my moses.ini as follows :
>>
>> 6 0 0 1 /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table
>> 6 0 0 1
>> /disque2/Preparation/syntactic/hierarchical_PSCT/model/glue-grammar
>>
>> But as I try to use it, I'm getting this :
>>
>> Defined parameters (per moses.ini or switch):
>> config: moses.ini
>> cube-pruning-pop-limit: 1000
>> input-factors: 0
>> inputtype: 3
>> lmodel-file: 0 0 5
>> /disque2/Preparation/syntactic/hierarchical/LM.sur.en.blm
>> mapping: 0 T 0 1 T 1
>> max-chart-span: 20 1000
>> non-terminals: X
>> search-algorithm: 3
>> ttable-file: 6 0 0 1
>> /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table 6 0 0 1
>> /disque2/Preparation/syntactic/hierarchical_PSCT/model/glue-grammar
>> ttable-limit: 20
>> weight-l: 0.5000
>> weight-t: 0.20 0.20 0.20 0.20 0.20 1.0
>> weight-w: -1
>> /mosesdecoder/bin
>> ScoreProducer: WordPenalty start: 0 end: 1
>> ScoreProducer: !UnknownWordPenalty start: 1 end: 2
>> Loading lexical distortion models...have 0 models
>> Start loading LanguageModel
>> /disque2/Preparation/syntactic/hierarchical/LM.sur.en.blm : [0.000] seconds
>> /disque2/Preparation/syntactic/hierarchical/LM.sur.en.blm: line 80492:
>> reached EOF before \end\
>> ScoreProducer: LM start: 2 end: 3
>> Finished loading LanguageModels : [0.056] seconds
>> Using uniform ttable-limit of 20 for all translation tables.
>> Start loading PhraseTable
>> /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table : [0.056]
>> seconds
>> filePath: /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table
>> ScoreProducer: PhraseModel start: 3 end: 4
>> Start loading PhraseTable
>> /disque2/Preparation/syntactic/hierarchical_PSCT/model/glue-grammar :
>> [0.056] seconds
>> filePath:
>> /disque2/Preparation/syntactic/hierarchical_PSCT/model/glue-grammar
>> ScoreProducer: PhraseModel:2 start: 4 end: 5
>> Finished loading phrase tables : [0.056] seconds
>> max-chart-span: 20
>> max-chart-span: 1000
>> Start loading phrase table from
>> /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table : [0.056]
>> seconds
>> Can't read /disque2/Preparation/syntactic/hierarchical_PSCT/rule-table
>>
>> So I wondering if I did something wrong with my training
>> command/binarisation or with the parameters in the moses.ini
>>
>>
>> Many thanks
>>
>> Regads
>>
>> MA
>>
>>
>>
>>
>> --
>>
>> [image: Description : Description : lingua_custodia_final full logo]
>>
>> *The Translation Trustee*
>>
>> *1, Place Charles de Gaulle*
>>
>> *78180 Montigny-le-Bretonneux*
>>
>> *Tel : +33 1 30 44 04 23 Mobile : +33 7 61 44 40 84*
>>
>> *Email :* *massinissa.ahmim@linguacustodia.com
>> <massinissa.ahmim@linguacustodia.com>*
>>
>> *Website :* *www.linguacustodia.com <http://www.linguacustodia.com/> -
>> www.thetranslationtrustee.com <http://www.thetranslationtrustee.com>*
>>
>> ? Pensez ? l'environnement, n'imprimez ce courriel que si n?cessaire.
>>
>> Please do not print this email unless it is absolutely necessary. Spread
>> environmental awareness.
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
>
> --
> Hieu Hoang
> Research Associate
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>
>


--

[image: Description : Description : lingua_custodia_final full logo]

*The Translation Trustee*

*1, Place Charles de Gaulle*

*78180 Montigny-le-Bretonneux*

*Tel : +33 1 30 44 04 23 Mobile : +33 7 61 44 40 84*

*Email :* *massinissa.ahmim@linguacustodia.com
<massinissa.ahmim@linguacustodia.com>*

*Website :* *www.linguacustodia.com <http://www.linguacustodia.com/> -
www.thetranslationtrustee.com <http://www.thetranslationtrustee.com>*

? Pensez ? l'environnement, n'imprimez ce courriel que si n?cessaire.

Please do not print this email unless it is absolutely necessary. Spread
environmental awareness.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140321/d27ac5a7/attachment.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 4421 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20140321/d27ac5a7/attachment.jpg

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 89, Issue 54
*********************************************

Tweet