Home » Uncategories » Moses-support Digest, Vol 98, Issue 14

Moses-support Digest, Vol 98, Issue 14

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Running Moses with a compact lexical reordering
(Marcin Junczys-Dowmunt)


----------------------------------------------------------------------

Message: 1
Date: Mon, 01 Dec 2014 17:26:43 +0100
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: Re: [Moses-support] Running Moses with a compact lexical
reordering
To: Massinissa Ahmim <massinissa.ahmim@linguacustodia.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <547C96C3.6070605@amu.edu.pl>
Content-Type: text/plain; charset="utf-8"

I am out of ideas now. I will give you access to one of my reordering
tables that I know are working in the evening. If it still fails it is
some configuration error somewhere.

W dniu 01.12.2014 o 17:21, Massinissa Ahmim pisze:
> Hi,
>
> Yes, I just finished processing another one and still I'm getting the
> same message :
>
> Loading table into memory...Exception: vector::_M_range_check
>
> 2014-12-01 17:13 GMT+01:00 Marcin Junczys-Dowmunt <junczys@amu.edu.pl
> <mailto:junczys@amu.edu.pl>>:
>
> Hm, OK,
> paths look good. Do you have another compact reordering table
> somewhere that you can substitute there and try? I am starting to
> suspect that the current one is corrupted for some reason. If it
> works with another table, then we would have proof.
>
> W dniu 01.12.2014 o 17:08, Massinissa Ahmim pisze:
>> Hi Marcin,
>>
>> Please find here attached the file
>>
>> Many thanks
>>
>> Massinissa
>>
>> 2014-12-01 17:02 GMT+01:00 Marcin Junczys-Dowmunt
>> <junczys@amu.edu.pl <mailto:junczys@amu.edu.pl>>:
>>
>> Hm, slighlty clueless by now.
>>
>> Can you execute
>>
>> find /home/train-degaulle-fren > paths.txt
>>
>> and send me the paths.txt file to my e-mails address? Do not
>> put that on the support-list.
>>
>> W dniu 01.12.2014 o 17:00, Massinissa Ahmim pisze:
>>> Hi,
>>>
>>> I'm getting the first error message now :
>>>
>>> Loading table into memory...Exception: vector::_M_range_check
>>>
>>> 2014-12-01 16:56 GMT+01:00 Marcin Junczys-Dowmunt
>>> <junczys@amu.edu.pl <mailto:junczys@amu.edu.pl>>:
>>>
>>> Put the suffix .minlexr back, that was a failed advise.
>>>
>>> W dniu 01.12.2014 o 16:56, Massinissa Ahmim pisze:
>>>> I'm getting the same message :
>>>>
>>>> Can't read
>>>> /home/train-degaulle-fren/training/model/reordering-table
>>>>
>>>> 2014-12-01 16:49 GMT+01:00 Marcin Junczys-Dowmunt
>>>> <junczys@amu.edu.pl <mailto:junczys@amu.edu.pl>>:
>>>>
>>>> Could you check whether this happens if you do not
>>>> use the phrase table combination? Just one normal
>>>> phrase table and a standard decoding path "0 T 0" ?
>>>>
>>>> W dniu 01.12.2014 o 16:43, Massinissa Ahmim pisze:
>>>>> Hi Marcin,
>>>>>
>>>>> the reordering table is over 12 GB, can I send a
>>>>> sample of the inital file instead?
>>>>>
>>>>> here is my moses.ini :
>>>>>
>>>>> ### MOSES CONFIG FILE ###
>>>>> #########################
>>>>>
>>>>> # input factors
>>>>> [input-factors]
>>>>> 0
>>>>>
>>>>> # mapping steps
>>>>> [mapping]
>>>>> 0 T 2
>>>>>
>>>>>
>>>>> [distortion-limit]
>>>>> 6
>>>>>
>>>>> # feature functions
>>>>> [feature]
>>>>> UnknownWordPenalty
>>>>> WordPenalty
>>>>> PhrasePenalty
>>>>> #PhraseDictionaryCompact tuneable=false
>>>>> name=TranslationModel0 num-features=4
>>>>> path=/home/Update4/train-pros-fren-update4/binarised-model/phrase-table-pros-fren-update4.minphr
>>>>> input-factor=0 output-factor=0 table-limit=20
>>>>>
>>>>>
>>>>> PhraseDictionaryCompact tuneable=false
>>>>> name=TranslationModel0 num-features=4
>>>>> path=/home/train-degaulle-fren/binarised-model/phrase-table-degaulle-firsthalf.minphr
>>>>> input-factor=0 output-factor=0 table-limit=20
>>>>>
>>>>>
>>>>> PhraseDictionaryCompact tuneable=false
>>>>> name=TranslationModel1 num-features=4
>>>>> path=/home/train-degaulle-fren/binarised-model/phrase-table-degaulle-secondhalf.minphr
>>>>> input-factor=0 output-factor=0 table-limit=20
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> PhraseDictionaryMultiModel num-features=4
>>>>> input-factor=0 output-factor=0 table-limit=20
>>>>> mode=interpolate lambda=0.5,0.5
>>>>> components=TranslationModel0,TranslationModel1
>>>>>
>>>>> LexicalReordering name=LexicalReordering0
>>>>> num-features=6 type=wbe-msd-bidirectional-fe-
>>>>> allff input-factor=0 output-factor=0
>>>>> path=/home/train-degaulle-fren/training/model/reordering-table.minlexr
>>>>>
>>>>>
>>>>> Distortion
>>>>> KENLM lazyken=0 name=LM0 factor=0
>>>>> path=/home/DATA/LM/Corps-lm.blm.en.mm
>>>>> <http://Corps-lm.blm.en.mm> order=5
>>>>>
>>>>> # dense weights for feature functions
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> [threads]
>>>>> all
>>>>> [weight]
>>>>>
>>>>> LexicalReordering0= 0.0564145 -0.00809962
>>>>> 0.0939308 0.114042 0.0630804 0.0683044
>>>>> Distortion0= 0.0452205
>>>>> LM0= 0.140316
>>>>> WordPenalty0= 0.0793205
>>>>> PhrasePenalty0= 0.113696
>>>>> PhraseDictionaryMultiModel0= 0.0608326 0.102939
>>>>> 0.0533373 0.000466626
>>>>> TranslationModel0= 0 0 1 0
>>>>> TranslationModel1= 0 0 1 0
>>>>> UnknownWordPenalty0= 1
>>>>>
>>>>>
>>>>> 2014-12-01 16:30 GMT+01:00 Marcin Junczys-Dowmunt
>>>>> <junczys@amu.edu.pl <mailto:junczys@amu.edu.pl>>:
>>>>>
>>>>> Hm,
>>>>> weird. In that case you should keep the
>>>>> suffix. Can you make the reordering table
>>>>> available for download somehow? Can I see your
>>>>> full INI-file?
>>>>>
>>>>> W dniu 01.12.2014 o 16:27, Massinissa Ahmim pisze:
>>>>>> Hi Marcin,
>>>>>>
>>>>>> I'm getting this now :
>>>>>>
>>>>>> Can't read
>>>>>> /home/train-degaulle-fren/training/model/reordering-table
>>>>>>
>>>>>> I double checked the paths, they seem to be
>>>>>> correct
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> Massinissa
>>>>>>
>>>>>> 2014-12-01 16:17 GMT+01:00 Marcin
>>>>>> Junczys-Dowmunt <junczys@amu.edu.pl
>>>>>> <mailto:junczys@amu.edu.pl>>:
>>>>>>
>>>>>> Hi,
>>>>>> try removing the ".minlexr" suffix from
>>>>>> the ini file. Does it work after that?
>>>>>>
>>>>>> W dniu 01.12.2014 o 16:14, Massinissa
>>>>>> Ahmim pisze:
>>>>>>> Dear all,
>>>>>>>
>>>>>>> I'm having troubles running moses with a
>>>>>>> compact lexical reordering table.
>>>>>>>
>>>>>>> After sorting the initial table, I ran
>>>>>>> the following command :
>>>>>>>
>>>>>>> /home/Moses/mosesdecoder/bin/processLexicalTableMin
>>>>>>> -in reordering-table.sort -out
>>>>>>> reordering-table -threads 32
>>>>>>>
>>>>>>> the process has been successfully
>>>>>>> completed so I've updated my moses.ini
>>>>>>> file :
>>>>>>>
>>>>>>> LexicalReordering
>>>>>>> name=LexicalReordering0 num-features=6
>>>>>>> type=wbe-msd-bidirectional-fe-allff
>>>>>>> input-factor=0 output-factor=0
>>>>>>> path=/home/train-deg
>>>>>>> aulle-fren/training/model/reordering-table.minlexr
>>>>>>>
>>>>>>>
>>>>>>> and ran moses like this :
>>>>>>>
>>>>>>> /home/Moses/mosesdecoder/bin/moses -f
>>>>>>> moses2.ini -minlexr-memory
>>>>>>>
>>>>>>> and I'm getting the following error :
>>>>>>>
>>>>>>> ....
>>>>>>> line=LexicalReordering
>>>>>>> name=LexicalReordering0 num-features=6
>>>>>>> type=wbe-msd-bidirectional-fe-allff
>>>>>>> input-factor=0 output-factor=0
>>>>>>> path=/home/train-degaulle-fren/training/model/reordering-table.minlexr
>>>>>>> FeatureFunction: LexicalReordering0
>>>>>>> start: 15 end: 20
>>>>>>> Initializing LexicalReordering..
>>>>>>> line=Distortion
>>>>>>> FeatureFunction: Distortion0 start: 21
>>>>>>> end: 21
>>>>>>> line=KENLM lazyken=0 name=LM0 factor=0
>>>>>>> path=/home/DATA/LM/Corps-lm.blm.en.mm
>>>>>>> <http://Corps-lm.blm.en.mm> order=5
>>>>>>> FeatureFunction: LM0 start: 22 end: 22
>>>>>>> Loading UnknownWordPenalty0
>>>>>>> Loading WordPenalty0
>>>>>>> Loading PhrasePenalty0
>>>>>>> Loading LexicalReordering0
>>>>>>> Loading table into memory...Exception:
>>>>>>> vector::_M_range_check
>>>>>>>
>>>>>>>
>>>>>>> Many thanks
>>>>>>>
>>>>>>> Massinissa
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Description : Description :
>>>>>>> lingua_custodia_final full logo
>>>>>>>
>>>>>>> */The Translation Trustee/*
>>>>>>>
>>>>>>> *1, Place Charles de Gaulle*
>>>>>>>
>>>>>>> *78180 Montigny-le-Bretonneux*
>>>>>>>
>>>>>>> *Tel : +33 1 30 44 04 23 Mobile : +33
>>>>>>> 7 61 44 40 84*
>>>>>>>
>>>>>>> *Email :****massinissa.ahmim@linguacustodia.com
>>>>>>> <mailto:massinissa.ahmim@linguacustodia.com>***
>>>>>>>
>>>>>>> *Website :****www.linguacustodia.com
>>>>>>> <http://www.linguacustodia.com/> -
>>>>>>> www.thetranslationtrustee.com
>>>>>>> <http://www.thetranslationtrustee.com>*
>>>>>>>
>>>>>>> A Young Innovative Company recognised by
>>>>>>> the French Research Ministry and Paris
>>>>>>> Finance Innovation cluster
>>>>>>>
>>>>>>> Logo labellis? FI small
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Moses-support mailing list
>>>>>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Moses-support mailing list
>>>>>> Moses-support@mit.edu
>>>>>> <mailto:Moses-support@mit.edu>
>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Description : Description :
>>>>>> lingua_custodia_final full logo
>>>>>>
>>>>>> */The Translation Trustee/*
>>>>>>
>>>>>> *1, Place Charles de Gaulle*
>>>>>>
>>>>>> *78180 Montigny-le-Bretonneux*
>>>>>>
>>>>>> *Tel : +33 1 30 44 04 23 Mobile : +33 7 61
>>>>>> 44 40 84*
>>>>>>
>>>>>> *Email :****massinissa.ahmim@linguacustodia.com
>>>>>> <mailto:massinissa.ahmim@linguacustodia.com>***
>>>>>>
>>>>>> *Website :****www.linguacustodia.com
>>>>>> <http://www.linguacustodia.com/> -
>>>>>> www.thetranslationtrustee.com
>>>>>> <http://www.thetranslationtrustee.com>*
>>>>>>
>>>>>> A Young Innovative Company recognised by the
>>>>>> French Research Ministry and Paris Finance
>>>>>> Innovation cluster
>>>>>>
>>>>>> Logo labellis? FI small
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Description : Description : lingua_custodia_final
>>>>> full logo
>>>>>
>>>>> */The Translation Trustee/*
>>>>>
>>>>> *1, Place Charles de Gaulle*
>>>>>
>>>>> *78180 Montigny-le-Bretonneux*
>>>>>
>>>>> *Tel : +33 1 30 44 04 23 Mobile : +33 7 61 44 40 84*
>>>>>
>>>>> *Email :****massinissa.ahmim@linguacustodia.com
>>>>> <mailto:massinissa.ahmim@linguacustodia.com>***
>>>>>
>>>>> *Website :****www.linguacustodia.com
>>>>> <http://www.linguacustodia.com/> -
>>>>> www.thetranslationtrustee.com
>>>>> <http://www.thetranslationtrustee.com>*
>>>>>
>>>>> A Young Innovative Company recognised by the
>>>>> French Research Ministry and Paris Finance
>>>>> Innovation cluster
>>>>>
>>>>> Logo labellis? FI small
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Description : Description : lingua_custodia_final full logo
>>>>
>>>> */The Translation Trustee/*
>>>>
>>>> *1, Place Charles de Gaulle*
>>>>
>>>> *78180 Montigny-le-Bretonneux*
>>>>
>>>> *Tel : +33 1 30 44 04 23 Mobile : +33 7 61 44 40 84*
>>>>
>>>> *Email :****massinissa.ahmim@linguacustodia.com
>>>> <mailto:massinissa.ahmim@linguacustodia.com>***
>>>>
>>>> *Website :****www.linguacustodia.com
>>>> <http://www.linguacustodia.com/> -
>>>> www.thetranslationtrustee.com
>>>> <http://www.thetranslationtrustee.com>*
>>>>
>>>> A Young Innovative Company recognised by the French
>>>> Research Ministry and Paris Finance Innovation cluster
>>>>
>>>> Logo labellis? FI small
>>>>
>>>
>>>
>>>
>>>
>>> --
>>>
>>> Description : Description : lingua_custodia_final full logo
>>>
>>> */The Translation Trustee/*
>>>
>>> *1, Place Charles de Gaulle*
>>>
>>> *78180 Montigny-le-Bretonneux*
>>>
>>> *Tel : +33 1 30 44 04 23 Mobile : +33 7 61 44 40 84*
>>>
>>> *Email :****massinissa.ahmim@linguacustodia.com
>>> <mailto:massinissa.ahmim@linguacustodia.com>***
>>>
>>> *Website :****www.linguacustodia.com
>>> <http://www.linguacustodia.com/> -
>>> www.thetranslationtrustee.com
>>> <http://www.thetranslationtrustee.com>*
>>>
>>> A Young Innovative Company recognised by the French Research
>>> Ministry and Paris Finance Innovation cluster
>>>
>>> Logo labellis? FI small
>>>
>>
>>
>>
>>
>> --
>>
>> Description : Description : lingua_custodia_final full logo
>>
>> */The Translation Trustee/*
>>
>> *1, Place Charles de Gaulle*
>>
>> *78180 Montigny-le-Bretonneux*
>>
>> *Tel : +33 1 30 44 04 23 Mobile : +33 7 61 44 40 84*
>>
>> *Email :****massinissa.ahmim@linguacustodia.com
>> <mailto:massinissa.ahmim@linguacustodia.com>***
>>
>> *Website :****www.linguacustodia.com
>> <http://www.linguacustodia.com/> - www.thetranslationtrustee.com
>> <http://www.thetranslationtrustee.com>*
>>
>> A Young Innovative Company recognised by the French Research
>> Ministry and Paris Finance Innovation cluster
>>
>> Logo labellis? FI small
>>
>
>
>
>
> --
>
> Description : Description : lingua_custodia_final full logo
>
> */The Translation Trustee/*
>
> *1, Place Charles de Gaulle*
>
> *78180 Montigny-le-Bretonneux*
>
> *Tel : +33 1 30 44 04 23 Mobile : +33 7 61 44 40 84*
>
> *Email :****massinissa.ahmim@linguacustodia.com
> <mailto:massinissa.ahmim@linguacustodia.com>***
>
> *Website :****www.linguacustodia.com
> <http://www.linguacustodia.com/> - www.thetranslationtrustee.com
> <http://www.thetranslationtrustee.com>*
>
> A Young Innovative Company recognised by the French Research Ministry
> and Paris Finance Innovation cluster
>
> Logo labellis? FI small
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141201/2a8b0417/attachment.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4421 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20141201/2a8b0417/attachment.jpg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 2720 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20141201/2a8b0417/attachment-0001.jpg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4421 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20141201/2a8b0417/attachment-0002.jpg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 2720 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20141201/2a8b0417/attachment-0003.jpg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4421 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20141201/2a8b0417/attachment-0004.jpg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 2720 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20141201/2a8b0417/attachment-0005.jpg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4421 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20141201/2a8b0417/attachment-0006.jpg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 2720 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20141201/2a8b0417/attachment-0007.jpg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4421 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20141201/2a8b0417/attachment-0008.jpg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 2720 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20141201/2a8b0417/attachment-0009.jpg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4421 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20141201/2a8b0417/attachment-0010.jpg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 2720 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20141201/2a8b0417/attachment-0011.jpg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4421 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20141201/2a8b0417/attachment-0012.jpg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 2720 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20141201/2a8b0417/attachment-0013.jpg

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 98, Issue 14
*********************************************

0 Response to "Moses-support Digest, Vol 98, Issue 14"

Post a Comment

Subscribe to: Post Comments (Atom)