Moses-support Digest, Vol 122, Issue 14

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Conversion of phrase model to PhraseDictionaryCompact
(Hieu Hoang)
2. Re: Conversion of phrase model to PhraseDictionaryCompact
(Shubham Khandelwal)


----------------------------------------------------------------------

Message: 1
Date: Thu, 8 Dec 2016 20:32:19 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Conversion of phrase model to
PhraseDictionaryCompact
To: Shubham Khandelwal <skhlnmiit@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <4f3e63e4-c0a7-6f3a-a5af-d0b2bc0114b0@gmail.com>
Content-Type: text/plain; charset="utf-8"

maybe try

-encoding None


On 08/12/2016 19:44, Shubham Khandelwal wrote:
> Hi Hieu,
>
> Thanks for your reply.
> Yes, I have used the absolute path and also I tried with -T but it did
> not work.
> Is there any other solution to this problem.
>
> Btw, Can anybody please upload the compact model of all pre-made
> models as this will take less space and also it will be very fast
> during decoding.
>
> Thanks.
>
> On Fri, Dec 9, 2016 at 12:50 AM, Hieu Hoang <hieuhoang@gmail.com
> <mailto:hieuhoang@gmail.com>> wrote:
>
> the previous email you referred to says that the directory
> /binarised-model/
> /
> /must exist before you run it, otherwise it will segfault. I would
> also use absolute path to make sure, ie. not
> ////////binarised-model/phrase-table
> ///
> /but
> /
> / /home/shubham/moses/binarised-model/phrase-table
>
> /
> /The previous email exchange also says you should try to add the
> argument
> /
> / -T .
>
> /
>
> Hieu Hoang
> http://www.hoang.co.uk/hieu
>
> On 8 December 2016 at 15:52, Shubham Khandelwal
> <skhlnmiit@gmail.com <mailto:skhlnmiit@gmail.com>> wrote:
>
> Hello,
>
> This is just the reminder of my previous email.
>
> Thanking You.
>
> Regards,
> Shubham
>
> On Thu, Dec 8, 2016 at 9:04 AM, Shubham Khandelwal
> <skhlnmiit@gmail.com <mailto:skhlnmiit@gmail.com>> wrote:
>
> Hello,
>
> I have just downloaded phrase-table.2.gz (18GB) de-en
> model and phrase-table.3.gz (22GB) fr-en model from the
> available pre-made models.
> Now, I am converting them to PhraseDictionaryCompact using
> following command (for exmaple):
> /
> /
> /~/mosesdecoder/bin/processPhraseTableMin -threads all -in
> ~/model/phrase-table.3.gz -nscores 4 -out
> binarised-model/phrase-table
> /
>
> But after passing 1/3, it gave following segementation
> fault error:
> /
> /
> /Pass 1/3: Creating hash function for rank assignment/
> /Segmentation fault (core dumped)/
>
> I have found almost same issue on this thread:
> http://comments.gmane.org/gmane.comp.nlp.moses.user/13033
> <http://comments.gmane.org/gmane.comp.nlp.moses.user/13033>
> However, I have provided the existing /binarised-model
> /folder in the command. Also, I have the write-access in
> /tmp but still it gave sementation fault.
>
> Can you please tell me what could be wrong here ?
>
> Thanking You.
>
> Regards,
> Shubham
>
>
>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
> <http://mailman.mit.edu/mailman/listinfo/moses-support>
>
>
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161208/b3450e99/attachment-0001.html

------------------------------

Message: 2
Date: Fri, 9 Dec 2016 13:48:05 +0530
From: Shubham Khandelwal <skhlnmiit@gmail.com>
Subject: Re: [Moses-support] Conversion of phrase model to
PhraseDictionaryCompact
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAHweNTtpjKrmdeAuizFYjSBHkps6BbsRY8ObBPmngdw6sjBr3A@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hello,

Thanks. It worked now. I have created compact phrase table.
Now, when I am running it using following command:

*~/mosesdecoder/bin/moses
-f ~/Translate/models/de-en/model/moses.ini.2 -threads all*

Then, after creating input-output object, it gave following segmentation
fault error:

Created input-output object : [14.796] seconds
Ich bin ein Student
Line 0: Initialize search took 0.000 seconds total
Translating: <s> Ich bin ein Student </s> ||| [0,0]=X (1) [0,1]=X (1)
[0,2]=X (1) [0,3]=X (1) [0,4]=X (1) [0,5]=X (1) [1,1]=X (1) [1,2]=X (1)
[1,3]=X (1) [1,4]=X (1) [1,5]=X (1) [2,2]=X (1) [2,3]=X (1) [2,4]=X (1)
[2,5]=X (1) [3,3]=X (1) [3,4]=X (1) [3,5]=X (1) [4,4]=X (1) [4,5]=X (1)
[5,5]=X (1)

Segmentation fault (core dumped)

In my machine, I have 40GB RAM but still I am confused why it gave this
error.
Can you please help me in this regard. I have attached moses.ini.2 for your
reference.

Thanks.

Regards,
Shubham

On Fri, Dec 9, 2016 at 2:02 AM, Hieu Hoang <hieuhoang@gmail.com> wrote:

> maybe try
>
> -encoding None
>
> On 08/12/2016 19:44, Shubham Khandelwal wrote:
>
> Hi Hieu,
>
> Thanks for your reply.
> Yes, I have used the absolute path and also I tried with -T but it did not
> work.
> Is there any other solution to this problem.
>
> Btw, Can anybody please upload the compact model of all pre-made models as
> this will take less space and also it will be very fast during decoding.
>
> Thanks.
>
> On Fri, Dec 9, 2016 at 12:50 AM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>
>> the previous email you referred to says that the directory
>>
>> * binarised-model/ *
>>
>> *must exist before you run it, otherwise it will segfault. I would also
>> use absolute path to make sure, ie. not *
>> *binarised-model/phrase-table *
>>
>> *but *
>>
>>
>> * /home/shubham/moses/binarised-model/phrase-table *
>>
>> *The previous email exchange also says you should try to add the argument
>> *
>>
>>
>> * -T . *
>>
>> Hieu Hoang
>> http://www.hoang.co.uk/hieu
>>
>> On 8 December 2016 at 15:52, Shubham Khandelwal <skhlnmiit@gmail.com>
>> wrote:
>>
>>> Hello,
>>>
>>> This is just the reminder of my previous email.
>>>
>>> Thanking You.
>>>
>>> Regards,
>>> Shubham
>>>
>>> On Thu, Dec 8, 2016 at 9:04 AM, Shubham Khandelwal <skhlnmiit@gmail.com>
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>> I have just downloaded phrase-table.2.gz (18GB) de-en model
>>>> and phrase-table.3.gz (22GB) fr-en model from the available pre-made
>>>> models.
>>>> Now, I am converting them to PhraseDictionaryCompact using following
>>>> command (for exmaple):
>>>>
>>>>
>>>> *~/mosesdecoder/bin/processPhraseTableMin -threads all -in
>>>> ~/model/phrase-table.3.gz -nscores 4 -out binarised-model/phrase-table *
>>>>
>>>> But after passing 1/3, it gave following segementation fault error:
>>>>
>>>> *Pass 1/3: Creating hash function for rank assignment*
>>>> *Segmentation fault (core dumped)*
>>>>
>>>> I have found almost same issue on this thread:
>>>> http://comments.gmane.org/gmane.comp.nlp.moses.user/13033
>>>> However, I have provided the existing *binarised-model *folder in the
>>>> command. Also, I have the write-access in /tmp but still it gave
>>>> sementation fault.
>>>>
>>>> Can you please tell me what could be wrong here ?
>>>>
>>>> Thanking You.
>>>>
>>>> Regards,
>>>> Shubham
>>>>
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161209/31f61645/attachment.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: moses.ini.2
Type: application/octet-stream
Size: 944 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20161209/31f61645/attachment.obj

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 122, Issue 14
**********************************************

Tweet